colin
/
onnx-mlir
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
onnx-mlir/.circleci/config.yml

57 lines
1.8 KiB
YAML
Raw Normal View History

enable ci
2019-12-24 05:33:08 +08:00
version: 2
jobs:
build:
docker:
use python image
2019-12-24 06:03:22 +08:00
- image: circleci/python
Enable e2e tests (#29) * Sync with latest MLIR. * Enable ONNX backend tests as a means to test ONNF lowering end-to-end. * Install ONNX using quiet mode. * Remove debug comments. * Install ONNX from third_party/onnx. * Check python version and fix pip command for installing ONNX. * Using --user install option to prevent permission denied. * Remove unused imports. * Try using stock ONNX pip package as there are more tests in them. * Pip got stuck building wheels, try sudo. * Use verbose install to debug. * Invalidate cache to build LLVM tools. * Fix mlir installation script location. * Debug to locate ONNF. * Sanity check. * Check out ONNF code first. * Use verbose LIT output. * 1. Update documentation to always use verbose LIT. 2. Update krnl ops to reflect new affine map attribute syntax. * See if conda exists * Install ONNX by manually cloning the repo. * Install cmake first. * Using sudo priviledge when installing. * Limit build parallelism. * Limit parallelism. * Larger memory. * Install onnx package with pip. * Build MLIR tools. * Invalidate cache. * Compile model.so with -fPIC. * Remove module dump to get concise debug output. * Print command before executing. * Use quiet install mode to reduce logging. * Use -relocation-model=pic to generate position independent code. * 1. Remove MAKEFLAGS because now buildbot has enough memory. 2. Run DocCheck as a last step. * 1. Add verbose mode for backtend test. * When dumping to LLVM bitcode, do not dump module IR, but print a message indicating that bitcode has been written to disk. * Do not pass MakeFlags to CMake. * Add more explaination for posible reasons of failing to identify tests.
2020-01-21 01:30:08 +08:00
resource_class: medium+
enable ci
2019-12-24 05:33:08 +08:00
steps:
running out of memory, limit parallelism
2019-12-24 07:19:55 +08:00
- run:
1. Simplify CircleCI build scripts. 2. Refactor mlir installation script to a standalone shell script.
2019-12-31 14:45:32 +08:00
name: Installing GCC, CMake, Ninja, Protobuf
1. Remove quotes from strings in yaml file for better consistency. 2. Use MAKEFLAGS env variable to limit build parallelism so that mlir installation script remains clean.
2020-01-01 02:28:26 +08:00
command: sudo apt-get update && sudo apt-get install -y gcc g++ cmake ninja-build protobuf-compiler
Enable e2e tests (#29) * Sync with latest MLIR. * Enable ONNX backend tests as a means to test ONNF lowering end-to-end. * Install ONNX using quiet mode. * Remove debug comments. * Install ONNX from third_party/onnx. * Check python version and fix pip command for installing ONNX. * Using --user install option to prevent permission denied. * Remove unused imports. * Try using stock ONNX pip package as there are more tests in them. * Pip got stuck building wheels, try sudo. * Use verbose install to debug. * Invalidate cache to build LLVM tools. * Fix mlir installation script location. * Debug to locate ONNF. * Sanity check. * Check out ONNF code first. * Use verbose LIT output. * 1. Update documentation to always use verbose LIT. 2. Update krnl ops to reflect new affine map attribute syntax. * See if conda exists * Install ONNX by manually cloning the repo. * Install cmake first. * Using sudo priviledge when installing. * Limit build parallelism. * Limit parallelism. * Larger memory. * Install onnx package with pip. * Build MLIR tools. * Invalidate cache. * Compile model.so with -fPIC. * Remove module dump to get concise debug output. * Print command before executing. * Use quiet install mode to reduce logging. * Use -relocation-model=pic to generate position independent code. * 1. Remove MAKEFLAGS because now buildbot has enough memory. 2. Run DocCheck as a last step. * 1. Add verbose mode for backtend test. * When dumping to LLVM bitcode, do not dump module IR, but print a message indicating that bitcode has been written to disk. * Do not pass MakeFlags to CMake. * Add more explaination for posible reasons of failing to identify tests.
2020-01-21 01:30:08 +08:00
- checkout:
[NFC] Change ONNF based names to ONNX-MLIR (#32) * Rename onnf to onnx-mlir. * Change workspace name.
2020-03-17 21:16:33 +08:00
path: onnx-mlir
Enable e2e tests (#29) * Sync with latest MLIR. * Enable ONNX backend tests as a means to test ONNF lowering end-to-end. * Install ONNX using quiet mode. * Remove debug comments. * Install ONNX from third_party/onnx. * Check python version and fix pip command for installing ONNX. * Using --user install option to prevent permission denied. * Remove unused imports. * Try using stock ONNX pip package as there are more tests in them. * Pip got stuck building wheels, try sudo. * Use verbose install to debug. * Invalidate cache to build LLVM tools. * Fix mlir installation script location. * Debug to locate ONNF. * Sanity check. * Check out ONNF code first. * Use verbose LIT output. * 1. Update documentation to always use verbose LIT. 2. Update krnl ops to reflect new affine map attribute syntax. * See if conda exists * Install ONNX by manually cloning the repo. * Install cmake first. * Using sudo priviledge when installing. * Limit build parallelism. * Limit parallelism. * Larger memory. * Install onnx package with pip. * Build MLIR tools. * Invalidate cache. * Compile model.so with -fPIC. * Remove module dump to get concise debug output. * Print command before executing. * Use quiet install mode to reduce logging. * Use -relocation-model=pic to generate position independent code. * 1. Remove MAKEFLAGS because now buildbot has enough memory. 2. Run DocCheck as a last step. * 1. Add verbose mode for backtend test. * When dumping to LLVM bitcode, do not dump module IR, but print a message indicating that bitcode has been written to disk. * Do not pass MakeFlags to CMake. * Add more explaination for posible reasons of failing to identify tests.
2020-01-21 01:30:08 +08:00
- run:
name: Pull Submodules
command: |
[NFC] Change ONNF based names to ONNX-MLIR (#32) * Rename onnf to onnx-mlir. * Change workspace name.
2020-03-17 21:16:33 +08:00
cd onnx-mlir
Enable e2e tests (#29) * Sync with latest MLIR. * Enable ONNX backend tests as a means to test ONNF lowering end-to-end. * Install ONNX using quiet mode. * Remove debug comments. * Install ONNX from third_party/onnx. * Check python version and fix pip command for installing ONNX. * Using --user install option to prevent permission denied. * Remove unused imports. * Try using stock ONNX pip package as there are more tests in them. * Pip got stuck building wheels, try sudo. * Use verbose install to debug. * Invalidate cache to build LLVM tools. * Fix mlir installation script location. * Debug to locate ONNF. * Sanity check. * Check out ONNF code first. * Use verbose LIT output. * 1. Update documentation to always use verbose LIT. 2. Update krnl ops to reflect new affine map attribute syntax. * See if conda exists * Install ONNX by manually cloning the repo. * Install cmake first. * Using sudo priviledge when installing. * Limit build parallelism. * Limit parallelism. * Larger memory. * Install onnx package with pip. * Build MLIR tools. * Invalidate cache. * Compile model.so with -fPIC. * Remove module dump to get concise debug output. * Print command before executing. * Use quiet install mode to reduce logging. * Use -relocation-model=pic to generate position independent code. * 1. Remove MAKEFLAGS because now buildbot has enough memory. 2. Run DocCheck as a last step. * 1. Add verbose mode for backtend test. * When dumping to LLVM bitcode, do not dump module IR, but print a message indicating that bitcode has been written to disk. * Do not pass MakeFlags to CMake. * Add more explaination for posible reasons of failing to identify tests.
2020-01-21 01:30:08 +08:00
git submodule update --init --recursive
1. Simplify CircleCI build scripts. 2. Refactor mlir installation script to a standalone shell script.
2019-12-31 14:45:32 +08:00
# Use cached mlir installation if possible.
cache MLIR build
2019-12-24 13:09:31 +08:00
- restore_cache:
Use the exact commit id specified in clang-ykt/llvm-project.
2020-03-03 00:18:59 +08:00
key: V9-LLVM-PROJECT-{{ arch }}
enable ci
2019-12-24 05:33:08 +08:00
- run:
running out of memory, limit parallelism
2019-12-24 07:19:55 +08:00
name: Install MLIR
set up buildbot
2019-12-24 05:53:08 +08:00
command: |
1. Simplify CircleCI build scripts. 2. Refactor mlir installation script to a standalone shell script.
2019-12-31 14:45:32 +08:00
# Check whether cache restoration succeeds by checking whether
# mlir-opt executable exists.
fix cache checking logic
2019-12-24 13:48:42 +08:00
if [ ! -f llvm-project/build/bin/mlir-opt ]; then
Resolve buildbot issue. (#64) * 1. Try limit parallelism. * Limit build parallelism to 4. * Fix doc-check failure. Co-authored-by: Gheorghe-Teodor Bercea <gt.bercea@gmail.com>
2020-01-31 23:24:45 +08:00
export MAKEFLAGS=-j4
[NFC] Change ONNF based names to ONNX-MLIR (#32) * Rename onnf to onnx-mlir. * Change workspace name.
2020-03-17 21:16:33 +08:00
source onnx-mlir/utils/install-mlir.sh
cache MLIR build
2019-12-24 13:09:31 +08:00
fi
- save_cache:
Use the exact commit id specified in clang-ykt/llvm-project.
2020-03-03 00:18:59 +08:00
key: V9-LLVM-PROJECT-{{ arch }}
cache MLIR build
2019-12-24 13:09:31 +08:00
paths:
- llvm-project
[RFC] Doc-check utility. (#12) * 1. Implement doc-check utility. * 1. Move ONNF installation script to a standalone script file. * 1. Modify build script to install llvm-project next to ONNF. The build script used to install llvm-project inside ONNF, which didn't make sense. * 1. Check out code to ONNF directory. * 1. Pass path parameter correctly. * 1. Debugging buildbot. * 1. Remove debug code. * 1. Update installation instructions in README.md. 2. Enforce consistency with scripts used in testing using doc-check. * 1. Fix error with respect to syntax to build multiple CMake targets. * 1. Move doc-check to doc_check. 2. Remove directive_config in top-level driver. * 1. Build onnf and check-mlir-lit separately because only CMake 3.15+ supports building multiple targets in one cmake --build run. * 1. Use new env variables to locate LLVM-Project. * 1. Documentation nits. * 1. Prettify buildbot scripts. * 1. Fix build script error. * 1. Support exclude_dirs in DocCheck. 2. Add README for DocCheck. * 1. Mark python3 interpreter as required. 2. Use imported interpreter target. * 1. Automatically deduce doc file extension in DocCheckCtx. 2. Rename ctx.open -> ctx.open_doc since it should only be used to open doc file. 3. Always read line in parser, instead of reading lines in driver and then passing it to parser.py. * 1. Rename parser -> doc_parser due to name conflict with python built-in module. 2. Explose doc_check module directory first before importing; otherwise if the doc_check utility is invoked by other script, importing will not work correctly. * 1. Keep renaming parser -> doc_parser. 2. Explicitly define a default configuration parser that parses the configuration into a python dictionary. * 1. Add test for doc-check. 2. Exclude doc-check tests from project dock-check because base directory is different. * 1. Raise ValueError if directive configuration fails to parse. 2. Format code. * Shorten test case documentation. Show example of using same-as-file directive, check with DocCheck. * 1. Shorten test case documentation. 2. More documentation, check documentation with DocCheck. * 1. Add copyright notice. * 1. Make documentation clearer. 2. Prettify build-scripts. * 1. Provide more documentation. 2. Fix some non-compliance with pep8 recommendations. Co-authored-by: Gheorghe-Teodor Bercea <gt.bercea@gmail.com>
2020-01-10 07:35:52 +08:00
- run:
[NFC] Change ONNF based names to ONNX-MLIR (#32) * Rename onnf to onnx-mlir. * Change workspace name.
2020-03-17 21:16:33 +08:00
name: Install ONNX MLIR
command: source onnx-mlir/utils/install-onnx-mlir.sh
Enable e2e tests (#29) * Sync with latest MLIR. * Enable ONNX backend tests as a means to test ONNF lowering end-to-end. * Install ONNX using quiet mode. * Remove debug comments. * Install ONNX from third_party/onnx. * Check python version and fix pip command for installing ONNX. * Using --user install option to prevent permission denied. * Remove unused imports. * Try using stock ONNX pip package as there are more tests in them. * Pip got stuck building wheels, try sudo. * Use verbose install to debug. * Invalidate cache to build LLVM tools. * Fix mlir installation script location. * Debug to locate ONNF. * Sanity check. * Check out ONNF code first. * Use verbose LIT output. * 1. Update documentation to always use verbose LIT. 2. Update krnl ops to reflect new affine map attribute syntax. * See if conda exists * Install ONNX by manually cloning the repo. * Install cmake first. * Using sudo priviledge when installing. * Limit build parallelism. * Limit parallelism. * Larger memory. * Install onnx package with pip. * Build MLIR tools. * Invalidate cache. * Compile model.so with -fPIC. * Remove module dump to get concise debug output. * Print command before executing. * Use quiet install mode to reduce logging. * Use -relocation-model=pic to generate position independent code. * 1. Remove MAKEFLAGS because now buildbot has enough memory. 2. Run DocCheck as a last step. * 1. Add verbose mode for backtend test. * When dumping to LLVM bitcode, do not dump module IR, but print a message indicating that bitcode has been written to disk. * Do not pass MakeFlags to CMake. * Add more explaination for posible reasons of failing to identify tests.
2020-01-21 01:30:08 +08:00
- run:
name: Run End-To-End Tests
command: |
[NFC] Change ONNF based names to ONNX-MLIR (#32) * Rename onnf to onnx-mlir. * Change workspace name.
2020-03-17 21:16:33 +08:00
sudo pip install -q -e ./onnx-mlir/third_party/onnx
cd onnx-mlir/build
Enable e2e tests (#29) * Sync with latest MLIR. * Enable ONNX backend tests as a means to test ONNF lowering end-to-end. * Install ONNX using quiet mode. * Remove debug comments. * Install ONNX from third_party/onnx. * Check python version and fix pip command for installing ONNX. * Using --user install option to prevent permission denied. * Remove unused imports. * Try using stock ONNX pip package as there are more tests in them. * Pip got stuck building wheels, try sudo. * Use verbose install to debug. * Invalidate cache to build LLVM tools. * Fix mlir installation script location. * Debug to locate ONNF. * Sanity check. * Check out ONNF code first. * Use verbose LIT output. * 1. Update documentation to always use verbose LIT. 2. Update krnl ops to reflect new affine map attribute syntax. * See if conda exists * Install ONNX by manually cloning the repo. * Install cmake first. * Using sudo priviledge when installing. * Limit build parallelism. * Limit parallelism. * Larger memory. * Install onnx package with pip. * Build MLIR tools. * Invalidate cache. * Compile model.so with -fPIC. * Remove module dump to get concise debug output. * Print command before executing. * Use quiet install mode to reduce logging. * Use -relocation-model=pic to generate position independent code. * 1. Remove MAKEFLAGS because now buildbot has enough memory. 2. Run DocCheck as a last step. * 1. Add verbose mode for backtend test. * When dumping to LLVM bitcode, do not dump module IR, but print a message indicating that bitcode has been written to disk. * Do not pass MakeFlags to CMake. * Add more explaination for posible reasons of failing to identify tests.
2020-01-21 01:30:08 +08:00
cmake --build . --target run-onnx-backend-test
[RFC] Doc-check utility. (#12) * 1. Implement doc-check utility. * 1. Move ONNF installation script to a standalone script file. * 1. Modify build script to install llvm-project next to ONNF. The build script used to install llvm-project inside ONNF, which didn't make sense. * 1. Check out code to ONNF directory. * 1. Pass path parameter correctly. * 1. Debugging buildbot. * 1. Remove debug code. * 1. Update installation instructions in README.md. 2. Enforce consistency with scripts used in testing using doc-check. * 1. Fix error with respect to syntax to build multiple CMake targets. * 1. Move doc-check to doc_check. 2. Remove directive_config in top-level driver. * 1. Build onnf and check-mlir-lit separately because only CMake 3.15+ supports building multiple targets in one cmake --build run. * 1. Use new env variables to locate LLVM-Project. * 1. Documentation nits. * 1. Prettify buildbot scripts. * 1. Fix build script error. * 1. Support exclude_dirs in DocCheck. 2. Add README for DocCheck. * 1. Mark python3 interpreter as required. 2. Use imported interpreter target. * 1. Automatically deduce doc file extension in DocCheckCtx. 2. Rename ctx.open -> ctx.open_doc since it should only be used to open doc file. 3. Always read line in parser, instead of reading lines in driver and then passing it to parser.py. * 1. Rename parser -> doc_parser due to name conflict with python built-in module. 2. Explose doc_check module directory first before importing; otherwise if the doc_check utility is invoked by other script, importing will not work correctly. * 1. Keep renaming parser -> doc_parser. 2. Explicitly define a default configuration parser that parses the configuration into a python dictionary. * 1. Add test for doc-check. 2. Exclude doc-check tests from project dock-check because base directory is different. * 1. Raise ValueError if directive configuration fails to parse. 2. Format code. * Shorten test case documentation. Show example of using same-as-file directive, check with DocCheck. * 1. Shorten test case documentation. 2. More documentation, check documentation with DocCheck. * 1. Add copyright notice. * 1. Make documentation clearer. 2. Prettify build-scripts. * 1. Provide more documentation. 2. Fix some non-compliance with pep8 recommendations. Co-authored-by: Gheorghe-Teodor Bercea <gt.bercea@gmail.com>
2020-01-10 07:35:52 +08:00
- run:
name: Run DocCheck
[NFC] Change ONNF based names to ONNX-MLIR (#32) * Rename onnf to onnx-mlir. * Change workspace name.
2020-03-17 21:16:33 +08:00
command: cd onnx-mlir/build && cmake --build . --target check-doc
Using Tablegen to Generate Op Documentation (#74) * Add dialect documentation. * Add a step in our CI to ensure documentation is up-to-date. * Add dialect documentation. * Fix config file mistake, using multi-line commands. * Fix a bug in DocCheck.
2020-02-11 03:18:54 +08:00
- run:
name: Ensure tablegen documentation is up-to-date
command: |
[NFC] Change ONNF based names to ONNX-MLIR (#32) * Rename onnf to onnx-mlir. * Change workspace name.
2020-03-17 21:16:33 +08:00
cd onnx-mlir/build
cmake --build . --target onnx-mlir-doc
Using Tablegen to Generate Op Documentation (#74) * Add dialect documentation. * Add a step in our CI to ensure documentation is up-to-date. * Add dialect documentation. * Fix config file mistake, using multi-line commands. * Fix a bug in DocCheck.
2020-02-11 03:18:54 +08:00
# Check whether dialect documentation is up-to-date.
diff doc/Dialects ../doc/Dialects
enable ci
2019-12-24 05:33:08 +08:00
- run:
name: Print the Current Time
command: date