onnx-mlir/.circleci/config.yml

57 lines
1.8 KiB
YAML
Raw Normal View History

2019-12-24 05:33:08 +08:00
version: 2
jobs:
build:
docker:
2019-12-24 06:03:22 +08:00
- image: circleci/python
Enable e2e tests (#29) * Sync with latest MLIR. * Enable ONNX backend tests as a means to test ONNF lowering end-to-end. * Install ONNX using quiet mode. * Remove debug comments. * Install ONNX from third_party/onnx. * Check python version and fix pip command for installing ONNX. * Using --user install option to prevent permission denied. * Remove unused imports. * Try using stock ONNX pip package as there are more tests in them. * Pip got stuck building wheels, try sudo. * Use verbose install to debug. * Invalidate cache to build LLVM tools. * Fix mlir installation script location. * Debug to locate ONNF. * Sanity check. * Check out ONNF code first. * Use verbose LIT output. * 1. Update documentation to always use verbose LIT. 2. Update krnl ops to reflect new affine map attribute syntax. * See if conda exists * Install ONNX by manually cloning the repo. * Install cmake first. * Using sudo priviledge when installing. * Limit build parallelism. * Limit parallelism. * Larger memory. * Install onnx package with pip. * Build MLIR tools. * Invalidate cache. * Compile model.so with -fPIC. * Remove module dump to get concise debug output. * Print command before executing. * Use quiet install mode to reduce logging. * Use -relocation-model=pic to generate position independent code. * 1. Remove MAKEFLAGS because now buildbot has enough memory. 2. Run DocCheck as a last step. * 1. Add verbose mode for backtend test. * When dumping to LLVM bitcode, do not dump module IR, but print a message indicating that bitcode has been written to disk. * Do not pass MakeFlags to CMake. * Add more explaination for posible reasons of failing to identify tests.
2020-01-21 01:30:08 +08:00
resource_class: medium+
2019-12-24 05:33:08 +08:00
steps:
- run:
name: Installing GCC, CMake, Ninja, Protobuf
command: sudo apt-get update && sudo apt-get install -y gcc g++ cmake ninja-build protobuf-compiler
Enable e2e tests (#29) * Sync with latest MLIR. * Enable ONNX backend tests as a means to test ONNF lowering end-to-end. * Install ONNX using quiet mode. * Remove debug comments. * Install ONNX from third_party/onnx. * Check python version and fix pip command for installing ONNX. * Using --user install option to prevent permission denied. * Remove unused imports. * Try using stock ONNX pip package as there are more tests in them. * Pip got stuck building wheels, try sudo. * Use verbose install to debug. * Invalidate cache to build LLVM tools. * Fix mlir installation script location. * Debug to locate ONNF. * Sanity check. * Check out ONNF code first. * Use verbose LIT output. * 1. Update documentation to always use verbose LIT. 2. Update krnl ops to reflect new affine map attribute syntax. * See if conda exists * Install ONNX by manually cloning the repo. * Install cmake first. * Using sudo priviledge when installing. * Limit build parallelism. * Limit parallelism. * Larger memory. * Install onnx package with pip. * Build MLIR tools. * Invalidate cache. * Compile model.so with -fPIC. * Remove module dump to get concise debug output. * Print command before executing. * Use quiet install mode to reduce logging. * Use -relocation-model=pic to generate position independent code. * 1. Remove MAKEFLAGS because now buildbot has enough memory. 2. Run DocCheck as a last step. * 1. Add verbose mode for backtend test. * When dumping to LLVM bitcode, do not dump module IR, but print a message indicating that bitcode has been written to disk. * Do not pass MakeFlags to CMake. * Add more explaination for posible reasons of failing to identify tests.
2020-01-21 01:30:08 +08:00
- checkout:
path: onnx-mlir
Enable e2e tests (#29) * Sync with latest MLIR. * Enable ONNX backend tests as a means to test ONNF lowering end-to-end. * Install ONNX using quiet mode. * Remove debug comments. * Install ONNX from third_party/onnx. * Check python version and fix pip command for installing ONNX. * Using --user install option to prevent permission denied. * Remove unused imports. * Try using stock ONNX pip package as there are more tests in them. * Pip got stuck building wheels, try sudo. * Use verbose install to debug. * Invalidate cache to build LLVM tools. * Fix mlir installation script location. * Debug to locate ONNF. * Sanity check. * Check out ONNF code first. * Use verbose LIT output. * 1. Update documentation to always use verbose LIT. 2. Update krnl ops to reflect new affine map attribute syntax. * See if conda exists * Install ONNX by manually cloning the repo. * Install cmake first. * Using sudo priviledge when installing. * Limit build parallelism. * Limit parallelism. * Larger memory. * Install onnx package with pip. * Build MLIR tools. * Invalidate cache. * Compile model.so with -fPIC. * Remove module dump to get concise debug output. * Print command before executing. * Use quiet install mode to reduce logging. * Use -relocation-model=pic to generate position independent code. * 1. Remove MAKEFLAGS because now buildbot has enough memory. 2. Run DocCheck as a last step. * 1. Add verbose mode for backtend test. * When dumping to LLVM bitcode, do not dump module IR, but print a message indicating that bitcode has been written to disk. * Do not pass MakeFlags to CMake. * Add more explaination for posible reasons of failing to identify tests.
2020-01-21 01:30:08 +08:00
- run:
name: Pull Submodules
command: |
cd onnx-mlir
Enable e2e tests (#29) * Sync with latest MLIR. * Enable ONNX backend tests as a means to test ONNF lowering end-to-end. * Install ONNX using quiet mode. * Remove debug comments. * Install ONNX from third_party/onnx. * Check python version and fix pip command for installing ONNX. * Using --user install option to prevent permission denied. * Remove unused imports. * Try using stock ONNX pip package as there are more tests in them. * Pip got stuck building wheels, try sudo. * Use verbose install to debug. * Invalidate cache to build LLVM tools. * Fix mlir installation script location. * Debug to locate ONNF. * Sanity check. * Check out ONNF code first. * Use verbose LIT output. * 1. Update documentation to always use verbose LIT. 2. Update krnl ops to reflect new affine map attribute syntax. * See if conda exists * Install ONNX by manually cloning the repo. * Install cmake first. * Using sudo priviledge when installing. * Limit build parallelism. * Limit parallelism. * Larger memory. * Install onnx package with pip. * Build MLIR tools. * Invalidate cache. * Compile model.so with -fPIC. * Remove module dump to get concise debug output. * Print command before executing. * Use quiet install mode to reduce logging. * Use -relocation-model=pic to generate position independent code. * 1. Remove MAKEFLAGS because now buildbot has enough memory. 2. Run DocCheck as a last step. * 1. Add verbose mode for backtend test. * When dumping to LLVM bitcode, do not dump module IR, but print a message indicating that bitcode has been written to disk. * Do not pass MakeFlags to CMake. * Add more explaination for posible reasons of failing to identify tests.
2020-01-21 01:30:08 +08:00
git submodule update --init --recursive
# Use cached mlir installation if possible.
2019-12-24 13:09:31 +08:00
- restore_cache:
key: V9-LLVM-PROJECT-{{ arch }}
2019-12-24 05:33:08 +08:00
- run:
name: Install MLIR
2019-12-24 05:53:08 +08:00
command: |
# Check whether cache restoration succeeds by checking whether
# mlir-opt executable exists.
2019-12-24 13:48:42 +08:00
if [ ! -f llvm-project/build/bin/mlir-opt ]; then
export MAKEFLAGS=-j4
source onnx-mlir/utils/install-mlir.sh
2019-12-24 13:09:31 +08:00
fi
- save_cache:
key: V9-LLVM-PROJECT-{{ arch }}
2019-12-24 13:09:31 +08:00
paths:
- llvm-project
[RFC] Doc-check utility. (#12) * 1. Implement doc-check utility. * 1. Move ONNF installation script to a standalone script file. * 1. Modify build script to install llvm-project next to ONNF. The build script used to install llvm-project inside ONNF, which didn't make sense. * 1. Check out code to ONNF directory. * 1. Pass path parameter correctly. * 1. Debugging buildbot. * 1. Remove debug code. * 1. Update installation instructions in README.md. 2. Enforce consistency with scripts used in testing using doc-check. * 1. Fix error with respect to syntax to build multiple CMake targets. * 1. Move doc-check to doc_check. 2. Remove directive_config in top-level driver. * 1. Build onnf and check-mlir-lit separately because only CMake 3.15+ supports building multiple targets in one cmake --build run. * 1. Use new env variables to locate LLVM-Project. * 1. Documentation nits. * 1. Prettify buildbot scripts. * 1. Fix build script error. * 1. Support exclude_dirs in DocCheck. 2. Add README for DocCheck. * 1. Mark python3 interpreter as required. 2. Use imported interpreter target. * 1. Automatically deduce doc file extension in DocCheckCtx. 2. Rename ctx.open -> ctx.open_doc since it should only be used to open doc file. 3. Always read line in parser, instead of reading lines in driver and then passing it to parser.py. * 1. Rename parser -> doc_parser due to name conflict with python built-in module. 2. Explose doc_check module directory first before importing; otherwise if the doc_check utility is invoked by other script, importing will not work correctly. * 1. Keep renaming parser -> doc_parser. 2. Explicitly define a default configuration parser that parses the configuration into a python dictionary. * 1. Add test for doc-check. 2. Exclude doc-check tests from project dock-check because base directory is different. * 1. Raise ValueError if directive configuration fails to parse. 2. Format code. * Shorten test case documentation. Show example of using same-as-file directive, check with DocCheck. * 1. Shorten test case documentation. 2. More documentation, check documentation with DocCheck. * 1. Add copyright notice. * 1. Make documentation clearer. 2. Prettify build-scripts. * 1. Provide more documentation. 2. Fix some non-compliance with pep8 recommendations. Co-authored-by: Gheorghe-Teodor Bercea <gt.bercea@gmail.com>
2020-01-10 07:35:52 +08:00
- run:
name: Install ONNX MLIR
command: source onnx-mlir/utils/install-onnx-mlir.sh
Enable e2e tests (#29) * Sync with latest MLIR. * Enable ONNX backend tests as a means to test ONNF lowering end-to-end. * Install ONNX using quiet mode. * Remove debug comments. * Install ONNX from third_party/onnx. * Check python version and fix pip command for installing ONNX. * Using --user install option to prevent permission denied. * Remove unused imports. * Try using stock ONNX pip package as there are more tests in them. * Pip got stuck building wheels, try sudo. * Use verbose install to debug. * Invalidate cache to build LLVM tools. * Fix mlir installation script location. * Debug to locate ONNF. * Sanity check. * Check out ONNF code first. * Use verbose LIT output. * 1. Update documentation to always use verbose LIT. 2. Update krnl ops to reflect new affine map attribute syntax. * See if conda exists * Install ONNX by manually cloning the repo. * Install cmake first. * Using sudo priviledge when installing. * Limit build parallelism. * Limit parallelism. * Larger memory. * Install onnx package with pip. * Build MLIR tools. * Invalidate cache. * Compile model.so with -fPIC. * Remove module dump to get concise debug output. * Print command before executing. * Use quiet install mode to reduce logging. * Use -relocation-model=pic to generate position independent code. * 1. Remove MAKEFLAGS because now buildbot has enough memory. 2. Run DocCheck as a last step. * 1. Add verbose mode for backtend test. * When dumping to LLVM bitcode, do not dump module IR, but print a message indicating that bitcode has been written to disk. * Do not pass MakeFlags to CMake. * Add more explaination for posible reasons of failing to identify tests.
2020-01-21 01:30:08 +08:00
- run:
name: Run End-To-End Tests
command: |
sudo pip install -q -e ./onnx-mlir/third_party/onnx
cd onnx-mlir/build
Enable e2e tests (#29) * Sync with latest MLIR. * Enable ONNX backend tests as a means to test ONNF lowering end-to-end. * Install ONNX using quiet mode. * Remove debug comments. * Install ONNX from third_party/onnx. * Check python version and fix pip command for installing ONNX. * Using --user install option to prevent permission denied. * Remove unused imports. * Try using stock ONNX pip package as there are more tests in them. * Pip got stuck building wheels, try sudo. * Use verbose install to debug. * Invalidate cache to build LLVM tools. * Fix mlir installation script location. * Debug to locate ONNF. * Sanity check. * Check out ONNF code first. * Use verbose LIT output. * 1. Update documentation to always use verbose LIT. 2. Update krnl ops to reflect new affine map attribute syntax. * See if conda exists * Install ONNX by manually cloning the repo. * Install cmake first. * Using sudo priviledge when installing. * Limit build parallelism. * Limit parallelism. * Larger memory. * Install onnx package with pip. * Build MLIR tools. * Invalidate cache. * Compile model.so with -fPIC. * Remove module dump to get concise debug output. * Print command before executing. * Use quiet install mode to reduce logging. * Use -relocation-model=pic to generate position independent code. * 1. Remove MAKEFLAGS because now buildbot has enough memory. 2. Run DocCheck as a last step. * 1. Add verbose mode for backtend test. * When dumping to LLVM bitcode, do not dump module IR, but print a message indicating that bitcode has been written to disk. * Do not pass MakeFlags to CMake. * Add more explaination for posible reasons of failing to identify tests.
2020-01-21 01:30:08 +08:00
cmake --build . --target run-onnx-backend-test
[RFC] Doc-check utility. (#12) * 1. Implement doc-check utility. * 1. Move ONNF installation script to a standalone script file. * 1. Modify build script to install llvm-project next to ONNF. The build script used to install llvm-project inside ONNF, which didn't make sense. * 1. Check out code to ONNF directory. * 1. Pass path parameter correctly. * 1. Debugging buildbot. * 1. Remove debug code. * 1. Update installation instructions in README.md. 2. Enforce consistency with scripts used in testing using doc-check. * 1. Fix error with respect to syntax to build multiple CMake targets. * 1. Move doc-check to doc_check. 2. Remove directive_config in top-level driver. * 1. Build onnf and check-mlir-lit separately because only CMake 3.15+ supports building multiple targets in one cmake --build run. * 1. Use new env variables to locate LLVM-Project. * 1. Documentation nits. * 1. Prettify buildbot scripts. * 1. Fix build script error. * 1. Support exclude_dirs in DocCheck. 2. Add README for DocCheck. * 1. Mark python3 interpreter as required. 2. Use imported interpreter target. * 1. Automatically deduce doc file extension in DocCheckCtx. 2. Rename ctx.open -> ctx.open_doc since it should only be used to open doc file. 3. Always read line in parser, instead of reading lines in driver and then passing it to parser.py. * 1. Rename parser -> doc_parser due to name conflict with python built-in module. 2. Explose doc_check module directory first before importing; otherwise if the doc_check utility is invoked by other script, importing will not work correctly. * 1. Keep renaming parser -> doc_parser. 2. Explicitly define a default configuration parser that parses the configuration into a python dictionary. * 1. Add test for doc-check. 2. Exclude doc-check tests from project dock-check because base directory is different. * 1. Raise ValueError if directive configuration fails to parse. 2. Format code. * Shorten test case documentation. Show example of using same-as-file directive, check with DocCheck. * 1. Shorten test case documentation. 2. More documentation, check documentation with DocCheck. * 1. Add copyright notice. * 1. Make documentation clearer. 2. Prettify build-scripts. * 1. Provide more documentation. 2. Fix some non-compliance with pep8 recommendations. Co-authored-by: Gheorghe-Teodor Bercea <gt.bercea@gmail.com>
2020-01-10 07:35:52 +08:00
- run:
name: Run DocCheck
command: cd onnx-mlir/build && cmake --build . --target check-doc
- run:
name: Ensure tablegen documentation is up-to-date
command: |
cd onnx-mlir/build
cmake --build . --target onnx-mlir-doc
# Check whether dialect documentation is up-to-date.
diff doc/Dialects ../doc/Dialects
2019-12-24 05:33:08 +08:00
- run:
name: Print the Current Time
command: date