Commit Graph

47 Commits

Author SHA1 Message Date
Tian Jin 5791cfdce7
Resolve buildbot issue. (#64)
* 1. Try limit parallelism.

* Limit build parallelism to 4.

* Fix doc-check failure.

Co-authored-by: Gheorghe-Teodor Bercea <gt.bercea@gmail.com>
2020-01-31 23:24:45 +08:00
Tian Jin 8665ecd998
Enable e2e tests (#29)
* Sync with latest MLIR.

* Enable ONNX backend tests as a means to test ONNF lowering end-to-end.

* Install ONNX using quiet mode.

* Remove debug comments.

* Install ONNX from third_party/onnx.

* Check python version and fix pip command for installing ONNX.

* Using --user install option to prevent permission denied.

* Remove unused imports.

* Try using stock ONNX pip package as there are more tests in them.

* Pip got stuck building wheels, try sudo.

* Use verbose install to debug.

* Invalidate cache to build LLVM tools.

* Fix mlir installation script location.

* Debug to locate ONNF.

* Sanity check.

* Check out ONNF code first.

* Use verbose LIT output.

* 1. Update documentation to always use verbose LIT.
2. Update krnl ops to reflect new affine map attribute syntax.

* See if conda exists

* Install ONNX by manually cloning the repo.

* Install cmake first.

* Using sudo priviledge when installing.

* Limit build parallelism.

* Limit parallelism.

* Larger memory.

* Install onnx package with pip.

* Build MLIR tools.

* Invalidate cache.

* Compile model.so with -fPIC.

* Remove module dump to get concise debug output.

* Print command before executing.

* Use quiet install mode to reduce logging.

* Use -relocation-model=pic to generate position independent code.

* 1. Remove MAKEFLAGS because now buildbot has enough memory.
2. Run DocCheck as a last step.

* 1. Add verbose mode for backtend test.

* When dumping to LLVM bitcode, do not dump module IR, but print a message indicating that bitcode has been written to disk.

* Do not pass MakeFlags to CMake.

* Add more explaination for posible reasons of failing to identify tests.
2020-01-20 12:30:08 -05:00
Tian Jin 1ebcc2eb64
[RFC] Doc-check utility. (#12)
* 1. Implement doc-check utility.

* 1. Move ONNF installation script to a standalone script file.

* 1. Modify build script to install llvm-project next to ONNF. The build script used to install llvm-project inside ONNF, which didn't make sense.

* 1. Check out code to ONNF directory.

* 1. Pass path parameter correctly.

* 1. Debugging buildbot.

* 1. Remove debug code.

* 1. Update installation instructions in README.md.
2. Enforce consistency with scripts used in testing using doc-check.

* 1. Fix error with respect to syntax to build multiple CMake targets.

* 1. Move doc-check to doc_check.
2. Remove directive_config in top-level driver.

* 1. Build onnf and check-mlir-lit separately because only CMake 3.15+ supports building multiple targets in one cmake --build run.

* 1. Use new env variables to locate LLVM-Project.

* 1. Documentation nits.

* 1. Prettify buildbot scripts.

* 1. Fix build script error.

* 1. Support exclude_dirs in DocCheck.
2. Add README for DocCheck.

* 1. Mark python3 interpreter as required.
2. Use imported interpreter target.

* 1. Automatically deduce doc file extension in DocCheckCtx.
2. Rename ctx.open -> ctx.open_doc since it should only be used to open doc file.
3. Always read line in parser, instead of reading lines in driver and then passing it to parser.py.

* 1. Rename parser -> doc_parser due to name conflict with python built-in module.
2. Explose doc_check module directory first before importing; otherwise if the doc_check utility is invoked by other script, importing will not work correctly.

* 1. Keep renaming parser -> doc_parser.
2. Explicitly define a default configuration parser that parses the configuration into a python dictionary.

* 1. Add test for doc-check.
2. Exclude doc-check tests from project dock-check because base directory is different.

* 1. Raise ValueError if directive configuration fails to parse.
2. Format code.

* Shorten test case documentation.
Show example of using same-as-file directive, check with DocCheck.

* 1. Shorten test case documentation.
2. More documentation, check documentation with DocCheck.

* 1. Add copyright notice.

* 1. Make documentation clearer.
2. Prettify build-scripts.

* 1. Provide more documentation.
2. Fix some non-compliance with pep8 recommendations.

Co-authored-by: Gheorghe-Teodor Bercea <gt.bercea@gmail.com>
2020-01-09 18:35:52 -05:00
Tian Jin 38e7d2d068 Update LLVM_SRC, LLVM_BUILD env vars to LLVM_PROJ_SRC, LLVM_PROJ_BUILD since MLIR is now parallel to LLVM in llvm-project repository. 2020-01-06 15:59:19 -05:00
Tian Jin 56e8c4c147 Pass MAKEFLAGS explicitly to CMake. 2019-12-31 14:13:02 -05:00
Tian Jin 12b51339cd Add quotation marks to debug. 2019-12-31 14:01:18 -05:00
Tian Jin 5c5efd142f Try setting Makeflags within installation scripts to debug. 2019-12-31 13:51:42 -05:00
Tian Jin 706ff24d33 1. Remove quotes from strings in yaml file for better consistency.
2. Use MAKEFLAGS env variable to limit build parallelism so that mlir installation script remains clean.
2019-12-31 13:28:26 -05:00
Tian Jin c4b990aed8 1. Simplify CircleCI build scripts.
2. Refactor mlir installation script to a standalone shell script.
2019-12-31 01:45:32 -05:00
Tian Jin b63b6bfd99 Limit build parallelism because CI is running out of memory. 2019-12-30 22:57:51 -05:00
Tian Jin 41b82556d6 Use new MLIR installation script. 2019-12-30 22:52:16 -05:00
Tian Jin d7a85cd7fa remove uncessary checks 2019-12-24 03:49:56 -05:00
Tian Jin 9eeb48addb more debug 2019-12-24 03:25:07 -05:00
Tian Jin 9281f82181 fix build 2019-12-24 03:15:49 -05:00
Tian Jin e4f95e2540 show all test results 2019-12-24 02:55:42 -05:00
Tian Jin 7e72bd3885 switch to lit test 2019-12-24 02:50:08 -05:00
Tian Jin b55387ae72 see what onnx-opt output is 2019-12-24 02:24:09 -05:00
Tian Jin ebdd580308 make all before make test 2019-12-24 02:13:35 -05:00
Tian Jin 2cf60bc4c8 verbose lit 2019-12-24 02:01:23 -05:00
Tian Jin 5fafe91380 verbose lit 2019-12-24 01:50:47 -05:00
Tian Jin c5f0e51d44 verbose build 2019-12-24 01:46:41 -05:00
Tian Jin e46b010abd add tests 2019-12-24 01:42:31 -05:00
Tian Jin 43afd5236b checkout specific commits of LLVM/MLIR 2019-12-24 01:17:20 -05:00
Tian Jin 747e2f17d0 install protobuf 2019-12-24 00:54:02 -05:00
Tian Jin e33192b14d pull submodule 2019-12-24 00:51:23 -05:00
Tian Jin 376bf215e4 fix cache checking logic 2019-12-24 00:48:42 -05:00
Tian Jin ffd58084f1 fix build path 2019-12-24 00:46:32 -05:00
Tian Jin 10183b27aa fix yaml file 2019-12-24 00:21:51 -05:00
Tian Jin ab10eb510a fix yaml file 2019-12-24 00:20:37 -05:00
Tian Jin 20fc4d963b fix yaml file 2019-12-24 00:17:45 -05:00
Tian Jin 40677f528d fix yaml file 2019-12-24 00:16:00 -05:00
Tian Jin 78dc3f6e93 cache MLIR build 2019-12-24 00:09:31 -05:00
Tian Jin 8dbca0cc7b build in release mode 2019-12-23 23:23:07 -05:00
Tian Jin e1f4fcd66e reduce hash table size 2019-12-23 23:18:38 -05:00
Tian Jin d4c59cc45f build all 2019-12-23 22:43:43 -05:00
Tian Jin 8e27d831e6 try tricks to reduce memory consumption of linkers 2019-12-23 22:37:23 -05:00
Tian Jin 71b27d555b use 4 threads, not much credit left 2019-12-23 22:01:16 -05:00
Tian Jin e909658279 use 2 threads 2019-12-23 21:59:26 -05:00
Tian Jin 01c88a3750 running out of memory, limit parallelism 2019-12-23 18:19:55 -05:00
Tian Jin 8f1af19e43 install ninja 2019-12-23 17:58:55 -05:00
Tian Jin 0a953629db use sudo 2019-12-23 17:53:31 -05:00
Tian Jin db4afef879 try without installing sudo 2019-12-23 17:51:52 -05:00
Tian Jin d601a34afa install gcc, cmake 2019-12-23 17:47:37 -05:00
Tian Jin 65240a1368 use python image 2019-12-23 17:03:22 -05:00
Tian Jin 52ca81f3ad find git 2019-12-23 17:00:55 -05:00
Tian Jin 13f9c83397 set up buildbot 2019-12-23 16:53:08 -05:00
Tian Jin 6c1d0d42c5 enable ci 2019-12-23 16:33:08 -05:00