Commit Graph

14 Commits

Author SHA1 Message Date
Tian Jin c9199c9061
Programmatically ensure ONNX Dialect related generated files are up-to-date. (#58)
* Generate ONNX Dialect TableGen Inc files & operation importing inc files when necessary.

* Ensure TableGen inc file is generated before TableGen is invoked.

* Nit: capitalize builder -> Builder.

* Use file-same-as-stdout directive to ensure generated files are always up-to-date in our codebase.

* Use more up-to-date version of ONNXOps.td.inc.

* Do not automatically invoke gen_doc.py.

* Support dry run in gen_doc.py.

* Fix case.

* Remove debug code.

* Add test for new doc_check primitive.

* Add documentation for file-same-as-stdout.

* Provide more comments.

* Add DocCheck to DocCheck README.

* Nit: format CMake script.

* Update comments.

Co-authored-by: Alexandre Eichenberger <alexe@us.ibm.com>
2020-04-08 15:00:34 +08:00
Gheorghe-Teodor Bercea 1622b9f161
[NFC] Change ONNF based names to ONNX-MLIR (#32)
* Rename onnf to onnx-mlir.

* Change workspace name.
2020-03-17 09:16:33 -04:00
Tian Jin 51b0f4c9dd
Chentong319 attribute with variant (#25)
* change the read-in of attribute, using variant

* Use backported variant.

* Reduce code duplication.

* 1. Make array attribute parsing more clear.
2. int -> int64_t.

* 1. Fix how array attributes are imported.

* 1. Fix clang-tidy warnings.

* 1. Nit: fix clang-tidy warnings.

* Fix MaxPool node construction.

* Fix call to MaxPool.

* Comment out backend tests that fail.

* Add path to variant submodule to enable include file detection.

* Allow unused argument to avoid special casing generator.

* Address attribute related e2e test failures for Hard sigmoid,Elu,LeakyRelu,Selu,Softmax

Co-authored-by: chentong319 <chentong@us.ibm.com>
Co-authored-by: Gheorghe-Teodor Bercea <gt.bercea@gmail.com>
2020-01-21 19:36:21 -07:00
Tian Jin 8665ecd998
Enable e2e tests (#29)
* Sync with latest MLIR.

* Enable ONNX backend tests as a means to test ONNF lowering end-to-end.

* Install ONNX using quiet mode.

* Remove debug comments.

* Install ONNX from third_party/onnx.

* Check python version and fix pip command for installing ONNX.

* Using --user install option to prevent permission denied.

* Remove unused imports.

* Try using stock ONNX pip package as there are more tests in them.

* Pip got stuck building wheels, try sudo.

* Use verbose install to debug.

* Invalidate cache to build LLVM tools.

* Fix mlir installation script location.

* Debug to locate ONNF.

* Sanity check.

* Check out ONNF code first.

* Use verbose LIT output.

* 1. Update documentation to always use verbose LIT.
2. Update krnl ops to reflect new affine map attribute syntax.

* See if conda exists

* Install ONNX by manually cloning the repo.

* Install cmake first.

* Using sudo priviledge when installing.

* Limit build parallelism.

* Limit parallelism.

* Larger memory.

* Install onnx package with pip.

* Build MLIR tools.

* Invalidate cache.

* Compile model.so with -fPIC.

* Remove module dump to get concise debug output.

* Print command before executing.

* Use quiet install mode to reduce logging.

* Use -relocation-model=pic to generate position independent code.

* 1. Remove MAKEFLAGS because now buildbot has enough memory.
2. Run DocCheck as a last step.

* 1. Add verbose mode for backtend test.

* When dumping to LLVM bitcode, do not dump module IR, but print a message indicating that bitcode has been written to disk.

* Do not pass MakeFlags to CMake.

* Add more explaination for posible reasons of failing to identify tests.
2020-01-20 12:30:08 -05:00
Tian Jin 1ebcc2eb64
[RFC] Doc-check utility. (#12)
* 1. Implement doc-check utility.

* 1. Move ONNF installation script to a standalone script file.

* 1. Modify build script to install llvm-project next to ONNF. The build script used to install llvm-project inside ONNF, which didn't make sense.

* 1. Check out code to ONNF directory.

* 1. Pass path parameter correctly.

* 1. Debugging buildbot.

* 1. Remove debug code.

* 1. Update installation instructions in README.md.
2. Enforce consistency with scripts used in testing using doc-check.

* 1. Fix error with respect to syntax to build multiple CMake targets.

* 1. Move doc-check to doc_check.
2. Remove directive_config in top-level driver.

* 1. Build onnf and check-mlir-lit separately because only CMake 3.15+ supports building multiple targets in one cmake --build run.

* 1. Use new env variables to locate LLVM-Project.

* 1. Documentation nits.

* 1. Prettify buildbot scripts.

* 1. Fix build script error.

* 1. Support exclude_dirs in DocCheck.
2. Add README for DocCheck.

* 1. Mark python3 interpreter as required.
2. Use imported interpreter target.

* 1. Automatically deduce doc file extension in DocCheckCtx.
2. Rename ctx.open -> ctx.open_doc since it should only be used to open doc file.
3. Always read line in parser, instead of reading lines in driver and then passing it to parser.py.

* 1. Rename parser -> doc_parser due to name conflict with python built-in module.
2. Explose doc_check module directory first before importing; otherwise if the doc_check utility is invoked by other script, importing will not work correctly.

* 1. Keep renaming parser -> doc_parser.
2. Explicitly define a default configuration parser that parses the configuration into a python dictionary.

* 1. Add test for doc-check.
2. Exclude doc-check tests from project dock-check because base directory is different.

* 1. Raise ValueError if directive configuration fails to parse.
2. Format code.

* Shorten test case documentation.
Show example of using same-as-file directive, check with DocCheck.

* 1. Shorten test case documentation.
2. More documentation, check documentation with DocCheck.

* 1. Add copyright notice.

* 1. Make documentation clearer.
2. Prettify build-scripts.

* 1. Provide more documentation.
2. Fix some non-compliance with pep8 recommendations.

Co-authored-by: Gheorghe-Teodor Bercea <gt.bercea@gmail.com>
2020-01-09 18:35:52 -05:00
Tian Jin da4527c961 flatten src directory structure 2019-12-23 00:13:52 -05:00
Tian Jin 685bf23b40 Enable ONNX Backend Test (#1)
* wip, commit before merging with upstream

* organize API, return wrapped output

* enable onnx backend test

* undo unintentional commit

* fix krnl ops tablegen

* format krnl ops

* reorder fillDynMemRefWithMemRef to be after fillPtrToMemRefWithDynMemRef, better comments

* more onnx backend tests

* ensure that test names refer to existing tests

* improve code readability by shortening type names

* nit

* restore unintentional changes

* more nits

* fix ; -> :

* split runtime implementation into header and body file, add support for data types

* comment on the onnx backend test

* make the comments read better

* do not dump when lowering
2019-12-22 23:14:57 -05:00
Tian Jin 5573cb39fe clean up, remove dependency for boost 2019-12-22 20:33:33 -05:00
Tian Jin b2a1103915 [MLIR] Refactor Krnl Dialect and Krnl Dialect Lowering (#375)
* Store bounds as affine map attributes & check in test cases with generic printer

* Upgrading MLIR

MLIR is outdated on Buildbot, rebuilding a newer version.

* work with new version of mlir

* check-in parser tests

* custom printer

* nit

* bug fix

* enable custom asm printer test

* enable custom asm printer test

* more consistent variable naming

* test max/min

* variable naming scheme change to MLIR style

* can lower krnl to llvm

* kernel -> llvm

* comments

* bug fix

* try fixing ci

* fix ci

* deactivate model test

* fix lit test

* nit

* fix z buildbot
2019-12-21 01:34:14 -05:00
GHEORGHE-TEOD BERCEA b02652dd76 [MLIR] Lowering of frontend dialect to KRNL dialect (#382)
* Partial support for lowering operations to KRNL dialect.

* Attempt to lower to KRNL IR.

* Update file.

* Add lowering.

* Address comments. Fix alloc dynamic dimensions. Correctly link StandardOps.

* Temporarily remove deallocation of locally allocated tensors.
2019-12-21 01:11:14 -05:00
Tian Jin d01ac7732f [MLIR] compartmentalize build script (#369)
* compartmentalize build script, temporarily remove dependency of onnf_opt on helper.cpp

* fix test includes

* fix op directory include

* compiler -> op

* compiler test depends on boost system

* fix function name

* specify libcompiler dependencies

* let cmake take care of transitive dependencies

* remove unnecessary includes

* use ONNF_SRC_ROOT and ONNF_BIN_ROOT

* allow whole-archive linked libraries to be appended

* [MLIR] Support filecheck (#371)

* support lit+FileCheck

* add lit into build script

* format MLIR.cmake

* format cmake

* [MLIR] Remove input/output ops (#372)

* remove input/output ops

* get output tensor type from symbol table
2019-12-21 00:34:51 -05:00
Tian Jin 00aee2e0b6 Use TableGen (#347)
* use table gen

* fix name of the dialect

* add old compilation path

* add some doc

* fix bug, sgir importer imports every op twice

* knl.visit -> knl.iterate
2019-12-19 15:18:35 -05:00
Doru Bercea 63427030ca Enable building of first commit. 2019-12-19 14:24:37 -05:00
Doru Bercea b6a2a50dd8 Add header. Clean-up file. Add CMake file. 2019-12-18 16:45:05 -05:00