Commit Graph

32 Commits

Author SHA1 Message Date
chentong319 32ceb6968a
Use map when Import onnx node (#230)
* add map

* generate map
2020-07-27 12:25:21 -04:00
Kevin O'Brien 8af5fdeb62
fix comment (#229)
* fix comment

* add the .inc files to the PR
2020-07-24 17:03:08 -04:00
Tung D. Le 0df421f7ad
Fix importing none inputs (#217) 2020-07-14 10:06:58 -04:00
chentong319 8848464a23
changes for mypipeline.onnx (#202)
* changes for mypipeline.onnx

* format

* rm MLOpBuildTable.inc

* copy string without free

* fix the memory issue

* restore change for STRING

* format

Co-authored-by: Tian Jin <tjingrant@gmail.com>
2020-07-08 16:52:56 -04:00
Tung D. Le 0a936edf79
Remove ImportNodeReshape (#208) 2020-07-08 10:27:14 -04:00
Tung D. Le 8b6cf7c661
Remove dangling constant op (#205) 2020-07-07 13:47:54 +08:00
chentong319 cc68f77d8d
Merge onnx ml into onnx (#176)
* merge onnx-ml into onnx

* delete onnx ml
2020-06-22 20:01:56 -04:00
Tian Jin e0ae583da0
Compiling Models with Large Constant Arrays (#146)
* PoC works.

* MNist works.

* Clean up.

* Fix test.

* Make Linux work.

* Use consistent symbol name.

* Fix variable name.

* Fix array addr access.

* Bug fix.

* Bug fix.

* install before running e2e tests.

* Fix build config.

* Use sudo when installing.

* Make embeddedDataLoader position independent.

* Enable ResNet50.

* Format code.

* Format MainUtil.

* Try not using sudo to install.

* Supply runtime dir via environment variable.

* Dump problematic operation.

* Dump entire function.

* Debug.

* Dump input.

* Dump constant op.

* Debug.

* Debug.

* Debug.

* Print to stderr.

* take care of endianness.

* Use endianness-aware execution session.

* Fix ZLinux error.

* Include warning when desired output endianness can't be deduced.

* Remove debug code.

* Remove debug code in shape inference.

* Support binary-decoder for testing constants packing.

* Support filename, move-to-file, elision-threshold configurations in constant packing pass for easy testing.

* Add lit test, fix lit test type mismatch.

* Add more consts packing tests.

* Ensure intermediate files are properly cleaned up.

* No need for constant elimination.

* Link with threading libraries.

* Remove debug code.

* Format code.

* More tests.

* test nit.

* Remove debug code.

* Reduce hard-coded constants.

* Use temporary and unique working directory for hosting model parameters.

* Test if it works.

* Try to find objcopy.

* Rename symbols using objcopy.

* Move sanitized name to linux section.

* Use verbose mode for debugging.

* Disambiguate pass constructor.

* Fix symbol name.

* Use Command API to build and execute commands.

* Move linux to use Command API.

* Fix reset args.

* Execute redefine sym.

* Format code.

* Do not use verbose mode for CircleCI.

* Remove debug code.

* Prettify code, add comments.

* getSegmentData -> getEmbeddedConstPool

* vector -> std::vector.

* Make sure we properly clean up intermediate files.

* Fix test cases.

* Add runtime directory.

* Trigger rebuild.

* [Merge with master] fix debug script.

* Diable affine fusion pass for now.

* Support generic fallback const packing mechanism.

* Remove debug code.

* Handle the case where objcopy is not available.

* Fix Windows missing types.

* Support int64.

* Copy packed constant to a local directory for non-Linux/Mac platforms.

* Nit: remove debug code, refactor const pack preprocessing out as a separate function.

* Cannot make preprocessConstPack a standalone function because file removers are stack-allocated, and they are deallocated prematurely when function stack gets popped, deleteing intermediate files too early.

* Don't require executable filename.

* Import ONNX data types directly.

* Fix LIT test.

* Bug fix, use moved string value.

* Remove redundant filenames.

* Fix CMake script.

* Embed endianness information as a symbol, and check during runtime.

* More comments, update lit tests.

* Fix lit test on BE machine.

* Copyright notices.
2020-06-12 10:27:05 +08:00
Aman LaChapelle ca185002f2
Add shape inference for several ops (#163)
* 1. Add shape inference for the following ops:
 - Atan
 - Tan
 - Sin
 - Cast
 - ConvTranspose
 - Flatten
 - DynamicQuantizeLinear
 - QuantizeLinear
 - DequantizeLinear
 - ConvInteger

2. Import attributes for generic nodes
3. Fixes for cases where .cast<> should be .isa<> (ONNXConcat::inferShapes)

* Fix foormatting issues

* Address comments:
 - SmallVector<> * -> SmallVectorImpl<> &
 - switch-case -> helper function
   - Inside helper function, preserve signed-ness
 - add TODOs

* Can't use signed integers yet in convertONNXTypeToMLIRType, add TODO

Co-authored-by: Tian Jin <tjingrant@gmail.com>
2020-06-09 14:55:49 +08:00
gongsu832 43dc1a1e01
Fix header dep (#150)
* Detect llvm-project commit change in utils/clone-mlir.sh and rebuild llvm-project
for zLinux Jenkins build bot

* Since many headers are generated and included indirectly through
other headers, there are often missing dependencies that break
parallel build. So we add header targets for KrnlOps, ONNXOps, and
MLONNXOps, and add explicit dependencies for all the relevant headers.

* fix copy-and-paste bug

Co-authored-by: Tian Jin <tjingrant@gmail.com>
2020-06-09 10:38:32 +08:00
Tung D. Le e2e1fbfd3b
Add type inference for CastOp (#156)
* Add type inference for CastOp

* Share type translation between op builder and onnx importer

* clang-format

* Format emitted code

* Remove unnecessary dependencies
2020-06-04 09:05:04 -04:00
Tung D. Le 2b6befce87
Fix importing variadic output (#152)
* Fix importing variadic output

* clang-format
2020-05-27 09:33:53 -04:00
chentong319 5a542e78a0
fix type inference (#144)
* fix type inference for ConstantOp and MaxPoolSingleOut

* modify interface

* use OpInterface

* change name to OpInterface

* Builder dependence

* Update CMakeLists.txt

* Update CMakeLists.txt

Co-authored-by: Tian Jin <tjingrant@gmail.com>
2020-05-26 09:54:19 +08:00
chentong319 6099efd91b
Express some basic features of an Operation in TableGen file (#103)
* change operation definition

* change importer

* default type inference

* file format

* generate types for input/output

* generate the mapping for operation output type

* remove debug message for gen_doc.py

* update the dialect doc

* add support Complex

* format

* update document

Co-authored-by: Gheorghe-Teodor Bercea <gt.bercea@gmail.com>
2020-05-21 22:03:16 -04:00
chentong319 a98c723b11
save and check the version of ONNX operation (#125)
* add version control

* fix the comments

* response to review and add onnx_ml

* update document

* add target and rename gen_doc.py

* small fix
2020-05-20 13:48:45 -04:00
Tung D. Le ad61eee908
Move to more recent LLVM commit ID (#131)
* Move to more recent LLVM ID (May 15)

* clang-format

* Bump cache version up

* Update readme

* Fix doc check

* Move to a newer commit id

* Update LoopToStandard -> SCFToStandard

* Change MLIRSideEffects to MLIRSideEffectInterfaces

Co-authored-by: Tian Jin <tjingrant@gmail.com>
2020-05-20 15:45:42 +08:00
chentong319 23bea50404
Implement PadOp based on attribute promotion (#71)
* enable promote attr for pad

* use optional arguments for pad

* shape infereance for pad

* Lowering Pad

* format file

* use DenseTensor for the attribute

* use Pad in ONNXRewrite

* fix the merge conflict

* fix the attr given to constantOp

* handle ONNXConstantOp in attribute promotion

* Fix bug when AttributePromotion is called more than once

* update ONNXOps.td.inc with correct version of onnx

* update onnx.md

* responses to review

* fix the build error

* change the implementation of Pad

* delete commented out code

* clang format

Co-authored-by: Tian Jin <tjingrant@gmail.com>
2020-05-15 13:19:28 +08:00
Tian Jin 4dd3c809c7
Support importing tensor proto. (#116)
* Support importing tensor proto.

* Use signed types, use template.

* Resotre using signless integer types because Reshape faults with integer signs; using traits to declare/define attribute types.

* Simplify attribute importing logic.

* Use existing code to import TensorProto.

* nit.
2020-05-14 11:05:02 +08:00
Tian Jin 7f2bffb27d
[NFC] Set up clang-format Github Action (#119)
* Run clang-format on all source code.

* Add Clang-Format Github Action.

* Apply patch produced by Clang-Format Bot.

* nit.

Co-authored-by: Gheorghe-Teodor Bercea <gt.bercea@gmail.com>
2020-05-13 22:37:51 +08:00
chentong319 7c29da191e
import onnx ml operations as a separate dialect (#97)
* generate MLONNX dialect

* Add MLONNX to Builder

* add MLONNX dialect operation document

* fix the format issues

Co-authored-by: Gheorghe-Teodor Bercea <gt.bercea@gmail.com>
2020-05-12 18:43:44 -04:00
Byron Changuion c567781fa3
Add support for Windows using Visual Studio Compiler (#86)
* Move to more recent LLVM commit ID

* Update LLVM cache version from V9 to V10

* Update to latest LLVM commit id from master, roll back conditions in util scripts

* Update circlci LLVM cache tag to ensure ci updates builds with latest LLVM commit id

* Update README.md to have matching LLVM commit id

* Update doc/Dialtects/onnx.md

* Enable onnx-mlir for VS builds on Windows

* Update README to include lit

* Update build command for Windows to include config

* Update build instructions, add cmd files for windows, enable single source of truth for MLIR commit-id (clone-mlir.sh)

* Add Visual Studio workload info

* Update ONNX op definitions

* Revert onnx submodule back to previous commit, disable warnings in CMakeLists to work around build issues with MSVC

* Update environment for path to PDcurses on Windows

* Fix directory strings to be compatible with Windows or Linux style slashes

* Fix install-mlir.sh so it works when sourced

* Ensure README and cmd files match and have correct paths

* Properly quote ONNX_MLIR_SRC_DIR

* Address PR feedback: Use llvm_unreachable to indicate failure to convert attribute proto to name/value pair

Co-authored-by: Tian Jin <tjingrant@gmail.com>
2020-04-19 22:11:24 +08:00
Tian Jin d06dbfefdd
Specify each lib only once; allow llvm build in shared libs mode. (#77)
* Specify each lib only once; allow llvm build in shared libs mode.

* Remove debug code.

* For library targets, retain dependency information using add_dependencies, but do not link using taget_link_libraries.

* Do not set LD_PRELOAD by default.

Co-authored-by: Gong Su <gongsu@us.ibm.com>
Co-authored-by: Gheorghe-Teodor Bercea <gt.bercea@gmail.com>
2020-04-14 17:40:05 +08:00
Tian Jin 7dba324404
Setup Documentation Page using Github Pages (#76)
* Setup documentation server, move doc files from /doc to /docs as per Github Pages convention.

* Include deleted files in patch.

* /doc -> /docs

* /doc -> /docs

* Update documentation on importing ONNX spec into ONNX Dialect; provide documentation on how to add new documentation pages.
2020-04-09 23:37:04 +08:00
Tian Jin c9199c9061
Programmatically ensure ONNX Dialect related generated files are up-to-date. (#58)
* Generate ONNX Dialect TableGen Inc files & operation importing inc files when necessary.

* Ensure TableGen inc file is generated before TableGen is invoked.

* Nit: capitalize builder -> Builder.

* Use file-same-as-stdout directive to ensure generated files are always up-to-date in our codebase.

* Use more up-to-date version of ONNXOps.td.inc.

* Do not automatically invoke gen_doc.py.

* Support dry run in gen_doc.py.

* Fix case.

* Remove debug code.

* Add test for new doc_check primitive.

* Add documentation for file-same-as-stdout.

* Provide more comments.

* Add DocCheck to DocCheck README.

* Nit: format CMake script.

* Update comments.

Co-authored-by: Alexandre Eichenberger <alexe@us.ibm.com>
2020-04-08 15:00:34 +08:00
Gheorghe-Teodor Bercea 8532a10614
Fix input argument indexing error (#69)
* Reorganize main function.

* Follow review comments.

* Emit constants are globals in Krnl and LLVM dialects.

* Fixes. Emit error before return in shape inference.

* Fix description.

* Fix emitted error message.

* Fix index name.
2020-04-06 11:35:17 -04:00
Byron Changuion b65e77305c
Move to more recent LLVM commit ID (#64)
* Move to more recent LLVM commit ID

* Update LLVM cache version from V9 to V10

* Update to latest LLVM commit id from master, roll back conditions in util scripts

* Update circlci LLVM cache tag to ensure ci updates builds with latest LLVM commit id

* Update README.md to have matching LLVM commit id

* Update doc/Dialtects/onnx.md
2020-04-01 12:38:34 -04:00
Tung D. Le c8758545e7
Import optional outputs as NoneType (#57)
* Import optional outputs as NoneType

* Allow NoneType results after the shape inference

* Use empty() to check an empty string
2020-03-30 21:21:18 -04:00
Alexandre Eichenberger 653fa69102
Unify Conv implementation (#54)
* fixed readme for new git repo

* conv with bias as an optional input
2020-03-26 11:03:19 -04:00
Tung D. Le ddff0f1256
Fix a bug in createArrayAttribute (#43)
* Fix a bug in createArrayAttribute

* Use size_t

* Use const auto&
2020-03-24 14:04:23 +08:00
Gheorghe-Teodor Bercea b3719d486b
Make path relative to onnx-mlir project only (#40)
* Make path relative to onnx-mlir project.

* Use binary root folder path for onnx-mlir.
2020-03-20 12:04:22 -04:00
Tian Jin febee542ee
[NFC] Breakdown CMake Build Scripts to Smaller Parts (#39)
* 1.Break down CMake scripts to smaller libraries per-directory.
2. Move some transformations and interfaces to the right folder.
3. Fix minor merge failure of the patch renaming files to use LLVM convention.

* Link OMBuilder with OMONNXOps.

* 1. Update the src location of generated ONNX dialect definition.
2. Link OMONNXRewrite with OMONNXOps.

* Fix path to tablegen for add_onnx_mlir_dialect_doc.

* Update build script for onnx_mlir_transform.

* 1. Remove comment code.
2. onnx_mlir_attribute_promotion -> OMAttributePromotion.

* Name tablegen generated files with LLVM convention.

* Nit: reorder libraries to link against.

* Nit: Link against MLIR first.
2020-03-20 22:40:51 +08:00
Tian Jin 0aafb3e72f
[WIP][NFC]Rename files to llvm style (#35)
* Change naming style for builder directory.

* Change naming style for conversion folder.

* Fix case sensitivity issue.

* Fix missing onnx header onnx_pb.h issue.

* Rename files in Conversion to llvm style.

* Rename files in Dialect to llvm style.

* Path fix.

* Rename files in Pass to llvm style.

* Rename files in Runtime to llvm style.

* Rename files in Tool to llvm style.

* Rename files in Transform to llvm style.

* Change comments about filenames.

* Fix case.

* Rename interface directory to use llvm file naming convention.
2020-03-19 16:48:09 +08:00