Setup Documentation Page using Github Pages (#76)

* Setup documentation server, move doc files from /doc to /docs as per Github Pages convention.

* Include deleted files in patch.

* /doc -> /docs

* /doc -> /docs

* Update documentation on importing ONNX spec into ONNX Dialect; provide documentation on how to add new documentation pages.

.circleci/config.yml

@@ -50,7 +50,7 @@ jobs:
             cd onnx-mlir/build
             cmake --build . --target onnx-mlir-doc
             # Check whether dialect documentation is up-to-date.
-            diff doc/Dialects ../doc/Dialects
+            diff docs/Dialects ../docs/Dialects
       - run:
           name: Print the Current Time
           command: date

CMakeLists.txt

@@ -27,5 +27,5 @@ add_subdirectory(third_party/variant)
 set(CMAKE_CXX_STANDARD 14)
 add_subdirectory(utils)
 add_subdirectory(src)
-add_subdirectory(doc)
+add_subdirectory(docs)
 add_subdirectory(test)

MLIR.cmake

@@ -238,7 +238,7 @@ function(add_onnx_mlir_dialect_doc dialect dialect_tablegen_file)
   # Generate Dialect Documentation
   set(LLVM_TARGET_DEFINITIONS ${dialect_tablegen_file})
   onnx_mlir_tablegen(${dialect}.md -gen-op-doc)
-  set(GEN_DOC_FILE ${ONNX_MLIR_BIN_ROOT}/doc/Dialects/${dialect}.md)
+  set(GEN_DOC_FILE ${ONNX_MLIR_BIN_ROOT}/docs/Dialects/${dialect}.md)
   add_custom_command(
           OUTPUT ${GEN_DOC_FILE}
           COMMAND ${CMAKE_COMMAND} -E copy

doc/ImportONNXDefs.md

@@ -1,32 +0,0 @@
-# Import ONNX specifications into ONNX MLIR
-The specifications of ONNX are defined under onnx/defs directory in ONNX projects. 
-There is a python script onnx/defs/gen_doc.py that automatically generate documents about operations in ONNX (docs/Operations.md). 
-ONNX MLIR modified this script to import ONNX specifications into ONNX MLIR. There are two files generated for ONNX MLIR with the modified gen_doc.py:
-1. src/Dialect/ONNX/ONNXOps.td.inc: Operation defintion for MLIR tablegen. Will be included in src/Dialect/ONNX/ONNXOps.td
-2. src/Builder/OpBuildTable.inc: c code for ONNX MLIR frontend to import operation nodes from ONNX model. Will be included in src/Builder/FrontendDialectTransformer.cpp
-
-## How to use the script
-1. Get ONNX. You can use onnx-mlir/third_party/onnx
-2. Perform the following steps (assume that we use onnx-mlir/third_party/onnx):
-```bash
-$ cd onnx-mlir
-$ cp doc/gen_doc.py third_party/onnx/onnx/defs/
-$ cd third_party/onnx
-$ python onnx/defs/gen_doc.py
-$ cd ../..
-$ cp third_party/onnx/onnx/defs/ONNXOps.td.inc src/Dialect/ONNX/
-$ cp third_party/onnx/onnx/defs/OpBuildTable.inc src/Builder/
-```
-
-## Consistency
-The Operators.md generated by gen_doc.py is copied into doc. Please refer to this specification, not the one in onnx github, to make sure operators are consistent in version with ONNXOps.td.inc.
-
-## Customization
-In addition to following the ONNX specification, the modified gen_doc.py provides some mechanism for you to customize the output. 
-Several tables are defined at the beginning of the script:
-1. special_attr_defaults: gives attribute special default value.
-2. special_op_handler: creates special import function in frontend_dialect_transformer.cpp. Currently special handler is used for operations with oprational arguments
-3. OpsWithShapeInference: list of operations which have shape inference defined
-4. OpsWithCanonicalizer: list of operations which have canonical form
-5. OpsWithPromotableConstOperands: list of operations which have operands that, if produced by constant operations, should be promoted to become an attribute (via attribute promotion)
-6. custom_builder_ops_list: list of operations which need custom build methods to deduce result types

docs/Documentation.md

@@ -0,0 +1,14 @@
+# About Documentation
+
+## How to add a new documentation page
+
+Firstly, `/docs` is the root directory of the documentation website, meaning that any
+documentation page you wish to display to a user must be located within `/docs`.
+
+Secondly, add the documentation page into the navigation configuration file located at
+`/docs/_data/navigation.yaml`. Edit the table of content to include the path to
+the newly created documentation page with a descriptive title.
+
+Then, capture the changes done in a patch and submit a pull request; once the patch is
+merged into `onnx-mlir` codebase, a link pointing to the file path specified with the
+descriptive title you provided will appear on the navigation panel.

docs/ImportONNXDefs.md

@@ -0,0 +1,32 @@
+# Import ONNX specifications into ONNX-MLIR
+
+ONNX specifications are defined under `onnx/defs` directory in the ONNX project repository. 
+There is a python script onnx/defs/gen_doc.py that automatically generate documents about operations in ONNX (docs/Operations.md). 
+ONNX-MLIR modified this script to import ONNX specifications into ONNX-MLIR. 
+There are two files generated for ONNX MLIR with the modified gen_doc.py:
+
+1. `src/Dialect/ONNX/ONNXOps.td.inc`: Operation definition for MLIR TableGen. `src/Dialect/ONNX/ONNXOps.td` includes this file.
+2. `src/Builder/OpBuildTable.inc`: C++ code for ONNX-MLIR frontend to import operation nodes from ONNX model. `src/Builder/FrontendDialectTransformer.cpp` includes this file.
+
+## How to use the script
+1. Install [ONNX](https://github.com/onnx/onnx). We highly recommend that you use the one located at `third_party/onnx.`
+2. Make target `OMONNXOpsIncTranslation`. For example,
+```
+make OMONNXOpsIncTranslation
+````
+Target `OMONNXOpsIncTranslation` invokes the script and places the generated files into the correct directories correspondingly.
+
+## Consistency
+For reference to the schema and semantics of an operation, please refer to [ONNX Dialect](Dialects/onnx.md). 
+Even though we strive to support the latest version of ONNX specification as quickly as we can, there will inevitably be a delay between the introduction of new changes in the ONNX specification and the adoption in our codebase. 
+Due to the possibility of such a delay, operator definition within the ONNX project repository may describe features and schemas that we do not yet support.
+
+## Customization
+In addition to following the ONNX specification, the modified gen_doc.py provides some mechanism for you to customize the output. 
+Several tables are defined at the beginning of the script:
+1. `special_attr_defaults`: gives attribute special default value.
+2. `special_op_handler`: creates special import function in frontend_dialect_transformer.cpp. Currently, a special handler is used for operations with operational arguments
+3. `OpsWithShapeInference`: list of operations which have shape inference defined
+4. `OpsWithCanonicalizer`: list of operations which have a canonical form
+5. `OpsWithPromotableConstOperands`: list of operations which have operands that, if produced by constant operations, should be promoted to become an attribute (via attribute promotion)
+6. `custom_builder_ops_list`: list of operations which need custom build methods to deduce result types

docs/README.md

@@ -0,0 +1,106 @@
+# ONNX MLIR
+The Open Neural Network Exchange implementation in MLIR.
+
+[![CircleCI](https://circleci.com/gh/onnx/onnx-mlir/tree/master.svg?style=svg)](https://circleci.com/gh/onnx/onnx-mlir/tree/master)
+
+## Prerequisites
+
+```
+gcc >= 6.4
+libprotoc >= 3.11.0
+cmake >= 3.15.4
+```
+
+## Installation
+
+Firstly, install MLIR (as a part of LLVM-Project):
+
+[same-as-file]: <> (utils/install-mlir.sh)
+``` bash
+git clone https://github.com/llvm/llvm-project.git
+# Check out a specific branch that is known to work with ONNX MLIR.
+cd llvm-project && git checkout 07e462526d0cbae40b320e1a4307ce11e197fb0a && cd ..
+mkdir llvm-project/build
+cd llvm-project/build
+cmake -G Ninja ../llvm \
+   -DLLVM_ENABLE_PROJECTS=mlir \
+   -DLLVM_BUILD_EXAMPLES=ON \
+   -DLLVM_TARGETS_TO_BUILD="host" \
+   -DCMAKE_BUILD_TYPE=Release \
+   -DLLVM_ENABLE_ASSERTIONS=ON \
+   -DLLVM_ENABLE_RTTI=ON
+
+cmake --build . --target -- ${MAKEFLAGS}
+cmake --build . --target check-mlir
+```
+
+Two environment variables need to be set:
+- LLVM_PROJ_SRC should point to the llvm-project src directory (e.g., llvm-project/).
+- LLVM_PROJ_BUILD should point to the llvm-project build directory (e.g., llvm-project/build).
+
+To build ONNX-MLIR, use the following command:
+
+[same-as-file]: <> ({"ref": "utils/install-onnx-mlir.sh", "skip-doc": 2})
+```
+git clone --recursive https://github.com/onnx/onnx-mlir.git
+
+# Export environment variables pointing to LLVM-Projects.
+export LLVM_PROJ_SRC=$(pwd)/llvm-project/
+export LLVM_PROJ_BUILD=$(pwd)/llvm-project/build
+
+mkdir onnx-mlir/build && cd onnx-mlir/build
+cmake ..
+cmake --build . --target onnx-mlir
+
+# Run FileCheck tests:
+export LIT_OPTS=-v
+cmake --build . --target check-onnx-lit
+```
+
+After the above commands succeed, an `onnx-mlir` executable should appear in the `bin` directory. 
+
+## Using ONNX MLIR
+
+The usage of `onnx-mlir` is as such:
+```
+OVERVIEW: ONNX MLIR modular optimizer driver
+
+USAGE: onnx-mlir [options] <input file>
+
+OPTIONS:
+
+Generic Options:
+
+  --help        - Display available options (--help-hidden for more)
+  --help-list   - Display list of available options (--help-list-hidden for more)
+  --version     - Display the version of this program
+
+ONNX MLIR Options:
+These are frontend options.
+
+  Choose target to emit:
+      --EmitONNXIR - Ingest ONNX and emit corresponding ONNX dialect.
+      --EmitMLIR   - Lower model to MLIR built-in transformation dialect.
+      --EmitLLVMIR - Lower model to LLVM IR (LLVM dialect).
+      --EmitLLVMBC - Lower model to LLVM IR and emit (to file) LLVM bitcode for model.
+```
+
+## Example
+
+For example, to lower an ONNX model (e.g., add.onnx) to ONNX dialect, use the following command:
+```
+./onnx-mlir --EmitONNXIR add.onnx
+```
+The output should look like:
+```
+module {
+  func @main_graph(%arg0: tensor<10x10x10xf32>, %arg1: tensor<10x10x10xf32>) -> tensor<10x10x10xf32> {
+    %0 = "onnx.Add"(%arg0, %arg1) : (tensor<10x10x10xf32>, tensor<10x10x10xf32>) -> tensor<10x10x10xf32>
+    return %0 : tensor<10x10x10xf32>
+  }
+}
+```
+
+## Troubleshooting
+
+If the latest LLVM project fails to work due to the latest changes to the MLIR subproject please consider using a slightly older version of LLVM. One such version, which we use, can be found [here](https://github.com/clang-ykt/llvm-project).

docs/_config.yml

@@ -0,0 +1,7 @@
+theme: jekyll-theme-minimal
+defaults:
+  -
+    scope:
+      path: "README.md"
+    values:
+      permalink: "README.html"

docs/_data/navigation.yml

@@ -0,0 +1,33 @@
+toc:
+# Restore when we have a proper
+# overview & installation page.
+#  - title: First Steps
+#    subfolderitems:
+#      - page: Overview
+#        url: /thing1.html
+#      - page: Installation
+#        url: /thing2.html
+#  - title: Tutorials
+#    subfolderitems:
+#      - page: Placeholder
+#        url: /piece1.html
+#  - title: How-tos
+#    subfolderitems:
+#      - page: Placeholder
+#        url: /piece1.html
+  - title: References
+    subfolderitems:
+      - page: ONNX Dialect
+        url: /Dialects/onnx.html
+      - page: Generating ONNX Dialect
+        url: /ImportONNXDefs.html
+      - page: About Documentation
+        url: /Documentation.html
+#  - title: Discussions
+#    subfolderitems:
+#      - page: Placeholder
+#        url: /piece1.html
+  - title: Tools
+    subfolderitems:
+      - page: DocCheck - Handling Necessary Code Duplication
+        url: /doc_check/

docs/_layouts/default.html

@@ -0,0 +1,74 @@
+<!DOCTYPE html>
+<html lang="{{ site.lang | default: "en-US" }}">
+  <head>
+    <meta charset="UTF-8">
+    <meta http-equiv="X-UA-Compatible" content="IE=edge">
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+
+{% seo %}
+    <link rel="stylesheet" href="{{ "/assets/css/style.css?v=" | append: site.github.build_revision | relative_url }}">
+    <!--[if lt IE 9]>
+    <script src="https://cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv.min.js"></script>
+    <![endif]-->
+  </head>
+  <body>
+    <div class="wrapper">
+      <header style="position: static; width: 350px">
+        <h1><a href="{{ "/" | absolute_url }}">{{ site.title | default: site.github.repository_name }}</a></h1>
+
+        {% if site.logo %}
+          <img src="{{site.logo | relative_url}}" alt="Logo" />
+        {% endif %}
+
+        <p>{{ site.description | default: site.github.project_tagline }}</p>
+        
+        {% if site.github.is_project_page %}
+        <p class="view"><a href="{{ site.github.repository_url }}">View the Project on GitHub <small>{{ site.github.repository_nwo }}</small></a></p>
+        {% endif %}
+
+        {% if site.github.is_user_page %}
+        <p class="view"><a href="{{ site.github.owner_url }}">View My GitHub Profile</a></p>
+        {% endif %}
+
+        {% if site.show_downloads %}
+        <ul class="downloads">
+          <li><a href="{{ site.github.zip_url }}">Download <strong>ZIP File</strong></a></li>
+          <li><a href="{{ site.github.tar_url }}">Download <strong>TAR Ball</strong></a></li>
+          <li><a href="{{ site.github.repository_url }}">View On <strong>GitHub</strong></a></li>
+        </ul>
+        {% endif %}
+        
+        {% for item in site.data.navigation.toc %}
+        <h3 style="margin: 20px 0 0;">{{ item.title }}</h3>
+        {% for entry in item.subfolderitems %}
+          <div><a href="{{ entry.url }}">{{ entry.page }}</a></div>
+        {% endfor %}
+      {% endfor %}
+      <br>
+      {% if site.github.is_project_page %}
+        <p>This project is maintained by <a href="{{ site.github.owner_url }}">{{ site.github.owner_name }}</a></p>
+      {% endif %}
+      <p><small>Hosted on GitHub Pages &mdash; Theme by <a href="https://github.com/orderedlist">orderedlist</a></small></p>
+      </header>
+      <section>
+
+      {{ content }}
+
+      </section>
+      <footer style="position: static">
+        
+      </footer>
+    </div>
+    <script src="{{ "/assets/js/scale.fix.js" | relative_url }}"></script>
+    {% if site.google_analytics %}
+    <script>
+      (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+      (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+      m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+      })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
+      ga('create', '{{ site.google_analytics }}', 'auto');
+      ga('send', 'pageview');
+    </script>
+    {% endif %}
+  </body>
+</html>

docs/doc_check/CMakeLists.txt

@@ -4,5 +4,5 @@ add_custom_target(check-doc
         COMMAND ${PYTHON_EXECUTABLE}
                     ${CMAKE_CURRENT_SOURCE_DIR}/check.py
                         ${ONNX_MLIR_SRC_ROOT}
-                        --exclude_dirs third_party doc/doc_check/test)
+                        --exclude_dirs third_party docs/doc_check/test)
 

docs/doc_check/README.md

@@ -58,7 +58,7 @@ to ensure the code snippet is always the same as its copy used in some unit test
 be fully specified using the name of the reference file to check against. For example, to ensure a code snippet is the 
 same as a unit-tested file `reference.cpp`, use the following directive as shown in the documentation snippet:
 
-[same-as-file]: <> (doc/doc_check/test/same-as-file/simple/README.md)
+[same-as-file]: <> (docs/doc_check/test/same-as-file/simple/README.md)
 ````markdown
 Lorem ipsum dolor sit amet, consectetur adipiscing elit.
 
@@ -85,7 +85,7 @@ In the canonical form of directive configuration (as a python dictionary literal
 
 For example, to ensure the following code snippet is the same as a unit-tested file `reference.cpp`, except for the first 2 lines of the code used in documentation, and the first 3 lines of code used in the reference file, the following directive configuration can be used:
 
-[same-as-file]: <> (doc/doc_check/test/same-as-file/skip-doc-ref/README.md)
+[same-as-file]: <> (docs/doc_check/test/same-as-file/skip-doc-ref/README.md)
 ````markdown
 Lorem ipsum dolor sit amet, consectetur adipiscing elit.
 
@@ -115,13 +115,13 @@ This directive supports these parameters in it:
 
 For example, to ensure that the content of a file `test.in`:
 
-[same-as-file]: <> (doc/doc_check/test/file-same-as-stdout/success/test.in)
+[same-as-file]: <> (docs/doc_check/test/file-same-as-stdout/success/test.in)
 ```
 dog
 ```
 
 is exactly the same as the output of command execution `echo dog`, one can use the following directive:
-[same-as-file]: <> (doc/doc_check/test/file-same-as-stdout/success/test.in.dc)
+[same-as-file]: <> (docs/doc_check/test/file-same-as-stdout/success/test.in.dc)
 ```
 file-same-as-stdout({"file": "test.in", "cmd": ["echo", "dog"]})
 ```

src/Builder/OpBuildTable.inc

@@ -1,7 +1,7 @@
 //********************************************************
 //   Do not modify this file directly.
 //   This file is automatically generated via script.
-//   Details can be found in doc/readonnxdefs.md .
+//   Details can be found in docs/readonnxdefs.md .
 //********************************************************
 
 if (opName == "Abs")

src/Dialect/ONNX/ONNXOps.td.inc

@@ -1,7 +1,7 @@
 //********************************************************
 //   Do not modify this file directly.
 //   This file is automatically generated via script.
-//   Details can be found in doc/readonnxdefs.md .
+//   Details can be found in docs/readonnxdefs.md .
 //********************************************************
 
 def ONNXAbsOp:ONNX_Op<"Abs",

utils/gen_doc.py

@@ -533,7 +533,7 @@ def main(args):  # type: (Type[Args]) -> None
         '//********************************************************\n'
         '//   Do not modify this file directly.\n'
         '//   This file is automatically generated via script.\n'
-        '//   Details can be found in doc/readonnxdefs.md .\n'
+        '//   Details can be found in docs/readonnxdefs.md .\n'
         '//********************************************************\n\n')
     autogen_warning = autogen_warning.format(curr_utc_time)