onnx-mlir/docs/doc_check/utils.py

92 lines
2.2 KiB
Python
Raw Permalink Normal View History

[RFC] Doc-check utility. (#12) * 1. Implement doc-check utility. * 1. Move ONNF installation script to a standalone script file. * 1. Modify build script to install llvm-project next to ONNF. The build script used to install llvm-project inside ONNF, which didn't make sense. * 1. Check out code to ONNF directory. * 1. Pass path parameter correctly. * 1. Debugging buildbot. * 1. Remove debug code. * 1. Update installation instructions in README.md. 2. Enforce consistency with scripts used in testing using doc-check. * 1. Fix error with respect to syntax to build multiple CMake targets. * 1. Move doc-check to doc_check. 2. Remove directive_config in top-level driver. * 1. Build onnf and check-mlir-lit separately because only CMake 3.15+ supports building multiple targets in one cmake --build run. * 1. Use new env variables to locate LLVM-Project. * 1. Documentation nits. * 1. Prettify buildbot scripts. * 1. Fix build script error. * 1. Support exclude_dirs in DocCheck. 2. Add README for DocCheck. * 1. Mark python3 interpreter as required. 2. Use imported interpreter target. * 1. Automatically deduce doc file extension in DocCheckCtx. 2. Rename ctx.open -> ctx.open_doc since it should only be used to open doc file. 3. Always read line in parser, instead of reading lines in driver and then passing it to parser.py. * 1. Rename parser -> doc_parser due to name conflict with python built-in module. 2. Explose doc_check module directory first before importing; otherwise if the doc_check utility is invoked by other script, importing will not work correctly. * 1. Keep renaming parser -> doc_parser. 2. Explicitly define a default configuration parser that parses the configuration into a python dictionary. * 1. Add test for doc-check. 2. Exclude doc-check tests from project dock-check because base directory is different. * 1. Raise ValueError if directive configuration fails to parse. 2. Format code. * Shorten test case documentation. Show example of using same-as-file directive, check with DocCheck. * 1. Shorten test case documentation. 2. More documentation, check documentation with DocCheck. * 1. Add copyright notice. * 1. Make documentation clearer. 2. Prettify build-scripts. * 1. Provide more documentation. 2. Fix some non-compliance with pep8 recommendations. Co-authored-by: Gheorghe-Teodor Bercea <gt.bercea@gmail.com>
2020-01-10 07:35:52 +08:00
# ===-------------------------- utils.py - Utility ------------------------===//
#
# Copyright 2019-2020 The IBM Research Authors.
#
# =============================================================================
#
# ===----------------------------------------------------------------------===//
import logging
import os
# Based on https://stackoverflow.com/a/6367075.
class WrappedFile(object):
"""
Complements the standard python file object in two ways:
- Implements a line number based EOF checker.
- Allows retriving the current line position of the cursor, which is
good for error localization.
"""
def __init__(self, f):
self.f = f
self.line = 0
# Compute the total number of lines.
self.num_lines = len(f.readlines())
f.seek(0)
def close(self):
return self.f.close()
def readline(self):
self.line += 1
return self.f.readline()
def next_non_empty_line(self):
while not self.eof():
line = self.readline()
if len(line.strip()):
return line
raise RuntimeError("Enf of file.")
def skip_lines(self, num_lines):
for i in range(num_lines):
self.readline()
def eof(self):
return self.line >= self.num_lines
# to allow using in 'with' statements
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.close()
class DocCheckerCtx(object):
def __init__(self, root_dir: str):
self.root_dir = root_dir
self.doc_file = None
def open_doc(self, file_name):
self.doc_file = WrappedFile(open(file_name, 'r'))
return self.doc_file
def doc_file_ext(self):
assert self.doc_file is not None, "hasn't opened any doc file"
_, file_extension = os.path.splitext(self.doc_file.f.name)
return file_extension
def success(states=None):
return "ok", states
def failure(states=None):
return "failed", states
def succeeded(states):
return states[0] == "ok"
def setup_logger(name):
handler = logging.StreamHandler()
logger = logging.getLogger(name)
logger.setLevel(logging.DEBUG)
logger.addHandler(handler)
return logger