onnx-mlir/.travis.yml

203 lines
9.1 KiB
YAML
Raw Normal View History

services:
- docker
language: cpp
os:
- linux
dist: bionic
arch:
- s390x
- ppc64le
- amd64
env:
- NIGHTLY=true
- NIGHTLY=false
Update llvm commit ID to 1d01fc1 (#292) * Fix for LLVM revision D85495 * Fix for LLVM revision DD86121 * Fix for LLVM revision D85622 (f9dc2b7) TODO: Change preloadDialectsInContext to false Memo for previous fixes: D86121 (250f43d), D85495 (575b22b) * clang-format * Update llvm commit ID of README and clone-mlir.sh * Updated llvm commit ID of README.md * Fix for passing backend tests * Removed the commented code * Empty commit for triggering rebuild * Test multi-stage travis build * Specify stage order. * Empty commit for triggering rebuild * Update prereq.s390x.Dockerfile Make it possible to execute s390x prereq docker multiple times. * Build prereq for each arch * Fix multi-arch prereq build. * timeout at 40m * Update .travis.yml * add ppc64le prereq builder * Run ppc docker prereq build multiple times * Do not test branch update unless it's mater. * Fix dockerfile. * Fix typo in travis.yml. * Fix ppc64 docker file * Update .travis.yml * turn off metacopy on ppc64le * Update .travis.yml * Turn off metacopy. * Turn off metacopy inside Dockerfile in ppc64. * No sudo in Docker. * Remove metacopy config from Dockerfile. * Change base image to be bionic. * Using newer linux distro for ppc64. * Turn off metacopy in before_install. * Fix sudo permission issue. * Run docker info. * Allow amd64 docker file to be built multiple times * Support building amd64 prereq. * Fix amd64 docker file typo. * fix ppc64le dockerfile typo. * timeout from 40m -> 30m * 40m->30m * 40m->30m * fix bug preventing incremental build. * fix bug preventing incremental build. * Bump CircleCI cache version. * Push to production prereq container repository and condition prereq docker rebuild on commit message. * Rebuild prereq docker. * Move default script to top-level. * Python not properly installed. * amd64 -> x86 * Rebuild prereq docker. * Rebuild prereq docker. * Rebuild prereq docker. * Restart all CI. * Disallow cache on Jenkins docker build. * Restart zJenkins. * Restart zJenkins. Co-authored-by: Haruki Imai <imaihal@jp.ibm.com> Co-authored-by: Alexandre Eichenberger <alexe@us.ibm.com>
2020-09-09 23:12:01 +08:00
# Do not test branch update unless it's mater.
branches:
only:
- master
stages:
- name: prereq-init
- name: prereq-0
- name: prereq-1
- name: prereq-2
- name: prereq-final
- name: test
script:
- export CPU_ARCH=$TRAVIS_CPU_ARCH
- if [ "$CPU_ARCH" == "amd64" ] ; then
export CPU_ARCH="x86";
fi
- echo "CPU Architecture is " $CPU_ARCH
- echo "commit is " $TRAVIS_COMMIT
- df -h
- if [ "$NIGHTLY" = true ] ; then
echo "Using nightly llvm-mlir image, build may fail.";
docker build --tag onnx-mlir-build:initial --build-arg BASE="onnxmlirczar/onnx-mlir-llvmimage:${CPU_ARCH}-nightly" -f ./docker/$CPU_ARCH.Dockerfile ./docker;
else
echo "Using latest compatible llvm-mlir image, build should not fail.";
docker build --tag onnx-mlir-build:initial --build-arg BASE="onnxmlirczar/onnx-mlir-llvmimage:$CPU_ARCH" -f ./docker/$CPU_ARCH.Dockerfile ./docker;
fi
- docker run -itd --name build onnx-mlir-build:initial
- echo "docker is running now"
- df -h
- docker cp $(pwd) build:/build/onnx-mlir
- cd docker
- chmod a+x compile-onnx-mlir.sh test-onnx-mlir.sh
- docker cp compile-onnx-mlir.sh build:/usr/bin/compile-onnx-mlir.sh
- docker cp test-onnx-mlir.sh build:/usr/bin/test-onnx-mlir.sh
- echo "about to execute build inside docker"
- docker exec build df -h
- docker exec build compile-onnx-mlir.sh
- docker exec build test-onnx-mlir.sh
- docker commit build onnxmlirczar/onnx-mlir-build:$CPU_ARCH
Update llvm commit ID to 1d01fc1 (#292) * Fix for LLVM revision D85495 * Fix for LLVM revision DD86121 * Fix for LLVM revision D85622 (f9dc2b7) TODO: Change preloadDialectsInContext to false Memo for previous fixes: D86121 (250f43d), D85495 (575b22b) * clang-format * Update llvm commit ID of README and clone-mlir.sh * Updated llvm commit ID of README.md * Fix for passing backend tests * Removed the commented code * Empty commit for triggering rebuild * Test multi-stage travis build * Specify stage order. * Empty commit for triggering rebuild * Update prereq.s390x.Dockerfile Make it possible to execute s390x prereq docker multiple times. * Build prereq for each arch * Fix multi-arch prereq build. * timeout at 40m * Update .travis.yml * add ppc64le prereq builder * Run ppc docker prereq build multiple times * Do not test branch update unless it's mater. * Fix dockerfile. * Fix typo in travis.yml. * Fix ppc64 docker file * Update .travis.yml * turn off metacopy on ppc64le * Update .travis.yml * Turn off metacopy. * Turn off metacopy inside Dockerfile in ppc64. * No sudo in Docker. * Remove metacopy config from Dockerfile. * Change base image to be bionic. * Using newer linux distro for ppc64. * Turn off metacopy in before_install. * Fix sudo permission issue. * Run docker info. * Allow amd64 docker file to be built multiple times * Support building amd64 prereq. * Fix amd64 docker file typo. * fix ppc64le dockerfile typo. * timeout from 40m -> 30m * 40m->30m * 40m->30m * fix bug preventing incremental build. * fix bug preventing incremental build. * Bump CircleCI cache version. * Push to production prereq container repository and condition prereq docker rebuild on commit message. * Rebuild prereq docker. * Move default script to top-level. * Python not properly installed. * amd64 -> x86 * Rebuild prereq docker. * Rebuild prereq docker. * Rebuild prereq docker. * Restart all CI. * Disallow cache on Jenkins docker build. * Restart zJenkins. * Restart zJenkins. Co-authored-by: Haruki Imai <imaihal@jp.ibm.com> Co-authored-by: Alexandre Eichenberger <alexe@us.ibm.com>
2020-09-09 23:12:01 +08:00
jobs:
allow_failures:
- env: NIGHTLY=true
include:
##########################################################################################
#
# Incrementally build s390 prereq docker images through 5 time-constrained steps.
#
##########################################################################################
- stage: prereq-init
if: commit_message = "Rebuild prereq docker."
arch:
- s390x
script:
- docker build -t onnxmlirczar/onnx-mlir-llvmimage-partial:s390x -f docker/prereq.s390x.Dockerfile --build-arg BASE_IMAGE="ubuntu:focal" ./utils
- docker login -u onnxmlirczar -p "$DOCKER_HUB_TOKEN"
- docker push onnxmlirczar/onnx-mlir-llvmimage-partial:s390x
- stage: prereq-0
if: commit_message = "Rebuild prereq docker."
arch:
- s390x
script:
- docker build -t onnxmlirczar/onnx-mlir-llvmimage-partial:s390x -f docker/prereq.s390x.Dockerfile --build-arg BASE_IMAGE="onnxmlirczar/onnx-mlir-llvmimage-partial:s390x" ./utils
- docker login -u onnxmlirczar -p "$DOCKER_HUB_TOKEN"
- docker push onnxmlirczar/onnx-mlir-llvmimage-partial:s390x
- stage: prereq-1
if: commit_message = "Rebuild prereq docker."
arch:
- s390x
script:
- docker build -t onnxmlirczar/onnx-mlir-llvmimage-partial:s390x -f docker/prereq.s390x.Dockerfile --build-arg BASE_IMAGE="onnxmlirczar/onnx-mlir-llvmimage-partial:s390x" ./utils
- docker login -u onnxmlirczar -p "$DOCKER_HUB_TOKEN"
- docker push onnxmlirczar/onnx-mlir-llvmimage-partial:s390x
- stage: prereq-2
if: commit_message = "Rebuild prereq docker."
arch:
- s390x
script:
- docker build -t onnxmlirczar/onnx-mlir-llvmimage-partial:s390x -f docker/prereq.s390x.Dockerfile --build-arg BASE_IMAGE="onnxmlirczar/onnx-mlir-llvmimage-partial:s390x" ./utils
- docker login -u onnxmlirczar -p "$DOCKER_HUB_TOKEN"
- docker push onnxmlirczar/onnx-mlir-llvmimage-partial:s390x
- stage: prereq-final
if: commit_message = "Rebuild prereq docker."
arch:
- s390x
script:
- docker build -t onnxmlirczar/onnx-mlir-llvmimage:s390x -f docker/prereq.s390x.Dockerfile --build-arg BASE_IMAGE="onnxmlirczar/onnx-mlir-llvmimage-partial:s390x" ./utils
- docker login -u onnxmlirczar -p "$DOCKER_HUB_TOKEN"
- docker push onnxmlirczar/onnx-mlir-llvmimage:s390x
##########################################################################################
#
# Incrementally build ppc64le prereq docker images through 5 time-constrained steps.
#
##########################################################################################
- stage: prereq-init
if: commit_message = "Rebuild prereq docker."
arch:
- ppc64le
script:
- docker build -t onnxmlirczar/onnx-mlir-llvmimage-partial:ppc64le -f docker/prereq.ppc64le.Dockerfile --build-arg BASE_IMAGE="ubuntu:bionic" ./utils
- docker login -u onnxmlirczar -p "$DOCKER_HUB_TOKEN"
- docker push onnxmlirczar/onnx-mlir-llvmimage-partial:ppc64le
- stage: prereq-0
if: commit_message = "Rebuild prereq docker."
arch:
- ppc64le
script:
- docker build -t onnxmlirczar/onnx-mlir-llvmimage-partial:ppc64le -f docker/prereq.ppc64le.Dockerfile --build-arg BASE_IMAGE="onnxmlirczar/onnx-mlir-llvmimage-partial:ppc64le" ./utils
- docker login -u onnxmlirczar -p "$DOCKER_HUB_TOKEN"
- docker push onnxmlirczar/onnx-mlir-llvmimage-partial:ppc64le
- stage: prereq-1
if: commit_message = "Rebuild prereq docker."
arch:
- ppc64le
script:
- docker build -t onnxmlirczar/onnx-mlir-llvmimage-partial:ppc64le -f docker/prereq.ppc64le.Dockerfile --build-arg BASE_IMAGE="onnxmlirczar/onnx-mlir-llvmimage-partial:ppc64le" ./utils
- docker login -u onnxmlirczar -p "$DOCKER_HUB_TOKEN"
- docker push onnxmlirczar/onnx-mlir-llvmimage-partial:ppc64le
- stage: prereq-2
if: commit_message = "Rebuild prereq docker."
arch:
- ppc64le
script:
- docker build -t onnxmlirczar/onnx-mlir-llvmimage-partial:ppc64le -f docker/prereq.ppc64le.Dockerfile --build-arg BASE_IMAGE="onnxmlirczar/onnx-mlir-llvmimage-partial:ppc64le" ./utils
- docker login -u onnxmlirczar -p "$DOCKER_HUB_TOKEN"
- docker push onnxmlirczar/onnx-mlir-llvmimage-partial:ppc64le
- stage: prereq-final
if: commit_message = "Rebuild prereq docker."
arch:
- ppc64le
script:
- docker build -t onnxmlirczar/onnx-mlir-llvmimage:ppc64le -f docker/prereq.ppc64le.Dockerfile --build-arg BASE_IMAGE="onnxmlirczar/onnx-mlir-llvmimage-partial:ppc64le" ./utils
- docker login -u onnxmlirczar -p "$DOCKER_HUB_TOKEN"
- docker push onnxmlirczar/onnx-mlir-llvmimage:ppc64le
##########################################################################################
#
# Incrementally build amd64 prereq docker images through 5 time-constrained steps.
#
##########################################################################################
- stage: prereq-init
if: commit_message = "Rebuild prereq docker."
arch:
- amd64
script:
- docker build -t onnxmlirczar/onnx-mlir-llvmimage-partial:x86 -f docker/prereq.amd64.Dockerfile --build-arg BASE_IMAGE="ubuntu:focal" ./utils
- docker login -u onnxmlirczar -p "$DOCKER_HUB_TOKEN"
- docker push onnxmlirczar/onnx-mlir-llvmimage-partial:x86
- stage: prereq-0
if: commit_message = "Rebuild prereq docker."
arch:
- amd64
script:
- docker build -t onnxmlirczar/onnx-mlir-llvmimage-partial:x86 -f docker/prereq.amd64.Dockerfile --build-arg BASE_IMAGE="onnxmlirczar/onnx-mlir-llvmimage-partial:x86" ./utils
- docker login -u onnxmlirczar -p "$DOCKER_HUB_TOKEN"
- docker push onnxmlirczar/onnx-mlir-llvmimage-partial:x86
- stage: prereq-1
if: commit_message = "Rebuild prereq docker."
arch:
- amd64
script:
- docker build -t onnxmlirczar/onnx-mlir-llvmimage-partial:x86 -f docker/prereq.amd64.Dockerfile --build-arg BASE_IMAGE="onnxmlirczar/onnx-mlir-llvmimage-partial:x86" ./utils
- docker login -u onnxmlirczar -p "$DOCKER_HUB_TOKEN"
- docker push onnxmlirczar/onnx-mlir-llvmimage-partial:x86
- stage: prereq-2
if: commit_message = "Rebuild prereq docker."
arch:
- amd64
script:
- docker build -t onnxmlirczar/onnx-mlir-llvmimage-partial:x86 -f docker/prereq.amd64.Dockerfile --build-arg BASE_IMAGE="onnxmlirczar/onnx-mlir-llvmimage-partial:x86" ./utils
- docker login -u onnxmlirczar -p "$DOCKER_HUB_TOKEN"
- docker push onnxmlirczar/onnx-mlir-llvmimage-partial:x86
- stage: prereq-final
if: commit_message = "Rebuild prereq docker."
arch:
- amd64
script:
- docker build -t onnxmlirczar/onnx-mlir-llvmimage:x86 -f docker/prereq.amd64.Dockerfile --build-arg BASE_IMAGE="onnxmlirczar/onnx-mlir-llvmimage-partial:x86" ./utils
- docker login -u onnxmlirczar -p "$DOCKER_HUB_TOKEN"
- docker push onnxmlirczar/onnx-mlir-llvmimage:x86
after_success:
- if [ $TRAVIS_PULL_REQUEST == false ] && [ $TRAVIS_BRANCH == "master" ] && [ $NIGHTLY == false ]; then
echo "Pushing new build to docker hub";
docker login -u onnxmlirczar -p "$DOCKER_HUB_TOKEN";
docker push onnxmlirczar/onnx-mlir-build:$CPU_ARCH;
Update llvm commit ID to 1d01fc1 (#292) * Fix for LLVM revision D85495 * Fix for LLVM revision DD86121 * Fix for LLVM revision D85622 (f9dc2b7) TODO: Change preloadDialectsInContext to false Memo for previous fixes: D86121 (250f43d), D85495 (575b22b) * clang-format * Update llvm commit ID of README and clone-mlir.sh * Updated llvm commit ID of README.md * Fix for passing backend tests * Removed the commented code * Empty commit for triggering rebuild * Test multi-stage travis build * Specify stage order. * Empty commit for triggering rebuild * Update prereq.s390x.Dockerfile Make it possible to execute s390x prereq docker multiple times. * Build prereq for each arch * Fix multi-arch prereq build. * timeout at 40m * Update .travis.yml * add ppc64le prereq builder * Run ppc docker prereq build multiple times * Do not test branch update unless it's mater. * Fix dockerfile. * Fix typo in travis.yml. * Fix ppc64 docker file * Update .travis.yml * turn off metacopy on ppc64le * Update .travis.yml * Turn off metacopy. * Turn off metacopy inside Dockerfile in ppc64. * No sudo in Docker. * Remove metacopy config from Dockerfile. * Change base image to be bionic. * Using newer linux distro for ppc64. * Turn off metacopy in before_install. * Fix sudo permission issue. * Run docker info. * Allow amd64 docker file to be built multiple times * Support building amd64 prereq. * Fix amd64 docker file typo. * fix ppc64le dockerfile typo. * timeout from 40m -> 30m * 40m->30m * 40m->30m * fix bug preventing incremental build. * fix bug preventing incremental build. * Bump CircleCI cache version. * Push to production prereq container repository and condition prereq docker rebuild on commit message. * Rebuild prereq docker. * Move default script to top-level. * Python not properly installed. * amd64 -> x86 * Rebuild prereq docker. * Rebuild prereq docker. * Rebuild prereq docker. * Restart all CI. * Disallow cache on Jenkins docker build. * Restart zJenkins. * Restart zJenkins. Co-authored-by: Haruki Imai <imaihal@jp.ibm.com> Co-authored-by: Alexandre Eichenberger <alexe@us.ibm.com>
2020-09-09 23:12:01 +08:00
else
echo "Skip pushing build";
fi