travis: add docker based cmake build job

Submitted by Andres Gomez on Dec. 29, 2017, 2:04 a.m.

Details

Message ID 20171229020436.10320-1-agomez@igalia.com
State Superseded
Headers show
Series "travis: add docker based cmake build job" ( rev: 1 ) in Piglit

Not browsing as part of any series.

Commit Message

Andres Gomez Dec. 29, 2017, 2:04 a.m.
Until now we were only running the python unit tests.

It seems desirable to also check that the CMake based build compiles
successfully. We do that now using docker.

The docker build can be tweaked with some environment variables and,
also, be stored in the docker hub if desired. Check the changes for
extra details regarding these variables.

Cc: Dylan Baker <dylan@pnwbakers.com>
Cc: Juan A. Suarez Romero <jasuarez@igalia.com>
Signed-off-by: Andres Gomez <agomez@igalia.com>
---
We choose to do this task using docker instead of directly in the
Travis CI VM because of various reasons.

First, the Travis env is configured for a python execution. Adding the
bits for a C compilation will mean some extra complication which is
not that big by just delegating this to a docker service.

Second, docker brings some additional benefits. For example, using
docker makes the CI indepent from the CI infrastructure; docker images
are storable and downloadable so we can get the same building setting
that was used in a specific moment; etc.

 .travis.yml              | 47 ++++++++++++++++++++++++--
 docker/Rockerfile.piglit | 88 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 132 insertions(+), 3 deletions(-)
 create mode 100644 docker/Rockerfile.piglit

Patch hide | download patch | download mbox

diff --git a/.travis.yml b/.travis.yml
index b47829ff9..32a6b53e5 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,5 +1,16 @@ 
+sudo: false
+os: linux
 language: python
-cache: pip
+cache:
+  - ccache
+  - pip
+services:
+  - docker
+
+env:
+  global:
+    - BUILD=pytest
+
 matrix:
   include:
     - python: 2.7
@@ -12,7 +23,37 @@  matrix:
       env: TOX_ENV="py35-{generator,noaccel,accel-nix,streams}"
     - python: 3.6
       env: TOX_ENV="py36-{generator,noaccel,accel-nix,streams}"
+    - env: BUILD=cmake
+
 install:
-    pip install tox
+  - |
+    if [[ $BUILD == pytest ]]; then
+      pip install tox
+    else
+      wget https://github.com/grammarly/rocker/releases/download/1.3.1/rocker-1.3.1-linux_amd64.tar.gz
+      tar xvf rocker-1.3.1-linux_amd64.tar.gz
+      rm rocker-1.3.1-linux_amd64.tar.gz
+    fi
+
+before_script:
+  - |
+    if [[ $BUILD != pytest ]]; then
+      mkdir -p -m777 ~/.ccache
+    fi
+
 script:
-    - tox -e $TOX_ENV
+  - |
+    if [[ $BUILD == pytest ]]; then
+      tox -e $TOX_ENV
+    else
+      ./rocker build -f docker/Rockerfile.piglit .
+    fi
+
+after_success:
+  - |
+    if [[ $BUILD != pytest ]]; then
+      if [[ -n $DOCKER_USERNAME && $TRAVIS_BRANCH == master ]]; then
+        docker login -u "$DOCKER_USERNAME" -p "$DOCKER_PASSWORD"
+        docker push ${DOCKER_IMAGE:-freedesktop/mesa}:piglit
+      fi
+    fi
diff --git a/docker/Rockerfile.piglit b/docker/Rockerfile.piglit
new file mode 100644
index 000000000..989b7d919
--- /dev/null
+++ b/docker/Rockerfile.piglit
@@ -0,0 +1,88 @@ 
+#
+# This builds and installs Piglit.
+#
+# ~~~
+#  rocker build -f Rockerfile.piglit [--attach]                 \
+#    [--var TAG=piglit]         # piglit-17.0, piglit-13.0, ... \
+#    [--var RELEASE=xenial]     # master, pre-release-17.0, pre-release-13.0, ...
+# ~~~
+#
+# Environment variables that are used in the build:
+#  - BASE_DOCKER_IMAGE: name of the image in which this will be based (default: ubuntu)
+#  - DOCKER_IMAGE: name of the final image to be tagged (default: freedesktop/mesa)
+#  - BASE_PIGLIT_REPO: custom piglit repository to use with a specific release
+#  - MAKEFLAGS: flags to pass to make (e.g., "-j8")
+#  - CCACHE_DIR: ccache directory (default: ~/.ccache)
+#
+# To run
+#
+# ~~~
+#   mkdir -p -m777 ~/my_results_dir
+#   docker run --privileged --rm -t -v ~/my_results_dir:/results:Z  \
+#              -v /tmp/.X11-unix:/tmp/.X11-unix freedesktop/mesa:piglit
+# ~~~
+#
+
+{{ $base_image := (or .Env.BASE_DOCKER_IMAGE "ubuntu") }}
+{{ $image := (or .Env.BASE_DOCKER_IMAGE "freedesktop/mesa") }}
+{{ $release := (or .RELEASE "xenial") }}
+{{ $ccachedir := (or .Env.CCACHE_DIR "~/.ccache") }}
+
+FROM {{ $base_image }}:{{ $release }}
+
+LABEL maintainer "Andres Gomez <agomez@igalia.com>"
+
+USER root
+
+ENV LC_ALL=C.UTF-8
+
+RUN apt-get update                                                      \
+  && apt-get -y --no-install-recommends install sudo gcc g++ ccache     \
+    git pkg-config bash-completion cmake                                \
+    libz-dev libpng-dev libgl-dev libegl1-mesa-dev libwaffle-dev        \
+    python3-setuptools python3-pip                                      \
+  && rm -fr /var/lib/apt/lists/*
+
+RUN pip3 install numpy six mako
+
+RUN getent passwd local > /dev/null || adduser --gecos "" local && passwd -d local && adduser local sudo
+
+USER local
+
+{{ if .Env.MAKEFLAGS }}
+ENV MAKEFLAGS={{ .Env.MAKEFLAGS }}
+{{ end }}
+
+WORKDIR /home/local
+
+MOUNT {{ $ccachedir }}:/home/local/.ccache:Z
+
+RUN sudo chown -R local:local /home/local/.ccache
+
+ENV PATH=/usr/lib/ccache:$PATH
+
+{{ if .RELEASE }}
+RUN (PIGLIT_RELEASE={{ .RELEASE }} && FULL_PIGLIT_RELEASE="${PIGLIT_RELEASE##*-}"   \
+      && MAJOR_PIGLIT_RELEASE=`echo "$FULL_PIGLIT_RELEASE" | cut -d . -f 1`         \
+      && MINOR_PIGLIT_RELEASE=`echo "$FULL_PIGLIT_RELEASE" | cut -d . -f 2`         \
+      && git clone --depth 1                                                        \
+               -b test/mesa-"$MAJOR_PIGLIT_RELEASE"."$MINOR_PIGLIT_RELEASE"         \
+               {{ .BASE_PIGLIT_REPO }} /home/local/piglit)             \
+    || git clone --depth 1                                                          \
+          git://anongit.freedesktop.org/piglit /home/local/piglit
+{{ else }}
+ADD . /home/local/piglit
+RUN sudo chown -R local:local /home/local/piglit
+{{ end }}
+
+WORKDIR /home/local/piglit
+
+ATTACH [ "/bin/bash" ]
+
+RUN cmake . && cmake --build .
+
+VOLUME /results
+
+{{ if .TAG }}
+TAG {{ $image }}:{{ .TAG }}
+{{ end }}

Comments

With below comments, Review-by: Juan A. Suarez <jasuarez@igalia.com>


On Fri, 2017-12-29 at 04:04 +0200, Andres Gomez wrote:
> Until now we were only running the python unit tests.
> 
> It seems desirable to also check that the CMake based build compiles
> successfully. We do that now using docker.
> 
> The docker build can be tweaked with some environment variables and,
> also, be stored in the docker hub if desired. Check the changes for
> extra details regarding these variables.
> 
> Cc: Dylan Baker <dylan@pnwbakers.com>
> Cc: Juan A. Suarez Romero <jasuarez@igalia.com>
> Signed-off-by: Andres Gomez <agomez@igalia.com>
> ---
> We choose to do this task using docker instead of directly in the
> Travis CI VM because of various reasons.
> 
> First, the Travis env is configured for a python execution. Adding the
> bits for a C compilation will mean some extra complication which is
> not that big by just delegating this to a docker service.
> 
> Second, docker brings some additional benefits. For example, using
> docker makes the CI indepent from the CI infrastructure; docker images
> are storable and downloadable so we can get the same building setting
> that was used in a specific moment; etc.
> 
>  .travis.yml              | 47 ++++++++++++++++++++++++--
>  docker/Rockerfile.piglit | 88 ++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 132 insertions(+), 3 deletions(-)
>  create mode 100644 docker/Rockerfile.piglit
> 
> diff --git a/.travis.yml b/.travis.yml
> index b47829ff9..32a6b53e5 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -1,5 +1,16 @@
> +sudo: false
> +os: linux
>  language: python
> -cache: pip
> +cache:
> +  - ccache
> +  - pip
> +services:
> +  - docker
> +
> +env:
> +  global:
> +    - BUILD=pytest
> +
>  matrix:
>    include:
>      - python: 2.7
> @@ -12,7 +23,37 @@ matrix:
>        env: TOX_ENV="py35-{generator,noaccel,accel-nix,streams}"
>      - python: 3.6
>        env: TOX_ENV="py36-{generator,noaccel,accel-nix,streams}"
> +    - env: BUILD=cmake
> +
>  install:
> -    pip install tox
> +  - |
> +    if [[ $BUILD == pytest ]]; then
> +      pip install tox
> +    else
> +      wget https://github.com/grammarly/rocker/releases/download/1.3.1/rocker-1.3.1-linux_amd64.tar.gz
> +      tar xvf rocker-1.3.1-linux_amd64.tar.gz
> +      rm rocker-1.3.1-linux_amd64.tar.gz
> +    fi
> +
> +before_script:
> +  - |
> +    if [[ $BUILD != pytest ]]; then
> +      mkdir -p -m777 ~/.ccache
> +    fi
> +
>  script:
> -    - tox -e $TOX_ENV
> +  - |
> +    if [[ $BUILD == pytest ]]; then
> +      tox -e $TOX_ENV
> +    else
> +      ./rocker build -f docker/Rockerfile.piglit .
> +    fi
> +
> +after_success:
> +  - |
> +    if [[ $BUILD != pytest ]]; then
> +      if [[ -n $DOCKER_USERNAME && $TRAVIS_BRANCH == master ]]; then
> +        docker login -u "$DOCKER_USERNAME" -p "$DOCKER_PASSWORD"
> +        docker push ${DOCKER_IMAGE:-freedesktop/mesa}:piglit
> +      fi
> +    fi
> diff --git a/docker/Rockerfile.piglit b/docker/Rockerfile.piglit
> new file mode 100644
> index 000000000..989b7d919
> --- /dev/null
> +++ b/docker/Rockerfile.piglit
> @@ -0,0 +1,88 @@
> +#
> +# This builds and installs Piglit.
> +#
> +# ~~~
> +#  rocker build -f Rockerfile.piglit [--attach]                 \
> +#    [--var TAG=piglit]         # piglit-17.0, piglit-13.0, ... \
> +#    [--var RELEASE=xenial]     # master, pre-release-17.0, pre-release-13.0, ...
> +# ~~~
> +#
> +# Environment variables that are used in the build:
> +#  - BASE_DOCKER_IMAGE: name of the image in which this will be based (default: ubuntu)
> +#  - DOCKER_IMAGE: name of the final image to be tagged (default: freedesktop/mesa)
> +#  - BASE_PIGLIT_REPO: custom piglit repository to use with a specific release
> +#  - MAKEFLAGS: flags to pass to make (e.g., "-j8")
> +#  - CCACHE_DIR: ccache directory (default: ~/.ccache)
> +#
> +# To run
> +#
> +# ~~~
> +#   mkdir -p -m777 ~/my_results_dir
> +#   docker run --privileged --rm -t -v ~/my_results_dir:/results:Z  \
> +#              -v /tmp/.X11-unix:/tmp/.X11-unix freedesktop/mesa:piglit
> +# ~~~
> +#
> +
> +{{ $base_image := (or .Env.BASE_DOCKER_IMAGE "ubuntu") }}
> +{{ $image := (or .Env.BASE_DOCKER_IMAGE "freedesktop/mesa") }}
> +{{ $release := (or .RELEASE "xenial") }}
> +{{ $ccachedir := (or .Env.CCACHE_DIR "~/.ccache") }}
> +
> +FROM {{ $base_image }}:{{ $release }}
> +
> +LABEL maintainer "Andres Gomez <agomez@igalia.com>"
> +
> +USER root
> +
> +ENV LC_ALL=C.UTF-8
> +
> +RUN apt-get update                                                      \
> +  && apt-get -y --no-install-recommends install sudo gcc g++ ccache     \
> +    git pkg-config bash-completion cmake                                \
> +    libz-dev libpng-dev libgl-dev libegl1-mesa-dev libwaffle-dev        \
> +    python3-setuptools python3-pip                                      \
> +  && rm -fr /var/lib/apt/lists/*
> +
> +RUN pip3 install numpy six mako
> +
> +RUN getent passwd local > /dev/null || adduser --gecos "" local && passwd -d local && adduser local sudo
> +
> +USER local
> +
> +{{ if .Env.MAKEFLAGS }}
> +ENV MAKEFLAGS={{ .Env.MAKEFLAGS }}
> +{{ end }}
> +
> +WORKDIR /home/local
> +
> +MOUNT {{ $ccachedir }}:/home/local/.ccache:Z
> +
> +RUN sudo chown -R local:local /home/local/.ccache
> +
> +ENV PATH=/usr/lib/ccache:$PATH
> +
> +{{ if .RELEASE }}
> +RUN (PIGLIT_RELEASE={{ .RELEASE }} && FULL_PIGLIT_RELEASE="${PIGLIT_RELEASE##*-}"   \
> +      && MAJOR_PIGLIT_RELEASE=`echo "$FULL_PIGLIT_RELEASE" | cut -d . -f 1`         \
> +      && MINOR_PIGLIT_RELEASE=`echo "$FULL_PIGLIT_RELEASE" | cut -d . -f 2`         \
> +      && git clone --depth 1                                                        \
> +               -b test/mesa-"$MAJOR_PIGLIT_RELEASE"."$MINOR_PIGLIT_RELEASE"         \
> +               {{ .BASE_PIGLIT_REPO }} /home/local/piglit)             \
> +    || git clone --depth 1                                                          \
> +          git://anongit.freedesktop.org/piglit /home/local/piglit

As the purpose of Travis CI is to test current commit, I would remove
this RELEASE stuff, and just keep testing the current commit.

> +{{ else }}
> +ADD . /home/local/piglit
> +RUN sudo chown -R local:local /home/local/piglit
> +{{ end }}
> +
> +WORKDIR /home/local/piglit
> +
> +ATTACH [ "/bin/bash" ]
> +
> +RUN cmake . && cmake --build .
> +
> +VOLUME /results
> +
> +{{ if .TAG }}
> +TAG {{ $image }}:{{ .TAG }}
> +{{ end }}