| Message ID | 20180129021117.9056-1-agomez@igalia.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series |
"travis: add docker based cmake build job"
( rev:
3
)
in
Piglit |
diff --git a/.travis.yml b/.travis.yml index b47829ff9..32a6b53e5 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,5 +1,16 @@ +sudo: false +os: linux language: python -cache: pip +cache: + - ccache + - pip +services: + - docker + +env: + global: + - BUILD=pytest + matrix: include: - python: 2.7 @@ -12,7 +23,37 @@ matrix: env: TOX_ENV="py35-{generator,noaccel,accel-nix,streams}" - python: 3.6 env: TOX_ENV="py36-{generator,noaccel,accel-nix,streams}" + - env: BUILD=cmake + install: - pip install tox + - | + if [[ $BUILD == pytest ]]; then + pip install tox + else + wget https://github.com/grammarly/rocker/releases/download/1.3.1/rocker-1.3.1-linux_amd64.tar.gz + tar xvf rocker-1.3.1-linux_amd64.tar.gz + rm rocker-1.3.1-linux_amd64.tar.gz + fi + +before_script: + - | + if [[ $BUILD != pytest ]]; then + mkdir -p -m777 ~/.ccache + fi + script: - - tox -e $TOX_ENV + - | + if [[ $BUILD == pytest ]]; then + tox -e $TOX_ENV + else + ./rocker build -f docker/Rockerfile.piglit . + fi + +after_success: + - | + if [[ $BUILD != pytest ]]; then + if [[ -n $DOCKER_USERNAME && $TRAVIS_BRANCH == master ]]; then + docker login -u "$DOCKER_USERNAME" -p "$DOCKER_PASSWORD" + docker push ${DOCKER_IMAGE:-freedesktop/mesa}:piglit + fi + fi diff --git a/docker/Rockerfile.piglit b/docker/Rockerfile.piglit new file mode 100644 index 000000000..d325205b0 --- /dev/null +++ b/docker/Rockerfile.piglit @@ -0,0 +1,74 @@ +# +# This builds Piglit. +# +# ~~~ +# rocker build -f Rockerfile.piglit [--attach] \ +# [--var TAG=piglit] # piglit-17.0, piglit-13.0, ... \ +# [--var PARENT=ubuntu:xenial] # ubuntu:xenial, freedesktop/mesa:17.3, ... +# ~~~ +# +# Environment variables that are used in the build: +# - DOCKER_IMAGE: name of the final image to be tagged (default: freedesktop/mesa) +# - MAKEFLAGS: flags to pass to make (e.g., "-j8") +# - CCACHE_DIR: ccache directory (default: ~/.ccache) +# +# To run +# +# ~~~ +# mkdir -p -m777 ~/my_results_dir +# docker run --privileged --rm -t -v ~/my_results_dir:/results:Z \ +# -v /tmp/.X11-unix:/tmp/.X11-unix freedesktop/mesa:piglit +# ~~~ +# + +{{ $image := (or .Env.DOCKER_IMAGE "freedesktop/mesa") }} +{{ $parent_image := (or .PARENT "ubuntu:xenial") }} +{{ $ccachedir := (or .Env.CCACHE_DIR "~/.ccache") }} + +FROM {{ $parent_image }} + +LABEL maintainer "Andres Gomez <agomez@igalia.com>" + +USER root + +ENV LC_ALL=C.UTF-8 + +RUN apt-get update \ + && apt-get -y --no-install-recommends install sudo gcc g++ ccache \ + git pkg-config bash-completion cmake \ + libz-dev libpng-dev libgl-dev libegl1-mesa-dev libwaffle-dev \ + python3-setuptools python3-pip \ + && rm -fr /var/lib/apt/lists/* + +RUN pip3 install numpy six mako + +RUN getent passwd local > /dev/null || adduser --gecos "" local && passwd -d local && adduser local sudo + +USER local + +{{ if .Env.MAKEFLAGS }} +ENV MAKEFLAGS={{ .Env.MAKEFLAGS }} +{{ end }} + +WORKDIR /home/local + +MOUNT {{ $ccachedir }}:/home/local/.ccache:Z + +RUN sudo chown -R local:local /home/local/.ccache + +ENV PATH=/usr/lib/ccache:$PATH + +ADD . /home/local/piglit +RUN sudo chown -R local:local /home/local/piglit + +WORKDIR /home/local/piglit + +ATTACH [ "/bin/bash" ] + +RUN cmake . && cmake --build . + +VOLUME /results + +{{ if .TAG }} +TAG {{ $image }}:{{ .TAG }} +{{ end }}
Andres Gomez <agomez@igalia.com> writes: > Until now we were only running the python unit tests. > > It seems desirable to also check that the CMake based build compiles > successfully. We do that now using docker. > > The docker build can be tweaked with some environment variables and, > also, be stored in the docker hub if desired. Check the changes for > extra details regarding these variables. > > v2: Removed other build possibilities other than just from inside > Travis-CI, as suggested by Juan. > v3: Replaced the "RELEASE" parameter to create the docker image with > "PARENT" and removed some unneeded documentation after v2, as > suggested by Juan. > > Cc: Dylan Baker <dylan@pnwbakers.com> > Cc: Juan A. Suarez <jasuarez@igalia.com> > Signed-off-by: Andres Gomez <agomez@igalia.com> > Reviewed-by: Juan A. Suarez <jasuarez@igalia.com> I'm definitely interested in extending our tests using Travis, but could you add some comments explaining what this system does? Do we do a bunch of image build work (numpy/six/mako install) on each Travis build, or does it get cached? Do I need to do any setup in Travis in order for this to take effect?
On Mon, 2018-01-29 at 14:06 -0800, Eric Anholt wrote: > Andres Gomez <agomez@igalia.com> writes: > > > Until now we were only running the python unit tests. > > > > It seems desirable to also check that the CMake based build compiles > > successfully. We do that now using docker. > > > > The docker build can be tweaked with some environment variables and, > > also, be stored in the docker hub if desired. Check the changes for > > extra details regarding these variables. > > > > v2: Removed other build possibilities other than just from inside > > Travis-CI, as suggested by Juan. > > v3: Replaced the "RELEASE" parameter to create the docker image with > > "PARENT" and removed some unneeded documentation after v2, as > > suggested by Juan. > > > > Cc: Dylan Baker <dylan@pnwbakers.com> > > Cc: Juan A. Suarez <jasuarez@igalia.com> > > Signed-off-by: Andres Gomez <agomez@igalia.com> > > Reviewed-by: Juan A. Suarez <jasuarez@igalia.com> > > I'm definitely interested in extending our tests using Travis, but could > you add some comments explaining what this system does? Do we do a > bunch of image build work (numpy/six/mako install) on each Travis build, > or does it get cached? Do I need to do any setup in Travis in order for > this to take effect? Yes, I should have been already more verbose with explanations when posting this patch in the first place. First, thanks to your reply I've sent a new v4 version after finding a couple of issues. With this v4 version, you can check this travis build, which is where I've tested the patch: https://travis-ci.org/Igalia/piglit/builds/335657962 There you will see that, in addition to the usual 5 python builds running the unit tests, there is an additional cmake build, this is the docker based build: https://travis-ci.org/Igalia/piglit/jobs/335657970 Without any special setup in Travis, the docker build will run with the default options. This will be done on every run: * Download the parent ubuntu:xenial image. * Install building dependencies from APT. * Install additional Python dependencies (numpy/six/mako). * Build piglit. The build is, however, quite quick since docker and travis have many of the needed bits cached and it also makes use of ccache, but still this process has to happen every time. Now, this patch allows a couple of additional tweaks: 1. You can select a different parent image by setting in Travis-CI the env variable DOCKER_PARENT. You could want to do this for different reasons. From the top of my head: * To speed up the build; using a parent image with the APT and Python dependencies already installed. * To create a build in which there are additional or alternative libraries installed; for example, at Igalia, we have the docker image "igalia/mesa:released-17.3.3.debug". This way, in case we want to later use the generated docker image to test piglit, we will be able to run the test suite against mesa-17.3.3 built with debug symbols. 2. You can define a docker hub repository to which upload the generated build as "$DOCKER_IMAGE:$DOCKER_TAG". For that, you have to set in Travis-CI the following env variables: * DOCKER_IMAGE: The path to the docker hub repository you want to use. We use "igalia/mesa". * DOCKER_TAG: The specific tag for the newly created image. For example, you could use "piglit". Therefore, in our case, a new build will be uploaded as "igalia/mesa:piglit". * DOCKER_USERNAME: Your user to the docker hub. * DOCKER_PASSWORD: Your password for the docker hub. Combining 1. and 2. you could have, for example, automatic builds of piglit's master uploaded to your personal docker hub with a specific mesa version installed. This is very useful for later running the test suite. It is also a good way of providing a specific environment for debugging/reproducing a bug and share with other developers. I hope this clarifies further the aim of this integration. PS: If this is accepted in piglit, my next step is to work in a patch proposal for having a similar use case for mesa. This will ease the combination of 1. and 2. that I was mentioning.
Until now we were only running the python unit tests. It seems desirable to also check that the CMake based build compiles successfully. We do that now using docker. The docker build can be tweaked with some environment variables and, also, be stored in the docker hub if desired. Check the changes for extra details regarding these variables. v2: Removed other build possibilities other than just from inside Travis-CI, as suggested by Juan. v3: Replaced the "RELEASE" parameter to create the docker image with "PARENT" and removed some unneeded documentation after v2, as suggested by Juan. Cc: Dylan Baker <dylan@pnwbakers.com> Cc: Juan A. Suarez <jasuarez@igalia.com> Signed-off-by: Andres Gomez <agomez@igalia.com> Reviewed-by: Juan A. Suarez <jasuarez@igalia.com> --- .travis.yml | 47 ++++++++++++++++++++++++++++-- docker/Rockerfile.piglit | 74 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 118 insertions(+), 3 deletions(-) create mode 100644 docker/Rockerfile.piglit