Bug 1247170 - Part 1: Add "android-gradle-build" Docker image. r=dustin,sebastian

We can't run Docker-in-Docker, so this is a clone of https://hub.docker.com/r/sonatype/nexus, modified to run additional commands. Sonatype Nexus officially supports the Oracle JDK but the OpenJDK appears to work as well, so we use it. This bakes a build.sh and Gradle-specific dependencies into the docker image. This makes sense in a future where the dependencies this fetches are baked into the image used to build Fennec. It makes less sense right now, when the dependencies are uploaded to tooltool and subsequently consumed. It's not critical that we get this right immediately. The actual build.sh script runs the Gradle wrapper for a special dependency project. The wrapper installs Gradle itself, downloads a large number of Gradle dependencies, and uses Jake Wharton's https://github.com/JakeWharton/sdk-manager-plugin/ to download and install the Android SDK. (We could use |mach bootstrap|, but that doesn't (yet) support non-interactive use, and I want to try to use the SDK Manager more generally.) These outputs are then packaged for further use: right now, they're exposed as artifacts, but eventually we'll upload them directly to tooltool. MozReview-Commit-ID: 7upsk1ANuxN --HG-- rename : testing/docker/desktop-build/Dockerfile => testing/docker/android-gradle-build/Dockerfile extra : source : 123c802d928a9a02040cc3316988d9b59db0ea45
2016-02-16 11:11:34 -08:00 · 2016-02-16 11:11:34 -08:00 · cc045e53ff
--- a/testing/docker/android-gradle-build/Dockerfile
+++ b/testing/docker/android-gradle-build/Dockerfile
@ -0,0 +1,57 @@
+FROM          centos:centos7
+MAINTAINER    Nick Alexander <nalexander@mozilla.com>
+
+# Reset user/workdir from parent image so we can install software.
+WORKDIR /
+USER root
+
+# Update base.
+RUN yum upgrade -y
+
+# Install JDK and Sonatype Nexus.  Cribbed directly from
+# https://github.com/sonatype/docker-nexus/blob/fffd2c61b2368292040910c055cf690c8e76a272/oss/Dockerfile.
+
+RUN yum install -y \
+  createrepo \
+  curl \
+  java-1.7.0-openjdk-devel \
+  java-1.7.0-openjdk \
+  sudo \
+  tar \
+  unzip \
+  wget \
+  zip \
+  && yum clean all
+
+ENV NEXUS_VERSION 2.12.0-01
+
+RUN mkdir -p /opt/sonatype/nexus
+
+WORKDIR /tmp
+RUN curl --fail --silent --location --retry 3 \
+    https://download.sonatype.com/nexus/oss/nexus-${NEXUS_VERSION}-bundle.tar.gz \
+    -o /tmp/nexus-${NEXUS_VERSION}-bundle.tar.gz
+RUN curl --fail --silent --location --retry 3 \
+    https://download.sonatype.com/nexus/oss/nexus-${NEXUS_VERSION}-bundle.tar.gz.sha1 \
+    -o /tmp/nexus-${NEXUS_VERSION}-bundle.tar.gz.sha1
+
+RUN echo $(cat nexus-${NEXUS_VERSION}-bundle.tar.gz.sha1) nexus-${NEXUS_VERSION}-bundle.tar.gz | sha1sum -c
+
+RUN tar zxf nexus-${NEXUS_VERSION}-bundle.tar.gz \
+  && mv /tmp/nexus-${NEXUS_VERSION}/* /opt/sonatype/nexus/ \
+  && rm -rf /tmp/nexus-${NEXUS_VERSION} \
+  && rm -rf /tmp/nexus-${NEXUS_VERSION}-bundle.tar.gz
+
+# Install tooltool directly from github.
+RUN mkdir /build
+ADD https://raw.githubusercontent.com/mozilla/build-tooltool/master/tooltool.py /build/tooltool.py
+RUN chmod +rx /build/tooltool.py
+
+# Add build scripts.
+ADD             build.sh /build/
+RUN             chmod +x /build/*
+
+ADD project /project
+
+# Invoke our build scripts by default, but allow other commands.
+CMD /build/build.sh
--- a/testing/docker/android-gradle-build/README.md
+++ b/testing/docker/android-gradle-build/README.md
@ -0,0 +1,2 @@
+This is a docker script for fetching Android Gradle dependenices for
+use in Mozilla's build clusters.
--- a/testing/docker/android-gradle-build/REGISTRY
+++ b/testing/docker/android-gradle-build/REGISTRY
@ -0,0 +1 @@
+taskcluster
--- a/testing/docker/android-gradle-build/VERSION
+++ b/testing/docker/android-gradle-build/VERSION
@ -0,0 +1 @@
+0.0.1
--- a/testing/docker/android-gradle-build/build.sh
+++ b/testing/docker/android-gradle-build/build.sh
@ -0,0 +1,62 @@
+#!/bin/bash -vex
+
+set -x -e
+
+: WORKSPACE ${WORKSPACE:=/workspace}
+: GRADLE_VERSION ${GRADLE_VERSION:=2.7}
+
+set -v
+
+# Frowned upon, but simplest.
+RUN_AS_USER=root NEXUS_WORK=${WORKSPACE}/nexus /opt/sonatype/nexus/bin/nexus start
+
+# Wait "a while" for Nexus to actually start.  Don't fail if this fails.
+wget --quiet --retry-connrefused --waitretry=2 --tries=100 \
+  http://localhost:8081/nexus/service/local/status || true
+rm -rf status
+
+# Verify Nexus has actually started.  Fail if this fails.
+curl --fail --silent --location http://localhost:8081/nexus/service/local/status | grep '<state>STARTED</state>'
+
+export JAVA_HOME=/usr/lib/jvm/jre-1.7.0-openjdk
+
+pushd /project
+./gradlew tasks
+popd
+
+# Package everything up.
+pushd ${WORKSPACE}
+# Not yet.  See notes on tooltool below.
+# cp -R /root/.android-sdk android-sdk-linux
+# tar cJf android-sdk-linux.tar.xz android-sdk-linux
+
+cp -R /workspace/nexus/storage/central jcentral
+tar cJf jcentral.tar.xz jcentral
+
+# The Gradle wrapper will have downloaded and verified the hash of exactly one
+# Gradle distribution.  It will be located at
+# ~/.gradle/wrapper/dists/gradle-2.7-all/$PROJECT_HASH/gradle-2.7-all.zip.  We
+# want to remove the version from the internal directory for use via tooltool
+# in a mozconfig.
+cp ~/.gradle/wrapper/dists/gradle-${GRADLE_VERSION}-all/*/gradle-${GRADLE_VERSION}-all.zip gradle-${GRADLE_VERSION}-all.zip
+unzip -q gradle-${GRADLE_VERSION}-all.zip
+mv gradle-${GRADLE_VERSION} gradle
+tar cJf gradle.tar.xz gradle
+
+mkdir -p /artifacts
+# We can't redistribute the Android SDK publicly just yet.  We'll
+# upload to (internal) tooltool eventually.  mv
+# android-sdk-linux.tar.xz /artifacts
+mv jcentral.tar.xz /artifacts
+mv gradle.tar.xz /artifacts
+popd
+
+# Bug 1245170: at some point in the future, we'll be able to upload
+# things directly to tooltool.
+# pushd /artifacts
+# /build/tooltool.py add --visibility=public jcentral.tar.xz
+# /build/tooltool.py add --visibility=public gradle.tar.xz
+# /build/tooltool.py add --visibility=internal android-sdk-linux.tar.xz
+# /build/tooltool.py upload -v --url=http://relengapi/tooltool/ \
+#   --message="No message - Gradle and jcentral archives uploaded from taskcluster."
+# popd
--- a/testing/docker/android-gradle-build/project/README.md
+++ b/testing/docker/android-gradle-build/project/README.md
@ -0,0 +1,6 @@
+This Gradle project exists only to list dependencies used to build Firefox for
+Android.  Eventually, this definition will live in ``mobile/android``, or it
+will just *be* ``mobile/android/app/build.gradle``.
+
+Until we determine the best Docker and Taskcluster patterns for dependency
+gathering and toolchain producing like this, this can live here.
--- a/testing/docker/android-gradle-build/project/build.gradle
+++ b/testing/docker/android-gradle-build/project/build.gradle
@ -0,0 +1,73 @@
+buildscript {
+  repositories {
+    maven {
+        url 'http://localhost:8081/nexus/content/repositories/central/'
+    }
+
+    maven {
+        url 'https://jitpack.io'
+    }
+  }
+
+  dependencies {
+    classpath 'com.android.tools.build:gradle:1.3.0'
+    classpath('com.stanfy.spoon:spoon-gradle-plugin:1.0.4') {
+        // Without these, we get errors linting.
+        exclude module: 'guava'
+    }
+
+    // See
+    // https://github.com/JakeWharton/sdk-manager-plugin/issues/73#issuecomment-106747867
+    // and
+    // https://github.com/JakeWharton/sdk-manager-plugin/issues/99#issuecomment-166438751.
+    classpath 'com.github.JakeWharton:sdk-manager-plugin:220bf7a88a7072df3ed16dc8466fb144f2817070'
+  }
+}
+
+allprojects {
+    repositories {
+        maven {
+            url 'http://localhost:8081/nexus/content/repositories/central/'
+        }
+    }
+}
+
+apply plugin: 'android-sdk-manager'
+
+// Optionally, require an emulator.
+// sdkManager {
+//   emulatorVersion 'android-19'
+//   emulatorArchitecture 'armeabi-v7a' // Optional, defaults to ARM.
+// }
+
+apply plugin: 'com.android.application'
+
+android {
+    compileSdkVersion 23
+    buildToolsVersion "23.0.1"
+
+    defaultConfig {
+        targetSdkVersion 23
+        minSdkVersion 15
+    }
+}
+
+// These are extracted from mobile/android/**/*.gradle.
+dependencies {
+    compile 'com.android.support:appcompat-v7:23.0.1'
+    compile 'com.android.support:design:23.0.1'
+    compile 'com.android.support:mediarouter-v7:23.0.1'
+    compile 'com.android.support:recyclerview-v7:23.0.1'
+    compile 'com.android.support:support-v4:23.0.1'
+    compile 'com.google.android.gms:play-services-base:8.1.0'
+    compile 'com.google.android.gms:play-services-basement:8.1.0'
+    compile 'com.google.android.gms:play-services-cast:8.1.0'
+    compile 'com.google.android.gms:play-services-gcm:8.1.0'
+    compile 'com.squareup.leakcanary:leakcanary-android-no-op:1.4-beta1'
+    compile 'com.squareup.leakcanary:leakcanary-android:1.4-beta1'
+    testCompile 'junit:junit:4.12'
+    testCompile 'org.mockito:mockito-core:1.10.19'
+    testCompile 'org.robolectric:robolectric:3.0'
+    testCompile 'org.simpleframework:simple-http:6.0.1'
+    androidTestCompile 'com.jayway.android.robotium:robotium-solo:4.3.1'
+}
--- a/testing/docker/android-gradle-build/project/gradle/wrapper/gradle-wrapper.jar
+++ b/testing/docker/android-gradle-build/project/gradle/wrapper/gradle-wrapper.jar
--- a/testing/docker/android-gradle-build/project/gradle/wrapper/gradle-wrapper.properties
+++ b/testing/docker/android-gradle-build/project/gradle/wrapper/gradle-wrapper.properties
@ -0,0 +1,7 @@
+#Tue Jan 26 20:42:06 PST 2016
+distributionBase=GRADLE_USER_HOME
+distributionPath=wrapper/dists
+zipStoreBase=GRADLE_USER_HOME
+zipStorePath=wrapper/dists
+distributionUrl=https\://services.gradle.org/distributions/gradle-2.7-all.zip
+distributionSha256Sum=2ba0aaa11a3e96ec0af31d532d808e1f09cc6dcad0954e637902a1ab544b9e60
--- a/testing/docker/android-gradle-build/project/gradlew
+++ b/testing/docker/android-gradle-build/project/gradlew
@ -0,0 +1,160 @@
+#!/usr/bin/env bash
+
+##############################################################################
+##
+##  Gradle start up script for UN*X
+##
+##############################################################################
+
+# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
+DEFAULT_JVM_OPTS=""
+
+APP_NAME="Gradle"
+APP_BASE_NAME=`basename "$0"`
+
+# Use the maximum available, or set MAX_FD != -1 to use that value.
+MAX_FD="maximum"
+
+warn ( ) {
+    echo "$*"
+}
+
+die ( ) {
+    echo
+    echo "$*"
+    echo
+    exit 1
+}
+
+# OS specific support (must be 'true' or 'false').
+cygwin=false
+msys=false
+darwin=false
+case "`uname`" in
+  CYGWIN* )
+    cygwin=true
+    ;;
+  Darwin* )
+    darwin=true
+    ;;
+  MINGW* )
+    msys=true
+    ;;
+esac
+
+# Attempt to set APP_HOME
+# Resolve links: $0 may be a link
+PRG="$0"
+# Need this for relative symlinks.
+while [ -h "$PRG" ] ; do
+    ls=`ls -ld "$PRG"`
+    link=`expr "$ls" : '.*-> \(.*\)$'`
+    if expr "$link" : '/.*' > /dev/null; then
+        PRG="$link"
+    else
+        PRG=`dirname "$PRG"`"/$link"
+    fi
+done
+SAVED="`pwd`"
+cd "`dirname \"$PRG\"`/" >/dev/null
+APP_HOME="`pwd -P`"
+cd "$SAVED" >/dev/null
+
+CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
+
+# Determine the Java command to use to start the JVM.
+if [ -n "$JAVA_HOME" ] ; then
+    if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
+        # IBM's JDK on AIX uses strange locations for the executables
+        JAVACMD="$JAVA_HOME/jre/sh/java"
+    else
+        JAVACMD="$JAVA_HOME/bin/java"
+    fi
+    if [ ! -x "$JAVACMD" ] ; then
+        die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
+
+Please set the JAVA_HOME variable in your environment to match the
+location of your Java installation."
+    fi
+else
+    JAVACMD="java"
+    which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
+
+Please set the JAVA_HOME variable in your environment to match the
+location of your Java installation."
+fi
+
+# Increase the maximum file descriptors if we can.
+if [ "$cygwin" = "false" -a "$darwin" = "false" ] ; then
+    MAX_FD_LIMIT=`ulimit -H -n`
+    if [ $? -eq 0 ] ; then
+        if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then
+            MAX_FD="$MAX_FD_LIMIT"
+        fi
+        ulimit -n $MAX_FD
+        if [ $? -ne 0 ] ; then
+            warn "Could not set maximum file descriptor limit: $MAX_FD"
+        fi
+    else
+        warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT"
+    fi
+fi
+
+# For Darwin, add options to specify how the application appears in the dock
+if $darwin; then
+    GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\""
+fi
+
+# For Cygwin, switch paths to Windows format before running java
+if $cygwin ; then
+    APP_HOME=`cygpath --path --mixed "$APP_HOME"`
+    CLASSPATH=`cygpath --path --mixed "$CLASSPATH"`
+    JAVACMD=`cygpath --unix "$JAVACMD"`
+
+    # We build the pattern for arguments to be converted via cygpath
+    ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null`
+    SEP=""
+    for dir in $ROOTDIRSRAW ; do
+        ROOTDIRS="$ROOTDIRS$SEP$dir"
+        SEP="|"
+    done
+    OURCYGPATTERN="(^($ROOTDIRS))"
+    # Add a user-defined pattern to the cygpath arguments
+    if [ "$GRADLE_CYGPATTERN" != "" ] ; then
+        OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)"
+    fi
+    # Now convert the arguments - kludge to limit ourselves to /bin/sh
+    i=0
+    for arg in "$@" ; do
+        CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -`
+        CHECK2=`echo "$arg"|egrep -c "^-"`                                 ### Determine if an option
+
+        if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then                    ### Added a condition
+            eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"`
+        else
+            eval `echo args$i`="\"$arg\""
+        fi
+        i=$((i+1))
+    done
+    case $i in
+        (0) set -- ;;
+        (1) set -- "$args0" ;;
+        (2) set -- "$args0" "$args1" ;;
+        (3) set -- "$args0" "$args1" "$args2" ;;
+        (4) set -- "$args0" "$args1" "$args2" "$args3" ;;
+        (5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;;
+        (6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;;
+        (7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;;
+        (8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;;
+        (9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;;
+    esac
+fi
+
+# Split up the JVM_OPTS And GRADLE_OPTS values into an array, following the shell quoting and substitution rules
+function splitJvmOpts() {
+    JVM_OPTS=("$@")
+}
+eval splitJvmOpts $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS
+JVM_OPTS[${#JVM_OPTS[*]}]="-Dorg.gradle.appname=$APP_BASE_NAME"
+
+exec "$JAVACMD" "${JVM_OPTS[@]}" -classpath "$CLASSPATH" org.gradle.wrapper.GradleWrapperMain "$@"
--- a/testing/docker/android-gradle-build/project/gradlew.bat
+++ b/testing/docker/android-gradle-build/project/gradlew.bat
@ -0,0 +1,90 @@
+@if "%DEBUG%" == "" @echo off
+@rem ##########################################################################
+@rem
+@rem  Gradle startup script for Windows
+@rem
+@rem ##########################################################################
+
+@rem Set local scope for the variables with windows NT shell
+if "%OS%"=="Windows_NT" setlocal
+
+@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
+set DEFAULT_JVM_OPTS=
+
+set DIRNAME=%~dp0
+if "%DIRNAME%" == "" set DIRNAME=.
+set APP_BASE_NAME=%~n0
+set APP_HOME=%DIRNAME%
+
+@rem Find java.exe
+if defined JAVA_HOME goto findJavaFromJavaHome
+
+set JAVA_EXE=java.exe
+%JAVA_EXE% -version >NUL 2>&1
+if "%ERRORLEVEL%" == "0" goto init
+
+echo.
+echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
+echo.
+echo Please set the JAVA_HOME variable in your environment to match the
+echo location of your Java installation.
+
+goto fail
+
+:findJavaFromJavaHome
+set JAVA_HOME=%JAVA_HOME:"=%
+set JAVA_EXE=%JAVA_HOME%/bin/java.exe
+
+if exist "%JAVA_EXE%" goto init
+
+echo.
+echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME%
+echo.
+echo Please set the JAVA_HOME variable in your environment to match the
+echo location of your Java installation.
+
+goto fail
+
+:init
+@rem Get command-line arguments, handling Windowz variants
+
+if not "%OS%" == "Windows_NT" goto win9xME_args
+if "%@eval[2+2]" == "4" goto 4NT_args
+
+:win9xME_args
+@rem Slurp the command line arguments.
+set CMD_LINE_ARGS=
+set _SKIP=2
+
+:win9xME_args_slurp
+if "x%~1" == "x" goto execute
+
+set CMD_LINE_ARGS=%*
+goto execute
+
+:4NT_args
+@rem Get arguments from the 4NT Shell from JP Software
+set CMD_LINE_ARGS=%$
+
+:execute
+@rem Setup the command line
+
+set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar
+
+@rem Execute Gradle
+"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %CMD_LINE_ARGS%
+
+:end
+@rem End local scope for the variables with windows NT shell
+if "%ERRORLEVEL%"=="0" goto mainEnd
+
+:fail
+rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
+rem the _cmd.exe /c_ return code!
+if  not "" == "%GRADLE_EXIT_CONSOLE%" exit 1
+exit /b 1
+
+:mainEnd
+if "%OS%"=="Windows_NT" endlocal
+
+:omega
--- a/testing/docker/android-gradle-build/project/src/main/AndroidManifest.xml
+++ b/testing/docker/android-gradle-build/project/src/main/AndroidManifest.xml
@ -0,0 +1,6 @@
+<manifest xmlns:android="http://schemas.android.com/apk/res/android"
+    package="org.mozilla.gecko">
+<!-- THIS IS NOT THE REAL MANIFEST!  This is for Gradle only.  See
+     AndroidManifest.xml.in. -->
+
+</manifest>