jitutils/doc/cijobs.md

CI jobs information

The .NET team maintains a "continuous integration" (CI) system for testing .NET, using Jenkins. In particular, each source code change submitted for consideration is tested. The "cijobs" command-line tool allows for querying the CI for per-job information, and for downloading archived per-job artifacts. These artifacts can be used for generating assembly code output, instead of building your own baseline JIT, for example.

cijobs

cijobs has two commands: (1) list, and (2) copy.

cijobs help message:

    $ cijobs --help
    usage: cijobs <command> [<args>]

        list    List jobs on ci.dot.net for the repo.
        copy    Copies job artifacts from ci.dot.net. This
                command copies a zip of artifacts from a repo (defaulted to
                dotnet_coreclr). The default location of the zips is the
                Product sub-directory, though that can be changed using the
                ContentPath(p) parameter

The "cijobs list" command has the following help message:

    $ cijobs list --help
    usage: cijobs list [-j <arg>] [-b <arg>] [-r <arg>] [-m <arg>]
                  [-n <arg>] [-l] [-c <arg>] [-a]

        -s, --server <arg>       Url of the server. Defaults to
                                 https://ci.dot.net/
        -j, --job <arg>          Name of the job.
        -b, --branch <arg>       Name of the branch (default is master).
        -r, --repo <arg>         Name of the repo (e.g. dotnet_corefx or
                                 dotnet_coreclr). Default is dotnet_coreclr
        -m, --match <arg>        Regex pattern used to select jobs output.
        -n, --number <arg>       Job number.
        -l, --last_successful    List last successful build.
        -c, --commit <arg>       List build at this commit.
        -a, --artifacts          List job artifacts on server.

The "cijobs copy" command has the following help message:

    usage: cijobs copy [-j <arg>] [-n <arg>] [-l] [-c <arg>] [-b <arg>]
                  [-r <arg>] [-o <arg>] [-u] [-p <arg>]

        -s, --server <arg>         Url of the server. Defaults to
                                   https://ci.dot.net/
        -j, --job <arg>            Name of the job.
        -n, --number <arg>         Job number.
        -l, --last_successful      Copy last successful build.
        -c, --commit <arg>         Copy this commit.
        -b, --branch <arg>         Name of the branch (default is master).
        -r, --repo <arg>           Name of the repo (e.g. dotnet_corefx or
                                   dotnet_coreclr). Default is
                                   dotnet_coreclr
        -o, --output <arg>         Output path.
        -u, --unzip                Unzip copied artifacts
        -p, --ContentPath <arg>    Relative product zip path. Default is
                                   artifact/bin/Product/*zip*/Product.zip

Using cijobs

A common question for a developer might be "what is the last successful OSX checked build?"

$ cijobs list --match "osx"

job checked_osx
job checked_osx_flow
job checked_osx_flow_prtest
job checked_osx_prtest
job checked_osx_tst
job checked_osx_tst_prtest
job debug_osx
job debug_osx_flow
job debug_osx_flow_prtest
job debug_osx_prtest
job debug_osx_tst
job debug_osx_tst_prtest
job release_osx
job release_osx_flow
...

The previous example shows searching for job names that match "osx". The checked_osx jobs is the one the developer wants. (Some familiarity with the jobs running on the server is helpful. Visit https://ci.dot.net to familarize yourself with what's available.)

Further querying the checked_osx job for the last successful build can be done with this command line.

$ cijobs list --job checked_osx --last_successful

Last successful build:
build 1609 - SUCCESS : commit 74798b5b95aca1b27050038202034448a523c9f9

With this in hand two things can be accomplished. First, new development for a feature could be started based on the commit hash returned. Second, the tools generated by this job can be downloaded for use locally.

$ cijobs copy --job checked_osx --last_successful --output ../output/mytools --unzip

Downloading: job/dotnet_coreclr/job/master/job/checked_osx/1609/artifact/bin/Product/*zip*/Product.zip

Results are unzipped in the output after they are downloaded.

$ ls ../output/mytools/
Product		Product.zip

One comment on the underlying Jenkins feature. The artifacts are kept and managed by a Jenkins plug-in used by the system. This plug-in will ZIP compress on demand at any point in the defined artifacts output tree. Today we only use the Product sub-directory but this could be extended in the future.