dnf is the default for current RPM-based distros now, so moving it
as first choice. We can probably remove the fallback to yum, but
leaving that for a follow-up.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
git ls-remote's <pattern> argument [1] is a glob [2], and matches anything
ending with the given string. This is problematic if multiple tags or
branches end with the given pattern. In containerd's case, this returns
both tags for the main module ("refs/tags/v1.7.19") and # the API module
("refs/tags/api/v1.7.19").
To prevent both of those being found, we check if the given reference starts
with a "v"; if it does, we can assume it's a tag, and prefix the pattern with
"refs/tags/" to make it less ambiguous.
We're using a case statement here to avoid introducing Bashisms.
[1]: https://git-scm.com/docs/git-ls-remote#Documentation/git-ls-remote.txt-ltpatternsgt82308203
[2]: https://git-scm.com/docs/gitglossary#Documentation/gitglossary.txt-glob
Before this patch:
make REF=v1.7.19 checkout
# ...
Initialized empty Git repository in /Users/thajeztah/go/src/github.com/docker/containerd-packaging/src/github.com/containerd/containerd/.git/
git -C src/github.com/containerd/containerd remote add origin "https://github.com/containerd/containerd.git"
./scripts/checkout.sh src/github.com/containerd/containerd "v1.7.19"
+ SRC=src/github.com/containerd/containerd
+ REF=v1.7.19
+ REF_FETCH=v1.7.19
++ git -C src/github.com/containerd/containerd ls-remote --refs --heads --tags origin v1.7.19
++ awk '{print $2}'
+ REF='refs/tags/api/v1.7.19
refs/tags/v1.7.19'
+ '[' -n 'refs/tags/api/v1.7.19
refs/tags/v1.7.19' ']'
+ REF_FETCH='refs/tags/api/v1.7.19
refs/tags/v1.7.19:refs/tags/api/v1.7.19
refs/tags/v1.7.19'
+ git -C src/github.com/containerd/containerd fetch --update-head-ok --depth 1 origin 'refs/tags/api/v1.7.19
refs/tags/v1.7.19:refs/tags/api/v1.7.19
refs/tags/v1.7.19'
fatal: invalid refspec 'refs/tags/api/v1.7.19
refs/tags/v1.7.19:refs/tags/api/v1.7.19
refs/tags/v1.7.19'
make: *** [checkout] Error 128
With this patch:
make REF=v1.7.19 checkout
# ...
Initialized empty Git repository in /Users/thajeztah/go/src/github.com/docker/containerd-packaging/src/github.com/containerd/containerd/.git/
git -C src/github.com/containerd/containerd remote add origin "https://github.com/containerd/containerd.git"
./scripts/checkout.sh src/github.com/containerd/containerd "v1.7.19"
+ SRC=src/github.com/containerd/containerd
+ REF=v1.7.19
+ REF_FETCH=v1.7.19
+ REF_GLOB=v1.7.19
+ case $REF_GLOB in
+ REF_GLOB=refs/tags/v1.7.19
++ git -C src/github.com/containerd/containerd ls-remote --refs --heads --tags origin refs/tags/v1.7.19
++ awk '{print $2}'
+ REF=refs/tags/v1.7.19
+ '[' -n refs/tags/v1.7.19 ']'
+ REF_FETCH=refs/tags/v1.7.19:refs/tags/v1.7.19
+ git -C src/github.com/containerd/containerd fetch --update-head-ok --depth 1 origin refs/tags/v1.7.19:refs/tags/v1.7.19
remote: Enumerating objects: 6397, done.
remote: Counting objects: 100% (6397/6397), done.
remote: Compressing objects: 100% (5114/5114), done.
Receiving objects: 100% (6397/6397), 10.09 MiB | 13.94 MiB/s, done.
remote: Total 6397 (delta 1376), reused 3349 (delta 816), pack-reused 0
Resolving deltas: 100% (1376/1376), done.
From https://github.com/containerd/containerd
* [new tag] v1.7.19 -> v1.7.19
+ git -C src/github.com/containerd/containerd checkout -q refs/tags/v1.7.19
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
containerd and runc (no longer) use the seccomp and apparmor build-tags, and runc
has removed the runc_nokmem build-tag (now the default).
seccomp is enabled by default for containerd and runc, but can be disabled on
runc by setting BUILDTAGS to an empty string;
https://github.com/opencontainers/runc/blob/v1.1.2/README.md#build-tags
Given that we always want to include seccomp (with non-static builds), this patch
removes the BUILDTAGS altogether for runc.
For containerd, we still need a buildtag to disable btrfs on CentOS 8 and up;
https://github.com/containerd/containerd/blob/v1.6.5/BUILDING.md#build-containerd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
While most of the dates generated were done from within a container (which defaults
to using UTC), some scripts could be run on the host, and therefore depend on the
host's configured timezone.
This patch makes sure that all calls to `date` use `TZ=UTC` to make sure the dates
are consistent.
Before this patch:
./scripts/new-deb-release 1.2.3 &> /dev/null && git diff -- debian/changelog | grep Sebastiaan
+ -- Sebastiaan van Stijn <github@gone.nl> Wed, 06 Apr 2022 12:22:09 +0200
With this patch:
./scripts/new-deb-release 1.2.3 &> /dev/null && git diff -- debian/changelog | grep Sebastiaan
+ -- Sebastiaan van Stijn <github@gone.nl> Wed, 06 Apr 2022 10:22:15 +0000
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
macOS uses the BSD flavor of `date`, which does not support the `--rfc-2822` option,
causing the script to fail and produce a changelog entry without a date.
The BSD flavor does have `-R` option, which is supported both on GNU and BSD,
which provides the equivalent. From the man page on macOS:
-R Use RFC 2822 date and time output format. This is equivalent to use
'%a, %d %b %Y %T %z' as output_fmt while LC_TIME is set to the
'C' locale .
Before this patch:
./scripts/new-deb-release 1.2.3
date: illegal option -- -
usage: date [-jnRu] [-d dst] [-r seconds] [-t west] [-v[+|-]val[ymwdHMS]] ...
[-f fmt date | [[[mm]dd]HH]MM[[cc]yy][.ss]] [+format]
...
containerd.io (1.2.3-1) release; urgency=medium
* TODO: Insert release changes here
-- Sebastiaan van Stijn <thajeztah@docker.com>
After this patch
./scripts/new-deb-release 1.2.3
containerd.io (1.2.3-1) release; urgency=medium
* TODO: Insert release changes here
-- Sebastiaan van Stijn <thajeztah@docker.com> Wed, 06 Apr 2022 12:06:17 +0200
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
While CI won't hit this situation, when running make validate
locally, the working directory may contain a src, build, or archive
directory, which is not part of the code in this repo, so does not
have to be validated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was originally used in e232259116 to download
the go binary, but starting with 6e4218ae9a, the
code switched to copy the go binary from a Docker Image, so there's no longer a
need to generate the download URL.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was introduced in e1b2c4a0e7, where
the version that was found is now stored in a variable (runc_ref) for
debugging.
However, I forgot to add an "echo" to actually print the version.
Because of that, the script debugs the version it found:
+ git -C src/github.com/containerd/containerd checkout -q refs/tags/v1.4.6
./scripts/checkout.sh src/github.com/opencontainers/runc "$(./scripts/determine-runc-version)"
INFO: detected runc version (v1.0.0-rc95) from script/setup/runc-version
But in the "build" target it shows that it didn't actually print it
(note the missing version missing between `runc :` and `(commit: ...)`):
--------------------------------------------------------------------
Building packages on docker.io/dockereng/rhel:7-s390x
containerd : v1.4.6 (commit: d71fcd7)
INFO: detected runc version (v1.0.0-rc95) from script/setup/runc-version
runc : (commit: e005fee)
architecture : s390x
build image : docker.io/dockereng/rhel:7-s390x
golang image : docker.io/library/golang:1.13.15-buster
--------------------------------------------------------------------
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The code that was used to automatically select the default version of
runc to use for containerd was still assuming the containerd code to
have a vendor.conf.
Containerd "master", and v1.5 have switched to go modules, so should
now look in the go.mod file to get the recommended / default version.
Also change the order in which we check out source-code, so that the
containerd source-code is checked out before we determine the runc
version; this allows us to always use the local source of containerd,
without having to use curl to get the version from GitHub.
Finally, a small modification was made for the "make validate" target,
which now uses a generic "alpine" Golang image, so that validation can
be done before the containerd source code was checked out (which was
used to detect the Go version to use).
Before this patch:
$ make docker.io/library/ubuntu:focal
...
curl: (22) The requested URL returned error: 404 Not Found
--------------------------------------------------------------------
Building packages on docker.io/library/ubuntu:focal
containerd : HEAD (commit: a72fe7d)
runc : (commit: 59ad417)
architecture : x86_64
build image : docker.io/library/ubuntu:focal
golang image : docker.io/library/golang:1.15.8-buster
With this patch:
$ make docker.io/library/ubuntu:focal
INFO: detected runc version from script/setup/runc-version
...
Building packages on docker.io/library/ubuntu:focal
containerd : HEAD (commit: a72fe7d)
runc : v1.0.0-rc93 (commit: 59ad417)
architecture : x86_64
build image : docker.io/library/ubuntu:focal
golang image : docker.io/library/golang:1.15.8-buster
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Debian "unstable" releases use apt caching information to get the codename
see discussion on https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=845651:
> That's all to say that this bug is (to my belief) actually expected behaviour;
> and fixing it through forcing the codename to be interpreted as "stretch" when
> apt-cache information is unavailable would be wrong. When /etc/debian_version
> contains "potato/sid", the codename is either potato xor sid, and only apt-
> cache can discriminate a testing host from a sid host. Therefore, in such a
> situation, the correct answer is actually "I can't tell", aka "n/a".
From testing, it reads the information from these files:
- /var/lib/apt/lists/deb.debian.org_debian_dists_bullseye_InRelease
- /var/lib/apt/lists/deb.debian.org_debian_dists_bullseye_main_binary-amd64_Packages.lz4
Removing these files (`rm -rf /var/lib/apt/lists/*`) causes 'lsb_release -sc`
to print 'n/a'. While we could use '/etc/debian_version' as a fallback for our
own scripts (stripping everything after '/' (e.g. bullseye/sid -> bullseye),
dpkg-buildpackage will still depend on this information to be present, and
if not present, renames packages to use 'n/a' in their path:
dpkg-buildpackage: info: full upload; Debian-native package (full source is included)
renamed '../containerd.io-dbgsym_0.20210219.014044~e58be59-1_amd64.deb' -> '/build/debian/n/a/amd64/containerd.io-dbgsym_0.20210219.014044~e58be59-1_amd64.deb'
renamed '../containerd.io_0.20210219.014044~e58be59-1_amd64.deb' -> '/build/debian/n/a/amd64/containerd.io_0.20210219.014044~e58be59-1_amd64.deb'
Given that we don't need the final image (as we only use it as a build environment
and copy the artifacts out), keeping some of the cache files should not be a problem.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 9706347c34 changed to shallow clones
but fetching a tag ref does not download the tag itself.
This patch makes sure that the tag (or branch) is downloaded otherwise
fallsback to fetching the commit hash as it did before.
Signed-off-by: Tibor Vass <tibor@docker.com>
This allows these scripts to be ran inside the Dockerfile
without having to use Bash as a shell.
Also using subshells because POSIX does not support
"local" variables.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Works around a difference in the macOS version of sed,
which caused this error;
sed: -i may not be used with stdin
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.11.13 (released 2019/08/13) includes security fixes to the net/http and net/url packages.
See the Go 1.11.13 milestone on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.13
- net/http: Denial of Service vulnerabilities in the HTTP/2 implementation
net/http and golang.org/x/net/http2 servers that accept direct connections from untrusted
clients could be remotely made to allocate an unlimited amount of memory, until the program
crashes. Servers will now close connections if the send queue accumulates too many control
messages.
The issues are CVE-2019-9512 and CVE-2019-9514, and Go issue golang.org/issue/33606.
Thanks to Jonathan Looney from Netflix for discovering and reporting these issues.
This is also fixed in version v0.0.0-20190813141303-74dc4d7220e7 of golang.org/x/net/http2.
net/url: parsing validation issue
- url.Parse would accept URLs with malformed hosts, such that the Host field could have arbitrary
suffixes that would appear in neither Hostname() nor Port(), allowing authorization bypasses
in certain applications. Note that URLs with invalid, not numeric ports will now return an error
from url.Parse.
The issue is CVE-2019-14809 and Go issue golang.org/issue/29098.
Thanks to Julian Hector and Nikolai Krein from Cure53, and Adi Cohen (adico.me) for discovering
and reporting this issue.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Dependencies needed a way to install in a more generic way since zypper
doesn't have the equivalent of yum-builddep
Also there needed to be an EXTRA_REPOS option so that we could include
repos for specific distributions like oraclelinux
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Changes the build process to be based on images instead of relying on
separate dockerfiles, simplifies maintenance to be just image based,
still need a good solution for windows though
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
go1.11.8 (released 2019/04/08) fixes an issue where using the prebuilt binary
releases on older versions of GNU/Linux led to failures when linking programs
that used cgo. Only Linux users who hit this issue need to update.
See https://github.com/golang/go/issues/31293 for details
Full diff: https://github.com/golang/go/compare/go1.11.7...go1.11.8
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This commit adds the necessary Makefile/Dockerfile changes to support
packaging on RHEL 8. Notably BTRFS is disabled for containerd builds on
this OS as the btrfs-progs package is not available during the beta. It
may also not be available at GA.
Signed-off-by: Dave Tucker <dt@docker.com>
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
New rpm version strings should look somewhat similar to
> build/centos-7/RPMS/x86_64/containerd.io-1.2.2-3.2.el7.x86_64.rpm
Note the 3.2 after the 1.2.2 versioning
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Allows a changeable spec file, and accounts for removing the RPM release
version when crafting a new one
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
This was a bug that if we wanted to do just a release of an rpm without
code changes to the underlying containerd we really couldn't.
This adds an extra number at the end of RPM_RELEASE so that it gets
numbered correctly upon release.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
Sebastiaan van Stijn
Sebastiaan van Stijn
Daniel Lipovetsky
Tibor Vass
Guillaume Lours
Derek McGowan
Eli Uriegas
zelahi
Dave Tucker
Jose Bigio