* Add prototype SELinux auto configure
* Add 'force_enforcing' option for SELinux
* Fix setools-console tools.
* Enable SELinux by default (permissive mode) on all images.
Drop build system unit test as it breaks with SELinux enabled on core-efi.
* selinux-policy: Update to 2.20210908.
* Update to 2.20220106.
Implement policy for systemd-homed and systemd-userdbd.
* Fix RPM changelog date.
* Finalize systemd-homed policy.
* Change SELinux enablement to not affect CONFIG_LSM.
* Document build settings
* Update cgmanifest
* Update toolkit/docs/formats/imageconfig.md
Co-authored-by: Christopher Co <35273088+christopherco@users.noreply.github.com>
* audit: Remove override so auditd starts by default.
* Add IsValid() call for SELinux inKkernelCommandLine
* Add unit test for missing selinux package
* Fix debug output for selinux setfiles
Co-authored-by: Daniel McIlvaney <damcilva@microsoft.com>
Co-authored-by: Christopher Co <35273088+christopherco@users.noreply.github.com>
* Update tools to build packages with dependencies hydrated
Hydrated build is a new approach where all the dependencies are hydrated from pre-built RPMs. This provides two advantages.
1. Faster build. As no package need to wait for its dependency's build to complete, all packages will be fired to build in parallel. (Practically with slight delay as firing of pre-built nodes takes place before).
2. Failure of any dependency build will not cause failure to the current package, as the dependency is fetched from cached RPMs.
Introduced a new flag HYDRATED_BUILD to enable this feature. Also it is pre-requisite to run `make hydrate-toolchain` and `make hydrate-rpms` with appropriate tar files before using this option.
This is achieved by replacing the BUILD to RUN node edge with BUILD to PREBUILT node edge (clone of the RUN node) nodes (if there is a corresponding RPM hydrated already).
* Fix go fmt check failures
* Fix typos
* Introduce graphscrubber
graphscrubber is for preprocessing the build graph to scheduler.
Currently it is only doing hydrated build related work. Ideally all the
graph optimization related jobs to be moved from scheduler to
graphscrubber.
* go fmt tidy
* Remove unnecessary functions from graphscrubber
* Fix Make rule to build graphscrubber before using it
* Use inline hydrated-build flag
* Update documentaion for HYDRATED_BUILD flag
* Update changes recommended in PR discussion
* go tidy
* Remove unnecessary arguments from graphPreprocessor
* Update toolkit/tools/graphPreprocessor/graphPreprocessor.go
Co-authored-by: Pawel Winogrodzki <pawelwi@microsoft.com>
* Update toolkit/tools/graphPreprocessor/graphPreprocessor.go
Co-authored-by: Pawel Winogrodzki <pawelwi@microsoft.com>
* Update toolkit/tools/graphPreprocessor/graphPreprocessor.go
Co-authored-by: Pawel Winogrodzki <pawelwi@microsoft.com>
* retrugger checks
* Update toolkit/scripts/pkggen.mk
Co-authored-by: Daniel McIlvaney <damcilva@microsoft.com>
* Update toolkit/scripts/pkggen.mk
Co-authored-by: Daniel McIlvaney <damcilva@microsoft.com>
* Update toolkit/scripts/pkggen.mk
Co-authored-by: Daniel McIlvaney <damcilva@microsoft.com>
* Update toolkit/scripts/pkggen.mk
Co-authored-by: Daniel McIlvaney <damcilva@microsoft.com>
* retrugger checks
* retrugger checks
Co-authored-by: Pawel Winogrodzki <pawelwi@microsoft.com>
Co-authored-by: Daniel McIlvaney <damcilva@microsoft.com>
* diskutils: Add partprobe after partition creation
There can be a timing issue where partition creation finishes but the
devtmpfs files are not populated in time for partition initialization.
So to deal with this, we call partprobe here to query and flush the
partition table information, which should enforce that the devtmpfs
files are created when partprobe returns control.
* diskutils: invoke partprobe with flock
Added flock because "partprobe -s" apparently doesn't always block.
flock is part of the util-linux package and helps to synchronize access
with other cooperating processes. The important part is it will block
if the fd is busy, and then execute the command. Adding a 5 second timeout
to prevent us from possibly waiting forever.
* diskutils: Update timeout variable name
* diskutils: clarify debug log message
* Add parted to documentation and quickstart runner
Signed-off-by: Chris Co <chrco@microsoft.com>
* Overlay based diff image creation prototype
Here is a link to the spec https://microsoft-my.sharepoint.com/:w:/g/personal/arvindka_microsoft_com1/ESrYHTpWUPBOgdi7LjDsE14Bf1mHSLG702551XctkFX1mA?e=CyCc2j. This is for early feedback on the approach.
It introduces a new element, BaseImage for each partition. Instead of creating a complete new partition image, a new diff layer is created using overlay file system. Overlay file system is a simple implementation of union file system. The changes files are completely copied in the upper level overlay. The implementation then copies the higher level files in a tgz.
This tgz can be transferred to the ADU agent which first rehydrates the base image and then uses SWUpdate to do the A/B switch.
Co-authored-by: Arvind Kandhare <arvindka@microsoft.com>
While the ISO initrd image does have a kernel, most rootfs images will
not have one. We need to support both flows.
Make KernelOptions an optional key for rootfs images, and only include
the kernel if it is set.
During independent build testing, it was found that three additional prerequisites were necessary to build the toolchain from scratch. This change corrects the documentation to add the missing prereqs.
* Change the password expire days to an int to allow for -1 to be passed
The -1 argument allows for user's passsword to never expire
Also added users.go and tests for invalid settings
Add documentation for the User field under SystemConfigs
* Update trademark section of the readme
Signed-off-by: Jim Perrin <Jim.Perrin@microsoft.com>
* Update building.md (#104)
* add wants=sshd-keygen.service to sshd (#58)
* add wants=sshd-keygen.service to sshd
Signed-off-by: Jim Perrin <Jim.Perrin@microsoft.com>
* modify signatures.json and bump release for pr
Signed-off-by: Jim Perrin <Jim.Perrin@microsoft.com>
* Fix libffi normal package build (#116)
* Fix libffi normal package build
* Add comment explaining the purpose of the sed call
* Upgrade golang to 1.13.15 (#93)
* Adding a small build tip to the quick start instructions. (#123)
* Add cloud-init-vmware-guestinfo package (#124)
* Add cloud-init-vmware-guestinfo package
* Updating 'ca-certificates' nssckbi.h header and unifying changelog entries with package version (#125)
* Updating changelog to be consistent with package version.
* Fixing missed update to 'nssckbi.h'.
* Updating manifests.
* Updating signatures.
* Markdown lint-induced clean-up of doc files. (#122)
* Makrdownlint-induced clean-up.
* Removing redundant lines.
* Removing redundant lines 2.
* Add IMA feature to the kernel, add config for it (#135)
* Add IMA feature to the kernel, add config for it
- Add IMA measurement configs to the x86_64, and aarch64 kernel configs (IMA_APPRAISE currently disabled).
- Add KernelCommandLine config field to control IMA, and allow additional configs to be passed.
Signed-off-by: Daniel McIlvaney <damcilva@microsoft.com>
Co-authored-by: Christopher Co <christopher.co@microsoft.com>
* Update tpm2 tools to 4.2, tss to 2.4.0 (#134)
Signed-off-by: Daniel McIlvaney <damcilva@microsoft.com>
* Enable Mellanox kernel configs
* Update tpm2-abrmd to 2.3.3 (#144)
* Update tpm2-abrmd to 2.3.3
* Create quickstart.yml (#119)
This patch adds a GitHub Action to verify our Quickstart instructions
* Nopatch httpd CVE-1999-0236, CVE-1999-1412 (#148)
* Nopatch httpd CVE-1999-0236, CVE-1999-1412
Signed-off-by: Daniel McIlvaney <damcilva@microsoft.com>
* Nopatch groff CVE-2000-0803 (#149)
* Nopatch groff CVE-2000-0803
Signed-off-by: Daniel McIlvaney <damcilva@microsoft.com>
* Nopatch apparmor CVE-2016-1585 (#150)
* Nopatch apparmor CVE-2016-1585
Signed-off-by: Daniel McIlvaney <damcilva@microsoft.com>
* Nopatch qemu CVE-2016-7161 (#152)
* Nopatch qemu CVE-2016-7161
Signed-off-by: Daniel McIlvaney <damcilva@microsoft.com>
* Nopatch lua CVE-2020-15889 (#153)
* nopatch lua CVE-2020-15889
Signed-off-by: Daniel McIlvaney <damcilva@microsoft.com>
* Nopatch unzip CVE-2008-0888 (#154)
Signed-off-by: Daniel McIlvaney <damcilva@microsoft.com>
* full: Always install the default kernel (#132)
Currently, when installing CBL-Mariner via ISO, the ISO will
install the standard kernel package or the kernel-hyperv package
depending on if installing on HyperV VM or not.
The HyperV kernel is still under evaluation so use the standard kernel
package across the board.
* Support downloading preview SRPMs (#160)
Replace SRPM_URL* with SRPM_URL_LIST
* Patch CVE-2020-14342 in cifs-utils
* Replace mariner-repos's %post script as %posttrans
- After looking at here, it shows that %post script for a new version runs before the %preun script for an old version. Which means, after an upgrade, the keys would be removed by the older version: https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/#ordering
* Update pkggen_core_aarch64.txt
* Update pkggen_core_x86_64.txt
* Update toolchain_aarch64.txt
* Update toolchain_x86_64.txt
* Add a more verbose changelog
* Remove chrony-wait as a boot service dependency (#166)
* Remove chrony-wait as a boot service dependency
* Add cgmanifest entry for chrony
* Address changelog and prep section comments
* initramfs: Regenerate initrd using host-only mode on file-based trigger (#170)
* initramfs: Always use host-only mode
kdump currently uses the host system's initrd when enrolling a crash kernel
and initrd. There is a limitation where the kdump initrd must be generated
with dracut in "host-only" mode.
The -k option forces a host-only initrd build.
The -q option suppresses verbose output
If mkinitrd is called without <image> and <kernel-version> parameters, it will
default to calling dracut in "host-mode" mode on every kernel version it can
find in /boot.
If mkinitrd is called with <image> and <kernel-version> parameters, it will
default to calling dracut in "generic host" mode for rebuilding the specific
initrd. Therefore we need to make sure to add the -k option when invoking
mkinitrd with an explicit <image> and <kernel version>
* Reword comment block
* Fix kernel specs' %postun scripts (#164)
* Fix `kernel.spec`'s `%postun` script
* Fix `kernel-signed-aarch64`'s `%postun` script
* Fix kernel-signed-x64.spec's %postun script
* Fix kernel-hyperv.spec's %postun script
* Adding new 'preview' repository. (#146)
* Adding new 'preview' repository.
* Addressing comments.
* Fix kernel aarch64 package build break due to missing CONFIG_IMA_KEXEC (#171)
* Fix kernel aarch64 package build break due to missing CONFIG_IMA_KEXEC (#171)
* Update fontconfig to 2.13.91 (#175)
* Extending 'strongswan' test timeout. (#173)
* Fix CVE-2020-14342 patch to not depend on PATH
* installutils: Supply blank /etc/machine-id file (#147)
From https://www.freedesktop.org/software/systemd/man/machine-id.html:
For operating system images which are created once and used on multiple
machines, for example for containers or in the cloud, /etc/machine-id
should be an empty file in the generic file system image. An ID will be
generated during boot and saved to this file if possible.
* installutils: Remove root password expiry when no root user is specified in imageconfig file (#161)
* Add SELinux packages to Mariner. (#100)
* Add SELinux packages to Mariner.
This commit add the following packages to Mariner to provide basic
SELinux support:
- checkpolicy
- libsemanage
- mcstrans
- policycoreutils
- secilc
- selinux-policy
- setools
The selinux-policy provided here is a generic base policy, which is not
specifically tuned for Mariner, therefore only permissive mode support
is enabled in this commit. (Although users could load a custom policy
to run in enforcing mode). Future phases have been discussed to add
SELinux enforcing mode support.
This commit does not enable SELinux by default. In order to enable
SELinux support, one must first install necessary packages (libselinux,
policycoreutils, secilc, selinux-policy), and then append "lsm=selinux
selinux=1" to the kernel command line. This will trigger an initial
boot to relabel the system, at which point the system will reboot, and
boot into an SELinux enabled system. SELinux state can be queried with
the "getenforce" command line tool. If SELinux has not been enabled, it
will report "Disabled" (the default). If SELinux support has been
enabled as described in this paragraph, it will report "permissive".
This commit also modifies the following packages to enabled SELinux
functionality in existing packages:
- coreutils
- cronie
- dbus
- openssh
- pam
- rpm
- shadow-utils
- systemd
- util-linux
This enables them to build with SELinux support so that when SELinux is
enabled, they have SELinux related functionality available.
Because coreutils is a basic package and requires building with
libselinux-devel present in order to enable key SELinux functionality,
several dependencies in other packages that rely on coreutils (namely
python2, python3 and systemd-bootstrap) had to be removed in order to
avoid circular dependencies. There does not appear to be a functional
impact from this change based on my testing.
* Remove "::set-env" commands in GitHub Actions (#178)
* Adding a .nopatch for CVE-2007-0086. (#176)
* Updating cert bundle paths. (#181)
* Updating cert bundle paths.
* Updating cgmanifest.json.
* Adding the `gflags` and `rocksdb` packages. (#183)
* Adding the 'rocksdb' package.
* Adding the 'gflags' package.
* Add missing %libsepolver definition in secilc.spec (#192)
* Removing 'TERMINAL_ISO_INSTALLER' from the docs. (#189)
* Add architecture at the end of toolkit archive (#182)
- Also add `version.txt` file in the toolkit archive as an easy way to verify toolkit version.
* Adding a missing '%{?dist}' tag. (#195)
* enable fetching RPMs from pacakges.microsoft.com for Docker based build (#198)
* Update README.md (#180)
* Update README.md (#180)
* Build Break Fix: Rollback selinux checkins. (#204)
* Revert "Add missing %libsepolver definition in secilc.spec (#192)"
This reverts commit 9cff088bec.
* Revert "Add SELinux packages to Mariner. (#100)"
This reverts commit b2d918efac.
* Natively support pulling from the preview repo (#199)
* Fix CVE-2020-26159 in oniguruma (#211)
* Fix CVE-2020-26159
* Increment release, fix autosetup.
* Adding the 'syslog-ng' package. (#205)
* Adding the 'tinyxml2' package. (#206)
* Adding the 'toml11' package. (#207)
* Adding the 'tracelogging' and 'zipper' packages. (#208)
* Add mm-common and libxml++ packages (#215)
* Add liblogging package (#214)
* Add nlohmann-json package (#217)
* Add msgpack package (#216)
* Adding the 'span-lite' and 'telegraf' packages. (#220)
* Remove toolchain-local-wget-list after use (#212)
* Remove toolchain-local-wget-list after use
- toolchain-local-wget-list has been left at the end of a toolchain build. It shows up on `git status` whene toolchain is built locally.
- Another solution would be adding it to `.gitignore`.
* Add temporary toolchain build files to toolkit/.gitignore
* Remove implicit git repository dependency from toolkit (#197)
* Remove implicit git repository dependency
* Remove the new GIT_REV variable
* Add jsonbuilder package (#223)
* update libffi to use https source0 (#227)
* Update libestr (#213)
* Add babeltrace2 and lttng-consume packages (#226)
* Add pugixml package (#222)
* Disable debug package for nlohmann-json (#228)
* Add rapidjson package (#225)
* Upgrade ruby to 2.6.6 to resolve CVE-2019-16255, CVE-2019-16201, CVE-2020-10933, CVE-2020-5247, CVE-2019-15845, CVE-2019-16254 (#224)
* Upgrade ruby to 2.6.6 to resolve CVEs
* Update cgmanifest
* Nopatch qemu CVE-2015-7504 CVE-2017-5931 CVE-2017-14167 (#162)
* Fix CVE-2020-26159 in oniguruma (#211)
* Fix CVE-2020-26159
* Increment release, fix autosetup.
* Enable QAT kernel configs in CBL-Mariner
* Nopatch kernel CVE-2020-10757, CVE-2020-12653, CVE-2020-12657, CVE-2010-3865, CVE-2020-11668, CVE-2020-12654, CVE-2020-24394, CVE-2020-8428 (#193)
* Address CVE-2020-10757, CVE-2020-12653, CVE-2020-12657, CVE-2010-3865, CVE-2020-11668, CVE-2020-12654, CVE-2020-24394, CVE-2020-8428
* Adding the `bond`, `fluent-bit`, and `ivykis` packages. (#234)
* Joslobo/add azure storage (#232)
* Add azure-storage spec file to mariner-core
* Register with legal and update map file
* Fixed #source0 link
* Updated per code review comments
* Fixed URL to use https
* Initial spec lint action commit (#172) (#191)
* Initial spec-cleaner commit for CBL-Mariner
* Add cgmanifest.json file for GitHub workflows folder
* Set continue-on-error to true for a trial period
* patch openssh (#238)
* Update pull_request_template.md (#236)
* Fix check tests for git, make, krb5 and libcap-ng (#241)
* fix check tests
* update toolchain manifests
* fix blank spaces and tabs in make.spec
* Fix CVE-2019-12735 in vim (#230)
* Fix CVE-2019-12735 in vim
* Update the changelog to address only one CVE.
* Switching to correct source for the Microsoft bundle. (#244)
* Fix check tests for brotli, gzip and python-certifi (#245)
* fix check test for brotli, gzip, python-cerifi
* update manifest release version for gzip
* skip check for vim
* Patch unbound CVE-2020-12662 and CVE-2020-12663 (#246)
* Portablectl patches for to support --now --enable and --no-block flags (#139)
* Portablectl patches for to support --now --enable and --no-block flags
* Portablectl patches for to support --now --enable and --no-block flags
* Patch lua CVE-2019-6706, CVE-2020-15888, nopatch CVE-2020-24342 (#169)
* Patch lua CVE-2019-6706, CVE-2020-15888, CVE-2020-15945, nopatch CVE-2020-24342
Signed-off-by: Daniel McIlvaney <damcilva@microsoft.com>
* Roll back CVE-2020-15945, patch ineffective
Signed-off-by: Daniel McIlvaney <damcilva@microsoft.com>
* Nopatch ed CVE-2015-2987 (#209)
ed CVE-2015-2987 applies to a different program named ed.
* Patch gnutls CVE-2020-24659 (#247)
Upstream CVE discussion: https://gitlab.com/gnutls/gnutls/-/issues/1071
* update ant verision
* fix changelog comment
* update cgmanifest
* Nopatch sqlite CVE-2015-3717 (#254)
* Added omi package
* Adding the `ccache` and `clamav` packages. (#251)
* Generate ant signatures (#260)
* Add auoms package (#258)
* add auoms package
* add auoms original source url comments
* fix changelog history
* fix auoms signatures
* fix changelog
* use %license
* update licenses-map
* add omi to LICENSES-MAP
* merge latest LICENSES-MAP
* Implement "distroless" containers (#252)
* Create distroless container without bash and surplus dependencies
* Remove RPM database for distroless
* Add busybox and uclibc. Add distroless-packages-debug
* Update cgmanifest
Co-authored-by: Jon Slobodzian <joslobo@microsoft.com>
Co-authored-by: MateuszMalisz <mamalisz@microsoft.com>
* Updated mariner-release package version (#262)
* fix setup (#263)
* fix missed merge file
* Fixed bad file merge
* Fixed poorly merged files
* Merge distroless container revert to 1.0 (#265)
* Revert "Implement "distroless" containers (#252)"
This reverts commit e41efdda19.
* Revert "Implement "distroless" containers (#252)" (#264)
This reverts commit e41efdda19.
* fix package manifest merge issues
* fix issues building input-srpms
* fix package manifest issues
* remove duplicate patch and sed cmd from lua spec
* revert package ignore list and graphoptimizer changes
* remove runc from LICENSES-MAP.md
* Update pkggen merge (#316)
* Clean up lua.spec 1.0 to dev merge (#318)
* update lua.spec and licenses-map.md per feedback
* revert gzip changes
* revert krb5 change
Co-authored-by: Jim Perrin <Jim.Perrin@microsoft.com>
Co-authored-by: Jason Goscinski <jasongos@users.noreply.github.com>
Co-authored-by: Mateusz Malisz <maliszmat@outlook.com>
Co-authored-by: Nicolas Ontiveros <54044510+niontive@users.noreply.github.com>
Co-authored-by: Pawel Winogrodzki <pawelwi@microsoft.com>
Co-authored-by: Daniel McIlvaney <damcilva@microsoft.com>
Co-authored-by: Christopher Co <christopher.co@microsoft.com>
Co-authored-by: chalamalasetty <chalamalasetty@live.com>
Co-authored-by: chalamalasetty <42326515+chalamalasetty@users.noreply.github.com>
Co-authored-by: Joe Schmitt <1146681+schmittjoseph@users.noreply.github.com>
Co-authored-by: Henry Beberman <henry.beberman@microsoft.com>
Co-authored-by: Emre Girgin <50592283+mrgirgin@users.noreply.github.com>
Co-authored-by: Thomas Crain <thcrain@microsoft.com>
Co-authored-by: Jon Slobodzian <joslobo@microsoft.com>
Co-authored-by: Emre Girgin <mrgirgin@microsoft.com>
Co-authored-by: Daniel Burgener <burgener.daniel@gmail.com>
Co-authored-by: nicolas guibourge <nicogbg@gmail.com>
Co-authored-by: Chirag Shah <chsha@microsoft.com>
Co-authored-by: Henry Li <lihl@microsoft.com>
Co-authored-by: Henry Li <69694695+henryli001@users.noreply.github.com>
Co-authored-by: rychenf1 <rychenf1@gmail.com>
Co-authored-by: Nick Samson <nick.samson@microsoft.com>
Co-authored-by: MateuszMalisz <mamalisz@microsoft.com>
rlmenge
Pawel Winogrodzki
Chris PeBenito
Bala
Ali Yousefi Sabzevar
Daniel McIlvaney
Andrew Phelps
jslobodzian
Thomas Crain
Andrew Phelps
Christopher Co
Henry Beberman
Joe Schmitt
arvindkandhare
Mateusz Malisz