8.0 KiB
Testing the macOS Cookbook
Requirements
- Chef Workstation
- Vagrant
- Packer
- A supported macOS hypervisor:
Syntax and style
cookstyle
Syntax testing is pretty straight forward. At the root of the cookbook, run:
cookstyle
Unit tests
For unit tests, we focus on testing the library files, which are written
in pure Ruby and tested with RSpec. The library files contain most of the core
business logic for each of the custom resources and are used as either mixins
for the custom resources or contain classes that act as helpers, except with
explicit namespacing. Some libary files are heavily unit tested, others are definitely
missing much-needed unit test coverage. The following command syntax assumes
you've made the Chef Workstation rspec your default by running the appropriate
chef shell-init for your shell. (https://docs.chef.io/workstation/ctl_chef/#chef-shell-init)
Clone this repo and in the root of the cookbook, run:
rspec spec
To run the unit tests in a specific file:
rspec spec/unit/libraries/xcode_spec.rb
Integration tests
For integration tests, we test custom resources using a test cookbook, found in
test/cookbooks/macos_test.
In general, each of the custom resources is used in a corresponding test recipe,
which is then added to a corresponding suite's runlist. For example, the xcode
resource is used in the xcode.rb recipe, which is called in the xcode suite.
For the specific suite, there are corresponding integration tests as specified
in the suite. Every suite is tested against all three platform version.
Building a macOS Vagrant Box
Due to Apple's Software License Agreement, you'll need to build your own boxes. There's a number of different resources on GitHub that provide some really great guides, but we're partial to osx-vm-templates.
This procedure is a bit of a pain to really nail down. We've been working on refining and automating it as much as possible, but regular changes to the macOS operating system by Apple (e.g. signing restrictions introduced in 10.12.3) have made this challenging.
Read the osx-vm-templates README thouroughly to get a clear understanding of what needs to be done to turn a "vanilla" macOS installer into a shiny new, barely-touched macOS Vagrant base box. The process is pretty different depending on which version you're building, so tread lightly.
It should be noted that we also maintain a fork of osx-vm-templates that contains a revised README and better support for building Parallels Desktop Vagrant boxes. We're working on getting those changes implemented, but there is a few issues that need to be addressed before doing so.
Running the tests
Once you have finished building and "adding" your box (with vagrant box add),
you'll need to modify the .kitchen.yml. The only modifications you should
need to make are replacing our box names with yours. For example, you would
replace microsoft/macos-monterey with my_macos_box. To double check the
available boxes and their names, execute vagrant box list. For example:
$ vagrant box list
microsoft/macos-big-sur (parallels, 11.6.1)
microsoft/macos-catalina (parallels, 10.15.7)
microsoft/macos-high-sierra (parallels, 10.13.6)
microsoft/macos-mojave (parallels, 10.14.6)
microsoft/macos-monterey (parallels, 12.2)
Next, make sure you're in the macOS cookbook root and run kitchen list to view
the available instances. It should look something like this:
$ kitchen list
Instance Driver Provisioner Verifier Transport Last Action Last Error
default-mojave-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
default-catalina-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
default-big-sur-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
default-monterey-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
software-updates-mojave-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
software-updates-catalina-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
software-updates-big-sur-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
software-updates-monterey-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
spotlight-mojave-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
spotlight-catalina-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
spotlight-big-sur-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
spotlight-monterey-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
xcode-from-apple-mojave-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
xcode-from-apple-catalina-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
xcode-from-apple-big-sur-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
xcode-from-apple-monterey-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
xcode-from-url-mojave-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
xcode-from-url-catalina-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
xcode-from-url-big-sur-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
xcode-from-url-monterey-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
command-line-tools-mojave-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
command-line-tools-catalina-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
command-line-tools-big-sur-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
command-line-tools-monterey-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
certificate-mojave-chef17 Vagrant ChefInfra Inspec Ssh <Not Created> <None>
The kitchen list command serves as a nearly-perfect way to validate the
.kitchen.yml syntax. For more info, check out kitchen help for commands and
run kitchen help COMMAND for help on a specific subcommand. When you're ready,
run kitchen test.
kitchen test
kitchen supports using regular expressions to only run a specific instance.
For example:
kitchen test xcode # test the xcode suite on all versions
kitchen test default.*101[23] # only test default suites on 10.12 and 10.13
macOS takes a little while to boot and the suites themselves (especially Xcode)
can take a while to run - some of our builds end up being 30-40 minutes per operating
system. If you've got the hardware, don't be afraid to run
kitchen test --concurrency n to save a little time (where n is the number of concurrent
instances you want to boot up).
Rake Tasks
Included are some convenient rake tasks for running particular batteries of tests. Just run rake to see a list of tasks available.
Continuous testing with guard
We've included a Guardfile custom-tailored to run the appropriate unit tests whenever a file is modified. To get started, simply bundle install && rake test:guard and watch the appropriate tests run automatically as you edit source files!