* Support for allowed ports for GCP firewall tests
Adds support for allowed ports in GCP firewall tests
As well, switch from the term "whitelisted" to "allowed"
* Add comments about allowed_ports* config opts
* Support Markdown in documents
Uses `myst_parser` to process Markdown files in the Sphinx workflow.
Using that, included the top level `README.md` as the "Overview" in the
rendered docs.
Some tweaks were needed to have `README.md` render correctly in rendered
docs. Still renders okay via GitHub.
Add a makefile target `doc-preview` to serve the locally built docs.
Fix some links in the README file.
Co-authored-by: Sven Marnach <sven@mozilla.com>
* Check an S3 bucket should have a life cycle policy
* Use an f-string
* Fix a Black error
* Pass ids to the parametrize call
Also fix get_s3_bucket_name function so that it can handle cases when `bucket`
is not a dictionary.
* Use the get_s3_bucket_name_only function
* Fix the function name
Co-authored-by: AJ Bahnken <1144310+ajvb@users.noreply.github.com>
* Check RDS storage type is not io1
* Rename a test file
Because we're checking not just for "io1" but also anything that starts with "io".
* Make black happy
* Adds topic test, and works except for the lambda function.
* Adds the test code.
* Contains a lambda fix as suggested by @ajvb, this is now working.
* Adds rationale.
* Black formatting fixes.
* Rework GCP code to support folder id arg
Reworks the GCP client, resources, and test code to add support for
passing in a Folder ID and running tests against all of the projects
within that folder.
Alongside that, this includes some stability fixes to the client code,
creates a new "frost" named cli arg group, and removes caching from the
GCP client.
* code review/clean up for #361
* setup.py: require python 3.8
* setup.py: add shebang
* add frost dir with VERSION and SOURCE_URL
* add click to requirements
* add frost module and pass through cli
* make: pytest -> frost test
* make: install as python package
* run pytest from the repo root directory
so we discover frost conftest and tests
* readme: update install directions and examples
* make: add test files to awsci target
* update dockerfile to install python package
* add requirements to setup.py
* bump version to 0.4.0
* make: remove python version check
setup.py specifies supported python versions
* ci: drop travis tests against python <3.8
* ci: drop travis ripgrep install
rg removed
* WIP: Adding sns tests.
* This now works and is relatively fast. It combines the suggestion from g-k, fixes from sven, and the slice from rds_db_snapshots_attributes().
* Reformat with black.
* Updates for latest pytest.
Co-authored-by: Sven Marnach <sven@mozilla.com>
Before, the test checked if the node version is in validNodeVersions. However, unlike the short validMasterVersions the list in validNodeVersions is very long and goes all the way back to k8s 1.6. Thus, this did not actually test that the node version was up-to-date.
I initially thought to just check that the node version matched the master version, but realized this is not a good idea since google doesn't upgrade clusters and nodes in lockstep.
Instead, I've changed the check to verify that the node version is in the list of valid master versions. That should ensure we're on a recent version and still pass in the gap between cluster and node upgrades.
* update pytest and pytest plugins
* conftest: add rationale and service markers
* s/get_marker/get_closest_marker/
> #4546: Remove Node.get_marker(name) the return value was not usable for more than a existence check.
>
> Use Node.get_closest_marker(name) as a replacement.
https://docs.pytest.org/en/6.0.2/changelog.html#pytest-4-1-0-2019-01-05
* replace removed _genid with nodeid
* aws: fix typo in iam_users_with_policies_and_groups docstring
Co-authored-by: Hal Wine <132412+hwine@users.noreply.github.com>
* add get_param_id to top level helpers
* handle test idfns failing with NotSetType for offline empty param lists
fix confusing pytest parmetrize doctest errors
* Fixes for pytest upgrade and new datetime parsing
* Create a new global for our CustomConfig obj
* Fix parameterize id calls where it's getting a list
* Use timezone info in datetime related tests
* Fix formatting + use get_param_id in redshift helpers
* switch ordering on rds id func for unit tests
Co-authored-by: Hal Wine <132412+hwine@users.noreply.github.com>
Co-authored-by: AJ Bahnken <aj@ajvb.me>
* Add check for use of provisioned iops in EC2
* Only try to convert cached strings to datetimes if they are in ISO format
Before this, we were converting volume types to datetimes.
```
>>> from dateutil.parser import parse
>>> volume_type="st1"
>>> parse(volume_type, ignoretz=True)
datetime.datetime(2020, 9, 1, 0, 0)
```
* Paint it black
* Even blacker
* Update test to match new json_iso_datetime_string_to_datetime func
Co-authored-by: AJ Bahnken <aj@ajvb.me>
* Replace use of 'rg' for CircleCI
'rg' isn't part of the base images, and may not be on some dev's
machines. So switch to the old standby of 'grep' -- with a bunch more
options for the same behavior.
* Reenable check_conftest_imports
Had been disabled until 'rg' replaced with 'grep'.
Docs!
* Add rendered docs for gh-pages delivery
Scheme based on
https://github.com/sphinx-doc/sphinx/issues/3382#issuecomment-409068915
* Makefile support for gh-pages hosted docs
N.B. we require user to have sphinx installed globally at the moment.
That is somewhat reasonable, as Rust language uses sphinx for
documentation.
If we end up wanting specific sphinx extensions, we'll have to add
those requirements to a (not yet in existence) requirements-dev.txt.
Fixes#304
We really care about the minimum version that is currently advertised as
being supported. The assumption is, anything that works on the minimum
version will continue to work on a later version.
Yes, CI will catch this -- the hope is this message may prevent a dev
from only finding out that late in the process. We could consider
removing this once tox testing is implemented.
AJ Bahnken
bqbn
Hal Wine
Hal Wine
Sven Marnach
Bob Micheletto
Greg Guthe
Brian Pitts
Brian Pitts
Sven Marnach