Some helper scripts to manage github orgs via API.
Перейти к файлу
Hal Wine 2d86afe2c7
Hwine/fewer acl recommendations (#110)
* Fixed potential infinate loop

When an iterator makes a GitHub API call, that call could hit an API
limit. Therefore, you must process each returned item separately (so
that when you restart, you're in the same place), AND you must handle
the StopIteration exception yourself.

After that, forward progress is made on the iterator!

Prior code tried to slurp all the items into a list -- so the restart
was from the beginning of the query. Which almost guaranteed you'd hit
the limit again.

* Don't suggest ACL searches for no-hit logins

There's no point in searching for potential GitHub logins that did not
get a member-or-contributor hit on the repos. This should prune the ACL
search to a max of 2 values: ldap & (real) GitHub login.
2023-06-20 07:18:01 -07:00
.jupyter Add ACL checks & py3 notebook 2021-08-04 18:11:43 -07:00
audit-log Update to Python 3 (#69) 2021-05-01 14:08:35 -07:00
cache Find old and small/unused repositories 2015-05-12 13:54:42 -07:00
notebooks Hwine/fewer acl recommendations (#110) 2023-06-20 07:18:01 -07:00
tests Update to Python 3 (#69) 2021-05-01 14:08:35 -07:00
.dockerignore Add debugging support to Dockerfile 2022-11-18 16:27:54 -08:00
.gitignore Works from fresh checkout 2021-07-30 14:18:15 -07:00
.pre-commit-config.yaml Update for new syntax in search commands 2022-11-18 17:40:38 -08:00
.secrets.baseline Hwine/fewer acl recommendations (#110) 2023-06-20 07:18:01 -07:00
CODE_OF_CONDUCT.md Update to Python 3 (#69) 2021-05-01 14:08:35 -07:00
Dockerfile Add debugging support to Dockerfile 2022-11-18 16:27:54 -08:00
LICENSE Update to Python 3 (#69) 2021-05-01 14:08:35 -07:00
Makefile Improve ACL results (#107) 2023-05-24 11:59:28 -07:00
README.md Remove dependence on repo2docker (#91) 2022-05-20 09:00:01 -07:00
block_user Hwine/block user (#65) 2021-01-11 08:57:22 -08:00
check_CoC.py File moved to Mozilla-GitHub-Standards repo 2019-03-31 15:55:01 -07:00
client.py Update to Python 3 (#69) 2021-05-01 14:08:35 -07:00
close_pull_requests.py Update to Python 3 (#69) 2021-05-01 14:08:35 -07:00
close_pull_requests.yaml Update URLs in close_pull_requests.yaml (#88) 2022-04-29 08:21:48 -07:00
code_search.py Update to Python 3 (#69) 2021-05-01 14:08:35 -07:00
contributing.py Update to work on GitHub3.py v1.3.0 (#71) 2021-05-02 07:32:11 -07:00
get_PAT_owner [get_PAT_owner] enhancements (#66) 2021-01-11 11:28:09 -08:00
get_active_hooks.py Update to Python 3 (#69) 2021-05-01 14:08:35 -07:00
get_olson_tz.sh Improve first name handling 2022-01-29 16:02:15 -08:00
get_org_info.py Improve output, add --owners-only option 2021-11-17 17:35:02 -08:00
manage_invitations.py Update to work on GitHub3.py v1.3.0 (#71) 2021-05-02 07:32:11 -07:00
old_repos.py Update to Python 3 (#69) 2021-05-01 14:08:35 -07:00
owner_actions_from_auditlog Update to Python 3 (#69) 2021-05-01 14:08:35 -07:00
poetry.lock Clearing up a lot of aged dependencies with modern versions. Note, should not affect the notebook functions 2023-05-03 15:58:25 -04:00
pyproject.toml Clearing up a lot of aged dependencies with modern versions. Note, should not affect the notebook functions 2023-05-03 15:58:25 -04:00
remove_member_from_org.py Update to work on GitHub3.py v1.3.0 (#71) 2021-05-02 07:32:11 -07:00
repo-admins Update to Python 3 (#69) 2021-05-01 14:08:35 -07:00
repo-pr-stats.py Update to Python 3 (#69) 2021-05-01 14:08:35 -07:00
requirements.in Remove dependence on repo2docker (#91) 2022-05-20 09:00:01 -07:00
requirements.txt Remove dependence on repo2docker (#91) 2022-05-20 09:00:01 -07:00
set_secrets_in_env.sh Improve error messages 2023-01-05 15:13:36 -08:00
sudo-all Add file used for debugging 2022-12-30 08:41:31 -08:00
team_update.py Hwine/py2 docker image (#74) 2021-07-28 15:20:43 -07:00

README.md

github org helper scripts

Current minimums: python 3.7; GitHub3.py 1.3.0

Please use poetry to manage virtual environments - requirements.txt may be out of date, and does not include development dependencies.

These are some API helper scripts for sanely managing a github org. For now this is somewhat hardcoded for the mozilla org; no need for it to remain that way though. Many scripts support the --help option. That information should be more up to date than information in this document.

Credentials

Supplying credentials for execution is done by passing a PAT token as the value of the environment variable GITHUB_TOKEN (preferred) or GITHUB_PAT.

The recommended way to set GITHUB_TOKEN is via cli access to your password manager. For example, using pass:

GITHUB_TOKEN=$(pass show myPAT) script args

Jupyter Notebooks

Docker Images

Our Jupyter Notebooks have a farely simple environment as regards dependencies. The recommended way to deal with this is by using a docker container.

The Makefile contains targets for building and running the docker images. Invoke make without arguments to see those targets

  • NOTE: the docker image allows credentials to be supplied via sops. The environment variable "SECOPS_SOPS_PATH" must be set appropriately.

When started, the docker container will serve notebooks from the notebooks/ directory, but they will be available at the top level. Current notebooks include:

  • User Search.ipynb -- Given a set of possible GitHub logins, determine if they might have any permissions in various organizations. Links are provided for hits, so easy to examine more closely.

    N.B.: Both this script and the GitHub search interface make assumptions. It is your responsibility to ensure any possible match is a valid match.

    There is now a section which will search for usernames in any non-documentation source file. The intent is to spot cases where app, login, or other permissions may have been granted via that file. Since such authorization usage is adhoc, there are likely to be many false positives. (However teams may choose to use the list for "cleanup" of unmaintained documents.) Typically, the user will want to supply both ldap and GitHub logins to be the search targets.

Scripts

Scripts should now work with Python 3. Please open issues for any problems you encounter.

auditlog.py

Download audit log for $ORG via headless firefox via selenium (geckodriver must be installed). Credentials as environment variables, and 2FA token passed as input when requested.

contributing.py

Analyze all the "sources" repositories (i.e., those that aren't forks) in a github org and list the repositories that do NOT have a CONTRIBUTING file.

get_active_hooks.py

Find all hooks configured for an organization -- see --help for details

get_org_info.py

Output basic info about an org, more if you have permissions. See --help for details

manage_invitations.py

Cancel all org & repository invitations older than a specified age (default 2 weeks). See --help for details.

team_update.py

Update administrative teams so they can be used for the new GitHub discussion feature. Use the --help option for more information.

hooks.py

Analyzes a list of audit log export files (from the JS script) for hook/service creation/deletion and provides a summary. Use it to show commonly used apps/services/webhooks across the org.

old_repos.py

Generate a list of empty (should be deleted) repositories as well as untouched repos (might need to be archived).

BUGS

  • Some of these scripts are no longer relevent.

License

This code is free software and licensed under an MPL-2.0 license. © 2015-2021 Fred Wenzel and others. For more information read the file LICENSE.