Support for filing issues as security bugs got added in bug 1369067. Having the
security group hardcoded and not matching the default one for the component
causes bugs to get missed during triage.
* Add model for file-to-bugzilla-component relation and its ingestion process
* add API endpoint to get Bugzilla product and component for source file path/search term
* bug filer: identify bugzilla product and component to use based on file path
* rename builds-4h to live_backing_log
* add foreign key to Job on TextLogError table
* remove TextLogErrorViewset and test
* add management command to backfill job ids and update artifact.py
* Add management task for fetching github commits based on repository fixes and
2 tables for storing that data
* Add changelog web api
* Add tests
* Refactor Github utilities into one file and move http utilities from common.py to new file
Since these Django management commands are infinite duration (they run
continuously listening for Pulse messages), they cannot be added to the
list of Django management commands in `newrelic.ini` that are automatically
tracked by the New Relic Python agent.
Instead, we need to wrap the finite duration function within the Pulse
listener command that represents a discrete unit of work that New Relic
can report as an individual non-web transaction.
See:
https://docs.newrelic.com/docs/agents/python-agent/supported-features/python-background-tasks#wrapping
The name of suitable (finite duration) management commands have to be
defined in the New Relic config file, since Django management commands
are not annotated by default:
https://docs.newrelic.com/docs/agents/python-agent/supported-features/python-background-tasks#django
We'll likely want to expand this list in the future, particularly if we
switch from celery beat schedules and towards the Heroku scheduler in
bug 1176492 (since it will run management commands directly, outside of
Celery).
It's been switched off via the New Relic web UI config page for some
time, however it makes sense to explicitly turn it off via the agent
config (which cannot then be overridden by the web UI).
The settings for the New Relic Python agent are defined via the
following methods (later items override the earlier ones):
1. Agent defaults
2. Environment variables
3. Local agent configuration file (iff `NEW_RELIC_CONFIG_FILE` set)
4. Server-side configuration (ie via the New Relic website)
5. Per-request configuration
Some settings can only be controlled by a subset of these methods, eg
the more security-sensitive ones (like whether request parameters should
be recorded) must be set via [3].
In addition, once [4] is activated, *all* settings that can be set via
their website will override those from 1-3, even if they are set to the
empty string. As such, it's recommended to *only* use the local agent
config file for settings that are unavailable on the website.
Stage/prod do currently have their own config file (managed via puppet,
forked from the standard IT config), however it turns out it's unused
since `NEW_RELIC_CONFIG_FILE` isn't defined. This explains why
bug 1141036 didn't actually make any difference.
To change New Relic settings (that can't be set via the website) on
Heroku, we're going to need to commit a config file to the repo anyway,
so we might as well use that same file for stage/prod too, so we can
modify the config there without requiring puppet changes. I've not
copied the unused stage/prod config file from puppet, since it's crufty
and mostly contains defaults.
This commit is a no-op until `NEW_RELIC_CONFIG_FILE` is defined in each
environment. The `log_file` setting within will make the NEW_RELIC_LOG
environment variable redundant. That variable is currently set to
'stderr' for stage/prod and 'stdout' for Heroku, so I've settled on
'stdout'.
In later bugs (eg for bug 1223496) more useful options will be added,
but we'll at least be able to start pointing at the config file using
`NEW_RELIC_CONFIG_FILE` in the meantime.
For the agent defaults and more info, see:
https://docs.newrelic.com/docs/agents/python-agent/installation-configuration/python-agent-configuration
Sebastian Hengst
beatrice-acasandrei
Sarah Clements
Tarek Ziade
Ed Morley
Ed Morley