Previously it was not possible to test features that required an
authenticated user when:
* using `yarn start` with Vagrant (bug 1363722), which meant slower
watch builds
* pointing the UI at the prod/stage API (bug 1317752), which was
extremely limiting
Now login works in all environments, since the frontend no longer uses a
URL prefix, but instead webpack-dev-server proxies non-webpack URLs to
the chosen `BACKEND_DOMAIN` - avoiding cross-domain issues. Cookies are
rewritten to remove any `secure` directive (which is set on production),
so that they can still be read from HTTP localhost. The `Referer` has to
also be changed to stop Django's CSRF checks from rejecting request.
The slower "build into `dist` and watch" mode is therefore no longer
necessary, so `yarn start:local` instead invokes webpack-dev-server just
like `yarn start` - and the `local-watch.js` workaround has been
removed.
Support for the "publish to GitHub with hardcoded `SERVICE_DOMAIN`"
workflow has been dropped, since it was already rarely used and there is
no way to make it support login.
The API domain environment variable was renamed to `BACKEND_DOMAIN` to
avoid potential confusion given it no longer behaves the same as
`SERVICE_DOMAIN` used to.
NB: For full stack Vagrant workflows users must now connect to port
*5000* on localhost, not 8000.
* Add hooks for actions
* Remove task from context for kind == hook
Also display hookGroupId/hookId for kind=hook
in the modal.
* Move taskcluster-lib-scopes to the vendor chunk
Since we've moving away from vendored packages.
The only differences between the vendored file and that in the NPM
package were the changes to `require('angular')` and export the
created AngularJS module. However Angular is already imported prior
to this package being imported, and we can just pass the module name
as a string instead of relying on the export.
Adds a new view to display intermittent test failure occurrences,
to replace the functionality currently provided by the legacy
OrangeFactor tool.
Includes the new API endpoints originally reviewed in #3271.
## Rough summary of the changes
### Front end
The auth callback is written in React and lives under the /login.html endpoint. It communicates with Treeherder using the localStorage.
### Credential expiration
The Django user session expiration is set to expire when the client access token or the id token expires (whichever one expires first). These values are controlled by the IAM team. Presently, the access token expires after 1 day and the id token expires after a week. That being said, the session will therefore expire after 1 day. If you want this value change, we simply need to send a request to the IAM team.
### Credential renewal
Renewals are set to happen every 15 minutes or so. The renewal is skewed slightly so that different open tabs don't renew at the same time. Once renewal happens, both tokens are renewed and the Django session is updated.
### Migration
If the userSession localStorage key is not set, then the user will be logged out including logging out from the Django session. In other words, all users will be automatically logged out when the merge to production happens.
Previously only 65 rules were enabled, since the `eslint:recommended`
and `plugin:react/recommended` entries in `extends` had no effect,
since when using ESLint's API rather than CLI, the options must be
passed inside the `baseConfig` property instead.
This commit corrects the usage of `extends` and switches us to AirBnb's
React ESLint preset rather than manually opting into rules:
https://github.com/airbnb/javascript
Even with the temporarily disabled rules (which can be gradually fixed
in the future), there are now over 200 ESLint rules enabled, giving
a significant increase in coverage.
Note: We're having to use v15 of `eslint-config-airbnb` rather than v16
until we update to newer Neutrino, since the latest preset has dropped
support for the ESLint v3 that comes with Neutrino 4.
Neutrino 4 configures `file-loader` as the loader for HTML (rather than
the more usual `html-loader`), which means the HTML is not parsed to
look for further dependencies such as `<img src="...">` tags. Our
custom Neutrino config overrode that to `raw-loader` (presumably to
work around bugs caused by the use of `file-loader`), which doesn't
parse HTML either.
Instead, these assets were being manually copied to `dist/img/` by
`neutrino-custom/production.js`'s `CopyPlugin` rule, effectively
circumventing the webpack build process.
Newer Neutrino correctly uses `html-loader`, causing our HTML to be
parsed during the webpack build for the first time. However now that
the images are being resolved at build time rather than runtime, the
relative paths need to be updated to account for the directory layout
differences between `src/` and `dist/`, to prevent build errors.
A significant benefit of this change is that images referenced from
HTML will now be output with hashed filenames, meaning they get given
long-lived `Cache-Control` headers by WhiteNoise.
See:
https://webpack.js.org/loaders/file-loader/https://webpack.js.org/loaders/raw-loader/https://webpack.js.org/loaders/html-loader/
It wasn't unvendored at the time of the switch to Neutrino/webpack
due to the official `flot` NPM package being out of date. However
there are now unofficial up to date flot packages on NPM that we can
use instead - which avoids the clutter and grep false-positives of
vendoring.
I've diffed the vendored files with those in `jquery.flot` and they
are identical, so this change should be a no-op in terms of
functionality.
The issue seen with the old Grunt+babel build process no longer
occurs, so we don't need the modified vendored version. As an added
bonus, the upstream version supports the ES6 import style which we
want to start using soon:
https://github.com/omichelsen/angular-clipboard#use-with-a-module-loader
This reverts commit b37ef957a2.
There appears to be a bug with the webpack module resolution, such that Treeherder's upgraded Hawk 7 is being used by taskcluster-client, even though that has a separate dependency on Hawk 6, and so should be using its own copy of Hawk.
For now, let's roll back to Hawk 7 for Treeherder's direct dependency (used by the login flow), to work around this.
Modify the code to:
* share assets and global settings wherever possible
* update links going both directions
* other small UI tweaks for uniformity with Treeherder
* Fixed a few routing dead-ends on the react side
* Removed the dead TestDetail file we weren't using anyway
* fix production domain urls
Since the node minor/patch versions are reliable enough that it's
not worth the hassle of pinning to an exact version. This only
affects Heroku/Travis, since Vagrant was already always using the
latest 8 series release via the APT repo.
This effectively upgrades node on Heroku/Travis from 8.9.0 to 8.9.1,
since it's the latest release at the moment:
https://github.com/nodejs/node/blob/master/doc/changelogs/CHANGELOG_V8.md#8.9.1
The yarn version specifier has been adjusted to use the `.x` format,
which gives the same end result as the caret range, given Heroku
doesn't cache binaries from one build to the next (only `node_modules`).
Also upgrade Enzyme from 2.7.1 to 3.1.1
Most notable change here is that React.PropTypes is now
moved to a separate package and referenced just by
PropTypes. So this needed some import and linting changes.
This adds the upgrade to Bootstrap 4, and some basic changes and
some CSS tweaks we needed to keep out UI consistent.
The simpler changes are things like:
* Classes that were renamed
* Adding classes that are now needed (dropdown-item, etc)
* Change an item from a button to a span
* Changing order of items (modal header close button, etc)
* CSS class syntax changes
The other changes are lots of CSS padding, margin, font and
other spacing tweaks.
Typically the `package.json` commands call scripts that exist in the
top-level `node_modules/.bin/` directory, which are (a) already on
`PATH` when invoked by yarn, and (b) on Windows have `.cmd` script
shims that work around Windows' lack of shebang support.
However due to symlinks being problematic inside Virtualbox, we have
to use `--no-bin-links` with `yarn install`, which means the top-level
`.bin/` directory doesn't exist. As such, the `<package>/bin/` scripts
have to be used directly, which require prefixing with `node` to work
around lack of shebang support on Windows.
See:
https://shapeshed.com/writing-cross-platform-node/#scripts-in-package-json
Note that even with these changes, the `start:stage` command will not
work due to the environment variable prefix. This will be solved by
the switch to newer Neutrino, which supports setting environment
variables using the `--options` command line parameter instead.
The `angular-marked` package is already listed in `package.json`,
which itself has a dependency on `marked`, so we don't need to list
it explicitly ourselves.
Packages have been updated as follows:
* `angular*`, `react*` and `enzyme`: Unchanged (latest patch version),
for the same reasons as in bug 1364010.
* all other packages have been bumped to their latest minor version.
* caret version ranges have been converted to exact versions, for
parity with the rest of the versions and to make it easier to
identify which version is being used without having to view the
lock file.
This was performed using:
```
sudo yarn global add npm-check-updates
ncu --upgrade --upgradeAll --semverLevel minor
ncu --upgrade --upgradeAll --semverLevel major --reject '/^(angular|react|enzyme).*/'
```
The lock file was then regenerated using:
```
rm -rf node_modules/ yarn.lock
yarn install --no-bin-links
```
Other bugs will handle the more involved `angular*`, `react*`,
`enzyme` and `neutrino*` updates (see deps of bug 1364010).
metrics-graphics is built over d3. So there is no need to add its pacakage separately.
Direct include of metrics-graphics is redundant with the angular fixture.
This replaces the old, "fake" JSON-e with the real JSON-e library.
The library is packaged as a service since the action spec will
eventually specify additional context values, and we want to apply those
values universally.
There were only two tiny issues that affected us:
* $.bind has been deprecated, so it is a switch to $.on
* Found a function that was dead code, so removed it
The webpack-md5-hash plugin attempts to make the generated hashes
less susceptible to churn when chunks haven't actually changed.
However it requires careful use and clearly wasn't set up correctly,
since the manifest file hashes were always the same regardless of
content, causing stale files and runtime errors in production.
Once Neutrino v6 is released, we can use its new in-built support
for caching to save having to implement this ourselves.
The only change is the move of `karma` from angular-toarrayfilter's
`dependencies` to `devDependencies`, which stops us unnecessarily
pulling in an ancient version of karma, and the resultant security
warnings.
Packages have been updated as follows:
* `angular*`: Latest patch version, since it doesn't follow semver.
* `react*`: Latest patch version, since in 15.5 some obtrusive
deprecation warnings were added, and `enzyme`'s peer-dependencies
need adjusting too.
* `enzyme`: Unchanged, since the newer minor version gives warnings.
* all other packages have been bumped to their latest minor version.
This was performed using:
```
sudo yarn global add npm-check-updates
ncu --upgrade --upgradeAll --semverLevel minor
ncu --upgrade --upgradeAll --semverLevel major --reject '/^(angular|react|enzyme).*/'
```
In addition, `karma-coverage-istanbul-reporter`, `taskcluster-client`
and `hawk` have been bumped to the latest major versions since their
changelogs don't show any breaking changes that impact us.
The lock file was then regenerated using:
```
rm -rf node_modules/ yarn.lock
yarn install --no-bin-links
```
At some point later we'll want to perform the more involved `angular*`,
`react*`, `enzyme`, `jquery` and `neutrino*` updates.
Node packages that are intended to be called from the command line can
request that symlinks be added to `node_modules/.bin/` as part of their
installation. When `npm/yarn run` is used, it automatically adds that
directory to the PATH, so commands listed in `scripts` in `package.json`
can normally be specified without the full filepath.
However when npm/yarn is used inside a Linux Vagrant instance running on
a Windows host, even though the guest OS supports symlinks, errors will
occur if symlinks are created in a directory that is shared with the
host using Virtualbox shared folders.
In this case, the workaround is to prevent the creation of symlinks
using `--no-bin-links`. However unfortunately instead of having a
sensible fallback (eg a shell script that acts like a symlink) instead
no files are created in `node_modules/.bin/` at all.
As such, we have to use full paths in `package.json` after all.
For the same reason as the previous commit.
Ideally we'd remove the grunt abstraction entirely and call eslint from
the `lint` command, but we might as well save that to the Neutrino PR.
Routing commands via npm/yarn is preferred, since it avoids
having to do global installs of grunt-cli, which simplifies contributor
setup, and means less effort when we switch to Yarn (since it requires
manual PATH setup for globally installed packages).
Vagrant uses the latest 7.x.x release, which is now 7.7.2. To reduce
differences between environments whilst the Neutrino/webpack work is
stabilised, it makes sense to update Heroku/Travis again too.
Introduces the ngreact-test-utils npm package to supply reusable
compile() and simulate() functions for testing ngreact components. This
requires browserify, which is now applied to tests in
tests/ui/unit/react only.
This is required because the older version of grunt-angular-templates
was hanging with component templates. In order to upgrade everything it
was also necessary to switch from grunt-cache-busting, which is no longer
maintained, to grunt-cache-bust, which is.
The move to eslint 1.x changes the way rules are managed, see:
http://eslint.org/docs/user-guide/migrating-to-1.0.0
There are slightly fewer rules enabled with "eslint:recommended" than
were by default before, but we'll enable more non-default rules later.
Normally grunt-cli (a wrapper around grunt to allow it to be easily run
from the command line) would be installed globally, so that it is on the
PATH. However to save having to do this on prod/Heroku/Travis, we
install it locally - and it will then be available via:
`./node_modules/.bin/grunt`
...or else via just `grunt`, if referenced inside a package.json scripts
section.
As of karma-jasmine 0.3.0, the jasmine library is not bundled with
karma-jasmine, and so jasmine-core has to be installed separately.
jasmine-jquery also has to be updated, since the old version was not
compatible. A duplicate jasmine-jquery file that was outside of the
tests vendor directory has also been removed.
To avoid Travis failing if new package authors don't follow semver and
break things in a point release.
This is effectively a no-op, compared to a clean install using the prior
package.json.
The karma package already comes with a wrapper script, so there's no
need to install another locally. The intended use for karma-cli is to
install it globally, which cannot be done from package.json.
The previous value of "^3.2.0" is equivalent to "4 > version >= 3.2.0".
We should pin to a specific version to stop deployments breaking if
a new release doesn't behave as expected. See:
https://nodesource.com/blog/semver-tilde-and-caret
We should also do the same for devDependencies, but let's do that in
another bug, since it doesn't affect production.
We were previously using '~' (which allows for N.N.*) and '^' (which
allows for N.*.*) for the grunt node package version numbers. This meant
that each person running grunt build could end up with different output,
since the versions were not pinned. Instead, all packages are now pinned
to a specific version - the latest currently available (which in many
cases is not far off the version someone running |npm install| in a
clean repo would have gotten anyway, due to the use of '^' and '~').
As part of merging the UI repo into this one, the following directory
moves were performed:
webapp/app/ -> ui/
webapp/test/ -> tests/ui/
webapp/config/ -> tests/ui/config/
webapp/scripts/ -> tests/ui/scripts/
webapp/scripts/web-server.js -> web-server.js
This means if |npm install foo --save-dev| is now used in the future,
the diff will be much clearer, since it won't also change the order of
existing entries.
Renovate Bot
Ed Morley
Cameron Dawson
Hassan Ali
renovate[bot]
Sarah Clements
Ed Morley
Brian Stack
Shruti Jasoria
Dustin J. Mitchell
Ed Morley
Casey Williams
James Graham
William Lachance
crosscent
Jonathan French
mdoglio
George Miroshnykov
Jonathan Eads