The internal kokoro build.sh currently runs the following:
devtools/ci/ci.sh
devtools/ci/nodejs.sh npm install --quiet
devtools/ci/nodejs.sh npm run lint
devtools/ci/nodejs.sh npm run test
devtools/ci/e2e.sh
Rather than having multiple files referenced in the internal kokoro
file, moving these tests to ci.sh. That way it is more opaque what we
are running in CI, and easier to add/remove files and comment out lines
for testing.
devtools/ci/nodejs.sh will be deleted in a future CL once it has been
removed from the internal build.sh.
Change-Id: I397a5d5cd2efb59da7a771617330f70b980911b0
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/326970
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
A copy of devtools/ci/nodejs.sh is made to devtools/nodejs.sh, and the
latter is now referenced where devtools/ci/nodejs.sh was previously used.
This makes it easier to comment out tests when testing configuration
changes to kokoro.
Also, since the contents of ./devtools/nodejs.sh are used in non-ci
parts of the repo, so it makes sense to have that script outside the ci/
folder.
In a future CL, rather than having these 3 lines in the kokoro config:
devtools/ci/nodejs.sh npm install --quiet
devtools/ci/nodejs.sh npm run lint
devtools/ci/nodejs.sh npm run test
They will be moved to devtools/ci/nodejs.sh, which will use
devtools/nodejs.sh.
Change-Id: I156324a625666ead31be18ae19e5608d38e0faf1
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/326989
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Pinning the nodejs image to the long term support
version and adding pseudo-tty to fix warning in CI.
Change-Id: Idd283029b0667a673b7f8f4844c7e92a605edb3b
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/326550
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Moves pages into domain specific directories
and updates the template parsing logic to
begin the switch to the new base layout. The
unit page migration will come in a later CL.
Change-Id: I424643146aacaefd61e88cb9a767b9d4b581aa5e
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/324391
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Julie Qiu <julie@golang.org>
Files used for kokoro are copied to devtools/ci along with a README.md,
so that it is clear what these files are used for.
The existing files will be deleted in the next CL, after an internal CL
is merged to change references.
Change-Id: I893ac9461d3f775460ba749da80e2b4ba71dcc1e
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/325129
Trust: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Sometimes some test DBs don't exist. Change the script so it continues
in that case.
Change-Id: I1e09632cdb47bb768cfd108d537c8b053b063ebe
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/323853
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Jamal Carvalho <jamal@golang.org>
Adds the GO_DISCOVERY_E2E prefix for all e2e env vars.
The docker container for nodejs now gets the relevant
variables passed along in the docker run command.
Change-Id: I43ae4c52fee7be47c6f6afc9265d0d7e01d54015
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/322831
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
We will need to run npx commands with docker when
running tests in CI. This change creates an npx
command in all.bash and modifies the docker_nodejs
script to share networking with the host so it can
connect to a local instance of the frontend.
Change-Id: Id8f32a7ff5bf017cb55c531b1e37a3f8aab0e73a
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/322635
Trust: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Julie Qiu <julie@golang.org>
That is what our frontend runs on.
Change-Id: Iae80f7141f5a88f76567cce21a115b3385fbb2b5
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/322009
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
The npm version that comes with node 14 is behind
the version we are using locally. Updates node to 15
to get an image with npm 7.x.x.
Change-Id: Ida4584693d86127ff145fde35146e51bda58da36
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/313029
Trust: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Adds the --rm flag to automatically clean up
docker containers after script runs. Removes
the -it flag for docker_nodejs as it causes
an error in CI.
Change-Id: If8cae56bc9dbe691ff216053e222b43e5591623a
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/311373
Reviewed-by: Julie Qiu <julie@golang.org>
Trust: Jamal Carvalho <jamal@golang.org>
The e2e tests will prefer a locally installed npm binary and
use a prebuilt headless chrome docker image. This will speed up
test runs for devs with npm installed but still leave the option
to opt out of installing node directly on the host machine.
Jest now uses a custom test environment that initializes global
variables, starts chrome, and adds authorization headers to
requests.
Change-Id: Ie43952ada6fd3e4df0fd522c84691186ef0d7f80
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/310314
Trust: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Julie Qiu <julie@golang.org>
Each e2e test file was starting and closing its
own instance of Chrome. To speed up the tests
we are now sharing a single instance of across
test files and properly exiting after the tests
have completed.
Change-Id: Ia7ad4acaf961cf89a177e293e971c94b43b76d05
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/304589
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
Adds a check feature to the static command
to ensure that the JS output is correct for
the current set of TS source files.
Change-Id: I4f805883f2ef6d30b888a915024585197049278d
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/300231
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
Sets up configuration for esbuild, a JavaScript bundler
written in Go that has support for TypeScript. This
tool will replace the TypeScript compiler in building
our frontend scripts. It is much faster and has no dependencies.
Run "go run devtools/cmd/static/main.go" to compile
TypeScript code for production or "go run cmd/pkgsite/main.go -dev"
to compile TypeScript code for development and rebuild
on file changes.
Fixesgolang/go#42540Fixesgolang/go#43359
Change-Id: I5ddc1b716587ee5bdd897891d4972158782dbb49
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/300074
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
Trust: Jamal Carvalho <jamal@golang.org>
This change gives us a slight improvement in the
speed of our tests and results in better error
messages when things break. Jest-circus is the default
test runner in the next major release of jest but
we can take advantage of it today.
Change-Id: I454449b18e1644a8d02faf276c2905b2a5879377
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/298429
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Refactors e2e tests to consolidate common setup steps
into setup.ts simplifying test maintenance and creation.
Change-Id: I886cd2f4fafc99308da2455942a6809d4eb1fe5a
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/298371
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Trust: Jamal Carvalho <jamal@golang.org>
For e2e test runs, the timing of when the db
starts accepting connections and the migrations
begin to run is not perfect. Restarting migrations
on failure should do the trick. Restarts are limited
to 5 attempts.
Change-Id: I91f992da62967c3fa31892af5ecedff37b264758
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/298370
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
Version 3 of the compose spec removes support
for the extends keyword.
Change-Id: Id2208252c1b62a940733214c70422e090bc3ca9b
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/298330
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
The -S option for /usr/bin/env is not supported in CI. Adjusting
the docker_compose.sh script header to match docker_ci.sh.
Change-Id: I77835e3d17442ef6122fb0ba7580f0101ce17fea
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/298329
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Trust: Jamal Carvalho <jamal@golang.org>
Adds a command to tail the logs of docker
services defined in docker-compose.yaml.
Change-Id: I3bdc0d0567c1a10e631932403d417539dfbf0cf1
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/297554
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
The file e2e-config will be mounted into
the frontend docker container to toggle
experiments for e2e tests. The file is
mounted instead of copied so that experiments
can be changed without rebuilding the frontend
container.
Change-Id: I7ff0db94b760594e501af0b747cd766934081d1c
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/297555
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
These files are no longer needed now that all.bash
has been updated to use devtools/docker-compose.sh.
Change-Id: I59b8fd7a195cc40f50834e6bfa27cd9b0f83ebc6
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/292697
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
The config creates the following services
- nodejs: used to run npm commands, a slight refactor of the
existing config for npm in docker-compose.yaml
- ci: extends nodejs, used to run the e2e tests
- frontend: runs the frontend server
- migrate: runs database migrations
- wait_for_db: a utility service, used to delay the start of the
migrations until the db is ready for connections
- db: exposes a postgres database
Using the devtools/docker_compose.sh script we can run these
services and perform the relevant cleanup. The script exposes
the nodejs and ci services. We'll use it in all.bash and CI
to run the linters and tests for frontend assets.
For golang/go#43361
Change-Id: Ie9c91058f9c6796b737e94c237b2d99f612f45a7
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/292695
Trust: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Julie Qiu <julie@golang.org>
Adds a setup file for e2e testing with jest. This
will let us consolidate setup code common to all
e2e tests.
Change-Id: I7f0ee7316b67e91345f7e47e507b5fd93da773c8
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/292699
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
These dockerfiles will create the images used during CI testing.
For golang/go#43361
Change-Id: I638b12709fd1a8b3f4bd06448d5e952345cd3917
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/292694
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Migrates playground.js code to TypeScript in preparation
of enhancements to the example playgrounds.
Change-Id: I185e9fe05c0e7bf3deb683f1583f9a26b4960237
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/288991
Trust: Jamal Carvalho <jamal@golang.org>
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Removes separate directory for TypeScript files and
updates frontend tools to match updated directory
structure. TypeScript files will be co-located with
the JavaScript files making easier to identify files
that have already been migrated to TypeScript.
Change-Id: I4deb5b22c550fc34c98529bbe19bec6d28759a02
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/282377
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
Removes PostCSS cli dependency and minified CSS files.
Reverts the changes from https://golang.org/cl/281772
to simplify the development process for CSS files. We
may revisit minifying and bundling CSS with a Go package
in the future.
Change-Id: I34936eefc0d9aced9a13badace2c950f53ac84da
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/282376
Reviewed-by: Julie Qiu <julie@golang.org>
Trust: Jamal Carvalho <jamal@golang.org>
If the npm script is run and the node_modules
directory does not exist the script will run
npm install first.
Change-Id: I8db7c6d980dddbbd685cf095652f6112b148c4ba
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/282375
Reviewed-by: Julie Qiu <julie@golang.org>
Trust: Jamal Carvalho <jamal@golang.org>
This change moves pkgsite page styles to lib/css directory
and writes processed and minified css to static/css.
For golang/go#43360
Change-Id: I42d61f4a270e3cde5016c6063555eb037f35c302
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/281772
Trust: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
This change updates the jest config to specify
test environments separately for unit and e2e
tests and adds the isolated modules option to
the ts-jest config to improve test startup time.
For golang/go#43361
Change-Id: I3fbfae2a42c26bd0eeb2d235dab1cced0c8701ea
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/280711
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Adds dependencies, configuration, and scripts for frontend
e2e and unit testing. Test examples to follow this change.
Change-Id: Ia3d246abfcc1411d5b00ca7e23829f1ede6e7207
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/280600
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
Trust: Jamal Carvalho <jamal@golang.org>
Creates build file for pkgsite npm container. This will enable
contributors to run prettier formatting without having npm and
node installed on their machines as long as they have docker and
docker-composed installed.
Change-Id: Ibb9ff375eabcb548d258907cf6f1111ca3654e7d
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/275447
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
The Postgres contribution guide (/doc/postgres.md) tells the user to
set specific environment variables and then run the create_local_db
script. However, the create_local_db script did not respect those
variables. This was causing issues for some local configurations.
Changes the script to respect the environment variables if set,
otherwise fall back on the postgres defaults detailed in the guide.
Change-Id: Ib84622266ae1a841f7a17a84ee020ac8b6ba30a6
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/273846
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
Trust: Jonathan Amsterdam <jba@google.com>
The Postgres contribution guide (/doc/postgres.md) tells the user to
set specific environment variables. When running migrations, the
migrate_db.sh script does not respect the environment variables
referenced. This was causing issues running local migrations.
Changes the script to respect the environment variables if set, and
fall back on postgres defaults if unset.
Change-Id: I5484d825ce2fa503290193aaea403f5cb5ef4cba
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/273927
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Removes static content used in the legacy package, directory, and module pages.
Moves sidenav and readme styles requests into details page so they are not
loaded on all page requests.
Change-Id: Ief93cb1ec21939dadf0f9f6a0c47abaf4284b2a9
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/272827
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
Updates css generator and creates new readme.css file that
preserves github heading styles when using the goldmark
parser.
Change-Id: Iddd2ea99711ffe11c14f6e9e2087b023b7d73325
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/266580
Trust: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Julie Qiu <julie@golang.org>
Julie Qiu
Jamal Carvalho
Jonathan Amsterdam
Charlotte Brandhorst-Satzkorn