Brad Fitzpatrick 70a5fe4614 cmd/coordinator: use historical test timing data for critical path scheduling We previously hard-coded a Linux-only static map of 2015 data in for the critical path scheduling of cmd/dist tests over N sharded buildlets. That worked well only for Linux and only in 2015. Instead, query BigQuery to find out what the recent timing data looks like for all builders. I'd started to work on this back in CL 30716 (Oct 2016) but apparently never finished. Yay me. But skip the writing-to-CSV step. BigQuery is much faster than I remember (maybe it got faster?), so just query it directly. The query takes about 2 seconds, and we only do it every hour (which is still overkill; daily is probably fine). Change-Id: I498fc09dfaf24fb1f11b2c0ab4b952b2f15f9c32 Reviewed-on: https://go-review.googlesource.com/c/160037 Reviewed-by: Andrew Bonventre <andybons@golang.org>	2019-01-30 21:44:06 +00:00
..
README.md	cmd/buildstats, cmd/coordinator: run buildstats sync in background	2017-08-03 20:46:40 +00:00
buildstats.go	cmd/coordinator: use historical test timing data for critical path scheduling	2019-01-30 21:44:06 +00:00

Brad Fitzpatrick 70a5fe4614 cmd/coordinator: use historical test timing data for critical path scheduling

We previously hard-coded a Linux-only static map of 2015 data in for
the critical path scheduling of cmd/dist tests over N sharded
buildlets. That worked well only for Linux and only in 2015.

Instead, query BigQuery to find out what the recent timing data looks
like for all builders.

I'd started to work on this back in CL 30716 (Oct 2016) but apparently
never finished. Yay me. But skip the writing-to-CSV step. BigQuery is
much faster than I remember (maybe it got faster?), so just query it
directly. The query takes about 2 seconds, and we only do it every
hour (which is still overkill; daily is probably fine).

Change-Id: I498fc09dfaf24fb1f11b2c0ab4b952b2f15f9c32
Reviewed-on: https://go-review.googlesource.com/c/160037
Reviewed-by: Andrew Bonventre <andybons@golang.org>

2019-01-30 21:44:06 +00:00

README.md

cmd/buildstats, cmd/coordinator: run buildstats sync in background

2017-08-03 20:46:40 +00:00

buildstats.go

cmd/coordinator: use historical test timing data for critical path scheduling

2019-01-30 21:44:06 +00:00

README.md

golang.org/x/build/internal/buildstats

Package buildstats contains code to sync the coordinator's build logs from Datastore to BigQuery.