Граф коммитов

47 Коммитов

Автор SHA1 Сообщение Дата
Roland Shoemaker 2c66d68b91 cmd/relui: add post-submit bypass to "tag single x/ repo" workflow
Requires adding a checkbox parameter type. This should allow the release
coordinator (or whoever is running this workflow) to tag a single x/
repo without needing to wait for a green post-submit commit, which is
necessary when doing a security release.

Change-Id: Id9e6743bc5d86730bb109cda91164190c34187c7
Reviewed-on: https://go-review.googlesource.com/c/build/+/517235
Reviewed-by: Heschi Kreinick <heschi@google.com>
Auto-Submit: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
2023-08-09 21:45:33 +00:00
Dmitri Shuralyov c42ce50790 internal/{relui,task,workflow}: check coordinator parameter sooner
It's not often that a bad parameter value is provided to a workflow.
But when it does happen, it'd be nicer if relui would tell you about
the problem right away. Instead, it quietly starts a workflow that
will predictably fail at some arbitrary later time, only after it
reaches the first task that uses said parameter.

This is especially true for cases that are trivial for relui to check
automatically, such as a typo in release coordinator username or name
or Gerrit email not being in gophers package. So add a check to report
right away when a release coordinator username parameter will result in
an error. This lets humans spend their attention on more interesting
things (in the long run).

Change-Id: If12b6275d56b6b03adc27958c1792391ba600712
Reviewed-on: https://go-review.googlesource.com/c/build/+/508435
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
2023-07-17 14:45:49 +00:00
Alex Rakoczy 67d624370c internal/relui: email on schedule failure
The user who scheduled a workflow may not be closely watching at the
time of failure. This adds functionality to send an email if a scheduled
workflow cannot progress.

The current implementation logs on stall, but it's trivial to
configure to email as some tasks do.

For golang/go#54476

Change-Id: Id0deefd3c1b07f569585600a583ba4e04f8f7be1
Reviewed-on: https://go-review.googlesource.com/c/build/+/444695
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2022-10-27 21:25:15 +00:00
Dmitri Shuralyov 2ca6ef6898 internal/relui: remove workflowParams
It's unused as of CL 422599.

For golang/go#54240.

Change-Id: I53628d167f0da8ae0bd255be230def3e8905aab4
Reviewed-on: https://go-review.googlesource.com/c/build/+/443059
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Jenny Rakoczy <jenny@golang.org>
2022-10-18 15:07:06 +00:00
Alex Rakoczy 6b41ff9f97 internal/relui: add delete schedules button
Add a button to permanently delete a workflow schedule.

For golang/go#54476

Change-Id: I14d5ba66e7ce609dfea97af9ecd2aa6173d03524
Reviewed-on: https://go-review.googlesource.com/c/build/+/432405
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Auto-Submit: Jenny Rakoczy <jenny@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-09-22 17:52:21 +00:00
Alex Rakoczy 1143a321bc internal/relui: show schedules on workflow lists
Add a small table to show scheduled workflows, including the last and
next time they will run.

For golang/go#54476

Change-Id: I82a7e34f58ff77eb59df330f52564aff6b6c581f
Reviewed-on: https://go-review.googlesource.com/c/build/+/432404
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Jenny Rakoczy <jenny@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2022-09-22 17:52:07 +00:00
Alex Rakoczy 15ca61f40b internal/relui: create cron-based workflow schedules
Adds functionality to schedule Workflows using a standard cron-spec.
Updates resume to skip RunOnce schedules that are in the past.

For golang/go#54476

Change-Id: I30f5bb45353a88ca9398b8787ee7ea44ae96d7e1
Reviewed-on: https://go-review.googlesource.com/c/build/+/432397
Auto-Submit: Jenny Rakoczy <jenny@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
2022-09-22 17:52:05 +00:00
Alex Rakoczy 6a129b0480 internal/relui: add ability to schedule workflows
This adds a new field on workflow creation to schedule the creation of a
workflow. Only immediate running and a single future-date schedule are
supported, but the vast majority of the implementation for duration
(every N minutes) and cron-based schedules is included.

Schedules are resumed from the database on start-up.

Future CLs will include support for the remaining schedule types, a UI
for viewing and deleting schedules, archiving defunct schedules, and
notifications for scheduled workflow failures.

For golang/go#54476

Change-Id: Ica6a9df85087657aeabd85dae7973487d648dd59
Reviewed-on: https://go-review.googlesource.com/c/build/+/429695
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Auto-Submit: Jenny Rakoczy <jenny@golang.org>
2022-09-22 17:47:50 +00:00
Alex Rakoczy 04792cf61d internal/relui: fix side-navigation highlight
The NameParam field was left unset accidentally when adding Others in CL
423043.

Updates golang/go#53859

Change-Id: Ibf3253273dda94d3308e6bb8fe1a09377bd65162
Reviewed-on: https://go-review.googlesource.com/c/build/+/425934
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
Auto-Submit: Jenny Rakoczy <jenny@golang.org>
2022-08-26 16:33:26 +00:00
Dmitri Shuralyov 84246809e8 cmd/relui: add pre-announcement workflows
These workflows do the work of filling in a pre-announcement template
with the user-provided values, taking ownership over email formatting,
and sending it to the right mailing lists.

Add a new parameter type to relui for convenience of entering a date.
It takes advantage of the <input type="date"> element available in
modern browsers.

Also add a select parameter type that allows selecting from a known
set of options, to avoid needing to type them manually.

Fixes golang/go#54063.

Change-Id: I041c2659db6bd384f3850b2df3bd31c8b096579d
Reviewed-on: https://go-review.googlesource.com/c/build/+/425196
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-08-25 18:01:46 +00:00
Alex Rakoczy 761bec5c19 internal/relui: show old workflows separately
This changes the left-hand navigation to show old workflows that we no
longer have runnable definitions for in a list of "Others".

Fixes golang/go#53859

Change-Id: I0c9a7a97347d149ee3c6e86d3e2d62dc05719a18
Reviewed-on: https://go-review.googlesource.com/c/build/+/423043
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-16 13:35:48 +00:00
Alex Rakoczy 595e59a61c internal/relui: add filtering and navigation to homepage
This adds a site-wide navigation bar which filters the workflows visible
on the homepage of relui. The navigation bar highlights the current
filter, or current workflow type being viewed on the show page.

For golang/go#53859

Change-Id: Ibcc9de343872110068bc6271c64e65e04c653be1
Reviewed-on: https://go-review.googlesource.com/c/build/+/423174
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Jenny Rakoczy <jenny@golang.org>
2022-08-11 21:23:01 +00:00
Heschi Kreinick ac2bd83359 internal/workflow: add native retry support
Waiting for the workflow to stop before retrying is very annoying. Add
support for retrying while the workflow runs.

Previously, the workflow stopped when no more tasks were runnable. This
raised difficult questions: does the workflow accept retry commands
while it's stopped? Should there be a way to force it to do at least one
task before giving up? Rather than deal with those, change the behavior:
the Run function now returns only on completion or context cancelation.
Update the tests to match. As far as I know, relui itself doesn't care.

Note, though: there is now no way to resume a stopped workflow. I don't
think there's much reason for us to stop them now so I don't think it's
a big problem?

Fixes golang/go#54304.

Change-Id: I4de56a6c50d71dddf0eaafce2e9c135c65e4cfec
Reviewed-on: https://go-review.googlesource.com/c/build/+/422098
Reviewed-by: Jenny Rakoczy <jenny@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
2022-08-11 21:17:35 +00:00
Alex Rakoczy dd76e044b5 internal/relui: fix layout for large parameters
Releases with a lot of security notes were difficult to read in the
interface. This change unifies params and output presentation for
workflows.

Fixes golang/go#54240
Updates golang/go#53382

Change-Id: I4602478aa245e509f305ed478898c3492bc6561b
Reviewed-on: https://go-review.googlesource.com/c/build/+/422599
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Jenny Rakoczy <jenny@golang.org>
2022-08-11 17:24:04 +00:00
Alex Rakoczy 25288d7720 cmd/relui,internal/relui: add database metrics
Add database middleware that records duration of queries, labeled by the
query name.

Change-Id: I01cd28356d8848a447edf8789f1932610aec7ccc
Reviewed-on: https://go-review.googlesource.com/c/build/+/422337
Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
Auto-Submit: Jenny Rakoczy <jenny@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-10 18:17:38 +00:00
Alex Rakoczy 2888d64174 cmd/relui,internal/relui: add HTTP server metrics
This adds metrics to all HTTP routes in relui. Similar to other
x/build services, these metrics are also available at /metrics.

Change-Id: Ibbe5e98965901787277251f48c80784bce225037
Reviewed-on: https://go-review.googlesource.com/c/build/+/422336
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Jenny Rakoczy <jenny@golang.org>
2022-08-10 17:21:39 +00:00
Dmitri Shuralyov 61de893eb5 internal/relui: display parameters, even if their type isn't string
Some parameters have a type other than string, so unmarshaling them into
a map with string value type would fail. Prefer to make such problems
visible by returning any non-nil errors from workflowParams to caller.

For now, display parameter values in their JSON form, since it's simple,
quite human readable, and has a nice side-effect of turning any leading
or trailing spaces apparent. (In the future we can consider investing
into a more elaborate parameter layout, as needed.)

Fixes golang/go#54240.

Change-Id: I2a6048e5b5b0d42257b6088c8ba01b26216a3c39
Reviewed-on: https://go-review.googlesource.com/c/build/+/422554
Reviewed-by: Jenny Rakoczy <jenny@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-10 14:24:34 +00:00
Alex Rakoczy d03101a05c internal/relui: add workflow details page and new list
Create a workflow details page, and a corresponding new list view for
all workflows. This makes the homepage dramatically easier to scan.
Also, having a page per workflow will make it easier to refresh and know
which workflow you are looking at.

For golang/go#53859

Change-Id: Ic67652345fb1ef3fb2cbda0fc05529959c9b4d77
Reviewed-on: https://go-review.googlesource.com/c/build/+/420808
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Jenny Rakoczy <jenny@golang.org>
2022-08-09 20:28:02 +00:00
Heschi Kreinick da77ce567a internal/workflow: generify
Finding out about type errors at runtime is no fun, and precisely the
kind of things generics are intended to prevent. Use them for the
workflow package. It should be impossible to make a type error now.

Due to limitations of Go generics, all methods become functions, many of
which take the Definition as their first argument. Task and Action
functions have to specify their arity.

To compensate for the increase in verbosity, import the package as 'wf' in a
few places. I might fully rename it later.

The implementation doesn't change much. The new Meta interfaces make it
possible to work with heterogenous Parameters and Values. I also took
advantage of knowing task arities to make After return a different type,
so that it's also a compilation error to forget to call it.

The way Params work is a little clunky; having to specify the type both
as a type parameter and as the ParamType struct is redundant. Room for
improvement there.

Change-Id: I1ed957ef441099d00842135243eb2eb9bfda705d
Reviewed-on: https://go-review.googlesource.com/c/build/+/411198
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Jenny Rakoczy <jenny@golang.org>
2022-08-02 21:39:18 +00:00
Heschi Kreinick e3899ba4fa internal/relui: persist retry-related fields
Store the retry count in the database and test that it all works.

I removed the definition argument from start; we can read it just as
well from the holder, and it makes it harder to forget to register it.

For golang/go#53886.

Change-Id: I4c3d48a96bae87c31629f76067e75ccd524381b0
Reviewed-on: https://go-review.googlesource.com/c/build/+/417590
Reviewed-by: Jenny Rakoczy <jenny@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Heschi Kreinick <heschi@google.com>
2022-07-19 17:27:28 +00:00
Alex Rakoczy 876116a7e6 internal: track starting of tasks
Update exported TaskState to include a "started" boolean. Record the
task started state in the database and show a different icon in the UI.

Tracking started enables several improvements. This change improves
resuming of workflows to mark tasks that started without finishing as
incomplete. This will prevent workflows resuming from an unknown state,
where a task may not be safe to resume by default, such as sending a
public announcement.

Also, this change makes minor tweaks to the UI for task ordering,
putting the most recently updated tasks at the top of the list. It also
fixes a minor but confusing bug around retrying "approval" tasks.

Updates golang/go#53382

Change-Id: Icff1c0df541a6e11a7bb0ab55beef60cd18a074a
Reviewed-on: https://go-review.googlesource.com/c/build/+/417221
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
Auto-Submit: Jenny Rakoczy <jenny@golang.org>
2022-07-13 20:23:51 +00:00
Alex Rakoczy de76fbed15 internal/relui: format workflow outputs
This change adds UI treatment for showing workflow outputs. Previously,
it was just rendered as a string. With this change, there are custom UI
elements for different output types.

For golang/go#53382

Change-Id: Ia0f875d8b3149652216ca2b82ba13a8ecd342eb7
Reviewed-on: https://go-review.googlesource.com/c/build/+/416222
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Jenny Rakoczy <jenny@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Jenny Rakoczy <jenny@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2022-07-12 15:27:36 +00:00
Alex Rakoczy 3ceb9f4e34 internal/relui: render more output types
Add support for slices and numbers when rendering outputs. Stop
handling artifact slices explicitly in resultDetail in order to support
more slice types.

For golang/go#53382

Change-Id: If16f8f4240e96a1f2f756743236134f8f637b079
Reviewed-on: https://go-review.googlesource.com/c/build/+/413582
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Alex Rakoczy <alex@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Alex Rakoczy <alex@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2022-06-23 21:33:49 +00:00
Alex Rakoczy 900dd3580d internal/relui: add approved_at to tasks
This change adds an approved_at column to the tasks table for checking
task approval, replacing the flimsy log-based determination for whether
a task has been approved.

Updates golang/go#53295

Change-Id: I3c2bd73134dc4a81e13c328aa085065a7b24a5d5
Reviewed-on: https://go-review.googlesource.com/c/build/+/413427
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Alex Rakoczy <alex@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Alex Rakoczy <alex@golang.org>
2022-06-22 16:52:16 +00:00
Alex Rakoczy bcadfe3dc1 internal/relui: improve task result formatting
Format most result types, somewhat. This implementation should be
reusable when formatting workflow outputs in the future.

For golang/go#53382

Change-Id: I4734cdefff85a37cb1047a6689411139f2ebfbc1
Reviewed-on: https://go-review.googlesource.com/c/build/+/412677
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Alex Rakoczy <alex@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-06-21 19:26:30 +00:00
Alex Rakoczy 2838fbb254 internal/relui: simplify rendering of homepage
This refactors the homepage rendering to use a nested template for each
task row. This will help simplify the template as we add more complex
layout to the outputs of workflows and tasks.

Updates golang/go#51797
Updates golang/go#40279
For golang/go#53382

Change-Id: I85a86b82bdc79c7fb4e837d884af922c7028295d
Reviewed-on: https://go-review.googlesource.com/c/build/+/412176
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Alex Rakoczy <alex@golang.org>
2022-06-15 18:16:44 +00:00
Alex Rakoczy fc56509fc5 internal/relui: record resume errors on workflows
When a workflow fails to resume from an error, record the error on the
workflow and mark it as finished. This typically happens when resuming a
task for an un-resumable workflow, either because the definition changed
or is missing.

With this change, the retryTaskHandler can simply redirect, as the user
will see the error on the workflow itself.

Change-Id: I934dc8b24983716820a3b19160052b173d3b519e
Reviewed-on: https://go-review.googlesource.com/c/build/+/412134
Run-TryBot: Alex Rakoczy <alex@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-06-14 19:49:09 +00:00
Alex Rakoczy f212a156ec internal/relui: add stop button to relui
Adds a stop button which cancels the workflow's context. Also, style the
buttons on the workflow page a little nicer.

Removes a guard around workflow cancellation that doesn't appear to be
strictly necessary. We still want to mark the task failed in this (and
probably most) scenarios.

Updates test docker target to include workflow package.

Updates golang/go#53317

Change-Id: Ib5df013d23345e9ebe0dc56c10ad27843efe3448
Reviewed-on: https://go-review.googlesource.com/c/build/+/411062
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Alex Rakoczy <alex@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Alex Rakoczy <alex@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-06-14 19:43:26 +00:00
Alex Rakoczy 9a59fdc1e4 internal/relui: simplify retryTask transaction
This is a minor refactoring to use BeginFunc to create, cancel, and
commit our transaction.

Change-Id: I7ea8327276b4d413f7d21de9967fd138ec4e05ae
Reviewed-on: https://go-review.googlesource.com/c/build/+/412135
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Alex Rakoczy <alex@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2022-06-14 18:26:26 +00:00
Alex Rakoczy 0f1fd27b5f internal/relui: add approval step to build workflow
This change adds a manual approval task to the build before making any
user-facing changes. It uses a sentinel log entry in the approval task's
log in order to verify whether a human has approved the workflow to
continue.

Ideally, we'd change the data model to allow for this, perhaps by adding
a specific type of task, or a special state column, but this should be
sufficient for testing the beta release. Also, there should be some
shared definition of approval, but leaving that for a follow-up CL.

Updates golang/go#53295
Updates golang/go#51797

Change-Id: Iacd8c8a3ba59f9e3a343916dbe1a3a8ba59ce80c
Reviewed-on: https://go-review.googlesource.com/c/build/+/411199
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Alex Rakoczy <alex@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2022-06-09 14:54:55 +00:00
Alex Rakoczy 96242bf858 internal/relui: log previous state when retrying tasks
This logs the task state and workflow state when retrying a task for a
workflow, so as not to lose history.

Eventually, we should store the full history for workflows and tasks.

Moves the transaction rollback to a defer, which is safe even if commit
or rollback is called first.

Fixes golang/go#53165

Change-Id: I6a9c42beb9352656b044dbe6e5adf6493d32873d
Reviewed-on: https://go-review.googlesource.com/c/build/+/411194
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Alex Rakoczy <alex@golang.org>
2022-06-09 02:46:32 +00:00
Alex Rakoczy e5336422d9 internal/relui: add manual retry button to failed tasks
Sometimes, tasks fail. This change adds an ability for the user to retry
specific tasks in a workflow. Retrying a task will reset the state of
the task, clear the state of the workflow, and restart the workflow to
retry the task.

This also fixes an issue resuming pending tasks that have no output to
unmarshal.

Adds httprouter for routing nested parameter routes.

Updates golang/go#53165

Change-Id: I9a899d1e6e0d5d9511648b478148a665459aae72
Reviewed-on: https://go-review.googlesource.com/c/build/+/411074
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Alex Rakoczy <alex@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2022-06-08 16:55:25 +00:00
Dmitri Shuralyov fb638e1ffb internal/relui, internal/workflow: add parameter type support
Previously, it was only possible to create workflow parameters
with the implicit "string" type hard-coded. Some workflows we're
creating either require or will benefit from more flexibility
in parameter types (slices of strings, single-line vs multi-line
strings, and so on). It was also not yet possible to associate
metadata to parameters (such as documentation, example values).

This change implements that flexibility for workflow parameters,
and uses it to better document the existing Twitter workflows.
The next change will add a workflow that uses new slice types.

For simplicity and clarity reasons, all parameter information
is contained in one place in the workflow.Parameter struct,
including some fields that control the HTML presentation of
said parameters, instead of trying to factor out HTML bits
into the relui package and creating a bridge between the two.

Also type check in more stages of the workflow processing.

For golang/go#47405.
Fixes golang/go#51191.

Change-Id: Ia805b3b355e65fcbf2397ad21800da448ccb826a
Reviewed-on: https://go-review.googlesource.com/c/build/+/404454
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-05-11 14:29:43 +00:00
Dmitri Shuralyov b002d65edb cmd/relui: redirect to base URL after creating a workflow
Also update the suggested command in README to serve over HTTP locally.
(This is needed now that relui uses x/build/internal/https for serving.)

Fixes golang/go#51466.

Change-Id: Iaf0e0b62f3cd6015c793a0a16cff895be5ea42b4
Reviewed-on: https://go-review.googlesource.com/c/build/+/401796
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-22 21:07:38 +00:00
Dmitri Shuralyov 00c8f6efb2 cmd/relui: register tweet workflow definitions
Define an external service configuration for tasks. Its starting point
has tasks running in dry-run mode. Adjust the site header accordingly,
so humans can refer to it to know what to expect from workflows.

Use this external service configuration to register twitter workflows.

After go.dev/issue/51122 is resolved and cmd/relui in the production
environment has access to secret credentials, we can flip the switch
so it starts posting tweets at twitter.com/golang.

Fixes golang/go#47402.

Change-Id: Icccd94518fd7bf39052f67ba1be4db3e42d3d686
Reviewed-on: https://go-review.googlesource.com/c/build/+/385294
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
2022-02-16 19:55:28 +00:00
Dmitri Shuralyov 3831dabe9d internal/relui: adjust new workflow page HTML and CSS
Make its presentation more friendly for entering single-line string
parameters of longer lengths, as will happen in the tweet workflows.

Add a confirmation prompt when submitting a workflow, to avoid the risk
that a single rogue misclick may otherwise immediately submit and start
the workflow (similar to how releasebot asks for Y/n confirmation right
before taking important action).

Add a new server parameter that allows controlling the site header
programmatically. This will be used in the next CL in the stack.

Change-Id: Icede1454c7b07341d139d953b9c4ea40d4f1c369
Reviewed-on: https://go-review.googlesource.com/c/build/+/385615
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
2022-02-16 19:54:28 +00:00
Dmitri Shuralyov e9b946c2ca internal/relui: add DefinitionHolder, remove package-scoped definitions
Up to this point, workflows have been simple and did not have external
configuration, so it was possible to define all of them them at package
initialization time in a package-scoped map. This doesn't scale well to
upcoming workflows that involve external service configuration being
provided to relui.

Make the trivial refactor to wrap the definitions map in a holder type
(similar to http.ServeMux) to get rid of the package-scoped variable,
without trying to make more invasive changes at this point.

A Worker now accepts a DefinitionHolder during creation, and uses it.
A Server already has a Worker, and uses its DefinitionHolder as needed.

Maybe in the future it'll make sense to merge the functionality of
DefinitionHolder into Worker, and it can stop being its own type.

Change-Id: Ia1539e0621db83c8fc5959b39445090d1f0034e1
Reviewed-on: https://go-review.googlesource.com/c/build/+/385614
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
2022-02-16 19:54:17 +00:00
Alexander Rakoczy ecf31e90db internal/relui: correct baseURL behavior
Nested http.ServeMux should be configured with the appropriate
trailing slash, but we should not use the trailing slash when using
http.StripPrefix. This simplifies ServeHTTP slightly, and corrects
behavior when handling nested mux.

For golang/go#47401

Change-Id: I2e6ab792e210a500025b8b07a28e3db67696397d
Reviewed-on: https://go-review.googlesource.com/c/build/+/363977
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-11-15 19:43:01 +00:00
Alexander Rakoczy 9076251d22 cmd/relui,internal/relui: serve from sub-paths
This change allows relui to correctly serve from a path, like
build.golang.org/releases. It adds a base-url flag which is used to
prefix all paths referenced in the application.

For golang/go#47401

Change-Id: Ib8f6fe429591ceabfaf0f419e5258a677b375ff8
Reviewed-on: https://go-review.googlesource.com/c/build/+/363975
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-11-15 18:05:32 +00:00
Alexander Rakoczy 19d8baf2fd cmd/relui,internal/relui: resume workflows
Resume all unfinished workflows on relui startup. Moves most workflow
running logic to a new Worker type.

This also adds a couple new columns to the workflow to record output and
whether a workflow has completed. In order to actually resume workflows,
we'll have to either add functionality to unmark them as "finished"
(restarting manually, which seems reasonable), or not marking a workflow
finished if it terminates with context.Cancelled.

For golang/go#47401

Change-Id: I3e0ed021d7a47fb125f1034df83dc3c6d95887f8
Reviewed-on: https://go-review.googlesource.com/c/build/+/353553
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-10-20 19:39:46 +00:00
Alexander Rakoczy 79bf688aec internal/relui: format task results on home
Render the workflow task results as a table using a css grid layout,
rather than a table, to allow for us to expand rows to show logs and
eventually other results without using javascript.

For golang/go#47401

Change-Id: I1ab1bb3a1b87d2e864b769eeb232336af98a306d
Reviewed-on: https://go-review.googlesource.com/c/build/+/353349
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-10-01 05:16:22 +00:00
Alexander Rakoczy 71603fa585 internal/relui: persist logs to database
Save log output from workflow tasks in the database. Render logs in the
UI alongside each task. This will almost certainly be a performance
problem if we have many logs until we implement pagination.

For golang/go#47401

Change-Id: I4ddc3d7845e4559cc636b3b7972e0adefe9fcec4
Reviewed-on: https://go-review.googlesource.com/c/build/+/353170
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-09-30 18:32:41 +00:00
Alexander Rakoczy 4c1ebefe74 cmd/relui,internal/relui: cleanup go:build comments
These are no longer necessary as embed is in all supported verions of
Go.

For golang/go#47401

Change-Id: I3b128a28028d0ca15088d71202984c8b8ef4469c
Reviewed-on: https://go-review.googlesource.com/c/build/+/352569
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-09-28 16:34:42 +00:00
Alexander Rakoczy 88204a4ef3 internal/relui: create workflows from parameters
This change updates the UI and creation endpoint to use the
parameters specified for the workflow.

For golang/go#47401

Change-Id: I46623a17161806c39cad98853e996f0fee5a0305
Reviewed-on: https://go-review.googlesource.com/c/build/+/352169
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-09-28 16:34:35 +00:00
Alexander Rakoczy cbb5112b70 cmd/relui,internal/relui: create and run workflows
Create and run workflows, storing state in a postgres database. Generate
database queries using sqlc, which allows the author to have complete
control over the query structure.

The basic structure of creating and running a workflow works, but
there is still much work to do in terms of selecting which workflow to
create, recording logs, maintaining a list of currently running
workflows, and recording workflow-level results.

Task persistence needs to be updated, as the current implementation
cannot be resumed due to losing the type during marshalling.

For golang/go#47401

Change-Id: I9ccddf117d023f70568de655c482820a1152d9cb
Reviewed-on: https://go-review.googlesource.com/c/build/+/350449
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-09-28 15:24:53 +00:00
Alexander Rakoczy d0268397ad cmd/relui,internal/relui: add database migrations
This change adds golang-migrate to manage database migrations for relui.
relui will run all database migrations on start, and return an error for
any failures.

The current developer tooling is managed through the Makefile. For ease
of use, common tasks will be moved to a script in a future CL.

For golang/go#47401

Change-Id: I678c3b21d7e89d7d5a66305f66075f02cc2f3284
Reviewed-on: https://go-review.googlesource.com/c/build/+/349951
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-09-17 16:15:38 +00:00
Alexander Rakoczy 3734795929 cmd/relui: move implementation to internal/relui
This makes it explicit that the code is not intended to be used outside
this module.

For golang/go#47401

Change-Id: I6c7ca7402b451dbcfac39e504d5fc767811c973d
Reviewed-on: https://go-review.googlesource.com/c/build/+/347292
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2021-09-02 21:31:40 +00:00