mozilla
/
gcp-ingestion
зеркало из https://github.com/mozilla/gcp-ingestion.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
2 020 коммитов 48 Ветки 14 Теги 40 MiB
Граф коммитов

16 Коммитов

Автор SHA1 Сообщение Дата
whd 9100a2d529
Remove pioneer/rally special cases (#2621) 
Co-authored-by: akkomar <akkomar@users.noreply.github.com>
 2024-10-16 19:26:16 +02:00
Daniel Thorn e9edfd30c0
Simplify proxy handling (#2271) 
and remove logic for x-pipeline-proxy

Co-authored-by: whd <whd@users.noreply.github.com>
 2023-06-26 17:19:02 +00:00
Jeff Klukas 7815c1264c
Bug 1742172 Add SanitizeAttributes for timestamp granularity (#1896) 
* Bug 1742172 Add SanitizeAttributes for timestamp granularity

See https://bugzilla.mozilla.org/show_bug.cgi?id=1742172 for more on the design here.
 2021-11-22 16:37:22 -05:00
Daniel Thorn fa98ac0c8f
Remove support for Dataflow classic templates (#1789) 2021-08-17 12:02:23 -07:00
whd 93ff621946
Rename default branch (#1636) 2021-04-06 20:58:49 +00:00
Jeff Klukas e1c97fdad5
Remove Memorystore and AET from architecture overview (#1610) 
Both of these features have been removed from the infrastructure.
 2021-03-18 14:31:40 -04:00
Jeff Klukas 2c494f48eb
Update docs about deduplication (#1577) 
* Update docs about deduplication

To match reality of https://bugzilla.mozilla.org/show_bug.cgi?id=1694764
and allow for clear communication to data users about the change.

A separate follow-up will be to remove code from the repository that
handles interaction with Redis.

* Apply suggestions from code review

Co-authored-by: Daniel Thorn <dthorn@mozilla.com>

Co-authored-by: Daniel Thorn <dthorn@mozilla.com>
 2021-02-25 15:57:37 -05:00
Daniel Thorn bc7854b0b5
Format markdown with prettier and enforce in CI (#1399) 2020-09-10 13:04:04 -07:00
Jeff Klukas e3f2d1c097 Update architecture overview 
This corrects info on how AET support is deployed, updates some outdated info
(sinks are now all Kube jobs rather than Dataflow), and now mentions the
concept of pipeline families.
 2020-06-12 10:47:47 -04:00
Jeff Klukas e5b390c7b3 Update docs 2020-06-10 10:46:09 -04:00
Jeff Klukas 32d6c45026 Implement proposal for flagging unwanted data 
See [the proposal Google
doc](https://docs.google.com/document/d/1VWI-PsD9tFaKVWp5hC75lu65lrtgDEBhO4S7XYvHkqQ/edit#)

We refactor some code here, fill in some documentation,
add a new exception type, and also add a single new case of unwanted data,
fixing https://bugzilla.mozilla.org/show_bug.cgi?id=1612933
 2020-02-27 15:49:29 -05:00
Jeff Klukas 5e345c193d Specify 413 2020-02-06 13:53:27 -05:00
Jeff Klukas ae09d47b68 Limit decoded payload size to 8 MB 
Closes #776 and also addresses documentation request in
https://bugzilla.mozilla.org/show_bug.cgi?id=1605503
 2020-02-06 13:53:27 -05:00
Jeff Klukas 30d4707d1d Update docs to match current state 
This updates the docs to better reflect the names we actually use for
components, adapt to how most of our sinks are now using `ingestion-edge`,
etc.
 2020-02-05 13:39:58 -05:00
Anna Scholtz cad125dab8 Update instructions to download GeoLite2-City dataset and fix markdown format 2020-01-09 14:01:38 -08:00
William Lachance 85863eef1e
Create an mkdocs-based site (#732) 
This substantially reorganizes the documentation as an mkdocs site. Main
changes:

* All documentation is now browseable and searchable in a single site, with
  handy table of contents on the side of each section
* Top-level README significantly slimmed down (just pointing to docs site)
* READMEs inside individual components removed (moved to subdirectories inside
  docs/ folder, accessible via top-level in docs site)
 2019-08-12 17:06:19 -04:00