12d236e0a4
Bump maven-surefire-plugin from 2.22.2 to 3.0.0 ( #2311 )
2023-03-15 16:34:38 +00:00
6f387bb28c
Bump types-protobuf from 4.21.0.3 to 4.21.0.4 in /ingestion-edge ( #2267 )
2023-02-06 10:05:00 -08:00
99de1e99f7
Rotate docs SSH key ( #2240 )
2023-01-17 13:07:36 -08:00
d0f752a768
Update pip-tools to 6.12.1
2023-01-17 12:13:01 -08:00
28cf0e3c79
Bump psutil from 5.9.3 to 5.9.4 in /ingestion-edge ( #2196 )
2022-11-28 09:50:47 -08:00
5c2d6ef2db
Upgrade ingestion-edge to python 3.10 ( #2199 )
...
and upgrade dependencies dependabot was missing
2022-11-16 14:06:17 -08:00
1054e57bae
Update codecov to new uploader and enable carryforward ( #1909 )
2021-12-08 16:38:46 -05:00
2bb2e53457
Run all maven tests as one matrix and include ingestion-core ( #1811 )
2021-09-01 13:47:47 -07:00
cfc826cf65
Upgrade ingestion-sink to java 11 ( #1791 )
2021-08-23 13:28:54 -04:00
cda1b661fc
Use java 11 for dataflow flex templates ( #1778 )
2021-08-05 13:55:44 -07:00
2ad3e413ed
Don't rely on unmaintatined docker image for mdspell ( #1637 )
2021-04-08 09:38:47 -07:00
93ff621946
Rename default branch ( #1636 )
2021-04-06 20:58:49 +00:00
f96fb3b8c6
Update rendered docs in line with bigquery-etl ( #1611 )
...
Uses the `material` theme and some additional plugins.
2021-03-19 06:17:37 -07:00
329c0fa048
Pin pip-tools to 5.4 until dependabot updates ( #1521 )
2021-01-04 15:37:20 -08:00
a9fa876db4
Fix yaml type of ssh key fingerprints ( #1443 )
2020-10-15 11:28:07 -07:00
bc79a1d58d
Set docker version in CircleCI config ( #1435 )
...
This explicitly sets the Docker version. If we don't set the version,
CircleCI defaults to using 17.03.0 which is a version that CircleCI is
deprecating.
https://discuss.circleci.com/t/old-linux-machine-image-remote-docker-deprecation/37572
Co-authored-by: Daniel Thorn <dthorn@mozilla.com>
2020-10-07 12:24:34 -07:00
bc7854b0b5
Format markdown with prettier and enforce in CI ( #1399 )
2020-09-10 13:04:04 -07:00
927228944f
Collect test metadata in CI ( #1326 )
2020-07-29 12:17:16 -07:00
df58c97e8c
Bump ingestion-edge requirements except google-cloud-pubsub ( #1325 )
2020-07-22 14:43:26 -07:00
04d0ac74dd
Only build docs for master ( #1272 )
2020-04-28 11:08:08 -07:00
07c869fdd9
Skip ci jobs on gh-pages branch deployed by mkdocs ( #1270 )
...
There is no circleci config on this branch, so it will always fail.
2020-04-28 13:54:22 -04:00
0291bb2065
Upgrade to python 3.8 ( #1131 )
2020-02-18 19:47:22 +01:00
9efd40cebe
Use pip-compile instead of constraints.txt ( #1128 )
2020-02-13 00:03:58 +01:00
4b5ddc551a
Add /app/version.json to sink ( #1039 )
2019-12-17 12:48:01 -05:00
192fc05d87
Replace maven inter-module deps with copying src ( #887 )
...
* Replace maven inter-module deps with copying src
* Update docs/ingestion-beam/sink-job.md
Co-Authored-By: Jeff Klukas <jeff@klukas.net>
2019-10-02 15:09:28 -07:00
1bfd73c05a
Skip integration tests for forked PRs
2019-09-25 16:07:13 -04:00
92ba503c4d
Share code between ingestion-beam and ingestion-sink ( #762 )
2019-08-21 11:07:41 -07:00
8a0fcb1b51
Fix ingestion-sink errors and performance ( #750 )
2019-08-20 11:18:47 -07:00
6606311bfa
Implement Streaming BQ Sink for Kubernetes ( #738 )
...
* Implement Streaming BQ Sink for Kubernetes
* Update ingestion-sink/src/test/java/com/mozilla/telemetry/ingestion/SinkIntegrationTest.java
Co-Authored-By: Jeff Klukas <jeff@klukas.net>
2019-08-14 10:49:55 -07:00
85863eef1e
Create an mkdocs-based site ( #732 )
...
This substantially reorganizes the documentation as an mkdocs site. Main
changes:
* All documentation is now browseable and searchable in a single site, with
handy table of contents on the side of each section
* Top-level README significantly slimmed down (just pointing to docs site)
* READMEs inside individual components removed (moved to subdirectories inside
docs/ folder, accessible via top-level in docs site)
2019-08-12 17:06:19 -04:00
bd423640c7
Setup maven for multiple modules ( #655 )
2019-06-18 16:08:51 -07:00
68a514f156
Skip tests for unmodified products ( #600 )
...
* Skip tests for unmodified products
* address review
2019-05-10 09:16:31 -07:00
d2e76af1c7
Include geo db version in metadata
...
Fixes #537
2019-05-01 14:18:01 -04:00
1aab5e6ee9
Test edge without docker-compose for better dependency caching ( #444 )
2019-02-27 11:17:06 -08:00
bbad098643
Pull schemas from GCS rather than building into jar ( #405 )
...
Closes #398
This new approach is consistent with how we pull in the geo database and how
we run tests for it. Instead of pulling from GitHub at maven build time,
developers are instructed to run a script to pull down an archive locally.
Then, we rely on Beam's Filesystems library to transparently handle a local
or GCS path.
2019-01-30 15:32:40 -05:00
7b3de2dd1e
Configure maven memory settings
...
The ingestion-beam-integration job was crashing the VM on CircleCI.
Working theory is that it was the jacoco agent crashing due to insufficient
memory.
See https://stackoverflow.com/a/38925634/1260237
2019-01-17 09:22:27 -05:00
c7a6dead54
Maven memory settings
2019-01-08 13:22:56 -05:00
56c8da2e53
Pin maven image
2019-01-08 13:22:56 -05:00
163cc7438c
Move black to same location as other style checks ( #302 )
2018-12-21 14:01:27 -08:00
e5379a1f5f
Remove redundant restore_cache step
2018-11-27 18:42:26 -05:00
4ecc751c66
Add StorageIntegationTest for beam GCS -> GCS pipeline
2018-11-27 18:42:26 -05:00
f841e3f314
Consolidate beam integration tests to one CircleCI job
2018-11-26 08:08:55 -05:00
8145640d8e
Add PubsubIntegrationTest to ingestion-beam
...
Adds one test case each for exercising InputType.pubsub and OutputType.pubsub
Closes #207
2018-11-20 15:18:24 -05:00
3441d37739
Use same codecov flag for both ingestion-beam jobs
2018-11-14 11:52:23 -05:00
ca511228ab
Add BigQueryIntegrationTest that accesses BigQuery in GCP
2018-11-14 11:52:23 -05:00
bd7c122e68
Report code coverage for ingestion-edge
...
Closes #123
2018-11-08 08:34:41 -05:00
5599c5be97
Make ingestion-edge metadata headers configurable ( #163 )
...
fixes #13
2018-11-07 15:38:59 -08:00
a582f7b3b8
Change flag name for sending code coverage report
...
Apparently, dashes are not allowed
https://docs.codecov.io/docs/flags#section-how-to-create-flags
2018-11-06 14:28:42 -05:00
c68a42e860
Add code coverage reporting for ingestion-beam
...
Closes #124
2018-11-06 13:27:31 -05:00
7f8f130d87
Add environment to dockerhub push ( #151 )
2018-10-31 11:13:20 -07:00
dependabot[bot]
whd
Daniel Thorn
Jeff Klukas
Will Kahn-Greene
William Lachance
Jeff Klukas