mozilla
/
docker-etl
зеркало из https://github.com/mozilla/docker-etl.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
303 коммитов 73 Ветки 0 Теги 6,3 MiB
Граф коммитов

303 Коммитов

Автор SHA1 Сообщение Дата
Anthony Miyaguchi a5edb9de87 Fix missing folder when scraping 2021-01-29 11:40:23 -08:00
Anthony Miyaguchi e0def3accf Add docker-context to ci_workflow templates 2021-01-29 11:17:24 -08:00
Anthony Miyaguchi 97c1f233d1 Add docker-context to etl-graph job 2021-01-29 11:16:56 -08:00
Anthony Miyaguchi 5c304ea698
Merge pull request #11 from mozilla/etl-graph 
Import etl-graph as subtree instead of git module
 2021-01-29 10:52:55 -08:00
Anthony Miyaguchi 91acdf0c8f Remove submodule references 2021-01-29 10:33:14 -08:00
Anthony Miyaguchi c84351c06a Update etl-graph configuration to follow vanilla template 2021-01-28 16:04:52 -08:00
Anthony Miyaguchi 5819d47872 Add 'jobs/etl-graph/' from commit '11166eced70f660447d23d5ab57380178ed759bf' 
git-subtree-dir: jobs/etl-graph
git-subtree-mainline: 148afe6427
git-subtree-split: 11166eced7
 2021-01-28 16:03:47 -08:00
Anthony Miyaguchi 148afe6427 Remove submodules 2021-01-28 16:03:15 -08:00
Anthony Miyaguchi 11166eced7
Merge pull request #12 from mozilla/fixup 
Specify both path and docker-context
 2021-01-28 15:42:02 -08:00
Anthony Miyaguchi 18f51948e6 Specify both path and docker-context 2021-01-28 14:59:57 -08:00
Anthony Miyaguchi a52fbf64e3
Bump etl-graph submodule for workspace location mishap (#7) 
* Add changes to a fixup branch of etl-graph

* Update etl-graph submodule for updated ci
 2021-01-28 14:42:21 -08:00
Anthony Miyaguchi ff7beed0fc
Merge pull request #11 from mozilla/fixup 
Update workspace attach location
 2021-01-28 11:16:38 -08:00
Anthony Miyaguchi 9f796fd4a7 Update workspace attach location 2021-01-27 14:27:00 -08:00
Anthony Miyaguchi e5933c71d7
Bump etl-graph for ci related fixes (#6) 
* Bump etl-graph for ci related fixes

* Add workspace related changes

* Bump circleci/gcp-gcr orb to 0.12.0
 2021-01-27 14:18:16 -08:00
Anthony Miyaguchi c1e54b5121
Add dependabot support for git submodules (#5) 2021-01-27 14:15:46 -08:00
Anthony Miyaguchi d545ee5699
Merge pull request #10 from mozilla/docker-etl 
Update docker-etl fragments with workspaces
 2021-01-27 13:21:58 -08:00
Anthony Miyaguchi caa4e4dd60 Update docker-etl fragments with workspaces 2021-01-27 11:03:56 -08:00
Anthony Miyaguchi 5306926d13
Add etl-graph as a job via submodule (#4) 
* Add etl-graph as a submodule job

* Update template to checkout submodules for verification checks

* Update to etl-graph#docker-etl branch to check CI

* Add generated ci configuration

* Add section in README about submodule workflow

* Add pull_request_template with small checklist

* Update subproject to main
 2021-01-27 10:35:06 -08:00
Anthony Miyaguchi d566cdb572
Merge pull request #9 from mozilla/docker-etl 
Add ci files for docker-etl
 2021-01-27 10:28:02 -08:00
Anthony Miyaguchi 7161a7ad8d Add submodule update call 2021-01-26 11:54:16 -08:00
Anthony Miyaguchi 14337b8fee Add ci files for docker-etl 2021-01-26 11:50:07 -08:00
Anthony Miyaguchi 8c7adfc29b
Merge pull request #8 from mozilla/docker 
Add configuration for docker and job scheduling
 2021-01-25 14:43:00 -08:00
Anthony Miyaguchi 59a48f1aaa Update readme for deploy scripts; add screenshot 2021-01-25 10:26:35 -08:00
Anthony Miyaguchi 03ae3acef9 Decouple updating data and site assets 2021-01-22 14:00:58 -08:00
Anthony Miyaguchi b675f1da9d Fix whitespacing 2021-01-21 15:38:47 -08:00
Anthony Miyaguchi 9cdbe7d97c Update frontmatter for last updated and new repo location 2021-01-21 15:38:22 -08:00
Anthony Miyaguchi 7c0196c7ac Add scripts for scraping and deploying with docker entrypoint 2021-01-21 15:23:56 -08:00
Anthony Miyaguchi 8b426583f0 Add configuration to docker-compose for testing gcloud commands 2021-01-21 14:14:34 -08:00
Anthony Miyaguchi 99a3a66bfa Add dockerfile and compose configuration 2021-01-21 12:26:49 -08:00
Anthony Miyaguchi 4ce2348b88
Merge pull request #7 from acmiyaguchi/query-nodes 
Update queries on the query logs and update the UI
 2021-01-19 16:32:17 -08:00
Anthony Miyaguchi 2d68fd9142 Update documentation 2021-01-19 16:26:55 -08:00
Anthony Miyaguchi a43ff62b1a Add network statistics on indexing 2021-01-19 16:20:48 -08:00
Anthony Miyaguchi b273ec76cb Add node information to the explorer page 2021-01-19 14:43:43 -08:00
Anthony Miyaguchi fab7fc4a92 Update to use only the query logs 2021-01-19 13:14:05 -08:00
Anthony Miyaguchi 7d330bbab0 Update queries to include weights and attributes on edges 2021-01-15 19:31:38 -08:00
Anthony Miyaguchi f335e844da Update shared-prod edgelist to query log dependencies 2021-01-15 15:41:30 -08:00
Anthony Miyaguchi 2505802446
Merge pull request #6 from acmiyaguchi/organize-scraping 
Refactor scraping
 2021-01-15 15:23:00 -08:00
Anthony Miyaguchi 97b05e8084 Use json instead of ndjson for queries 2021-01-15 15:12:20 -08:00
Anthony Miyaguchi a8e081591d Move commands into a separate file 2021-01-15 13:29:30 -08:00
Anthony Miyaguchi d4cc11c099
Merge pull request #5 from acmiyaguchi/summary-page 
Add routing using page.js and summaries
 2021-01-15 13:22:09 -08:00
Anthony Miyaguchi ed810120da Add build as part of the deploy process 2021-01-15 13:18:54 -08:00
Anthony Miyaguchi 8ef8936d0d Add manifest and artifacts to the page 2021-01-15 12:11:24 -08:00
Anthony Miyaguchi c3dea76404 Add a statistics page 2021-01-15 11:46:38 -08:00
Anthony Miyaguchi 013889ee7d Add routing and a navbar for multiple pages 2021-01-15 10:47:15 -08:00
Anthony Miyaguchi 9c00807a26 Fix typos and reorganize text 2021-01-15 10:31:32 -08:00
Anthony Miyaguchi a16a1dac98 Reorganize code with components and routes 2021-01-15 10:25:37 -08:00
Anthony Miyaguchi b2701ba2f2
Merge pull request #3 from acmiyaguchi/quality-of-life 
Improve usability of the web UI
 2021-01-14 17:48:03 -08:00
Anthony Miyaguchi 78f12f3ed5 Add bootstrap 2021-01-14 17:39:45 -08:00
Anthony Miyaguchi 0425a4c422 Update deploy script and fix bundles to be relative 2021-01-14 17:19:45 -08:00
Anthony Miyaguchi 34b49a085e Add text to sections 2021-01-14 17:11:42 -08:00