microsoft
/
ghcrawler
зеркало из https://github.com/microsoft/ghcrawler.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
440 коммитов 15 Ветки 0 Теги 1,7 MiB
Граф коммитов

440 Коммитов

Автор SHA1 Сообщение Дата
Gene Hazan de09c0314e Merge remote-tracking branch 'origin/develop' into jm/fixTest 2017-04-07 13:21:04 -07:00
Gene Hazan 2851389fa6 Gh/deleted entities (#74) 
* Initial changes for deleted entities support. Correct repo test.

* Add tests

* Do not queue certain events in case of deletion

* Add placeholders for handling relation changes

* Relation handling for team and member events

* Initial code review

* update deletion handling and repo test

* Correct indentation

* Process repository only if TeamEvent contains repository payload

* Correct TeamEvent & deletion tests

* Try queueing repo for MemberEvent's added action which does not contain repository in the payload

* Decrease qlimit for one time job.

* Add history. Capture removed MemberEvent.

* Added storage count script. Added token arg.

* Final corrections

* Correct tests

* Add test

* More tweaks

* Mark review_comment deleted - could be via PullRequestReviewCommentEvent

* Add OrgBlockEvent

* Code cleanup

* Remove duplicate functions

* Crawl a repo when it is first created.

* Update MemberEvent handling

* Crawl team when it is first created

* Update fetch policy for relations.

* Correct linting warnings

* Correct merge

* Correct merge

* Move deleteAt marking out to crawler

* Correct context copying

* tweak processing outcome for delete

* Carry history through
 2017-04-07 11:34:16 -07:00
Gene Hazan 93723291ec Fix in memory test configuration 2017-04-07 11:30:55 -07:00
Jeff McAffer bc73b4bc16 Fix remaining tests 2017-04-06 16:51:00 -07:00
Jeff McAffer defd4f0da2 fix a mess of test errors 2017-04-06 15:53:11 -07:00
Gene Hazan 05cd6005b0 Merge pull request #80 from Microsoft/gh/processCount 
Replace iisnode.yml with web.config
 2017-04-06 15:28:35 -07:00
Gene Hazan d68e426fe6 Replace iisnode.yml with web.config 2017-04-06 15:22:15 -07:00
Jeff McAffer 1127dc212d Merge pull request #79 from Microsoft/jm/etagFix 
fix event type in etag issue
 2017-04-04 21:28:55 -07:00
Jeff McAffer 10b45402ff fix event type in etag issue 2017-04-04 21:25:39 -07:00
Jeff McAffer deb6604c4d Merge pull request #78 from Microsoft/jm/logRootRequest 
add root request to log meta object
 2017-04-04 20:28:26 -07:00
Jeff McAffer b145c960e2 add root request to log meta object 2017-04-04 20:18:25 -07:00
Gene Hazan 6a511e46b3 Merge pull request #76 from Microsoft/gh/urlUrnMapping 
Replace Redis URL & URN mapping store with Azure table
 2017-04-03 11:32:51 -07:00
Gene Hazan 138d4158d1 Replace Redis URL & URN mapping store with Azure table 2017-03-31 08:17:08 -07:00
Jeff McAffer b1dde26455 update name of dashboard 2017-03-23 10:30:45 -07:00
Gene Hazan b541ad5910 Merge pull request #75 from Microsoft/gh/perf 
Improve app service performance
 2017-03-22 10:06:04 -07:00
Gene Hazan 3a351ebba5 Improve app service performance 2017-03-21 15:23:44 -07:00
William Bartholomew 3b9aa4b7cd Put metabase data in a volume 2017-03-20 11:26:02 -07:00
William Bartholomew 8a5626064a Add missing copyright headers and remove test file 2017-03-17 13:46:50 -07:00
William Bartholomew acd96e1a95 Reduce log noise during NPM install 2017-03-16 10:15:48 -07:00
Gene Hazan 15ef285d73 Merge pull request #71 from Microsoft/gh/oneTimePopulateMapping 
Gh/one time populate mapping
 2017-03-16 08:53:51 -07:00
Gene Hazan 2b077a9397 Code review & logging update 2017-03-16 08:43:59 -07:00
Jeff McAffer 14237490ce trim ciab doc 2017-03-15 22:53:58 -07:00
Jeff McAffer e1c7486f9b add webhook config info in readme 2017-03-15 22:51:11 -07:00
Gene Hazan fbdf523128 Print error 2017-03-15 18:53:42 -07:00
Gene Hazan d0ade70694 Correct counter 2017-03-15 18:51:46 -07:00
Gene Hazan 0f48ba4263 Add one time table mapping job 2017-03-15 18:47:29 -07:00
William Bartholomew d31e9ffa6c Fixed incorrect storage API 2017-03-15 16:52:29 -07:00
William Bartholomew 86662dad8e Merge pull request #68 from Microsoft/willbar/mongodeadletter 
Add MongoDB support for deadletter refactor
 2017-03-15 16:08:02 -07:00
William Bartholomew 21789799a8 Add comments on deadletter-only APIs 2017-03-15 16:07:30 -07:00
William Bartholomew b1af03a946 Make stores return a consistent format for list 2017-03-15 15:40:46 -07:00
William Bartholomew 220a3eee85 Store metadata lowercase 
Azure Storage will store it in lowercase so minimize surprises by having the code read that way.
 2017-03-15 13:58:15 -07:00
William Bartholomew 59ff3a07fb Make deadletter APIs consistent in use of URN 2017-03-15 13:37:47 -07:00
William Bartholomew fff2a743c2 Make argument names consistent 2017-03-15 12:28:41 -07:00
William Bartholomew 11b3e3a013 Add MongoDB store support for deadletter refactor 
There's some oddities attempting to mimic the behavior of the Azure Blob store. I want to come back and resolve these but it's more to have CIAB running successfully.
 2017-03-14 16:16:10 -07:00
William Bartholomew 42a7429546 First pass at MongoDB APIs to support deadletter 2017-03-14 16:16:09 -07:00
Gene Hazan c72c4cde40 Merge pull request #67 from Microsoft/gh/queueNotFoundError 
Gh/queue not found error
 2017-03-14 14:39:16 -07:00
Gene Hazan a485b5d189 Code review 2017-03-14 14:38:13 -07:00
Gene Hazan a9a55780b8 Dashboard fails if queue does not exist 2017-03-14 13:41:20 -07:00
Gene Hazan 380be04da3 Remove unused variable 2017-03-14 13:29:46 -07:00
Gene Hazan ada6304651 Merge pull request #66 from Microsoft/gh/tableStore 
Gh/table store
 2017-03-14 12:30:30 -07:00
Gene Hazan e5469fcfda Insert into both Redis & Azure table stores. Read only from Redis for now. 2017-03-14 11:20:33 -07:00
Jeff McAffer db86e737ca Update README.md 2017-03-14 08:32:05 -07:00
Jeff McAffer 32445e620b Update README.md 2017-03-14 08:26:26 -07:00
Gene Hazan caaef167a0 Correct _getBlobNameForUrl 2017-03-13 17:33:28 -07:00
Gene Hazan 2ec5863c11 Add Azure table mapping store 2017-03-13 16:17:23 -07:00
Jeff McAffer c052a36545 update readme for wiki and repo links 2017-03-12 00:29:53 -08:00
William Bartholomew 93b749787c Merge pull request #52 from Microsoft/willbar/reorg 
Merge ospo-ghcrawler and ghcrawler
 2017-03-10 11:32:27 -08:00
William Bartholomew 7f5b291055 Make Dockerfile consistent naming 2017-03-10 09:36:40 -08:00
William Bartholomew 4986cb9647 Rationalize merged README 2017-03-10 09:34:07 -08:00
William Bartholomew ad884805d5 Use modules directly 2017-03-10 09:28:54 -08:00