microsoft
/
mwt-ds
зеркало из https://github.com/microsoft/mwt-ds.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
5 087 коммитов 74 Ветки 0 Теги 110 MiB
Граф коммитов

5087 Коммитов

Автор SHA1 Сообщение Дата
Markus Cozowicz a657872c1c added hosted REST example 2017-11-21 13:19:35 -05:00
Marco Rossi 48e282abb7 RankRewardAnalyzer.py: Check reward with relative toll and use same reward for post and local file 2017-11-13 15:37:40 -05:00
Marco Rossi c1762e844f RankRewardAnalyzer.py: Fixes and improvements (#108) 
- Fixed bug when x-msdecision-src is not in header (write full header to output file)
- Fixed bug in printout when no_events_idx is empty
- Fixed bug using float for time_sleep
- Fixed bug with format in printouts
- Scan dir tree when --azure_path is a dir
- Added --verbose in print_stats
- Added printout of duplicates in --verbose
- Added --plot_hist to plot missing ranking and reward requests timeline
- Added check for status_code errors and skipped lines
- Speedup
 2017-11-09 14:57:22 -05:00
Markus Cozowicz 171961e52d version bump VW and nuget 2017-11-07 09:01:32 -05:00
Markus Cozowicz 73b4ccea3d
Merge pull request #107 from marco-rossi29/DataScience 
Data science
 2017-11-06 15:14:59 -05:00
Marco Rossi 055ac730e7 DataScience: modified Azure Storage authentication in ds.config 
- using Azure Storage connectionString
- added ability to save multiple connectionString, indexed by appId
- removed usage of pickle for Azure Storage authentication
 2017-11-06 15:07:17 -05:00
Marco Rossi 2ca16fcae1 AzureStorageDownloader.py: Avoid prompting user when overwrite_mode=2 2017-11-06 13:48:28 -05:00
Marco Rossi 6333f7ba77 AzureStorageDownloader.py: Try importing azure.storage and install if necessary 2017-11-06 13:38:14 -05:00
Marco Rossi dd45471953 RankRewardAnalyzer.py: Testing ranking and reward requests 2017-11-06 11:47:53 -05:00
Marco Rossi 4825d9a475 AzureStoageDownloader.py: Cosmetic changes 2017-11-03 19:23:37 -04:00
Marco Rossi 5851e5efb4 AzureStoageDownloader.py: Fixed crash when azure blob is modified during download 
Removed --skip_current flag now unnecessary
 2017-11-03 19:23:37 -04:00
Marco Rossi e74aa1b6c9 AzureStorageDownloader.py: If --auth_fp present, program fails if file not found 2017-11-03 19:23:24 -04:00
Markus Cozowicz db752f5bd3
Merge pull request #106 from marco-rossi29/DataScience 
DataScience: Visualization and AzureStorageDownloader major improvements
 2017-11-03 09:07:48 -04:00
Marco Rossi a2ebf606ba DataScience: Visualization and AzureStorageDownloader improvements 
1) Using argparse for parsing cmd line
2) Added progress_callback for verion=2
3) Expanded overwrite to overwrite_mode
 2017-11-02 18:52:36 -04:00
Marco Rossi e96a6e7411 DataScience: Visualization and AzureStorageDownloader major improvements 
1) Decoupled logDownloader logic from Visualization.py. logD logic is now part of AzureStorageDownloader.py (version=1 flag)
2) Refactored cmd line parsing to avoid code duplication
3) Enabled reading Account name and key from ds.config (when pickle file doesn't exist)
4) ds.config: Removed unused settings and moved needed ones to cmd line input
5) Check that AzureStorage account_key entered has length 88
6) Using get_blob_to_path() to prevent saturating memory when downloading large files
7) Changed legends labels for Online performance and Baseline estimate
8) Cosmetic improvements
 2017-10-31 14:31:27 -04:00
Markus Cozowicz 0d3ffc4b11 Merge pull request #105 from marco-rossi29/DataScience 
Data science - AzureStorageDownloader
 2017-10-18 13:53:27 -04:00
Marco Rossi f470bfb354 AzureStorageDownloader.py 2017-10-18 13:29:39 -04:00
Marco Rossi 31c6671bec Visualization.py: Added two extra figures plotting requests number 2017-10-18 13:29:39 -04:00
Marco Rossi a823bf4f46 Visualization.py: added ips estimate on constant action policy 2017-10-18 13:29:39 -04:00
Marco Rossi cddd1512b5 Visualization.py: simplification in parse_logs() api 2017-10-18 13:27:58 -04:00
Marco Rossi 182a5fc0df Visualization.py: 6x speed-up by discarding "_multi" info before json parsing 2017-10-18 13:27:58 -04:00
Markus Cozowicz 789be368c9 Merge pull request #103 from sidsen/master 
Model update in local mode was reading wrong probability for chosen a…
 2017-10-16 11:55:08 -04:00
U-NORTHAMERICA\sidsen e53d63fdf8 Address marcozo comments for pull request 103 2017-10-16 11:27:15 -04:00
Jacob Alber 632ca02ad6 Silence ServerProtocolFailure WebExceptions 
In the last change, we added more tracing to ServerProtocolViolation WebExceptions in the Crawler. Unfortunately, the ones we are seeing aren't triggering this code because they do not get far enough to get a Response object. I don't expect that we will be able to get useful information from this telemetry, so add this to the set of dependency failures we silently ignore, after adding a trace.
 2017-10-11 10:14:36 -04:00
Jacob Alber 1cdf062154 Silence ServerProtocolViolation errors 
In the last change, we added more tracing to ServerProtocolViolation WebExceptions in the Crawler. Unfortunately, the ones we are seeing aren't triggering this code because they do not get far enough to get a Response object. I don't expect that we will be able to get useful information from this telemetry, so add this to the set of dependency failures we silently ignore, after adding a trace.
 2017-10-11 10:07:33 -04:00
Jacob Alber 6c03aa0d05 Merge pull request #1 from Microsoft/DecisionService 
RI from upstream
 2017-10-11 10:03:02 -04:00
U-NORTHAMERICA\sidsen fdc1bb5c01 Make DS local mode thread-safe 2017-10-07 22:58:07 -04:00
U-NORTHAMERICA\sidsen 9d5b15c413 Handle string (json) contexts specially within DS local class 2017-10-07 00:57:22 -04:00
U-NORTHAMERICA\sidsen 98d3c7604f Handle string (json) contexts specially in DS localmode 2017-10-06 20:57:32 -04:00
U-NORTHAMERICA\sidsen 4bb8f52491 Adding unit test for model learning under localmode 2017-10-06 17:00:09 -04:00
Markus Cozowicz c2df9e5dd7 Merge pull request #102 from lokitoth/lokitoth/dev/HardenCrawler 
Crawler pipeline hardening
 2017-10-06 13:22:24 -04:00
Jacob Alber 139a42521a Switch to using Dictionary initializer, and add check for SupportsHeaders 2017-10-06 13:20:08 -04:00
Jacob Alber 6bf6201cad Add tracing for ServerProtocolViolation errors 2017-10-06 11:37:05 -04:00
Jacob Alber bf9fccafdd Add 404 and 503 to targeted exclusions from hard failure at Download stage of Crawl 2017-10-06 11:05:32 -04:00
U-NORTHAMERICA\sidsen 689e381bdf Merge remote-tracking branch 'upstream/master' 2017-10-05 00:25:47 -04:00
U-NORTHAMERICA\sidsen fc3c5cbbdc Model update in local mode was reading wrong probability for chosen action; also fixed edge case in random sampling (benign in most cases, bug in case of multiactionhelper 2017-10-05 00:22:14 -04:00
Markus Cozowicz 908216e869 Merge branch 'DecisionService' of https://github.com/Microsoft/mwt-ds into DecisionService 2017-10-04 17:21:17 -04:00
Markus Cozowicz 794fcd64a6 added interface for unit tests 2017-10-04 17:21:02 -04:00
Markus Cozowicz 6c214dda74 Merge pull request #101 from lokitoth/DecisionService 
Fix ArgumentNullException in VideoIndexer
 2017-10-04 10:22:41 -04:00
Jacob Alber a8874661f7 Fix style: Use built-in keyword for string static method calls 2017-10-04 10:21:13 -04:00
Jacob Alber 9d1ea67f17 Fix ArgumentNullException when VideoIndexer knows about a video, but cannot get data about it 
Also avoid queueing indexing when the video URL is not a valid URL.
 2017-10-03 17:26:39 -04:00
Markus Cozowicz b222fafabb Merge pull request #100 from marco-rossi29/DataScience 
Data Science python code
 2017-10-02 15:47:05 -04:00
Marco Rossi bc5e5ff539 Experimentation.py: Added comments for shared/action features 2017-10-02 14:24:34 -04:00
Marco Rossi 9c05af4b8d Using ds.config and .format 2017-10-02 14:09:33 -04:00
Marco Rossi 77965fe2d7 Data Science python code 
- Visualization.py: visualize data in Azure Storage
- Summary.py: Few improvements to the code
- Experimentation.py: Given ds_json file, swipe over possibile vw cmd
configs
 2017-10-02 11:51:38 -04:00
Markus Cozowicz 63c99c2d9e added documentation 2017-09-20 19:01:22 -04:00
Markus Cozowicz 87630d9173 upgrade msbuild task to fix parallel build issues 2017-09-18 15:45:51 -04:00
Markus Cozowicz 687ef0ae70 upgraded gitlink 2017-09-18 15:24:32 -04:00
Markus Cozowicz e25a055fbf fixed warnings 2017-09-18 15:08:41 -04:00
Markus Cozowicz 0f51caf506 upstream merge 2017-09-15 19:48:52 -04:00