c079e8fd73
Merge pull request #38 from Microsoft/kiril/improve_encryption
...
Kiril/improve encryption
2018-03-16 11:54:55 -07:00
010f4cc086
Update config example
2018-03-14 18:00:23 -07:00
4e809e32f5
Fix urandom
2018-03-14 17:41:13 -07:00
3d16ff253b
Use new IV for each encryption
2018-03-14 17:27:02 -07:00
accebfecfc
Added a configurable data parameter to the deployment process ( #37 )
...
* Configurable data file to process during deployment
* When computing final processing time, need to find first instance where % is 100%.
* add data to params file
2017-12-12 17:54:08 -06:00
defbb5b3a9
Dedicated Workload Tracker ( #36 )
...
* Refactored code to represent a workload and print summary
* Kick off an azure deployment
* add and consolidate results to azure queue oppose to azure blob
* Use separate queue service for results queue
* use configuration for consolidation size
* Update getWorkloadStatus to compute more accurate time stamp
* Track when jobs are done
* Add workload tracker, queue, sas, and dedicated logs
* Remove some unneeded logs and track after all jobs are queued
* Requeue failed jobs
* fix minor PR comments
2017-12-11 10:04:30 -06:00
6c12f4a7f8
Added VMSS metrics support and refactored. ( #35 )
...
* Added VMSS metrics support and refactored.
* Refactored retrieving resources in a resource group to remove duplication and simplify.
2017-11-30 15:14:44 -08:00
bb0021954b
Validator and Results will Track Workload Progress ( #34 )
...
* Track workload progress during results processing
* Add comments for helper scripts
* Adding comment and wait for container to be deleted before recreating.
* Look for specific events and print them
* Add stopping condition for scheduler
refactor separate function to get status
2017-11-30 14:03:38 -08:00
78819938bd
Improved docs for metrics logging, created metrics logger execution loop and fixed a few bugs ( #33 )
2017-11-28 15:49:56 -08:00
611b0d3e39
Generate larger records and combine build scripts ( #32 )
...
* remove tracking of data.encrypted
* Add configuration property for size and number of records
* ignore encrypted files
* Combine generation scripts into one build.sh
* Clean-up generation code
2017-11-28 14:03:45 -08:00
af67111953
update readme and deployment securestrings ( #31 )
2017-11-21 12:59:06 -05:00
e614067d9f
Major Clean-Up Part 1 ( #30 )
...
* Remove unneeded files
* Renamed and reorganized config values
* Add the attributes back into the class for auto-complete
2017-11-17 15:43:29 -08:00
683c94a907
Add ssh support and fix missing imports ( #29 )
2017-11-16 17:19:56 -08:00
e52482fd09
Merge pull request #28 from Microsoft/ali/fix
...
Cleanups
2017-11-15 14:01:35 -08:00
3bac332d02
updated readme
2017-11-14 23:55:39 -08:00
ae2feaabd0
added config.example.json
2017-11-14 23:55:39 -08:00
dc107db6cc
removed config and added to ignore
2017-11-14 23:55:39 -08:00
3ede1699d2
create blob if doesnt exist
2017-11-14 23:55:39 -08:00
70e9408da2
Fix how we encode results and append to the consolidation blob ( #27 )
2017-11-14 17:10:45 -08:00
3cc4d328f4
Updated logging string to include module name ( #26 )
2017-11-14 15:23:53 -08:00
ab9c98362e
Changes to move Azure Storage artificats to the correct storage account ( #25 )
2017-11-13 17:34:57 -08:00
1133f9805b
Update structure of code deployment to use app, data, and config ( #24 )
2017-11-13 10:50:21 -08:00
3ccb3e8565
Merge pull request #23 from Microsoft/ajobanpu/config
...
Queuelogger and metricslogger use the main storage account
2017-11-12 23:46:24 -05:00
b18700cc9b
Metrics logger now uses the main storage account
2017-11-12 15:53:16 -08:00
b88996a28a
Added code to create queue if it does not exist
2017-11-12 10:25:40 -08:00
1171c13d12
Added code to create a queue if it does not already exist
2017-11-12 08:20:03 -08:00
e47619d2bb
Removed separate logger storage account
2017-11-12 07:53:14 -08:00
0627075a6d
Added logging to storage queue for schedulernode and processornode in docker ( #22 )
2017-11-10 11:43:48 -08:00
2feaccdbc4
Create results module ( #21 )
...
* Create results module
* Added new config values and switched encryption to use AESHelper.
* test for 10k
* Fixes to integrate AESHelper
* Added comments and logging
2017-11-08 13:25:15 -08:00
bb2524f6af
Merge pull request #19 from Microsoft/ali/cpp
...
added cpp code for busy work
2017-11-08 15:21:36 -05:00
67800bff31
Anluong/generate larger data set ( #20 )
...
* Add data generator and update functions.py
* Update GenerateEncryptedFiles to use an existing AES key
2017-11-08 15:21:16 -05:00
58f0229e1a
Merge pull request #17 from Microsoft/ajobanpu/stdout
...
Added queuelogging to processor docker container
2017-11-08 14:17:31 -05:00
d2366e85b5
added cpp code for busy work
2017-11-08 14:14:53 -05:00
19a3974c5b
Merge pull request #18 from Microsoft/kiril/fix_deployment_scripts
...
Fix errors in deployment scripts
2017-11-08 13:59:51 -05:00
d17698aacb
Fix errors in deployment scripts
2017-11-08 13:53:11 -05:00
2759a651e5
Added queuelogging to processor docker container
2017-11-08 07:56:26 -08:00
85048772c0
Merge pull request #15 from Microsoft/jeff
...
Adding cron job to run validator once per minute; removed extra
2017-11-08 10:51:12 -05:00
8749f0b259
Merge pull request #16 from Microsoft/kiril/custom_data
...
Support for CustomData
2017-11-08 10:49:15 -05:00
12b70d4ece
Update custom data variable name
2017-11-08 10:27:38 -05:00
ba1043f739
Updated to run everything explicitly...
...
...in the current working directory
2017-11-08 10:23:17 -05:00
6a7c5fc89d
Merge branch 'jeff' of https://github.com/Microsoft/azure-python-redis-queue-processor into jeff
2017-11-08 10:08:25 -05:00
93a6e32ad6
Reordering to assure cron job is running before scheduler starts; other
...
script edits
2017-11-08 10:06:04 -05:00
13b7d27693
Update scheduler_bootstrap.sh
2017-11-08 09:58:26 -05:00
a5b81b0527
getting rid of extra character
2017-11-08 09:53:54 -05:00
b1c074105d
Update config.json
2017-11-08 09:52:05 -05:00
e1505a0f4a
Merge
2017-11-08 09:49:50 -05:00
b8887d7828
Refactor config.py
2017-11-08 09:48:47 -05:00
04c3a8ed51
Adding cron job to run validator once per minute; removed extra
...
parameters from validator.py
2017-11-08 09:28:15 -05:00
3779dc3a72
Merge pull request #14 from Microsoft/ajobanpu/stdout
...
Updated QueueLogger to use pipeout Stdout
2017-11-07 17:28:28 -05:00
ec5dc0a697
ignore env and vs files
2017-11-07 14:20:02 -08:00
Kirill Logachev
Kirill Logachev
aluong
Aaron Schnieder
Ali Hajimirza
ali92hm
jobanputra
Kirill Logachev
Arjun Jobanputra
Jeff Day
dawg89