Граф коммитов

95 Коммитов

Автор SHA1 Сообщение Дата
Peter Felts 🍺 4ef8175faa Jan 2020 dependency updates. 2020-01-08 10:52:56 -08:00
Peter Felts 1ada7f41ea Updating IoT Hub Client and Service SDK versions. (#358) 2019-07-17 11:34:06 -07:00
Peter Felts b3de8ff5ce Adding diagnostics instrumentation around CosmosDB failures and process termination. (#341) 2019-03-29 15:41:22 -07:00
Avani Patel 8a31fcf64b Integrate and merge replay feature to master (#338)
* Create replay thread (#325)

* Create empty replay thread

* Remove connection using

* Remove references to connections in the replay task

* Update name of replay task test

* Move const to config file

* Add device replay actors  (#329)

* Create empty replay thread

* Remove connection using

* Remove references to connections in the replay task

* Update name of replay task test

* Add device replay actors

* Read replay file and stream telemetry (#334)

* update

* read replay file from storage

* replay file settings

* update

* update

* update validation

* PR comments

* fix test
2019-01-18 14:30:29 -08:00
Avani Patel 942b2d5593 Replay file APIs (#333)
* Replay file  APIs

* update settings

* PR comments

* update owners

* typo

* Update DeviceModelScriptsTest.cs
2019-01-16 10:59:21 -08:00
Devis Lucato 9a67305899 Add options to disable device twins and methods (#331) 2018-12-12 15:35:59 -08:00
Jill Bender 5754cf4e0f Add template and config value for Remote Monitoring compatibility (#330) 2018-12-12 10:34:29 -08:00
Devis Lucato 0ced724464 Add support for storing data in Azure Tables (#327) 2018-12-10 14:59:37 -08:00
sasathy 18b3a6eb98 Delete devices even after the simulation is stopped (#313) 2018-12-04 12:52:39 -08:00
Devis Lucato 8fa180f715
Reliability improvements
**Major:**
1. Backoff in case of d2c messaging throttling, stopping telemetry and twin operations
2. Improve perf reducing load on garbage collector and disposing IoT SDK resources
3. Improve device twin management, e.g. try to avoid twin writes when data is not changed, handle more error scenarios
4. Add "development" feature flag to enable/disable expensive runtime checks
5. Change partitions size from 1000 to 5000 devices to reduce the load on storage

**Minor:**
1. Backoff in case of device count throttling (e.g. when reaching 8000 devices in the free SKU)
2. Remove unused daily counter for telemetry
3. Improve perf reducing the number of no-op async tasks
4. Update IoT SDK and other dependencies
5. Clean up logging code, reimplement log filtering (removed in past PRs)
6. Print SDK version at startup
7. Add some scripts for development, see /scripts/development (create/delete simulation, start storage adapter)

**Bug fixes:**
1. Handle and recover from exceptions in the partitioning agent
2. Fix some swallowed errors/exceptions and unnecessary try/catch
3. Share script interpreter between methods and state to ensure device state consistency
4. Fix logged throughput in case of no traffic, i.e. show 0.0 msg/sec, and round value to 3 decimals
5. Fix logging from ConfigData
6. Change the dev endpoint used to delete simulations: don't delete devices (the endpoint was not working) - no user impact, dev only
2018-11-29 17:29:58 -08:00
Harleen Thind 4cc7936e4e
Updating description of sample simulation (#320) 2018-11-28 11:14:45 -08:00
Devis Lucato c1bc7e9f8a Revert PR 296, due to errors seen during the release process (#319) 2018-11-21 16:21:29 -08:00
Harleen Thind 2680d01bcf
updating default simulation template (#318) 2018-11-21 15:09:15 -08:00
Harleen Thind e28d871936
Updates for exposing throttling setting to user for changes (#259) 2018-11-19 23:11:10 -08:00
Devis Lucato 8a1dbb5f7f Stop the application when one of the threads fails (#316)
* Stop the application when one of the threads fails

* Fix and refactor IoT hub connection string handling
2018-11-19 11:44:28 -08:00
Isaac 4765e05256 Status API update - [Hold Merge until Java] (#296)
* Status API update.

* unitTest fix

* Storage Adapter

* consistency with other services

* unit Test fix

* Refactoring

* ping diagnostics Service from simulation

* serviceResult

* remove StatusResultServiceModel constructor
2018-11-19 10:38:01 -08:00
sasathy 677c0453f9 Replacing deprecated method 'GetDevicesAsync' with 'GetJobsAsync' (#308) 2018-11-13 21:52:18 -08:00
Devis Lucato 44a612c5d2 New (admin) endpoint to delete all devices (#310) 2018-11-08 14:59:18 -08:00
Avani Patel 01cfbcfc4f Fix to exclude active device count for expired nodes (#309) 2018-11-05 09:55:09 -08:00
Avani Patel 5a7875e7c6 Fix for incorrect active device count calculation. (#305) 2018-10-30 15:57:38 -07:00
Hugh Xiong bb62b8f017 Changes to create default simulations (#298) 2018-10-30 00:10:37 -07:00
sasathy 4f90c13b42 Tagging telemetry messages with user agent (#294) 2018-10-29 17:06:57 -07:00
Avani Patel 20f8059093 Add ActiveDevices counter to statistics and other updates (#304) 2018-10-29 16:36:27 -07:00
Avani Patel 55bff1c184 Statistics for simulations spanning multiple nodes (#287)
*
2018-10-26 15:49:22 -07:00
Hugh Xiong f65415776b Add null check before disposing the device client (#295)
* Add null check before disposing the device client.
2018-10-25 22:06:15 -07:00
Hugh Xiong a5c896f40e Reset simulation deletion state on toggle(enable/disable) (#291)
* Reset simulation deletion states.
2018-10-23 17:28:38 -07:00
Avani Patel 88678cc1ef Changes for autoscaling VMSS on simulation start and stop (#268) 2018-10-23 10:00:14 -07:00
Devis Lucato 31094648bb
When reconnecting a device, make sure the existing client is disposed (#289)
* When reconnecting a device, make sure the existing client is disposed
2018-10-19 16:58:18 -07:00
Hugh Xiong 070663dfcc Bug fixes (#286)
* fixing dateformat in simulation Api response
2018-10-18 11:08:19 -07:00
Peter Felts e2854f6782 Integrate SimulationManager (#270)
* Switching to SimulationManager from SimulationRunner
2018-10-17 17:16:05 -07:00
Hugh Xiong e4f1c79e69 Support for bulk deletion of devices (#275) 2018-10-12 16:49:29 -07:00
Devis Lucato d1a986f1dd bugfix: use correct Device IDs (#274) 2018-10-09 17:59:19 -07:00
Peter Felts 8f66a67290 Adding SimulationManager + tests (#263) 2018-10-05 14:39:33 -07:00
Devis Lucato a7c0b021bf Create devices bulk import job (#255) 2018-10-01 10:08:31 -07:00
Peter Felts 1658ce7664 Changing the 'copy to output directory' property for data files to ensure that they are always copied to the output directory. This is to fix a previous change that stopped them from being exported. (#266) 2018-09-28 15:25:10 -07:00
Hugh Xiong c4a0ce796e Changes to support fetching Iothub metrics (#250)
* Fetch IoTHub metrics
* These metrics are used to display chart on simulation detail page
2018-09-26 18:00:59 -07:00
Harleen Thind 746054d018
Fix for creating Default Simulation (#264) 2018-09-26 12:29:40 -07:00
Hugh Xiong c13614e9c6 Add duplicate check for the iothub connect string (#262) 2018-09-24 22:02:54 -07:00
Hugh Xiong 2abcd73c33 When upserting a simulation, keep the original/internal/readonly prop… (#261) 2018-09-21 14:56:13 -07:00
Harleen Thind 939333e700
Bug fixes (#247) 2018-09-21 10:23:51 -07:00
Peter Felts a2e01dbacb Changes to start consuming new storage library by simulation and some configs (#243) 2018-09-18 22:22:47 -07:00
Devis Lucato df5d3023e7 Add partitions management (#254)
* Add partitions management, so that multiple VMs can work on a different set of devices.
2018-09-17 22:16:21 -07:00
Devis Lucato 92aa06a4eb
Manage device simulation nodes (#253)
When a node starts it generates a unique ID to identify itself, and writes the ID into the Nodes collection. The ID expires unless the process keeps updating the record, as a result the nodes collection contains an eventually consistent list of active nodes, which can collaborate to run simulations. The list size for instance is used to calculate the throttling limits.

To avoid contentions, only one node takes care of cleaning up the nodes collection, and in order to decide which node, there is a master-node election process. If the master node crashes another node eventually becomes master. The master election process will be used also to decide which node creates devices and partitions.
2018-09-15 13:17:36 -07:00
Devis Lucato a8971a68ee Maintenance and clean up some build warnings (#252)
* Maintenance and clean up build warnings
2018-09-13 10:25:29 -07:00
Devis Lucato 792fdfd345
Add agent for creating and partitioning devices (#251)
Add agent for creating and partitioning devices

1. Add a new agent to the solution, which will be responsible for creating
the devices in IoT hub and for partitioning the list of devices, so
that multiple VMs can each simulate a different set of devices.
The agent is not started yet because the new storage library is not
ready.
2. Refactor concurrency settings and add clustering settings
3. Temporary add retry logic to a couple of flaky tests
2018-09-12 15:37:56 -07:00
Jill Bender 61c76dd188 Add delivery truck device model that runs on a predefined loop (#249)
* Adding delivery truck device model that runs on a predefined loop.
2018-09-12 11:26:31 -07:00
Peter Felts 30a9529574 Storage Library + Tests (#241)
* Adding new Storage Library . This new storage library is not consumed yet, it will be used instead of Storage Adapter in subsequent updates.
2018-09-11 21:47:03 -07:00
sasathy 27ddbbeb67 Webinsights - Sending Diagnostics Data from Microservice to Diagnostics Backend (#236)
* Adding customer insights for different simulation events (start, stop, error)
2018-09-11 15:40:24 -07:00
Harleen Thind a5da099a01
Multiple simulation support and adding stats to simulation definition (#234)
* Support for configuring multiple simulations but running only 1 at a time
* Support for saving Simulation statistics
2018-08-31 20:34:27 -07:00
Peter Felts f3fbc220aa Storage settings (#237)
* Adding new storage config settings
2018-08-28 09:53:47 -07:00