Граф коммитов

31 Коммитов

Автор SHA1 Сообщение Дата
Avani Patel 8a31fcf64b Integrate and merge replay feature to master (#338)
* Create replay thread (#325)

* Create empty replay thread

* Remove connection using

* Remove references to connections in the replay task

* Update name of replay task test

* Move const to config file

* Add device replay actors  (#329)

* Create empty replay thread

* Remove connection using

* Remove references to connections in the replay task

* Update name of replay task test

* Add device replay actors

* Read replay file and stream telemetry (#334)

* update

* read replay file from storage

* replay file settings

* update

* update

* update validation

* PR comments

* fix test
2019-01-18 14:30:29 -08:00
Jill Bender 5754cf4e0f Add template and config value for Remote Monitoring compatibility (#330) 2018-12-12 10:34:29 -08:00
Devis Lucato 8fa180f715
Reliability improvements
**Major:**
1. Backoff in case of d2c messaging throttling, stopping telemetry and twin operations
2. Improve perf reducing load on garbage collector and disposing IoT SDK resources
3. Improve device twin management, e.g. try to avoid twin writes when data is not changed, handle more error scenarios
4. Add "development" feature flag to enable/disable expensive runtime checks
5. Change partitions size from 1000 to 5000 devices to reduce the load on storage

**Minor:**
1. Backoff in case of device count throttling (e.g. when reaching 8000 devices in the free SKU)
2. Remove unused daily counter for telemetry
3. Improve perf reducing the number of no-op async tasks
4. Update IoT SDK and other dependencies
5. Clean up logging code, reimplement log filtering (removed in past PRs)
6. Print SDK version at startup
7. Add some scripts for development, see /scripts/development (create/delete simulation, start storage adapter)

**Bug fixes:**
1. Handle and recover from exceptions in the partitioning agent
2. Fix some swallowed errors/exceptions and unnecessary try/catch
3. Share script interpreter between methods and state to ensure device state consistency
4. Fix logged throughput in case of no traffic, i.e. show 0.0 msg/sec, and round value to 3 decimals
5. Fix logging from ConfigData
6. Change the dev endpoint used to delete simulations: don't delete devices (the endpoint was not working) - no user impact, dev only
2018-11-29 17:29:58 -08:00
Harleen Thind 4cc7936e4e
Updating description of sample simulation (#320) 2018-11-28 11:14:45 -08:00
Harleen Thind 2680d01bcf
updating default simulation template (#318) 2018-11-21 15:09:15 -08:00
Hugh Xiong bb62b8f017 Changes to create default simulations (#298) 2018-10-30 00:10:37 -07:00
Peter Felts 8f66a67290 Adding SimulationManager + tests (#263) 2018-10-05 14:39:33 -07:00
Jill Bender 61c76dd188 Add delivery truck device model that runs on a predefined loop (#249)
* Adding delivery truck device model that runs on a predefined loop.
2018-09-12 11:26:31 -07:00
Harleen Thind a7ef4c3d24
Renaming Protobuf device model name (#239)
* Renaming Protobuf device model
2018-08-22 18:44:51 -07:00
Harleen Thind 7858054e5a
Protobuf bug fixes (#238) 2018-08-22 10:50:59 -07:00
sasathy d812d110a4 Adding Protobuf support for truck (#235)
* Changes to Support sending encrypted Protobuf messages in addition to Json
2018-08-13 14:08:52 -07:00
Devis Lucato 6f0d073713
Performance improvements (#223)
* Refactor code writing twin tags for better performance
* Refactor device properties code for better performance
* Refactor simulation runner for better performance
* Improve connections performance
* Reduce time to start simulation, by creating devices using the same conn string and fetching device from registry only if the authentication fails.
* Limit the number of pending threads attempting to deliver telemetry
* Change all stock device models to use AMQP
* Move SDK timeout setting and other hard coded magic numbers to config file
* Use default timeout value from SDK instead of hardcoding it
* Improve documentation in the configuration file
* Upgrade IoT SDK
* Improve logging in device client
* Reduce noise in the "Info" log level
* When building, remove "deleted files" left over from previous builds
* Improve error detection when creating devices with batch operations
* Recreate device client when the internal AMQP client is disposed (lib bug workaround)
* VS code config update
* Remove simulation version, not used anywhere
* Catch and log when a device model doesn't exist
* Log missing config settings
* Make sure the simulation object is not lost in case of errors while checking for simulation changes
* Limit async tests duration
* Remove IoT hub registry workaround
* Fix boolean conf values parsing
* Remove "en-us" from links in the comments
* Fix invalid parsing of environment variables
2018-07-17 16:37:58 -07:00
Peter Felts ca0c8bb396 Remove duplicate firmware-update related status keys in twin (#209)
* Updating the key name of the reported device firmware update property, so that we won't have two reported properties: 'FirmwareUpdateStatus' and 'DeviceMethodStatus', one of which is always null.

* PR feedback.

* PR feedback.
2018-05-04 09:43:49 -07:00
Peter Felts 2d2a92d17e Removing the 'Restart' method from the engine device models to align with the spec. Also made some minor clean up to the Empty and Fill tank methods. (#205) 2018-04-30 17:25:10 -07:00
Zhang Hua 7637f1302e Revert "Change fuellevel and vibration value to non-zero value (#206)" (#207)
This reverts commit 2f8598654e.
2018-04-30 16:58:24 -07:00
Zhang Hua 2f8598654e Change fuellevel and vibration value to non-zero value (#206)
* Change fuellevel and vibration value to non-zero value

* ASA query can not process zero value in UDF function, we avoid to use
default zero value for engine device model
* TODO: Revert this commit once ASA fix the bug

* Fix based on comments
2018-04-30 15:58:28 -07:00
Peter Felts 5fab02209b Implement js device methods (#201)
* initial commit

* Cleaning up unneeded telemetry schema changes

* Removing jslint todo options

* Restoring launchSettings.json changes.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Updated JavaScript methods to throw an error if sent invalid input.

* PR feedback. Minor changes to log format for consistency. Updated docs.

* Updating state script files to use a consistent pattern for updating state, and corresponding documentation and interperter changes.

* PR feedback.

* Restoring TempIncrease, TempDecrease, FillTank, and EmptyTank methods, as RemoteMonitoring doesn't support method arguments now. Also: implemented these methods and made some corresponding tweaks to state files that can be impacted by the state that these methods change.

* Added state-restoration to temperature methods so that they won't error out if a previous state hadn't been set by the time they were called, and did some whitespace clean up and other minor clean up.

* Updated FillTank and EmptyTank to incrementally increase and decrease fuel level, respectively.
2018-04-27 10:06:00 -07:00
Jill Bender b211cb7805 Merge azure-iot-pcs-simulation into master (#197)
* Rewrite to use fewer threads and increase throughput
* Expose simulation metrics in the status endpoint
* When available, return a URL to the Azure Portal metrics of the pre-provisioned IoT Hub
* Allow to run a simulation against a custom IoT Hub, passing the connection string
* Allow to schedule a simulation, defining start and end time
* Improve security by running the service as a non-root user
* Support multiple behavior scripts in a device model
* Support custom device with custom sensors, behavior and frequency
* Allow to override the initial device state when creating a simulation
* When a simulation starts, create all the devices in batches
* When a simulation is deleted, delete also the devices
* Refactor timers to start sending telemetry as soon as possible
* Refactor and improve how simulation scripts access state and properties
* Change stock models to use AMQP (#189)
* Overall improvements to exceptions handling
* Disable SDK retry, and change timeout from default (4 minutes) to 10 seconds
* Do not retry sending telemetry on failure, skip message
* Use IoT Hub S2 SKU limits by default
* Upgrade to .NET Core 2.0.3
* Upgrade Azure IoT SDK: Devices 1.4.1 to 1.6.0, Devices Client 1.5.2 to 1.7.0
* Run simulation engine in the same process used for the web service
* Move docs, move wiki, new API specs, fix scripts for Windows Bash
* Fix the spelling of “ETag” and allow ETag=*
* Add internal scripts for increasing/decreasing/random telemetry
* Add env vars documentation ENVIRONMENT_VARIABLES.md
* Add more optional logging by device and by actor
* Use logging level and other logging settings from configuration
* Adjust unit test precision to mitigate flaky test
* Add system properties to telemetry messages
* Removing the squash flag from Docker scripts
* Use env vars in launchSettings.json
2018-04-12 17:42:46 -07:00
Devis Lucato fd19210445
Allow demos to fix/break the simulated chillers (#104)
* Change the Chiller simulation to support fixing/breaking the state via methods
2017-10-27 17:17:03 -07:00
Tim Laverty d5e95c8e82 Set method result to empty string @ execution completion. Add exception handler to callback function called by hub. (#99) 2017-10-20 11:22:33 -07:00
Devis Lucato 961bc428f2 Update scripts to use “testing” version 2017-10-04 19:35:40 -07:00
Devis Lucato c3dbf9d646 Merge sensors messages (#65) 2017-09-28 01:43:49 -07:00
Tim Laverty 51630ec0cc Switch devices to MQTT, Add exception handler to Actor::Stop (#62)
* Switch devices to use MQTT

* Add exception handler to Actor::Stop

* update version
2017-09-26 14:27:36 -07:00
Devis Lucato 13e6ebf863 Code style 2017-09-22 18:43:21 -07:00
Tim Laverty 3ff4ffae0a Add capability to change simulation at runtime. (#54)
* reboot & firmware update support for other device models

* Add capability to change simulation.

* actor not existing cleanly

* typo

* update version

* Refactoring and nits per PR review.

* PR nits
2017-09-18 18:33:50 -07:00
Tim Laverty c33241a5cd Methods and docs (#40)
This PR provides method support for chillers in the default simulation: reboot, firmware update, increase pressure, and decrease pressure.  It also keeps reported properties in sync with desired properties (by polling for value changes).

* DeviceMethod implmentation class

* First round of methods, added: 1) infra for registering methods for devices, 2) walking avail. methods for each device.  Main functionality missing is allowing specifation and then running the method script.

* remove todo from actor

* nits to code style

* Move methods ownership from devicebootstrap to deviceclient.

* error handling and todos.

* retry method registration, minor fixes.

* nits, logging, removed exception handling for registering methods (it's handled by the parent method)

* bumped version

* code style nits

* Methods use scripts.

Desired properties are polled in UpdateDeviceState and delta is copied into Reported property values.

Telemetry is not sent if the device is marked other than online == "True"

* code style/cleanup changes.

change get query for simulations to linq query rather than iterating the list.

* update version

* Exception handling for updatedevicestate

nit for rename method execution endpoint to include Async suffix.

* nit fix comment code style

* Inject scriptinterpreter. refactoring.

* nits

* state update bug

* no need for scriptengine null check

* Bootstrap individual methods, implement decreasepressure aka EMergencyValveRelease

* remove connection string :(

* fixed threading problem w/ methods, implemented & tested first four methods

* Implement switch to turn random telemetry off; e.g. when increasepressure is called, wait until decreasepressure is called to turn it back on.

* remove clearing of DeviceMethodStatus

* Code style changes + fix some jslint errors

* Fix js methods files

* DeviceMethods code style changes

* Devices.cs code style

* JavascriptInterpreter.cs code style

* script changes for messages, docs

* Cleanup up Javascript functions, create Issue to track implementing functions not yet done.

* Halve the amount of telemetry sent

* Address PR comments.

* issue for script interpreter todo

* todos, nits, pr feedback
2017-09-07 15:08:26 -07:00
Devis Lucato 570645bf2e Simulation seed data (#33)
* Complete seed data, complete all the device models as per specs
* Rename “device type” to “device model”
* Clean JS syntax, check with JS Lint
* Generate device IDs from device model ID, instead of device model Name
* Complete health check endpoint
2017-08-16 12:37:56 -07:00
Devis Lucato abbd305492 Implement device bootstrap scenario and decouple from IoT Hub manager (#30)
* When a new device is created, mark it as a simulated device and 
* When a device connects report some properties like device type, messages schema, and initial location. Use MQTT for twin operations
* Remove dependency on IoT Hub manager and access IoT Hub directly
* Refactor state machine to reduce complexity and reuse code
* Add launch settings for Visual Studio
* Remove env var used for the web service TCP port
* Improve logging of exceptions to avoid log flooding
* Fix messages format, to always use the “_unit” convention
* Bump version to 0.1.4
2017-08-08 18:10:43 -07:00
Devis Lucato 454ffe86ae Allow to inject custom device types (#24)
* Allow to inject custom device models
* Improve parsing of Jint response to support dictionaries and objects
* Sync scripts with template
* Change Truck geolocation format
* Update docker scripts
2017-07-21 12:20:25 -07:00
Devis Lucato 892a55e7e4 Generate telemetry using external scripts (#22)
Use external Javascript functions to generate the telemetry for simulated devices. The path of folder that contain the Javascript files can be configured, so it's possible to mount JS files from anywhere. Files are loaded at runtime. The name of the files is specified in the Device Type JSON files, which are managed in the same way, with a configurable path and loaded at runtime.
Javascript files are interpreted using Jint https://github.com/sebastienros/jint .
Documentation here: https://github.com/Azure/device-simulation-dotnet/wiki/Device-Types
2017-07-11 10:28:55 -07:00
Devis Lucato 16a851651f Add device types configuration and endpoint
This adds a new endpoint to list all the supported device types. Device types are configured via JSON files external to the assembly, so that users can add custom types without rebuilding the service. The files include a README describing the format of these files.
I was going to use HOCON for these files too, for a cleaner look, but I'm actually reconsidering that approach and the use of Newtonsoft greatly reduces the code required to parse these files (at least in .NET).
2017-05-30 19:50:03 -07:00