Merge azure-iot-pcs-simulation into master (#197)
* Rewrite to use fewer threads and increase throughput
* Expose simulation metrics in the status endpoint
* When available, return a URL to the Azure Portal metrics of the pre-provisioned IoT Hub
* Allow to run a simulation against a custom IoT Hub, passing the connection string
* Allow to schedule a simulation, defining start and end time
* Improve security by running the service as a non-root user
* Support multiple behavior scripts in a device model
* Support custom device with custom sensors, behavior and frequency
* Allow to override the initial device state when creating a simulation
* When a simulation starts, create all the devices in batches
* When a simulation is deleted, delete also the devices
* Refactor timers to start sending telemetry as soon as possible
* Refactor and improve how simulation scripts access state and properties
* Change stock models to use AMQP (#189)
* Overall improvements to exceptions handling
* Disable SDK retry, and change timeout from default (4 minutes) to 10 seconds
* Do not retry sending telemetry on failure, skip message
* Use IoT Hub S2 SKU limits by default
* Upgrade to .NET Core 2.0.3
* Upgrade Azure IoT SDK: Devices 1.4.1 to 1.6.0, Devices Client 1.5.2 to 1.7.0
* Run simulation engine in the same process used for the web service
* Move docs, move wiki, new API specs, fix scripts for Windows Bash
* Fix the spelling of “ETag” and allow ETag=*
* Add internal scripts for increasing/decreasing/random telemetry
* Add env vars documentation ENVIRONMENT_VARIABLES.md
* Add more optional logging by device and by actor
* Use logging level and other logging settings from configuration
* Adjust unit test precision to mitigate flaky test
* Add system properties to telemetry messages
* Removing the squash flag from Docker scripts
* Use env vars in launchSettings.json
2018-04-13 03:42:46 +03:00
|
|
|
#!/usr/bin/env bash
|
|
|
|
# Copyright (c) Microsoft. All rights reserved.
|
|
|
|
# Note: Windows Bash doesn't support shebang extra params
|
|
|
|
set -e
|
2017-05-01 04:17:01 +03:00
|
|
|
|
2017-07-07 23:49:03 +03:00
|
|
|
# Usage:
|
|
|
|
# Build the project in the local environment: ./scripts/build
|
|
|
|
# Build the project inside a Docker container: ./scripts/build -s
|
|
|
|
# Build the project inside a Docker container: ./scripts/build --in-sandbox
|
|
|
|
|
2017-05-01 04:17:01 +03:00
|
|
|
# Debug|Release
|
|
|
|
CONFIGURATION=Release
|
|
|
|
|
|
|
|
APP_HOME="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && cd .. && pwd )/"
|
|
|
|
source "$APP_HOME/scripts/.functions.sh"
|
|
|
|
|
2017-07-07 23:49:03 +03:00
|
|
|
# Folder where PCS sandboxes cache data. Reuse the same folder to speed up the
|
|
|
|
# sandbox and to save disk space.
|
|
|
|
# Use PCS_CACHE="$APP_HOME/.cache" to cache inside the project folder
|
|
|
|
PCS_CACHE="/tmp/azure/iotpcs/.cache"
|
|
|
|
|
2018-07-18 02:37:58 +03:00
|
|
|
cleanup_tmp_files() {
|
|
|
|
check_dependency_dotnet
|
|
|
|
|
|
|
|
cd $APP_HOME
|
|
|
|
header "Removing temporary folders and files..."
|
|
|
|
|
|
|
|
rm -fR packages
|
|
|
|
rm -fR target
|
|
|
|
rm -fR out
|
|
|
|
|
|
|
|
PROJECTS=$(dotnet sln list | grep 'csproj$')
|
|
|
|
for PROJ in $PROJECTS; do
|
|
|
|
PROJ=$(dirname "$PROJ")
|
|
|
|
cd $PROJ
|
|
|
|
rm -fR bin/
|
|
|
|
rm -fR obj/
|
|
|
|
cd $APP_HOME
|
|
|
|
done
|
|
|
|
|
|
|
|
echo -e "\nDone"
|
|
|
|
}
|
|
|
|
|
2017-05-01 04:17:01 +03:00
|
|
|
compile() {
|
Migrate to .NET Core (#9)
Upgrade source code, project files and scripts to use .NET Core. Mono is not required anymore, and also the Docker image will now use only .NET Core.
The configuration format has changed to be in line with ASP.NET Core configuration providers.
The simulation runner is temporarily disabled, because Akka.NET is not ready for .NET Core, and I'm in the middle of removing Akka from the project as well.
Due to the move from .NET Core, some internal APIs are different, for instance you might notice changes in the web service exception filter, in the Autofac dependency injection setup, in the use of Kestrel as a HTTP server.
Since the microservice is now built and packaged with the `dotnet` command, some paths have changed, and scripts have been updated accordingly. I tested the scripts in Windows and MacOS, hopefully catching all the edge cases. The scripts are now actually simpler, which is a nice and unexpected result of the migration.
2017-06-21 04:52:50 +03:00
|
|
|
check_dependency_dotnet
|
2017-05-26 04:22:07 +03:00
|
|
|
|
|
|
|
cd $APP_HOME
|
2017-08-09 04:10:43 +03:00
|
|
|
./scripts/env-vars-check
|
|
|
|
|
2017-05-26 04:22:07 +03:00
|
|
|
header "Downloading dependencies..."
|
Migrate to .NET Core (#9)
Upgrade source code, project files and scripts to use .NET Core. Mono is not required anymore, and also the Docker image will now use only .NET Core.
The configuration format has changed to be in line with ASP.NET Core configuration providers.
The simulation runner is temporarily disabled, because Akka.NET is not ready for .NET Core, and I'm in the middle of removing Akka from the project as well.
Due to the move from .NET Core, some internal APIs are different, for instance you might notice changes in the web service exception filter, in the Autofac dependency injection setup, in the use of Kestrel as a HTTP server.
Since the microservice is now built and packaged with the `dotnet` command, some paths have changed, and scripts have been updated accordingly. I tested the scripts in Windows and MacOS, hopefully catching all the edge cases. The scripts are now actually simpler, which is a nice and unexpected result of the migration.
2017-06-21 04:52:50 +03:00
|
|
|
dotnet restore
|
2017-05-26 04:22:07 +03:00
|
|
|
|
|
|
|
header "Compiling code..."
|
Migrate to .NET Core (#9)
Upgrade source code, project files and scripts to use .NET Core. Mono is not required anymore, and also the Docker image will now use only .NET Core.
The configuration format has changed to be in line with ASP.NET Core configuration providers.
The simulation runner is temporarily disabled, because Akka.NET is not ready for .NET Core, and I'm in the middle of removing Akka from the project as well.
Due to the move from .NET Core, some internal APIs are different, for instance you might notice changes in the web service exception filter, in the Autofac dependency injection setup, in the use of Kestrel as a HTTP server.
Since the microservice is now built and packaged with the `dotnet` command, some paths have changed, and scripts have been updated accordingly. I tested the scripts in Windows and MacOS, hopefully catching all the edge cases. The scripts are now actually simpler, which is a nice and unexpected result of the migration.
2017-06-21 04:52:50 +03:00
|
|
|
dotnet build --configuration $CONFIGURATION
|
2017-05-01 04:17:01 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
run_tests() {
|
2017-05-26 04:22:07 +03:00
|
|
|
check_dependency_dotnet
|
|
|
|
|
|
|
|
cd $APP_HOME
|
|
|
|
header "Running tests..."
|
Migrate to .NET Core (#9)
Upgrade source code, project files and scripts to use .NET Core. Mono is not required anymore, and also the Docker image will now use only .NET Core.
The configuration format has changed to be in line with ASP.NET Core configuration providers.
The simulation runner is temporarily disabled, because Akka.NET is not ready for .NET Core, and I'm in the middle of removing Akka from the project as well.
Due to the move from .NET Core, some internal APIs are different, for instance you might notice changes in the web service exception filter, in the Autofac dependency injection setup, in the use of Kestrel as a HTTP server.
Since the microservice is now built and packaged with the `dotnet` command, some paths have changed, and scripts have been updated accordingly. I tested the scripts in Windows and MacOS, hopefully catching all the edge cases. The scripts are now actually simpler, which is a nice and unexpected result of the migration.
2017-06-21 04:52:50 +03:00
|
|
|
PROJECTS=$(dotnet sln list | grep 'csproj$' | grep '\.Test')
|
2017-05-01 04:17:01 +03:00
|
|
|
for PROJ in $PROJECTS; do
|
2017-05-26 04:22:07 +03:00
|
|
|
echo "-- $PROJ"
|
2017-07-07 23:49:03 +03:00
|
|
|
dotnet test --configuration $CONFIGURATION $PROJ
|
2017-05-01 04:17:01 +03:00
|
|
|
done
|
|
|
|
}
|
|
|
|
|
2017-08-16 22:37:56 +03:00
|
|
|
json_lint() {
|
|
|
|
cd $APP_HOME
|
|
|
|
set +e
|
|
|
|
TEST=$(which jsonlint)
|
|
|
|
set -e
|
|
|
|
if [[ ! -z "$TEST" ]]; then
|
|
|
|
for foo in Services/data/devicemodels/*.json; do
|
|
|
|
set +e
|
|
|
|
TEST=$(cat $foo | jsonlint -s 2> /dev/null)
|
|
|
|
if [[ -z "$TEST" ]]; then
|
|
|
|
error "$foo:"
|
|
|
|
cat $foo | jsonlint -s
|
|
|
|
exit 1
|
|
|
|
fi
|
|
|
|
set -e
|
|
|
|
done
|
|
|
|
fi
|
|
|
|
}
|
|
|
|
|
|
|
|
js_lint() {
|
|
|
|
cd $APP_HOME
|
|
|
|
set +e
|
|
|
|
TEST=$(which jslint4java)
|
|
|
|
set -e
|
|
|
|
if [[ ! -z "$TEST" ]]; then
|
|
|
|
for foo in Services/data/devicemodels/scripts/*.js; do
|
|
|
|
jslint4java $foo
|
|
|
|
done
|
|
|
|
fi
|
|
|
|
}
|
|
|
|
|
2017-07-07 23:49:03 +03:00
|
|
|
setup_sandbox_cache() {
|
|
|
|
mkdir -p $PCS_CACHE/sandbox/.config
|
|
|
|
mkdir -p $PCS_CACHE/sandbox/.dotnet
|
|
|
|
mkdir -p $PCS_CACHE/sandbox/.nuget
|
2017-07-21 22:20:25 +03:00
|
|
|
echo "Note: caching build files in $PCS_CACHE"
|
2017-07-07 23:49:03 +03:00
|
|
|
}
|
|
|
|
|
2017-05-26 04:22:07 +03:00
|
|
|
build_in_sandbox() {
|
2017-07-07 23:49:03 +03:00
|
|
|
|
|
|
|
setup_sandbox_cache
|
|
|
|
|
2017-05-26 04:22:07 +03:00
|
|
|
cd $APP_HOME
|
2017-07-07 23:49:03 +03:00
|
|
|
|
2017-07-21 22:20:25 +03:00
|
|
|
# In Windows this script should use docker.exe, in which case
|
2017-07-07 23:49:03 +03:00
|
|
|
# the parameters syntax is different, e.g. volumes path
|
|
|
|
# (i.e. C:\path\path\... vs /c/path/path/...).
|
|
|
|
#
|
|
|
|
# Note that this script is also used for the git precommit hook.
|
|
|
|
set +e
|
|
|
|
IS_WINDOWS=$(which cmd.exe)
|
|
|
|
set -e
|
|
|
|
if [[ -z "$IS_WINDOWS" ]]; then
|
|
|
|
check_dependency_docker
|
2017-07-11 20:28:55 +03:00
|
|
|
docker run -it \
|
2017-10-05 05:26:24 +03:00
|
|
|
-e PCS_IOTHUB_CONNSTRING \
|
2017-07-07 23:49:03 +03:00
|
|
|
-v "$PCS_CACHE/sandbox/.config:/root/.config" \
|
|
|
|
-v "$PCS_CACHE/sandbox/.dotnet:/root/.dotnet" \
|
|
|
|
-v "$PCS_CACHE/sandbox/.nuget:/root/.nuget" \
|
|
|
|
-v "$APP_HOME:/opt/code" \
|
2017-07-21 22:20:25 +03:00
|
|
|
azureiotpcs/code-builder-dotnet:1.0-dotnetcore /opt/code/scripts/build
|
2017-07-07 23:49:03 +03:00
|
|
|
else
|
|
|
|
# Note 'winpty' is required to provide a TTY to Docker
|
2017-07-21 22:20:25 +03:00
|
|
|
echo "Launching cmd.exe /c winpty ..."
|
|
|
|
cmd.exe /c "winpty .\scripts\build.cmd --in-sandbox"
|
2017-07-07 23:49:03 +03:00
|
|
|
fi
|
2017-05-26 04:22:07 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
if [[ "$1" == "--in-sandbox" || "$1" == "-s" ]]; then
|
|
|
|
build_in_sandbox
|
|
|
|
else
|
2017-07-07 23:49:03 +03:00
|
|
|
# workaround for https://github.com/dotnet/cli/issues/3995
|
|
|
|
unset home
|
|
|
|
|
2018-07-18 02:37:58 +03:00
|
|
|
cleanup_tmp_files
|
2017-08-16 22:37:56 +03:00
|
|
|
json_lint
|
|
|
|
js_lint
|
2017-05-26 04:22:07 +03:00
|
|
|
compile
|
|
|
|
run_tests
|
|
|
|
fi
|
|
|
|
|
|
|
|
set +e
|