microsoft
/
bonsai-anylogic
зеркало из https://github.com/microsoft/bonsai-anylogic.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

Fixes and comment updates to ABCA model and Inkling (#9) 

* Remove unused parameters

A previous commit disconnected bonsai-related parameters from the Connector but mistakenly did not delete them from the model (they appear in the experiment properties, as if still functional)

* Improve comments & code

- Remove unused code

- Add small workaround code for locally hosted sims

- Add/edit clarifying comments

* Remove stop options from sim

Stop/terminal conditions offset entirely to Inkling, as current AnyLogic integration does not support this (may cause training issues for users who are experimenting)

* Clean/fix inkling

- Improve comments: removed an inaccuracy (reference to some nonexistent "ModelAction" class), consistency of where they're written, clarified purposes
- Replace 'Action' type with 'SimAction' type (the latter was previously unused)
- Remove unused "Math" import

* Update README.md

- Removed a sentence only applicable to the original simulation model
- Replaced explanations about the old, "wrapper" workflow with the new RL experiment workflow
- Updated explanations about how to perform local assessments
- Updated sample inkling based on previous commit
- Added more detailed instructions
- Clarified various parts
- Cleaned some grammar

* Update exported.zip

Exported with the latest changes

* Minor comment updates

* Update README.md

- Manually merge some phrases made by @mzat-msft, adjusting slightly for grammar
- Reapply line width
This commit is contained in:
Tyler 2022-02-23 16:23:07 -06:00 коммит произвёл GitHub
Родитель dc70d307d8
Коммит 8a7b811bbc
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
4 изменённых файлов: 226 добавлений и 364 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

241
samples/abca/README.md
Просмотреть файл

@ -1,183 +1,222 @@
# Activity Based Costing Analysis Overview
In this simplified factory floor model, cost associated with product
processing is calculated and analyzed.
In this simplified factory floor model, cost associated with product processing
is calculated and analyzed.
Each incoming product seizes one unit of resource A, then one unit of
resource B, then is processed by a machine. Afterwards, A is released, the
product is conveyed to the exit, and then B is released just before the exit.
Each incoming product seizes one unit of resource A, then one unit of resource
B, then is processed by a machine. Afterwards, resource A is released, the
product is conveyed to the exit, and then resource B is released just before the
exit.
Whenever the product is in the system, the “Existence” cost applies ($ per
hour). While a resource unit is being seized by a product, its “Busy” cost is
allocated to the product, otherwise the “Idle” cost is applied (which is
uniformly distributed to all products). Processing at the machine and
conveying have direct fixed costs, which are different for equipment with
different performances.
uniformly distributed to all products). Processing at the machine and conveying
have direct fixed costs, which are different for equipment with different
performances.
Cost accumulated by a product is broken down into several categories for
analysis and optimization. You can change the factory floor parameters on the
fly and see how they affect the product cost.
analysis and optimization.
### Complexity
- Production rate, cost of resources A & B are configurable.
- Capacities of resource A, B, processing time of the machine and
conveyor speed can be adjusted dynamically in order to keep the total
cost per product at minimum.
- Regular simulation optimization (SO) is not capable of adaptive
change of parameters in order to produce optimum results over time
- Rate of product arrivals can be thought as exogenous and is made configurable.
- Capacities of resource A and B, processing time of the machine and conveyor
speed can be adjusted dynamically in order to keep the total cost per product
at minimum.
- Regular simulation optimization (SO) is not capable of adaptively changing of
parameters in order to produce optimum results for unpredictable arrival
rates.
### Observation space
Although there are many aspects of the simulator, the `arrival rate` is what
is used to by the brain to make decisions.
- Arrival rate
Although there are many aspects of the simulator, the frequency at which
products arrive into the factory was found to be the most influential.
### Action space
- Number of resource A and B
- process time, conveyor speed (these could be continuous values)
- Number of "A" and "B" resources
- Process time
- Conveyor speed
### Reward
The reward is optimized based on minimizing the total cost per product (this
may need modification to maintain a desired production goal in a particular
scenario).
- The reward is optimized based on minimizing the total cost per product (this
may need modification to maintain a desired production goal in a particular
scenario).
- A penalty is given if the chosen actions cause the system to overload.
# Create a Brain
To start a new brain for this model:
1. Create an account or sign into Bonsai.
2. Click **Create brain** button in the top left, then select **Empty brain**
in the dialog.
2. Click **Create brain** button in the top left, then select **Empty brain** in
the dialog.
3. Name your new brain (e.g., “costing-analysis”).
4. Click **Create Brain**. This will create a new brain for you.
Copy the contents of <a href="abca.ink">abca.ink</a> in to the *Teach* tab
for your new brain.
Copy the contents of <a href="abca.ink">abca.ink</a> in to the _Teach_ tab for
your new brain.
Do not click Train yet.
# Running the Model
To run the model for training, you have to first setup the connector to work
with the Bonsai Platform. Pick your `workspace-id` and `access-key` and
insert them in the main Bonsai connector in the Anylogic simulation. For this
simulation, you can find the connector in the *Project* tab under
`Activity Based Costing Analysis (Bonsai) -> Main -> Agents -> bonsaiConnector`.
After that is correctly set up, you can right-click on **TrainingSimulation**
and then click the **Run** button. This will register the simulation with the
Bonsai platform. You can now go back to the Bonsai UI where you created your
brain.
with the Bonsai Platform by inserting your your `workspace-id` and `access-key`
into the appropriate fields of the Bonsai Connector found within the model.
Click the **Train** button. The simulator with the name matching your
simulator will appear (in the example above, this is called *AnyLogic -
ABCA*). Click the name of your simulator.
Start by opening the provided model inside of AnyLogic. From the **Projects**
panel, double click on the "Main" agent to navigate inside of it. Locate the
"bonsaiConnector" object within the dotted red box above the initial view; click
on its icon to view its properties. In the **Properties** panel, replace the
placeholders for the "Workspace ID" and "Access key" fields with your own
credentials (they should be placed inside quotation marks).
If this is the first start of your brain it may take a few minutes for the
brain to generate the appropriate parameters and connect to the simulator.
Once the connection is made you will see your first episodeStart event fire
in the ModelExecuter handler.
From the **Projects** panel, right click on the **TrainingSimulation**
experiment, then click the **Run** button. This will start to register the
simulation with the Bonsai platform. Once registration is complete (it will only
take a few seconds), go back to the Bonsai UI where you created your brain.
You may decide to let your training run for a bit, particularly across
multiple episode start events, to get an understanding of how the model
behaves under various configuration parameters provided by the brain. You
will also want to make sure your number of iterations stay below 1000, or the
brain will struggle to learn. If needed, you can implement custom logic in
the **halted()** method in ModelExecuter to help drive behavior. Halted
indicates to the brain that the simulator has reached a state that it cannot
progress from.
Within Bonsai, click the **Train** button. The simulator with the name matching
your simulator will appear (default: "ABCA sim"). Click the name of your
simulator.
After you have tested locally, stop your model. Then click **Stop Training**
in the Bonsai UI for the brain.
If this is the first time starting your brain, it may take a few minutes for the
brain to generate the appropriate parameters and connect to the simulator. Once
the connection is made, you will see the model beginning to run on its own and
occasionally reset itself.
During preparation and training, do not attempt to make any meaningful
interactions with the running model, as it may disrupt the training process.
You may decide to let your training run for a bit, particularly across multiple
episode start events, to get an understanding of how the model behaves under
various configuration parameters provided by the brain.
In general, you will also want to make sure the number of iterations per episode
stays below 1000, or the brain will struggle to learn. The default
implementation of this model only has 2 iterations: one at the start for the
brain to choose the action (based on the arrival rate) and a second, 6 months
into the simulation, where it will evaluate its performance and restart the
episode to try again.
After you have tested locally, stop your model. Then click **Stop Training** in
the Bonsai UI for the brain.
# Export Your Model
After you have confirmed your model can connect to the platform locally, it's
time to scale your model.
time to scale your model. This will be done by exporting the model as a zip file
and uploading it in the Bonsai UI. This feature is available to _all_ editions
of AnyLogic, including Personal Learning Edition (PLE).
AnyLogic Professional users can export their model by going to **File** >
**Export...** > **to standalone Java application** in the menu bar.
Inside of AnyLogic, under the **Projects** panel, click on the "RLExperiment"
experiment. At the top of the **Properties** panel, select "Export to Microsoft
Bonsai".
Select **HeadlessExperiment** in the dialog and the directory where the
exported files will reside.
In the prompt, choose a destination for the zip file, then click **Next** to
begin the export. When it's finished, you may follow the steps shown and then
click the **Finish** button.
If you need additional assistance with exporting a model, please see the <a
href="https://help.anylogic.com/index.jsp?topic=%2Fcom.anylogic.help%2Fhtml%2Fstandalone%2FExport_Java_Application.html">Exporting
a model as a standalone Java application</a> topic in the AnyLogic Help
topics.
If you are not able to export your model in this way, you may use the example <a
href="exported.zip">exported.zip</a> file.
If you are not able to export your model to a standalone Java application you
may use the example <a href="exported.zip">exported.zip</a> file to use for
scaling.
Note that, unlike the export feature available in AnyLogic Professional, this
exported model will _only_ function on the Microsoft Bonsai platform.
# Scale Your Model
Once you have exported your model, you can zip the entire contents of the
folder that contains the exported application.
Once you have exported your model, return to the Bonsai UI and next to
**Simulators**, click the **Add sim** button.
For example, if your folder structure is:
```
Activity Based Costing Analysis Exported
└─── lib
| |── AnyLogic Model End User Agreement.pdf
| └── ... jar files ...
|─── Activity Based Costing Analysis_linux.sh
|─── ... jar files ...
└─── readme.txt
```
Then you only need to zip the parent **Activity Based Costing Analysis
Exported** folder.
Back in the Bonsai UI, next to **Simulators**, click the **Add sim** button.
This will open a dialog
This will open a dialog.
<img src="images/add_sim.png" alt="Add Sim Prompt" width="500" border="1"/>
Select AnyLogic.
<img src="images/add_sim_al_nozip.png" alt="Add Sim Prompt 2" width="500" border="1"/>
<img src="images/add_sim_al_nozip.png" alt="Add Sim Prompt 2" width="500"
border="1"/>
Select or drag the zip file containing the exported model.
<img src="images/add_sim_al_zip.png" alt="Add Sim Prompt 3" width="500" border="1"/>
<img src="images/add_sim_al_zip.png" alt="Add Sim Prompt 3" width="500"
border="1"/>
Give your simulator a name, then click **Create simulator**.
After the simulator is created you will see the new simulator appear under
the **Simulators** section.
After the simulator is created you will see the new simulator appear under the
**Simulators** section.
Now click the *Teach* tab.
Now click the _Teach_ tab.
In the simulator definition, just after the open brackets, add a <a
href="#">package</a> statement using the name of the simulator you gave
during the Add Simulator dialog above.
href="#">package</a> statement using the name of the simulator you gave during
the Add Simulator dialog above.
```
simulator Simulator(action: Action, config: SimConfig): SimState {
simulator Simulator(action: SimAction, config: SimConfig): SimState {
package "<simulator_name_from_upload>"
}
```
Now click **Train**. Since you indicated the package name you do not need to
Now click **Train**. Since you indicated the package name, you do not need to
select a simulator from the dropdown like you did when you started locally.
In a few minutes time you will see several simulators connect to and train
your brain.
In a few minutes time you will see several simulators connect to and train your
brain.
# Sample Results
You can read about how the Bonsai brain performed against the default
AnyLogic optimizer in the `ABCA-Optimization-Writeup.docx` file.
You can read about how the Bonsai brain performed against the default AnyLogic
optimizer in the `ABCA-Optimization-Writeup.docx` file.
# Using Bonsai Assessment with Your Model
Starting an Assessment session is similar to starting a training session.
Start your AnimatedExperiment and wait for it to register. In the Bonsai UI,
using your already-trained brain, click the **Assessment** button. Then
select the name of your simulator
You can perform Bonsai Assessments either with the unmanaged (locally hosted) or
managed (uploaded) sim. Instructions for each can be found in the following two
Bonsai documentation articles:
- Unmanaged:
https://docs.microsoft.com/en-us/bonsai/guides/assess-with-local-sim
- Managed: https://docs.microsoft.com/en-us/bonsai/guides/assess-brain
Additionally, you can export a partially or fully trained brain and add it back
into the simulation model through the use of the Bonsai Connector's "Playback"
functionality. To do this, start by clicking the _Train_ tab in the Bonsai UI
and then the **Export brain** button. Enter a desired export name and preserve
the default processor architecture, then click **Export**.
After a few minutes, you should see a pop-up with instructions for deployment
via Docker.
If you have Docker on your system, you may follow the instructions in the pop-up
to host your brain on your local machine. In this case, skip the following
paragraph; otherwise, read on.
If you don't have Docker on your system or wish to have a publicly available
deployment, you can create an Azure Web App that hosts your exported brain (done
from the Azure Portal). For a detailed visual walkthrough, see the "Exporting
the model" section of a previous AnyLogic-Bonsai webinar (<a
href="https://youtu.be/-iTSZaG3lg8?t=3968">timestamped link</a>) -
**important:** the core focus of the linked webinar is for a now-deprecated
workflow; however, the web app deployment process is mostly the same.
Once you have a brain running locally via Docker or hosted on a web app, return
to the model in AnyLogic and, once again, view the properties of the
"bonsaiConnector" object. In the **Exported brain address** field, enter the
prediction endpoint as a quoted string. The specific format of the endpoint will
be slightly different depending on your chosen deployment strategy:
- Docker: http://localhost:5000/v1/prediction
- Web App: https://MYWEBAPP.azurewebsites.net/v1/prediction
- (where "MYWEBAPP" is the name of your Web App)
Next, right click the model's "PlaybackSimulation" experiment (in the
**Projects** panel) and click **Run**. When the model starts, it is setup such
that it will query your exported brain whenever you change the slider for
"Arrival rate".

52
samples/abca/abca.ink
Просмотреть файл

@ -1,8 +1,8 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.
inkling "2.0"
using Math
# SimState has the same properties as the Observation fields in the RLExperiment
# SimState has the same properties as the Observation fields in the RL experiment.
type SimState {
arrivalRate: number<0.5 .. 2.0>,
recentNProducts: number,
@ -20,54 +20,48 @@ type SimState {
exceededCapacityFlag: number<0,1,>,
simTimeMonths: number<0 .. 7>
}
# The ObservableState is what the brain sees from the simulator.
# In this case, it's just the arrival rate.
type ObservableState {
arrivalRate: number
}
type Action {
# SimAction has the same properties as the Action fields in the RL experiment.
# The ranges are based on the sliders in the sim (the ones the user typically controls).
type SimAction {
numResourceA: number<1 .. 20>,
numResourceB: number<1 .. 20>,
processTime: number<1.0 .. 12.0>,
conveyorSpeed: number<0.01 .. 1.0>,
}
# SimConfig has the same properties as the Configuration fields in the RL experiment.
# Each training episode can potentially vary the arrival rate,
# and the size of the buffer queues in the first part of the process.
type SimConfig {
arrivalRate: number,
sizeBufferQueues: number
}
simulator Simulator(action: Action, config: SimConfig): SimState {
}
# SimAction is the values translated for the sim.
# We do not need ranges here.
# These are the same as the ModelAction class.
type SimAction {
numResourceA: number,
numResourceB: number,
processTime: number,
conveyorSpeed: number,
simulator Simulator(action: SimAction, config: SimConfig): SimState {
# (package statement for managed sims are placed here)
}
function Terminal(obs:SimState)
{
if(obs.exceededCapacityFlag == 1)
{
return true
}
# The brain gets one chance at the answer
return obs.simTimeMonths >= 6
# Reset the episode if the buffer queue capacity was exceeded (1 = true),
# or the simulated time is at/after when the second action is taken (giving the brain one chance)
function Terminal(obs:SimState) {
return obs.exceededCapacityFlag == 1 or obs.simTimeMonths >= 6
}
# Large penalty for exceeding the buffer queue's capacity.
# Otherwise, try to maximize the cost per product value.
function Reward(obs: SimState) {
# Large penalty for exceeding the buffer queue's capacity.
# Otherwise, try to maximize the cost per product value.
return -obs.costPerProduct - 1000 * obs.exceededCapacityFlag
}
graph (input: ObservableState): Action {
concept optimize(input): Action {
graph (input: ObservableState): SimAction {
concept optimize(input): SimAction {
curriculum {
source Simulator
reward Reward

Двоичные данные
samples/abca/exported.zip
Просмотреть файл

Двоичный файл не отображается.

297
samples/abca/model/Activity Based Costing Analysis (Bonsai).alp
Просмотреть файл

@ -4,7 +4,7 @@
AnyLogic Project File
*************************************************
-->
<AnyLogicWorkspace WorkspaceVersion="1.9" AnyLogicVersion="8.7.2.202103191534" AlpVersion="8.7.0">
<AnyLogicWorkspace WorkspaceVersion="1.9" AnyLogicVersion="8.7.4.202104291518" AlpVersion="8.7.2">
<Model>
<Id>1208168597833</Id>
<Name><![CDATA[Activity Based Costing Analysis (Bonsai)]]></Name>
@ -308,7 +308,7 @@
<Variable Class="Parameter">
<Id>1588008430360</Id>
<Name><![CDATA[bonsaiMode]]></Name>
<X>210</X><Y>-100</Y>
<X>320</X><Y>-100</Y>
<Label><X>10</X><Y>0</Y></Label>
<PublicFlag>false</PublicFlag>
<PresentationFlag>true</PresentationFlag>
@ -352,138 +352,6 @@
</ParameterEditor>
</Properties>
</Variable>
<Variable Class="Parameter">
<Id>1606784137636</Id>
<Name><![CDATA[playbackURL]]></Name>
<X>210</X><Y>-80</Y>
<Label><X>10</X><Y>0</Y></Label>
<PublicFlag>false</PublicFlag>
<PresentationFlag>true</PresentationFlag>
<ShowLabel>true</ShowLabel>
<Properties SaveInSnapshot="true" ModificatorType="STATIC">
<Type><![CDATA[String]]></Type>
<UnitType><![CDATA[NONE]]></UnitType>
<SdArray>false</SdArray>
<DefaultValue Class="CodeValue">
<Code><![CDATA["https://<WEBAPP_NAME>.azurewebsites.net/v1/prediction"]]></Code>
</DefaultValue>
<ParameterEditor>
<Id>1606784137634</Id>
<Label><![CDATA[Exported brain address]]></Label>
<EditorContolType>TEXT_BOX</EditorContolType>
<HideCondition>
<Id>0</Id>
<ParameterId>1588008430360</ParameterId>
<Operation>NOT_EQUALS</Operation>
<Value Class="CodeValue">
<Code><![CDATA[1]]></Code>
</Value>
</HideCondition>
<MinSliderValue><![CDATA[0]]></MinSliderValue>
<MaxSliderValue><![CDATA[100]]></MaxSliderValue>
<DelimeterType>SEPARATOR</DelimeterType>
</ParameterEditor>
</Properties>
</Variable>
<Variable Class="Parameter">
<Id>1606784147383</Id>
<Name><![CDATA[workspaceID]]></Name>
<X>320</X><Y>-80</Y>
<Label><X>10</X><Y>0</Y></Label>
<PublicFlag>false</PublicFlag>
<PresentationFlag>true</PresentationFlag>
<ShowLabel>true</ShowLabel>
<Properties SaveInSnapshot="true" ModificatorType="STATIC">
<Type><![CDATA[String]]></Type>
<UnitType><![CDATA[NONE]]></UnitType>
<SdArray>false</SdArray>
<DefaultValue Class="CodeValue">
<Code><![CDATA["<YOUR_WORKSPACE_ID>"]]></Code>
</DefaultValue>
<ParameterEditor>
<Id>1606784147381</Id>
<Label><![CDATA[Workspace ID]]></Label>
<EditorContolType>TEXT_BOX</EditorContolType>
<HideCondition>
<Id>0</Id>
<ParameterId>1588008430360</ParameterId>
<Operation>NOT_EQUALS</Operation>
<Value Class="CodeValue">
<Code><![CDATA[0]]></Code>
</Value>
</HideCondition>
<MinSliderValue><![CDATA[0]]></MinSliderValue>
<MaxSliderValue><![CDATA[100]]></MaxSliderValue>
<DelimeterType>SEPARATOR</DelimeterType>
</ParameterEditor>
</Properties>
</Variable>
<Variable Class="Parameter">
<Id>1606784157740</Id>
<Name><![CDATA[accessKey]]></Name>
<X>320</X><Y>-60</Y>
<Label><X>10</X><Y>0</Y></Label>
<PublicFlag>false</PublicFlag>
<PresentationFlag>true</PresentationFlag>
<ShowLabel>true</ShowLabel>
<Properties SaveInSnapshot="true" ModificatorType="STATIC">
<Type><![CDATA[String]]></Type>
<UnitType><![CDATA[NONE]]></UnitType>
<SdArray>false</SdArray>
<DefaultValue Class="CodeValue">
<Code><![CDATA["<YOUR_ACCESS_KEY>"]]></Code>
</DefaultValue>
<ParameterEditor>
<Id>1606784157738</Id>
<Label><![CDATA[Access key]]></Label>
<EditorContolType>TEXT_BOX</EditorContolType>
<HideCondition>
<Id>0</Id>
<ParameterId>1588008430360</ParameterId>
<Operation>NOT_EQUALS</Operation>
<Value Class="CodeValue">
<Code><![CDATA[0]]></Code>
</Value>
</HideCondition>
<MinSliderValue><![CDATA[0]]></MinSliderValue>
<MaxSliderValue><![CDATA[100]]></MaxSliderValue>
<DelimeterType>NO_DELIMETER</DelimeterType>
</ParameterEditor>
</Properties>
</Variable>
<Variable Class="Parameter">
<Id>1606784198501</Id>
<Name><![CDATA[simulatorName]]></Name>
<X>320</X><Y>-100</Y>
<Label><X>10</X><Y>0</Y></Label>
<PublicFlag>false</PublicFlag>
<PresentationFlag>true</PresentationFlag>
<ShowLabel>true</ShowLabel>
<Properties SaveInSnapshot="true" ModificatorType="STATIC">
<Type><![CDATA[String]]></Type>
<UnitType><![CDATA[NONE]]></UnitType>
<SdArray>false</SdArray>
<DefaultValue Class="CodeValue">
<Code><![CDATA["Simulator"]]></Code>
</DefaultValue>
<ParameterEditor>
<Id>1606784198499</Id>
<Label><![CDATA[Simulator name]]></Label>
<EditorContolType>TEXT_BOX</EditorContolType>
<HideCondition>
<Id>0</Id>
<ParameterId>1588008430360</ParameterId>
<Operation>NOT_EQUALS</Operation>
<Value Class="CodeValue">
<Code><![CDATA[0]]></Code>
</Value>
</HideCondition>
<MinSliderValue><![CDATA[0]]></MinSliderValue>
<MaxSliderValue><![CDATA[100]]></MaxSliderValue>
<DelimeterType>NO_DELIMETER</DelimeterType>
</ParameterEditor>
</Properties>
</Variable>
<Variable Class="Parameter">
<Id>1208169522728</Id>
<Name><![CDATA[BusyCostPerHourA]]></Name>
@ -808,8 +676,8 @@
<Variable Class="Parameter">
<Id>1605111366738</Id>
<Name><![CDATA[sizeBufferQueues]]></Name>
<X>100</X><Y>-260</Y>
<Label><X>-50</X><Y>20</Y></Label>
<X>320</X><Y>-70</Y>
<Label><X>10</X><Y>0</Y></Label>
<PublicFlag>false</PublicFlag>
<PresentationFlag>true</PresentationFlag>
<ShowLabel>true</ShowLabel>
@ -1172,7 +1040,9 @@
<PublicFlag>false</PublicFlag>
<PresentationFlag>true</PresentationFlag>
<ShowLabel>true</ShowLabel>
<Action><![CDATA[RLExperiment.takeAction(this);]]></Action>
<Action><![CDATA[// The `takeAction` function takes any valid agent as an argument.
// It's placed in this dedicated event to avoid conflict with other events.
RLExperiment.takeAction(this);]]></Action>
</DynamicEventClass>
</DynamicEvents>
<Functions>
@ -1199,20 +1069,7 @@
<PresentationFlag>true</PresentationFlag>
<ShowLabel>true</ShowLabel>
<Body><![CDATA[double dt = time() - timeUpdateIdleCosts; //period duration
/*for (int i = 0; i < resourceA.capacity; i++) {
if (!resourceA.getUnit(i).isBusy()) {
accumIdleCostA += dt * resourceA.getUnit(i).idleCostPerHour;
}
}
*/
accumIdleCostA += dt * resourceA.idle() * IdleCostPerHourA;
/*for (int i = 0; i < resourceB.capacity; i++) {
//if (!resourceB.getUnit(i).isBusy()) {
accumIdleCostB += dt * resourceB.getUnit(i).idleCostPerHour;
}
}
*/
accumIdleCostB += dt * resourceB.idle() * IdleCostPerHourB;
timeUpdateIdleCosts = time();]]></Body>
</Function>
@ -1266,18 +1123,27 @@ groupMainMenu.setPos( viewArea.getX(), viewArea.getY() );]]></Body>
<PublicFlag>false</PublicFlag>
<PresentationFlag>true</PresentationFlag>
<ShowLabel>true</ShowLabel>
<Body><![CDATA[// only force RL events if in training
// (playback events are triggered by changing the slider)
<Body><![CDATA[// This function schedules the triggers for bonsai to take actions in the model.
/*
Only execute when the 'bonsaiMode' parameter is in training mode (0);
in deactivated mode (-1), the model will execute as if without bonsai;
in playback mode (1), calls to the brain are triggered on slider updates.
*/
if (bonsaiMode != 0)
return;
// #1 on startup to set configuration
// When using an unmanaged (locally hosted) sim, the first action is replaced with the configuration.
// In this case, create an extra event at time 0.
// This behavior may be changed in a future release.
if (getEngine().getExperiment() instanceof ExperimentSimulation) {
create_RLEvent(0, DAY);
}
// Schedule the first prediction at the model start.
create_RLEvent(0, DAY);
// #2 on startup to make first prediction
create_RLEvent(0, DAY);
// #3 at 6 months to evaluate prediction
// Schedule an event at the end to evaluate the prediction.
create_RLEvent(6, MONTH);]]></Body>
</Function>
</Functions>
@ -3419,7 +3285,8 @@ productCount++;]]></Code>
if (auxQueueA.size() >= auxQueueA.capacity) {
// set the flag
exceededCapacity = true;
// and force-stop the simulation (if not in training)
// and force-stop the simulation when not in training
// (during training, the terminal condition will come from bonsai)
if (bonsaiMode != 0)
finishSimulation();
}]]></Code>
@ -3599,7 +3466,7 @@ if (auxQueueA.size() >= auxQueueA.capacity) {
<Parameter>
<Name><![CDATA[enable]]></Name>
<Value Class="CodeValue">
<Code><![CDATA[bonsaiMode >= 0]]></Code>
<Code><![CDATA[bonsaiMode >= 0 // training (0) or playback (1)]]></Code>
</Value>
</Parameter>
<Parameter>
@ -3608,7 +3475,7 @@ if (auxQueueA.size() >= auxQueueA.capacity) {
<Parameter>
<Name><![CDATA[playback]]></Name>
<Value Class="CodeValue">
<Code><![CDATA[bonsaiMode == 1]]></Code>
<Code><![CDATA[bonsaiMode == 1 // playback]]></Code>
</Value>
</Parameter>
<Parameter>
@ -3616,15 +3483,9 @@ if (auxQueueA.size() >= auxQueueA.capacity) {
</Parameter>
<Parameter>
<Name><![CDATA[workspace]]></Name>
<Value Class="CodeValue">
<Code><![CDATA["<bonsai_workspace_id>"]]></Code>
</Value>
</Parameter>
<Parameter>
<Name><![CDATA[accessKey]]></Name>
<Value Class="CodeValue">
<Code><![CDATA["<bonsai_access_key>"]]></Code>
</Value>
</Parameter>
<Parameter>
<Name><![CDATA[simulatorName]]></Name>
@ -7663,15 +7524,11 @@ STRUCTURE]]></Text>
<EmbeddedIcon>false</EmbeddedIcon>
<Enabled>true</Enabled>
<EnableExpression><![CDATA[bonsaiMode != 0 // bonsai deactivated or playback]]></EnableExpression>
<ActionCode><![CDATA[if (bonsaiMode == 1) { // playback (costs reset as part of action)
try {
RLExperiment.takeAction(this);
} catch (NullPointerException e) { // in case any issues
error("There was an issue with either the sent observation or the received action.\nConfirm the output of the provided brain matches with the fields in the RLExperiment.");
}
} else { // bonsai deactivated
resetCosts();
}]]></ActionCode>
<ActionCode><![CDATA[if (bonsaiMode == 1) { // playback
RLExperiment.takeAction(this);
}
resetCosts();]]></ActionCode>
</BasicProperties>
<ExtendedProperties>
<DefaultValueCode><![CDATA[ArrivalRate]]></DefaultValueCode>
@ -7695,7 +7552,7 @@ STRUCTURE]]></Text>
<BasicProperties Width="95" Height="25">
<EmbeddedIcon>false</EmbeddedIcon>
<Enabled>false</Enabled>
<EnableExpression><![CDATA[bonsaiMode == -1 // bonsai deactivated]]></EnableExpression>
<EnableExpression><![CDATA[bonsaiMode == -1 // enabled when running the "original" sim (bonsai deactivated)]]></EnableExpression>
<ActionCode><![CDATA[resetCosts();]]></ActionCode>
</BasicProperties>
<ExtendedProperties>
@ -7720,7 +7577,7 @@ STRUCTURE]]></Text>
<BasicProperties Width="95" Height="25">
<EmbeddedIcon>false</EmbeddedIcon>
<Enabled>false</Enabled>
<EnableExpression><![CDATA[bonsaiMode == -1 // bonsai deactivated]]></EnableExpression>
<EnableExpression><![CDATA[bonsaiMode == -1 // enabled when running the "original" sim (bonsai deactivated)]]></EnableExpression>
<ActionCode><![CDATA[resetCosts();]]></ActionCode>
</BasicProperties>
<ExtendedProperties>
@ -7745,7 +7602,7 @@ STRUCTURE]]></Text>
<BasicProperties Width="95" Height="25">
<EmbeddedIcon>false</EmbeddedIcon>
<Enabled>false</Enabled>
<EnableExpression><![CDATA[bonsaiMode == -1 // bonsai deactivated]]></EnableExpression>
<EnableExpression><![CDATA[bonsaiMode == -1 // enabled when running the "original" sim (bonsai deactivated)]]></EnableExpression>
<ActionCode><![CDATA[resetCosts();]]></ActionCode>
</BasicProperties>
<ExtendedProperties>
@ -7770,7 +7627,7 @@ STRUCTURE]]></Text>
<BasicProperties Width="95" Height="25">
<EmbeddedIcon>false</EmbeddedIcon>
<Enabled>false</Enabled>
<EnableExpression><![CDATA[bonsaiMode == -1 // bonsai deactivated]]></EnableExpression>
<EnableExpression><![CDATA[bonsaiMode == -1 // enabled when running the "original" sim (bonsai deactivated)]]></EnableExpression>
<ActionCode><![CDATA[resetCosts();]]></ActionCode>
</BasicProperties>
<ExtendedProperties>
@ -7828,24 +7685,28 @@ STRUCTURE]]></Text>
</Labels>
<ShowLegend>true</ShowLegend>
<DataItem>
<Id>1625859479979</Id>
<Expression><![CDATA[zidz(accumIdleCostA + accumIdleCostB, productCount)]]></Expression>
<Color>-989556</Color>
<Expression2Flag>false</Expression2Flag>
<Title><![CDATA[Idle]]></Title>
</DataItem>
<DataItem>
<Id>1625859479980</Id>
<Expression><![CDATA[zidz(accumWaitingCost, productCount)]]></Expression>
<Color>-360334</Color>
<Expression2Flag>false</Expression2Flag>
<Title><![CDATA[Waiting]]></Title>
</DataItem>
<DataItem>
<Id>1625859479981</Id>
<Expression><![CDATA[zidz(accumProcessCost, productCount)]]></Expression>
<Color>-6632142</Color>
<Expression2Flag>false</Expression2Flag>
<Title><![CDATA[Processing]]></Title>
</DataItem>
<DataItem>
<Id>1625859479982</Id>
<Expression><![CDATA[zidz(accumMoveCost, productCount)]]></Expression>
<Color>-7114536</Color>
<Expression2Flag>false</Expression2Flag>
@ -7904,6 +7765,7 @@ STRUCTURE]]></Text>
</Labels>
<ShowLegend>false</ShowLegend>
<DataItem>
<Id>1625859479983</Id>
<Expression><![CDATA[zidz(accumWaitingCost, productCount)]]></Expression>
<Color>-360334</Color>
<Expression2Flag>false</Expression2Flag>
@ -7963,6 +7825,7 @@ STRUCTURE]]></Text>
</Labels>
<ShowLegend>false</ShowLegend>
<DataItem>
<Id>1625859479984</Id>
<Expression><![CDATA[zidz(accumSeizeAQCost, productCount)]]></Expression>
<Color>-360334</Color>
<Expression2Flag>false</Expression2Flag>
@ -8022,6 +7885,7 @@ STRUCTURE]]></Text>
</Labels>
<ShowLegend>false</ShowLegend>
<DataItem>
<Id>1625859479985</Id>
<Expression><![CDATA[zidz(accumSeizeBQCost, productCount)]]></Expression>
<Color>-360334</Color>
<Expression2Flag>false</Expression2Flag>
@ -8081,6 +7945,7 @@ STRUCTURE]]></Text>
</Labels>
<ShowLegend>false</ShowLegend>
<DataItem>
<Id>1625859479986</Id>
<Expression><![CDATA[zidz(accumConveyorQCost, productCount)]]></Expression>
<Color>-360334</Color>
<Expression2Flag>false</Expression2Flag>
@ -8140,6 +8005,7 @@ STRUCTURE]]></Text>
</Labels>
<ShowLegend>false</ShowLegend>
<DataItem>
<Id>1625859479987</Id>
<Expression><![CDATA[zidz(accumProcessCost, productCount)]]></Expression>
<Color>-6632142</Color>
<Expression2Flag>false</Expression2Flag>
@ -8199,6 +8065,7 @@ STRUCTURE]]></Text>
</Labels>
<ShowLegend>false</ShowLegend>
<DataItem>
<Id>1625859479988</Id>
<Expression><![CDATA[zidz(accumMoveCost, productCount)]]></Expression>
<Color>-7114536</Color>
<Expression2Flag>false</Expression2Flag>
@ -8258,6 +8125,7 @@ STRUCTURE]]></Text>
</Labels>
<ShowLegend>false</ShowLegend>
<DataItem>
<Id>1625859479989</Id>
<Expression><![CDATA[zidz(accumIdleCostA, productCount)]]></Expression>
<Color>-989556</Color>
<Expression2Flag>false</Expression2Flag>
@ -8317,6 +8185,7 @@ STRUCTURE]]></Text>
</Labels>
<ShowLegend>false</ShowLegend>
<DataItem>
<Id>1625859479990</Id>
<Expression><![CDATA[zidz(accumIdleCostB, productCount)]]></Expression>
<Color>-989556</Color>
<Expression2Flag>false</Expression2Flag>
@ -8376,6 +8245,7 @@ STRUCTURE]]></Text>
</Labels>
<ShowLegend>false</ShowLegend>
<DataItem>
<Id>1625859479991</Id>
<Expression><![CDATA[zidz( resourceA.busy(), resourceA.capacity )]]></Expression>
<Color>-10185235</Color>
<Expression2Flag>false</Expression2Flag>
@ -8435,6 +8305,7 @@ STRUCTURE]]></Text>
</Labels>
<ShowLegend>false</ShowLegend>
<DataItem>
<Id>1625859479992</Id>
<Expression><![CDATA[zidz( resourceB.busy(), resourceB.capacity )]]></Expression>
<Color>-10185235</Color>
<Expression2Flag>false</Expression2Flag>
@ -8494,6 +8365,7 @@ STRUCTURE]]></Text>
</Labels>
<ShowLegend>false</ShowLegend>
<DataItem>
<Id>1625859479993</Id>
<Expression><![CDATA[zidz(accumIdleCostA + accumIdleCostB, productCount)]]></Expression>
<Color>-989556</Color>
<Expression2Flag>false</Expression2Flag>
@ -8669,7 +8541,7 @@ STRUCTURE]]></Text>
<Rectangle>
<Id>1587397855299</Id>
<Name><![CDATA[rectangle3]]></Name>
<X>180</X><Y>-130</Y>
<X>290</X><Y>-130</Y>
<Label><X>10</X><Y>10</Y></Label>
<PublicFlag>true</PublicFlag>
<PresentationFlag>true</PresentationFlag>
@ -8682,7 +8554,7 @@ STRUCTURE]]></Text>
<LineColor>-65536</LineColor>
<LineMaterial>null</LineMaterial>
<LineStyle>DOTTED</LineStyle>
<Width>520</Width>
<Width>410</Width>
<Height>90</Height>
<Rotation>0.0</Rotation>
<FillColor/>
@ -9287,15 +9159,6 @@ return cost; ]]></Body>
<ItemName><![CDATA[bonsaiMode]]></ItemName>
</InputReference>
</RunConfigurationInput>
<RunConfigurationInput>
<Id>1606843730678</Id>
<Name><![CDATA[Exported brain address]]></Name>
<InputReference>
<PackageName><![CDATA[activity_based_costing_analysis_bonsai]]></PackageName>
<ClassName><![CDATA[Main]]></ClassName>
<ItemName><![CDATA[playbackURL]]></ItemName>
</InputReference>
</RunConfigurationInput>
<RunConfigurationInput>
<Id>1606797956378</Id>
<Name><![CDATA[Arrival rate]]></Name>
@ -9716,18 +9579,6 @@ the product cost.]]></Text>
<Parameter>
<ParameterName><![CDATA[bonsaiMode]]></ParameterName>
</Parameter>
<Parameter>
<ParameterName><![CDATA[playbackURL]]></ParameterName>
</Parameter>
<Parameter>
<ParameterName><![CDATA[workspaceID]]></ParameterName>
</Parameter>
<Parameter>
<ParameterName><![CDATA[accessKey]]></ParameterName>
</Parameter>
<Parameter>
<ParameterName><![CDATA[simulatorName]]></ParameterName>
</Parameter>
<Parameter>
<ParameterName><![CDATA[IdleCostPerHourA]]></ParameterName>
</Parameter>
@ -10028,7 +9879,7 @@ the product cost.]]></Text>
<Z>0</Z>
<Rotation>0.0</Rotation>
<Color>-16777216</Color>
<Text><![CDATA[To use this model, paste the URL of your exported brain into the properties of this experiment as a String (in "double quotes").
<Text><![CDATA[To use this experiment, paste the URL of your exported brain as a string (in double quotes) into the properties of the bonsai connector object inside of Main.
After starting the simulation run, you can use the slider to adjust the arrival rate.
Once you update the slider, the code `RLExperiment.takeAction(this);` will be executed. This single line of code will do the following:
@ -10054,18 +9905,6 @@ Note: If your defined observation contains more fields than the brain was traine
<Code><![CDATA[1]]></Code>
</ParameterValue>
</Parameter>
<Parameter>
<ParameterName><![CDATA[playbackURL]]></ParameterName>
</Parameter>
<Parameter>
<ParameterName><![CDATA[workspaceID]]></ParameterName>
</Parameter>
<Parameter>
<ParameterName><![CDATA[accessKey]]></ParameterName>
</Parameter>
<Parameter>
<ParameterName><![CDATA[simulatorName]]></ParameterName>
</Parameter>
<Parameter>
<ParameterName><![CDATA[BusyCostPerHourA]]></ParameterName>
</Parameter>
@ -10380,8 +10219,9 @@ Note: If your defined observation contains more fields than the brain was traine
<Z>0</Z>
<Rotation>0.0</Rotation>
<Color>-16777216</Color>
<Text><![CDATA[To use this model, paste your bonsai workspace ID and access key into the properties of this experiment.
You may also optionally set the simulator name - this is what will appear in the bonsai platform; its purpose is only to identify the local simulator when starting training.
<Text><![CDATA[To use this experiment, paste your bonsai workspace ID and access key into the fields of the bonsai connector in the Main agent.
In the connector, you can also optionally set the simulator name - this is what will appear in the bonsai platform;
its purpose is only to identify the local simulator when you start the training.
After starting this simulation, follow the listed instructions that appear.
@ -10390,7 +10230,7 @@ As this control includes starting/stopping your model, it's advised not to make
Attempting to pause the model may interrupt the connection.
This method of training is meant primarily for debugging purposes. For optional connection and training speed, it's advised to upload your model to Azure.]]></Text>
This method of training is meant primarily for debugging purposes. For optional connection and training speed, it's advised to upload your model to Bonsai.]]></Text>
<Font>
<Name>SansSerif</Name>
<Size>12</Size>
@ -10407,18 +10247,6 @@ This method of training is meant primarily for debugging purposes. For optional
<Code><![CDATA[0]]></Code>
</ParameterValue>
</Parameter>
<Parameter>
<ParameterName><![CDATA[playbackURL]]></ParameterName>
</Parameter>
<Parameter>
<ParameterName><![CDATA[workspaceID]]></ParameterName>
</Parameter>
<Parameter>
<ParameterName><![CDATA[accessKey]]></ParameterName>
</Parameter>
<Parameter>
<ParameterName><![CDATA[simulatorName]]></ParameterName>
</Parameter>
<Parameter>
<ParameterName><![CDATA[BusyCostPerHourA]]></ParameterName>
</Parameter>
@ -10486,14 +10314,14 @@ This method of training is meant primarily for debugging purposes. For optional
</SimulationExperiment>
<!-- ========= Reinforcemenet Learning Experiment ======== -->
<ReinforcementLearningExperiment ActiveObjectClassId="1208168597834">
<Id>0</Id>
<Id>1625859480089</Id>
<Name><![CDATA[RLExperiment]]></Name>
<MaximumMemory>512</MaximumMemory>
<RandomNumberGenerationType>randomSeed</RandomNumberGenerationType>
<CustomGeneratorCode>new Random()</CustomGeneratorCode>
<SeedValue>1</SeedValue>
<ModelTimeProperties>
<StopOption><![CDATA[Stop at specified time]]></StopOption>
<StopOption><![CDATA[Never]]></StopOption>
<InitialDate><![CDATA[1577865600000]]></InitialDate>
<InitialTime><![CDATA[0.0]]></InitialTime>
<FinalDate><![CDATA[1593504000000]]></FinalDate>
@ -10574,7 +10402,7 @@ ratioCostMoving = zidz(root.accumMoveCost, totalCost);
exceededCapacityFlag = root.exceededCapacity ? 1 : 0;
simTimeMonths = root.time(MONTH);]]></ObservationCode>
<StopCondition><![CDATA[root.exceededCapacity || root.time(MONTH) >= 6]]></StopCondition>
<StopCondition><![CDATA[false]]></StopCondition>
<ActionField>
<Name><![CDATA[numResourceA]]></Name>
<Type><![CDATA[int]]></Type>
@ -10608,7 +10436,8 @@ root.resourceB.resetStats();]]></ActionCode>
<Name><![CDATA[sizeBufferQueues]]></Name>
<Type><![CDATA[int]]></Type>
</ConfigurationField>
<ConfigurationCode><![CDATA[root.ArrivalRate = arrivalRate;
<ConfigurationCode><![CDATA[root.bonsaiMode = 0; // training
root.ArrivalRate = arrivalRate;
root.sizeBufferQueues = sizeBufferQueues;]]></ConfigurationCode>
</ReinforcementLearningExperiment>
</Experiments>