36d2e67319
Fixup a vscode config file. ( #643 )
...
The rest of the repo configs are not prepared to handle separate
workspaces for the different modules (e.g., linting rules break and fail
to find the right config files).
So for now we back this change out in a way that seems more friendly
with existing vscodes.
2024-01-18 18:15:50 -06:00
713510f3f3
Add unit tests for `MlosCoreoptimizer._to_df()` functionality; use special values in tunables ( #638 )
...
* [x] Use special values outside of the regular range for some tunables
in the unit tests
* [x] Add unit tests to validate the data conversion between tunables
and ConfigSpace-based inputs to mlos_core optimizers
Closes #636
---------
Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>
2024-01-18 23:15:52 +00:00
acddd4579e
Tell pylint to ignore FIXME comments in our code. ( #639 )
...
We can find them other ways.
2024-01-18 17:04:56 -06:00
5daaa26399
Support for special tunable values outside of the range ( #617 )
...
Enables special values outside of the range (e.g., `-1` with a range of
`[0, 100]`).
To do we make use of "conditionals" in ConfigSpace to constrain the
space. This has a number of implementation implications, addressed
below:
* [x] Add support for special values to the `Tunable` class
* [x] Add unit tests for assigning special values outside of the range
to the `Tunable` objects
* [x] Add special values outside of the range to the unit tests for
`ConfigSpace` conversion
* [x] Implement proper `TunableGroups` to `ConfigSpace` conversion for
tunables with special values
* [x] Update `mlos_core` optimizers to support conditionals and special
values in `ConfigSpace`
* [x] Add more unit tests to check the conversion
* [x] Make LlamaTune adapter support conditionals in `ConfigSpace`
---------
Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>
2024-01-17 13:29:53 -08:00
0678c439a5
Bump actions/cache from 3 to 4 in /.github/workflows ( #637 )
...
Bumps [actions/cache](https://github.com/actions/cache ) from 3 to 4.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/cache/releases ">actions/cache's
releases</a>.</em></p>
<blockquote>
<h2>v4.0.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Update action to node20 by <a
href="https://github.com/takost "><code>@takost</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1284 ">actions/cache#1284</a></li>
<li>feat: save-always flag by <a
href="https://github.com/to-s "><code>@to-s</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1242 ">actions/cache#1242</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/takost "><code>@takost</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/cache/pull/1284 ">actions/cache#1284</a></li>
<li><a href="https://github.com/to-s "><code>@to-s</code></a> made their
first contribution in <a
href="https://redirect.github.com/actions/cache/pull/1242 ">actions/cache#1242</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/cache/compare/v3...v4.0.0 ">https://github.com/actions/cache/compare/v3...v4.0.0 </a></p>
<h2>v3.3.3</h2>
<h2>What's Changed</h2>
<ul>
<li>Cache v3.3.3 by <a
href="https://github.com/robherley "><code>@robherley</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1302 ">actions/cache#1302</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/robherley "><code>@robherley</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/cache/pull/1302 ">actions/cache#1302</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/cache/compare/v3...v3.3.3 ">https://github.com/actions/cache/compare/v3...v3.3.3 </a></p>
<h2>v3.3.2</h2>
<h2>What's Changed</h2>
<ul>
<li>Fixed readme with new segment timeout values by <a
href="https://github.com/kotewar "><code>@kotewar</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1133 ">actions/cache#1133</a></li>
<li>Readme fixes by <a
href="https://github.com/kotewar "><code>@kotewar</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1134 ">actions/cache#1134</a></li>
<li>Updated description of the lookup-only input for main action by <a
href="https://github.com/kotewar "><code>@kotewar</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1130 ">actions/cache#1130</a></li>
<li>Change two new actions mention as quoted text by <a
href="https://github.com/bishal-pdMSFT "><code>@bishal-pdMSFT</code></a>
in <a
href="https://redirect.github.com/actions/cache/pull/1131 ">actions/cache#1131</a></li>
<li>Update Cross-OS Caching tips by <a
href="https://github.com/pdotl "><code>@pdotl</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1122 ">actions/cache#1122</a></li>
<li>Bazel example (Take <a
href="https://redirect.github.com/actions/cache/issues/2 ">#2</a>️⃣) by
<a href="https://github.com/vorburger "><code>@vorburger</code></a> in
<a
href="https://redirect.github.com/actions/cache/pull/1132 ">actions/cache#1132</a></li>
<li>Remove actions to add new PRs and issues to a project board by <a
href="https://github.com/jorendorff "><code>@jorendorff</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1187 ">actions/cache#1187</a></li>
<li>Consume latest toolkit and fix dangling promise bug by <a
href="https://github.com/chkimes "><code>@chkimes</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1217 ">actions/cache#1217</a></li>
<li>Bump action version to 3.3.2 by <a
href="https://github.com/bethanyj28 "><code>@bethanyj28</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1236 ">actions/cache#1236</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/vorburger "><code>@vorburger</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/cache/pull/1132 ">actions/cache#1132</a></li>
<li><a
href="https://github.com/jorendorff "><code>@jorendorff</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/cache/pull/1187 ">actions/cache#1187</a></li>
<li><a href="https://github.com/chkimes "><code>@chkimes</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/cache/pull/1217 ">actions/cache#1217</a></li>
<li><a
href="https://github.com/bethanyj28 "><code>@bethanyj28</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/cache/pull/1236 ">actions/cache#1236</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/cache/compare/v3...v3.3.2 ">https://github.com/actions/cache/compare/v3...v3.3.2 </a></p>
<h2>v3.3.1</h2>
<h2>What's Changed</h2>
<ul>
<li>Reduced download segment size to 128 MB and timeout to 10 minutes by
<a href="https://github.com/kotewar "><code>@kotewar</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1129 ">actions/cache#1129</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/cache/compare/v3...v3.3.1 ">https://github.com/actions/cache/compare/v3...v3.3.1 </a></p>
<h2>v3.3.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Bug: Permission is missing in cache delete example by <a
href="https://github.com/kotokaze "><code>@kotokaze</code></a> in <a
href="https://redirect.github.com/actions/cache/pull/1123 ">actions/cache#1123</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/actions/cache/blob/main/RELEASES.md ">actions/cache's
changelog</a>.</em></p>
<blockquote>
<h1>Releases</h1>
<h3>3.0.0</h3>
<ul>
<li>Updated minimum runner version support from node 12 -> node
16</li>
</ul>
<h3>3.0.1</h3>
<ul>
<li>Added support for caching from GHES 3.5.</li>
<li>Fixed download issue for files > 2GB during restore.</li>
</ul>
<h3>3.0.2</h3>
<ul>
<li>Added support for dynamic cache size cap on GHES.</li>
</ul>
<h3>3.0.3</h3>
<ul>
<li>Fixed avoiding empty cache save when no files are available for
caching. (<a
href="https://redirect.github.com/actions/cache/issues/624 ">issue</a>)</li>
</ul>
<h3>3.0.4</h3>
<ul>
<li>Fixed tar creation error while trying to create tar with path as
<code>~/</code> home folder on <code>ubuntu-latest</code>. (<a
href="https://redirect.github.com/actions/cache/issues/689 ">issue</a>)</li>
</ul>
<h3>3.0.5</h3>
<ul>
<li>Removed error handling by consuming actions/cache 3.0 toolkit, Now
cache server error handling will be done by toolkit. (<a
href="https://redirect.github.com/actions/cache/pull/834 ">PR</a>)</li>
</ul>
<h3>3.0.6</h3>
<ul>
<li>Fixed <a
href="https://redirect.github.com/actions/cache/issues/809 ">#809</a> -
zstd -d: no such file or directory error</li>
<li>Fixed <a
href="https://redirect.github.com/actions/cache/issues/833 ">#833</a> -
cache doesn't work with github workspace directory</li>
</ul>
<h3>3.0.7</h3>
<ul>
<li>Fixed <a
href="https://redirect.github.com/actions/cache/issues/810 ">#810</a> -
download stuck issue. A new timeout is introduced in the download
process to abort the download if it gets stuck and doesn't finish within
an hour.</li>
</ul>
<h3>3.0.8</h3>
<ul>
<li>Fix zstd not working for windows on gnu tar in issues <a
href="https://redirect.github.com/actions/cache/issues/888 ">#888</a> and
<a
href="https://redirect.github.com/actions/cache/issues/891 ">#891</a>.</li>
<li>Allowing users to provide a custom timeout as input for aborting
download of a cache segment using an environment variable
<code>SEGMENT_DOWNLOAD_TIMEOUT_MINS</code>. Default is 60 minutes.</li>
</ul>
<h3>3.0.9</h3>
<ul>
<li>Enhanced the warning message for cache unavailablity in case of
GHES.</li>
</ul>
<h3>3.0.10</h3>
<ul>
<li>Fix a bug with sorting inputs.</li>
<li>Update definition for restore-keys in README.md</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="13aacd865chttps://redirect.github.com/actions/cache/issues/1242 ">#1242</a>
from to-s/main</li>
<li><a
href="53b35c543965b8989fabhttps://redirect.github.com/actions/cache/issues/1284 ">#1284</a>
from takost/update-to-node-20</li>
<li><a
href="d0be34d54466cf064d471326563738e71876755e01229828ff3185ecfd6125618a0a67https://github.com/actions/cache/compare/v3...v4 ">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores )
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-17 11:50:04 -06:00
8d5239cc32
Add more info on const_args vs. tunable_params; start the DEVNOTES ( #630 )
...
Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>
2024-01-16 16:16:13 -06:00
769e43e96f
Add labels to the optimizer trends plot ( #634 )
...
Co-authored-by: Sergiy Matusevych <sergiym@microsoft.com>
2024-01-16 15:18:30 -06:00
52fdc3c2e9
Github Action syntax fixup ( #632 )
...
Fixup to #631
2024-01-16 13:05:24 -08:00
1f3b26d9f0
Introduce basic experiment visualization module mlos_viz via dabl ( #624 )
...
Adds a basic `mlos_viz.plot(exp)` style API for simple visualizations of
`ExperimentData` results relative to the experiment's objectives
(building off of #628 and https://github.com/dabl/dabl/pull/335 ).
Note: this PR currently omits unit tests for the new module due to the
complexity of testing visualizations. We intend to add this in future
PRs. There is however, a working example of its use here right now:
https://github.com/Microsoft-CISL/sqlite-autotuning/pull/41
---------
Co-authored-by: Sergiy Matusevych <sergiy.matusevych@gmail.com>
2024-01-16 18:48:20 +00:00
221cee31ee
Add support for storing the optimization targets and direction of an experiment ( #628 )
...
This PR is useful for mlos-viz and dabl wrapper (#624 ) to be able to
automatically graph the results for a given optimization target, for
instance via something like the following:
```python
for opt_target in exp.objectives:
dabl.plot(exp.results, opt_target)
```
Since the prior efforts on capturing this data in the Trial metadata are
somewhat problematic (allow conflicting changes between runs of an
experiment, don't support multi-objective), we extend them to also store
values directly as a part of the Experiment, which is a somewhat more
appropriate location. Upon retrieval, an attempt is also made to merge
the two data sources for backwards compatibility.
This PR does not enforce strictness on that metadata, but future
versions could (e.g., disallow resuming an Experiment if it looks like
the objective targets have changed. In that case the prior Trial results
can potentially still be used to prewarm a new Experiment's optimizer).
---------
Co-authored-by: Sergiy Matusevych <sergiy.matusevych@gmail.com>
2024-01-16 18:11:01 +00:00
42a149f87f
CI tweak: use "release" event instead of "tag push" for pypi publishing ( #631 )
...
Minor tweak on #626 : instead of acting on tag push events, we only act
on "release" events. This gives us a chance to fix bad tag pushes (which
otherwise don't have much gated checking on them) prior to attempting to
publish a version on pypi, which cannot be overwritten (only
superseded).
2024-01-16 11:13:07 -06:00
c8639c09a4
Rename and improve tunable vs config API and documentation ( #629 )
...
Minor tweaks to make it easier to understand the difference between a
(tunable) config (for an Environment) and a config (part of the
framework) when interacting with the storage layer (who's schema
switches terminology slightly than the code).
2024-01-11 11:12:36 -08:00
2032d7908f
Readme tweaks ( #627 )
...
Backreference links to/from pypi
2024-01-10 12:37:30 -08:00
c7df7ff99a
Pypi packaging ( #626 )
...
Co-authored-by: Sergiy Matusevych <sergiy.matusevych@gmail.com>
2024-01-09 22:40:15 +00:00
3415bcc93d
Cleanup some deprecated VSCode settings ( #622 )
...
Co-authored-by: Sergiy Matusevych <sergiym@microsoft.com>
2024-01-09 15:59:43 -06:00
564df0e041
Tunable fixups ( #623 )
...
Improvements on #549
- Handles the case where the default for a tunable is `False` (else it
wouldn't be converted to a `str`).
- Adds more details during debug and error output to make
troubleshooting easier
2024-01-03 13:35:54 -08:00
e02c036501
Bump github/codeql-action from 2 to 3 in /.github/workflows ( #620 )
...
Bumps [github/codeql-action](https://github.com/github/codeql-action )
from 2 to 3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/github/codeql-action/releases ">github/codeql-action's
releases</a>.</em></p>
<blockquote>
<h2>CodeQL Bundle v2.15.4</h2>
<p>Bundles CodeQL CLI v2.15.4</p>
<ul>
<li>(<a
href="https://github.com/github/codeql-cli-binaries/blob/HEAD/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql-cli-binaries/releases/tag/v2.15.4 ">release</a>)</li>
</ul>
<p>Includes the following CodeQL language packs from <a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4 "><code>github/codeql@codeql-cli/v2.15.4</code></a>:</p>
<ul>
<li><code>codeql/cpp-queries</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/cpp/ql/src/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/cpp/ql/src ">source</a>)</li>
<li><code>codeql/cpp-all</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/cpp/ql/lib/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/cpp/ql/lib ">source</a>)</li>
<li><code>codeql/csharp-queries</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/csharp/ql/src/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/csharp/ql/src ">source</a>)</li>
<li><code>codeql/csharp-all</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/csharp/ql/lib/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/csharp/ql/lib ">source</a>)</li>
<li><code>codeql/go-queries</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/go/ql/src/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/go/ql/src ">source</a>)</li>
<li><code>codeql/go-all</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/go/ql/lib/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/go/ql/lib ">source</a>)</li>
<li><code>codeql/java-queries</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/java/ql/src/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/java/ql/src ">source</a>)</li>
<li><code>codeql/java-all</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/java/ql/lib/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/java/ql/lib ">source</a>)</li>
<li><code>codeql/javascript-queries</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/javascript/ql/src/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/javascript/ql/src ">source</a>)</li>
<li><code>codeql/javascript-all</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/javascript/ql/lib/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/javascript/ql/lib ">source</a>)</li>
<li><code>codeql/python-queries</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/python/ql/src/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/python/ql/src ">source</a>)</li>
<li><code>codeql/python-all</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/python/ql/lib/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/python/ql/lib ">source</a>)</li>
<li><code>codeql/ruby-queries</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/ruby/ql/src/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/ruby/ql/src ">source</a>)</li>
<li><code>codeql/ruby-all</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/ruby/ql/lib/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/ruby/ql/lib ">source</a>)</li>
<li><code>codeql/swift-queries</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/swift/ql/src/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/swift/ql/src ">source</a>)</li>
<li><code>codeql/swift-all</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/swift/ql/lib/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.4/swift/ql/lib ">source</a>)</li>
</ul>
<h2>CodeQL Bundle</h2>
<p>Bundles CodeQL CLI v2.15.3</p>
<ul>
<li>(<a
href="https://github.com/github/codeql-cli-binaries/blob/HEAD/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql-cli-binaries/releases/tag/v2.15.3 ">release</a>)</li>
</ul>
<p>Includes the following CodeQL language packs from <a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3 "><code>github/codeql@codeql-cli/v2.15.3</code></a>:</p>
<ul>
<li><code>codeql/cpp-queries</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/cpp/ql/src/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/cpp/ql/src ">source</a>)</li>
<li><code>codeql/cpp-all</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/cpp/ql/lib/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/cpp/ql/lib ">source</a>)</li>
<li><code>codeql/csharp-queries</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/csharp/ql/src/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/csharp/ql/src ">source</a>)</li>
<li><code>codeql/csharp-all</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/csharp/ql/lib/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/csharp/ql/lib ">source</a>)</li>
<li><code>codeql/go-queries</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/go/ql/src/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/go/ql/src ">source</a>)</li>
<li><code>codeql/go-all</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/go/ql/lib/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/go/ql/lib ">source</a>)</li>
<li><code>codeql/java-queries</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/java/ql/src/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/java/ql/src ">source</a>)</li>
<li><code>codeql/java-all</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/java/ql/lib/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/java/ql/lib ">source</a>)</li>
<li><code>codeql/javascript-queries</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/javascript/ql/src/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/javascript/ql/src ">source</a>)</li>
<li><code>codeql/javascript-all</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/javascript/ql/lib/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/javascript/ql/lib ">source</a>)</li>
<li><code>codeql/python-queries</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/python/ql/src/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/python/ql/src ">source</a>)</li>
<li><code>codeql/python-all</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/python/ql/lib/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/python/ql/lib ">source</a>)</li>
<li><code>codeql/ruby-queries</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/ruby/ql/src/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/ruby/ql/src ">source</a>)</li>
<li><code>codeql/ruby-all</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/ruby/ql/lib/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/ruby/ql/lib ">source</a>)</li>
<li><code>codeql/swift-queries</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/swift/ql/src/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/swift/ql/src ">source</a>)</li>
<li><code>codeql/swift-all</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/swift/ql/lib/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.3/swift/ql/lib ">source</a>)</li>
</ul>
<h2>CodeQL Bundle</h2>
<p>Bundles CodeQL CLI v2.15.2</p>
<ul>
<li>(<a
href="https://github.com/github/codeql-cli-binaries/blob/HEAD/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql-cli-binaries/releases/tag/v2.15.2 ">release</a>)</li>
</ul>
<p>Includes the following CodeQL language packs from <a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.2 "><code>github/codeql@codeql-cli/v2.15.2</code></a>:</p>
<ul>
<li><code>codeql/cpp-queries</code> (<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.2/cpp/ql/src/CHANGELOG.md ">changelog</a>,
<a
href="https://github.com/github/codeql/tree/codeql-cli/v2.15.2/cpp/ql/src ">source</a>)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/github/codeql-action/blob/main/CHANGELOG.md ">github/codeql-action's
changelog</a>.</em></p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="b374143c11https://redirect.github.com/github/codeql-action/issues/2034 ">#2034</a>
from github/update-v3.22.11-64e61baea</li>
<li><a
href="e2b5cc75ce64e61baeachttps://redirect.github.com/github/codeql-action/issues/2006 ">#2006</a>
from github/nickfyson/node-20</li>
<li><a
href="c757f9f6de7898bc20416b5b958063ea1e72c669b974542e9fb995212303https://redirect.github.com/github/codeql-action/issues/2024 ">#2024</a>)</li>
<li><a
href="3c1878d8f9https://redirect.github.com/github/codeql-action/issues/2029 ">#2029</a>
from github/mergeback/v2.22.10-to-main-305f6546</li>
<li>Additional commits viewable in <a
href="https://github.com/github/codeql-action/compare/v2...v3 ">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores )
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-15 15:27:14 -06:00
3824f0c320
Move deploymentName from the Service config to the Environment config ( #616 )
...
Needed so that we can support multiple Environment deployments under the
same Service.
Closes #610
2023-12-01 13:58:08 -06:00
67ccdac441
Abstract OS environment variable case sensitivity name handling across platforms ( #618 )
...
Fixes a bug introduced in #614 for Windows tests.
On Windows,
```python
os.environ["string"] = "str"
```
internally gets replaced as
```python
os.environ["STRING"]
```
The workaround is to use `nt.environ` on Windows instead.
This change introduces an abstraction to use a new simple drop in
replacement library everywhere we need to handle environment variables.
2023-12-01 13:30:03 -06:00
bab3ca5d49
Add NetworkEnv and AzureNetworkService ( #606 )
...
Adds support for modeling virtual networks as an Environment in order to
deploy virtual networks separately from VMs.
This can be useful if we want to, for instance, leave and reuse virtual
networks accessible across different experiments.
To do this, we also move common code from `azure_vm_services` to a base
class in `azure_services`.
- [x] `NetworkEnv` implementation
- [x] `NetworkEnv` schema
- [ ] `NetworkEnv` tests (kinda hard to do with the current test infra,
may skip for now)
- [x] `AzureNetworkService` implementation
- [x] `AzureNetworkService` schema
- [x] `AzureNetworkService` tests
- [x] Add example config
---------
Co-authored-by: Sergiy Matusevych <sergiym@microsoft.com>
2023-11-29 23:29:30 +00:00
3b9d0b443a
Add support for recursive dictionary templating to more places ( #614 )
...
Fixes extracted from #606
This expands support for doing secondary `$var` string template
expansion to two additional places beyond `global_config`:
- `const_args`
- `deploymentTemplateParameters` in `AzureVmService` (and soon
`AzureNetworkService`)
Expands a few test cases to check for this and relies on one prior
existing one to still pass (the `launch_parse_args` tests which already
checked for this behavior in `global_config` expansion).
2023-11-29 23:16:21 +00:00
59cc179a58
Enforce correct code coverage threshold ( #615 )
...
Corrects this error:
```
---------- coverage: platform linux, python 3.11.5-final-0 -----------
Coverage HTML written to dir htmlcov
Coverage XML written to file coverage.xml
Required test coverage of 0.92% reached. Total coverage: 93.12%
=========================== short test summary info ============================
```
2023-11-29 21:37:48 +00:00
ea594eb112
Code coverage tweaks ( #609 )
2023-11-28 09:41:46 -06:00
9cad1aee8d
Bump conda-incubator/setup-miniconda from 2 to 3 in /.github/workflows ( #611 )
...
Bumps
[conda-incubator/setup-miniconda](https://github.com/conda-incubator/setup-miniconda )
from 2 to 3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/conda-incubator/setup-miniconda/releases ">conda-incubator/setup-miniconda's
releases</a>.</em></p>
<blockquote>
<h2>Version 3.0.0</h2>
<h3>Features</h3>
<ul>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/308 ">#308</a>
Update to node20</li>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/291 ">#291</a>
Add conda-solver option (defaults to libmamba)</li>
</ul>
<h3>Fixes</h3>
<ul>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/299 ">#299</a>
Fix condaBasePath when useBundled is false, and there's no pre-existing
conda</li>
</ul>
<h3>Documentation</h3>
<ul>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/309 ">#309</a>
Switch to main branch based development</li>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/313 ">#313</a>
Specify team conda-incubator/setup-miniconda as codeowners</li>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/318 ">#318</a>
README: update actions in examples, add security section, similar
actions</li>
</ul>
<h3>Tasks and Maintenance</h3>
<ul>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/307 ">#307</a>
Run dependabot against main branch and also update node packages</li>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/311 ">#311</a>
Bump actions/checkout from 2 to 4</li>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/310 ">#310</a>
Bump actions/cache from 1 to 3</li>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/314 ">#314</a>
Strip/update dependencies</li>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/315 ">#315</a>
Split lint into check and build, switch from <code>npm install</code> to
<code>npm ci</code></li>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/317 ">#317</a>
Bump normalize-url from 4.5.1 to 8.0.0</li>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/316 ">#316</a>
Faster workflow response / saving resources via timeout/concurrency
policy</li>
</ul>
<p><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/308 ">#308</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/308 ">conda-incubator/setup-miniconda#308</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/291 ">#291</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/291 ">conda-incubator/setup-miniconda#291</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/299 ">#299</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/299 ">conda-incubator/setup-miniconda#299</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/309 ">#309</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/309 ">conda-incubator/setup-miniconda#309</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/313 ">#313</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/313 ">conda-incubator/setup-miniconda#313</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/318 ">#318</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/318 ">conda-incubator/setup-miniconda#318</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/307 ">#307</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/307 ">conda-incubator/setup-miniconda#307</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/311 ">#311</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/311 ">conda-incubator/setup-miniconda#311</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/310 ">#310</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/310 ">conda-incubator/setup-miniconda#310</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/314 ">#314</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/314 ">conda-incubator/setup-miniconda#314</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/315 ">#315</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/315 ">conda-incubator/setup-miniconda#315</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/317 ">#317</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/317 ">conda-incubator/setup-miniconda#317</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/316 ">#316</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/316 ">conda-incubator/setup-miniconda#316</a></p>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/isuruf "><code>@isuruf</code></a> made
their first contribution in <a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/299 ">conda-incubator/setup-miniconda#299</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/conda-incubator/setup-miniconda/compare/v2...v3.0.0 ">https://github.com/conda-incubator/setup-miniconda/compare/v2...v3.0.0 </a></p>
<h2>Version 2.3.0</h2>
<h3>Documentation</h3>
<ul>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/263 ">#263</a>
Update links to GitHub shell docs</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/conda-incubator/setup-miniconda/blob/main/CHANGELOG.md ">conda-incubator/setup-miniconda's
changelog</a>.</em></p>
<blockquote>
<h2><a
href="https://github.com/conda-incubator/setup-miniconda/releases/tag/v2.3.0 ">v3.0.0</a>
(2023-11-27)</h2>
<h3>Features</h3>
<ul>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/308 ">#308</a>
Update to node20</li>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/291 ">#291</a>
Add conda-solver option (defaults to libmamba)</li>
</ul>
<h3>Fixes</h3>
<ul>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/299 ">#299</a>
Fix condaBasePath when useBundled is false, and there's no pre-existing
conda</li>
</ul>
<h3>Documentation</h3>
<ul>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/309 ">#309</a>
Switch to main branch based development</li>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/313 ">#313</a>
Specify team conda-incubator/setup-miniconda as codeowners</li>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/318 ">#318</a>
README: update actions in examples, add security section, similar
actions</li>
</ul>
<h3>Tasks and Maintenance</h3>
<ul>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/307 ">#307</a>
Run dependabot against main branch and also update node packages</li>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/311 ">#311</a>
Bump actions/checkout from 2 to 4</li>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/310 ">#310</a>
Bump actions/cache from 1 to 3</li>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/314 ">#314</a>
Strip/update dependencies</li>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/315 ">#315</a>
Split lint into check and build, switch from <code>npm install</code> to
<code>npm ci</code></li>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/317 ">#317</a>
Bump normalize-url from 4.5.1 to 8.0.0</li>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/316 ">#316</a>
Faster workflow response / saving resources via timeout/concurrency
policy</li>
</ul>
<p><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/308 ">#308</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/308 ">conda-incubator/setup-miniconda#308</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/291 ">#291</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/291 ">conda-incubator/setup-miniconda#291</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/299 ">#299</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/299 ">conda-incubator/setup-miniconda#299</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/309 ">#309</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/309 ">conda-incubator/setup-miniconda#309</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/313 ">#313</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/313 ">conda-incubator/setup-miniconda#313</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/318 ">#318</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/318 ">conda-incubator/setup-miniconda#318</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/307 ">#307</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/307 ">conda-incubator/setup-miniconda#307</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/311 ">#311</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/311 ">conda-incubator/setup-miniconda#311</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/310 ">#310</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/310 ">conda-incubator/setup-miniconda#310</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/314 ">#314</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/314 ">conda-incubator/setup-miniconda#314</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/315 ">#315</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/315 ">conda-incubator/setup-miniconda#315</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/317 ">#317</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/317 ">conda-incubator/setup-miniconda#317</a>
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/316 ">#316</a>:
<a
href="https://redirect.github.com/conda-incubator/setup-miniconda/pull/316 ">conda-incubator/setup-miniconda#316</a></p>
<h2>[v2.3.0] (2023-11-22)</h2>
<h3>Documentation</h3>
<ul>
<li><a
href="https://redirect.github.com/conda-incubator/setup-miniconda/issues/263 ">#263</a>
Update links to GitHub shell docs</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="2defc80cc6https://redirect.github.com/conda-incubator/setup-miniconda/issues/320 ">#320</a>)</li>
<li><a
href="0d5a56b9ebhttps://redirect.github.com/conda-incubator/setup-miniconda/issues/319 ">#319</a>)</li>
<li><a
href="45fd3f9089https://redirect.github.com/conda-incubator/setup-miniconda/issues/316 ">#316</a>
from dbast/timeout</li>
<li><a
href="d1e04fc267https://redirect.github.com/conda-incubator/setup-miniconda/issues/299 ">#299</a>
from isuruf/condaBasePath</li>
<li><a
href="fab0073840https://redirect.github.com/conda-incubator/setup-miniconda/issues/318 ">#318</a>
from dbast/readme</li>
<li><a
href="fa6bdf9643d42f8b884a624cf702708874247bd8https://redirect.github.com/conda-incubator/setup-miniconda/issues/291 ">#291</a>)</li>
<li><a
href="ce210e19d5https://redirect.github.com/conda-incubator/setup-miniconda/issues/317 ">#317</a>
from conda-incubator/dependabot/npm_and_yarn/main/nor...</li>
<li>Additional commits viewable in <a
href="https://github.com/conda-incubator/setup-miniconda/compare/v2...v3 ">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores )
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-27 13:32:23 -06:00
a1288249e9
Allow specifying services in CLI args or top level cli config. ( #608 )
...
Closes #607
2023-11-22 21:29:18 +00:00
28b377e275
Update schema docs to reflect new unified schema ( #605 )
...
Minor updates to docs following #596 .
- [x] ~Pending https://github.com/SchemaStore/schemastore/pull/3402
approval for some of the content.~
See Also: #522
2023-11-21 11:12:07 -08:00
cd2a9e6061
Rename azure_services to azure_vm_services ( #604 )
...
Working on adding `azure_network_services` so this is mostly a cosmetic
change, but helps keep things logically separate for that upcoming
change.
---------
Co-authored-by: Sergiy Matusevych <sergiym@microsoft.com>
2023-11-20 22:14:11 +00:00
ffa3dd7ba3
Retries for HTTP requests ( #598 )
...
This PR enables automatic retry and backoffs for HTTP requests made
using requests, mainly targeting GETs.
This is to address intermittent connection issues observed during
long-running operations such as polling, which would have resulted in
OSErrors crashing MLOS and ending experiments prematurely.
The main patterns used are:
-
https://docs.python-requests.org/en/latest/user/advanced/#example-automatic-retries
-
https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html#module-urllib3.util.retry
---------
Co-authored-by: Eu Jing Chua <eujingchua@microsoft.com>
Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>
2023-11-20 21:48:47 +00:00
ceab0d26d1
Explicitly add the project root to pytest arguments ( #603 )
...
My VSCode adds `"."` to pytest arguments in config every time I rebuild
the container - and deletes the `--log-level=DEBUG` argument at teh same
time. With this small fix, vscode can accept the project config as we
have it in git without modifying it
Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>
2023-11-20 21:30:09 +00:00
65ad423a47
Remove textwrap from help docstrings ( #602 )
...
Apply suggestions from #601
2023-11-20 21:02:59 +00:00
6bb0124bf9
experiment_id documentation improvements ( #601 )
...
Closes #599
2023-11-20 14:50:49 -06:00
0ae67ab751
Single schema definition support ( #596 )
...
Part of #522
2023-11-17 00:32:42 +00:00
2e80139f69
Allow storage RG override for separate control plane RG vs experiment RG ( #597 )
...
Allow overriding which RG to look for the storage account in when
separating experiment RG from control plane RG
2023-11-16 13:55:59 -08:00
2d77d96827
Fix typo in json.dumps ( #595 )
...
Co-authored-by: Eu Jing Chua <eujingchua@microsoft.com>
2023-11-14 15:27:44 -06:00
a8915c2dcb
Make FIO benchmark a configurable parameter ( #593 )
...
Also, fix the shell env variables declarations and rename the output
files for readability
Closes #500
Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>
2023-11-14 19:30:25 +00:00
9102ce5c8b
Mypy version fixups ( #594 )
2023-11-14 19:16:54 +00:00
9a4f90fe12
More fixups to setup control plane rg arm template ( #592 )
...
Allows deploying a premium fileshare as well.
2023-11-09 09:05:40 -08:00
e90fb39f03
Azure RG setup script fixups ( #590 )
...
Co-authored-by: Sergiy Matusevych <sergiym@microsoft.com>
2023-11-08 21:28:08 +00:00
5f56c462ed
Add unit tests for LocalFileShareEnv $var expansion in upload and download paths ( #587 )
2023-11-08 20:06:21 +00:00
125417b179
include the links to the coverage images for cross link fixups ( #588 )
...
Fixups to #524
2023-11-08 11:48:28 -08:00
85fa2d0977
allow using all parameters in paths witout declaring them in `shell_env_params` ( #586 )
...
Closes #585
2023-11-08 00:35:18 +00:00
231c5e61e2
Some documentation fixups ( #524 )
...
- fix a quoting issue in an example
- cross link the `sqlite-autotuning` repo
- reuse the content from the main README.md's for the github pages site
- general improvements to the landing page docs
---------
Co-authored-by: Sergiy Matusevych <sergiym@microsoft.com>
2023-11-08 00:11:57 +00:00
6b4cf3ee5f
SaaS Environment + Azure Flex DB configuration service ( #520 )
...
Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>
2023-11-07 15:31:34 -08:00
4a8e54aae7
Implement get_auth_headers() method in AzureAuthService ( #584 )
...
...and use it in `AzureVMService`. We will use it in #520 once it is
merged in.
Closes #581
2023-11-06 15:47:19 -08:00
25e144e444
Pylint and test fixups ( #583 )
2023-11-06 23:22:07 +00:00
82af33b3af
Clean-up temp files created by mlos_core optimizers ( #579 )
2023-11-06 15:09:29 -08:00
108f5ded3c
Tests for shell env var population ( #578 )
...
Follow up to #573
2023-11-03 16:32:12 -07:00
35f5968dc0
Fix asyncio issues ( #577 )
...
Tries to handle a few issues we've seen with the event loop code.
In particular, Windows sometimes hangs and seems to have lingering
coroutines (sometimes added by it's default ProactorEventLoopPolicy).
This change adds some additional tests and debug code and switches to
the SelectorEventLoopPolicy for Windows.
Note: this comes with some drawbacks like not being able to handle
subprocesses in async code, which could prove to be a problem for us
with parallel executors in the future:
https://docs.python.org/3/library/asyncio-platforms.html#windows
---------
Co-authored-by: Sergiy Matusevych <sergiym@microsoft.com>
2023-11-03 14:04:22 -07:00
64d7cd5f87
Fixup issues with VSCode and pytest working directory paths in different environments ( #576 )
...
Previously #553 solved this for pytest inside VSCode for WSL,
devcontainer within WSL, and Windows directly dev environments by
looking at the `$VSCODE_CWD` environment variable that the Pytest VSCode
extension sets to note the difference between the expected environment
variable `$PWD` and the result of `os.getcwd()`.
However, when the devcontainer is launched from Windows, the VSCode
pytest plugin takes a different approach and launches a proxy server in
WSL to run the tests. In that mode, `$VSCODE_CWD` is still set but is no
longer the path used.
To avoid fighting with all these different scenarios, we change the test
to set `$CUSTOM_PATH_FROM_ENV` via `os.environ` prior to the start of
the test, and then make sure that it's expanded when we use it inside
`global_test_config.jsonc`
Closed #575
2023-11-03 20:00:30 +00:00
e2b8564588
Parallelize ssh tests ( #570 )
...
Note: current includes changes from #557 as well.
Closes #563
Removes reliance on `--dist=loadgroup` and
`@pytest.mark.xdist_group(...)` to run SSH tests in a single worker by
implementing a portable file-lock based serialization method for test
service setup/teardown and separating out services that might interact
between tests.
In my local testing, this seems to solve both the "successful but
non-reported tests" issue, limited discoverability of tests in vscode
upon failure, and lack of parallel execution in all dev environments
(Windows, WSL Devcontainer, Linux).
On occassion in Windows I have seen the following error, but it is after
tests have passed, so seems to be part of pytest cleanup:
```
...
[gw4] [100%] PASSED mlos_bench/mlos_bench/tests/services/remote/ssh/test_ssh_host_service.py::test_ssh_service_reboot
=================================================================================================================================== warnings summary ====================================================================================================================================
mlos_bench/mlos_bench/tests/services/remote/ssh/test_ssh_fileshare.py::test_ssh_fileshare_upload_file_dne
C:\Users\bpkroth\.conda\envs\mlos\lib\site-packages\_pytest\threadexception.py:73: PytestUnhandledThreadExceptionWarning: Exception in thread Thread-9 (_run_event_loop)
Traceback (most recent call last):
File "C:\Users\bpkroth\.conda\envs\mlos\lib\asyncio\windows_events.py", line 494, in finish_recv
return ov.getresult()
OSError: [WinError 995] The I/O operation has been aborted because of either a thread exit or an application request
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\bpkroth\.conda\envs\mlos\lib\asyncio\windows_events.py", line 826, in _poll
value = callback(transferred, key, ov)
File "C:\Users\bpkroth\.conda\envs\mlos\lib\asyncio\windows_events.py", line 498, in finish_recv
raise ConnectionResetError(*exc.args)
ConnectionResetError: [WinError 995] The I/O operation has been aborted because of either a thread exit or an application request
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\bpkroth\.conda\envs\mlos\lib\threading.py", line 1016, in _bootstrap_inner
self.run()
File "C:\Users\bpkroth\.conda\envs\mlos\lib\threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\bpkroth\src\MLOS\mlos_bench\mlos_bench\event_loop_context.py", line 48, in _run_event_loop
self._event_loop.run_forever()
File "C:\Users\bpkroth\.conda\envs\mlos\lib\asyncio\windows_events.py", line 321, in run_forever
super().run_forever()
File "C:\Users\bpkroth\.conda\envs\mlos\lib\asyncio\base_events.py", line 603, in run_forever
self._run_once()
File "C:\Users\bpkroth\.conda\envs\mlos\lib\asyncio\base_events.py", line 1871, in _run_once
event_list = self._selector.select(timeout)
File "C:\Users\bpkroth\.conda\envs\mlos\lib\asyncio\windows_events.py", line 444, in select
self._poll(timeout)
File "C:\Users\bpkroth\.conda\envs\mlos\lib\asyncio\windows_events.py", line 828, in _poll
f.set_exception(e)
File "C:\Users\bpkroth\.conda\envs\mlos\lib\asyncio\windows_events.py", line 88, in set_exception
super().set_exception(exception)
asyncio.exceptions.InvalidStateError: invalid state
warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))
```
---------
Co-authored-by: Sergiy Matusevych <sergiym@microsoft.com>
Co-authored-by: Sergiy Matusevych <sergiy.matusevych@gmail.com>
2023-11-02 20:57:10 +00:00
Brian Kroth
Sergiy Matusevych
dependabot[bot]
Eu Jing Chua