* Try to find the .NET SDK PATH on Mac if it Installs to Unexpected Location
Our highest failure for .NET SDK Installs is that the `InstallationValidator` which checks the hard-coded expected path of Installation does not find .NET in the expected location. Unfortunately, nobody has reported this.
To get to this point in the code of `Validation`, everything has to have succeeded, including the installer only returning and exiting 0, which it would only do upon installation success.
I'm guessing there is some undocumented case where the .NET SDK is installed to a different location, but I couldn't find where.
Here is our path check. https://github.com/dotnet/vscode-dotnet-runtime/blob/main/vscode-dotnet-runtime-library/src/Acquisition/WinMacGlobalInstaller.ts#L239
According to the installer source code and our documentation, it should only install to this location or with x64 during emulation.
```
<PkgInstallDirectory>/usr/local/share/dotnet</PkgInstallDirectory>
<x64EmulationPkgInstallDirectory>/usr/local/share/dotnet/x64</x64EmulationPkgInstallDirectory>
```
The 64 directory however, is only for when you install a 64-bit SDK on an ARM Mac. We wouldn't expect this to happen because we only install the SDK that matches os.platform(), so we should never install an x64 SDK on an ARM machine.
I'm curious to see where this path is and this will also help us track that.
It's a bit of a hack, but I think it will have the desired outcome.
* Remove extra check for global since this only happens in global code
* Change 'InstallKey' terminology to 'InstallId'
VS Code has a filter for the string 'key' -- this causes our telemetry to be dropped when referencing the term 'install key' as it thinks it may be a secret. 48ba01b589/src/vs/platform/telemetry/common/telemetryUtils.ts (L336)
Id is safe. I talked with the vscode team and there is no good alternative besides turning off all telemetry safety scanning, which I didn't want to do.
Hence, 'key' --> 'id'
* rename key to id
* rename remaining instances of 'id'
* Improve names of types
* Interpret Installer Exit Codes
These will give the user more actionable items, and also increase the chances the post a log so we can help them if something goes wrong and understand the issue better.
* Add page fault message
* Add Timeout error code
* Use Native Code Message
* Use err message from https://learn.microsoft.com/en-us/windows/win32/debug/system-error-codes--1300-1699-
* Use particular error code from windows
* Use the official error message.
* Add failure handling for file integrity check
This is now what is hitting the EPERM and ENOENT errors.
We need to skip the check and see what happens from here, such as allowing to elevate via windows.
This also adds specific handlers for when we fail to download the SDK.
* missing semicolon
* Make Command Executor return an object instead of string
This prevents us from having to run commands twice to get the status and stdout.
Caution must be taken to make sure that the assumptions here in what is returned as stdout vs stderr are correct.
Furthermore, when looking at the code, I saw something that looked just plain incorrect, so I will need to investigate that by adding the TODO before merge
* Fix Execution Type Parsing
* fix linting issue
* Fix line parsing logic
* Fix tests
* Run icacls to set execute permissions
* move to sha256
* remove unnecessary code
* Remove todo that should go in another PR
* Remove redundant trim
* rename class CommandProcessorOutput since it didnt match CommandExecutor
* Rename class file
* Report json object for failed permissions.
* Fix test to use correct folder
* fix lint
* Fix installer file deletion
* respond to linter
* Try to Add a Timeout for Installer.Exe
This will allow us to not count times when users never respond to the installer as a failure and or remediate when the installer is taking too long.
* Add .editorconfig for style settings
* Update Linux Apt Commands to Handle Lock Timeouts
We also pass the -y flag when it can help push through updates.
Apt will return 100 whenever there is any failure. aa56836331/apt-private/private-cmndline.cc (L602)
This means the errors we are seeing are not just due to locking errors. But, perhaps this will help.
Also this will try to run update twice, to see if that 2nd try will work. It seems a bit silly, but it does actually seem to improve the chance of success when something else is running on the system.
* Update distro failure message b to have the same information as the other failure message
* Fix package is available check
* Remove verbosity as it is not helpful or needed information to help customers
* Remove -y
it doesnt do anything for update lol
* fix json
* try forking
* just rely on node timeout
* update test data
* Handle .net installer timeouts
* remove unnecessary import
* Fix tests
* Include stderr for more failure detail
* fix options check
* Fix option
* Include options that are needed to preserve the working env on a general command execution
* Remove hash as it already gets anonymized by the regular expression
* Increase lock timeout time as now with waiting for lock it can take longer
* Add passive option to automate dialog
* Skip extra hashing thats handled by the regular expression
* Add a timeout to kill our subprocesses that hold onto the lock
* Timeout default is seconds not minutes
* Make code more clear to read by moving else after return into else
* Improve lock file handling on install tracker
* Adhere to style convention with release() block
* add back in -y to automate
---------
Co-authored-by: Ubuntu 1804 <you@example.com>
* Make Command Executor return an object instead of string
This prevents us from having to run commands twice to get the status and stdout.
Caution must be taken to make sure that the assumptions here in what is returned as stdout vs stderr are correct.
Furthermore, when looking at the code, I saw something that looked just plain incorrect, so I will need to investigate that by adding the TODO before merge
* Fix Execution Type Parsing
* fix linting issue
* Fix line parsing logic
* Fix tests
* Run icacls to set execute permissions
* move to sha256
* remove unnecessary code
* Remove todo that should go in another PR
* Remove redundant trim
* rename class CommandProcessorOutput since it didnt match CommandExecutor
* Rename class file
* Report json object for failed permissions.
* Fix test to use correct folder
* fix lint
* Fix installer file deletion
* respond to linter
* Try to Add a Timeout for Installer.Exe
This will allow us to not count times when users never respond to the installer as a failure and or remediate when the installer is taking too long.
* Add .editorconfig for style settings
* try forking
* just rely on node timeout
* Handle .net installer timeouts
* remove unnecessary import
* fix options check
* Fix option
* Include options that are needed to preserve the working env on a general command execution
* Add passive option to automate dialog
* dont use english url
* Make Command Executor return an object instead of string
This prevents us from having to run commands twice to get the status and stdout.
Caution must be taken to make sure that the assumptions here in what is returned as stdout vs stderr are correct.
Furthermore, when looking at the code, I saw something that looked just plain incorrect, so I will need to investigate that by adding the TODO before merge
* Fix Execution Type Parsing
* fix linting issue
* Fix line parsing logic
* Fix tests
* Run icacls to set execute permissions
* move to sha256
* remove unnecessary code
* Remove todo that should go in another PR
* Remove redundant trim
* rename class CommandProcessorOutput since it didnt match CommandExecutor
* Rename class file
* Report json object for failed permissions.
* Fix test to use correct folder
* fix lint
* Fix installer file deletion
* respond to linter
Axios (web request library we use) has a bug with no connection timeouts. Its timeout handler only works with request timeouts, not no connection timeouts. More at https://stackoverflow.com/questions/36690451/timeout-feature-in-the-axios-library-is-not-working. So yeah, it's going to be stuck forever waiting for a connection.
I fixed#2 in the past by adding a custom timeout with a cancellation token. dotnet/vscode-dotnet-runtime@ced82f3#diff-ad11027eb5a8406f68425da0e49b1c8a4aa83394444043e247c10e7bb1976999
It looks like I got rid of that because I forgot about the axios timeout issue. dotnet/vscode-dotnet-runtime@3b1c251
* wip - separate out worker paths so contexts are needed for every call, but there is less potential for things to go wrong and less code duplication
* Further isolate code, need to isolate the install tracker now
* migrate install tracker to use event stream instead of being coupled to the context
* move installtracker to a singleton and fix tests
the install tracker really only needs to exist once since the event stream and extension state are global, this will prevent errors from different states of promises and simplify our code with this design pattern to decouple the installation tracking mechanism from the context object, which will reduce code duplication for aspnet install feature additions
* Fix some issues with the test during code migration
* Fix issue with accessing instance
I forgot to ctrl s!
* Clean up logic with install key generation
* Remove duplicate source of truth for architecture
* Improve logging and error handling of install tracker
This class is super annoying to debug without better logging
* Update some broken components of the tests
* Prevent saying object object in the log output by expanding the object
* Tests working
We used to need to concat these tests results with different contexts but not any longer
* FIx uninstall all commands
* Fix the file rename
* Improve logging messages
* Add mode handling for aspnet
* Add ASP NET Support
* Add Generic -> Mode Specific Event Stream Class Hierarchy
* [wip] create events for all 'modal' events
* Dont hold a generic arg obj because it becomes too unwieldy
* Fix some issues with the test and sdk extension
* Add 'mode' to the acquire context
This will allow API callers to set the mode to call aspnet without having to write a new endpoint call. We can default to runtime for when it wasnt provided that would be the old behavior.
* Move 'installMode' into the acquire context
This allows API callers to provide the mode,
We remove it from the worker context to dedupe this and prevent 2x sources of truth.
Now this is a bit weird because some API endpoints will take the mode even though the API endpoint is only for one mode. We will just ignore the mode for that. Eventually we can try to transition all of the related calls to a single endpoint based on 'mode' which would simplify API usage.
* Code cleanup
* De-Dupe Test Code
* Add Requested Events to Republisher
* Update InstallKey pattern to include ~aspnetcore
This is a bit strange since - used to be for 'global' and ~ for architecture. There are only so many path safe characters to use, more importantly though we dont rely on the install key as the information for the install anymore, it is stored in the DotnetInstall object. ~ has logic in place to not account for a 'legacy' install, so ~ with aspnet (which would not have existed before arch was added) is also a bet that will work with pre-existing logic.
* fix build issues
* Add ASP.NET Directory Provider
* Only Report Total Success if Path exists
The path can be undefined if an error throws so its not a success
* Add ASP.NET API to sample extension
* Resolve Asp Net Runtime versions as Runtime versions
* Fix mistake on test
* Fix bug with install script param
* add source-map-support to allow build in CI
this got removed and passed earlier somehow
* respond to linter
* add source map support
* Respond to linter
* Add test for specific telemetry messages
* 2.0.7 branding
* Prevent circular import
* rewrite completion event so we can test the modaleventrepublisher
* fix linter issue
* Revert new test since the object is not mockable
* Get rid of check for event that cannot be published via mocks
* Fix bug where logging was not up to date
to prevent i/o costs the file is dumped at the end of error handling, but this means other events past error handling dont get added to the log. This fixes this.
* respond to pr feedback
* Remove 'Debug' As it's not used
* Remove 'Diff' as it's not used
* Remove 'es-abstract' as its not used
* Remove 'got' as its not used
* Remove http cache semantics from the main dependencies list
its just a sub dependency
* Remove 'p-retry' as its not used
* Remove 'public-ip' as its not used
* Remove tslint from dependencies, its not a dependency
* Increase attribution information for library dependencies
* Remove a ton of unused dependencies
* Clean out dependency tree
* Clean up sample extension dependencies
* Fix SDK Yarn.Lock
* Fix sample yarn.lock
* Reorder remaining licenses to match package.json
* add missing semver package that used to be included elsewhere by other dependencies
* Delete vscode-dotnet-runtime-extension/yarn-error.log
* Update readme.md with notices per industry standard
* Add back in ts-loader as it is required
* wip - separate out worker paths so contexts are needed for every call, but there is less potential for things to go wrong and less code duplication
* Further isolate code, need to isolate the install tracker now
* migrate install tracker to use event stream instead of being coupled to the context
* move installtracker to a singleton and fix tests
the install tracker really only needs to exist once since the event stream and extension state are global, this will prevent errors from different states of promises and simplify our code with this design pattern to decouple the installation tracking mechanism from the context object, which will reduce code duplication for aspnet install feature additions
* Fix some issues with the test during code migration
* Fix issue with accessing instance
I forgot to ctrl s!
* Clean up logic with install key generation
* Remove duplicate source of truth for architecture
* Improve logging and error handling of install tracker
This class is super annoying to debug without better logging
* Update some broken components of the tests
* Prevent saying object object in the log output by expanding the object
* Tests working
We used to need to concat these tests results with different contexts but not any longer
* FIx uninstall all commands
* Fix the file rename
* Improve logging messages
* fix indentation
* Introduce error to wrap all global sdk errors
* Wrap installation in success event to improve trackability
* Expose the original event name for property expansion
* Respond to linter
* Improve Error Message Specificity
* Update the event stream to use the right install key
* Update Errors to be EventBasedErrors that contain the Event Name
This is a mildly ugly pattern. See the PR description for more detail.
* Make legacy code throw an error instead of throwing a string
Throwing other object types up makes it harder for us to get the callstack etc
* Fix bug with install key management where it would not have -global or the architecture would be unspecified
* respond to linter
* Fix whitespace
* Fix whitespace
* Don't rely on english output for registry lookups
This will fail with another display language.
* Update reg test to account for 2 command execution calls
* Account for executing multiple commands with status output on the mock object
* Remove whitespace xd
* Scan command args for potential matches to mock commands
* Fix registry lookup test logic
* Make sure the return status is not null for mock objects
* WIP - Migrate Install Key strings to type to allow future flexibility with installs -- e.g. non-versioned install folder names so we can auto update. Start collecting more information during install about who installed it so we can ref count them for uninstalls. Does not migrate telemetry events or other code yet which needs to be experimented on further
* WIP - Migrate all code to use InstallKey Type & Fix Bugs
There are several big things here that might seem like small code changes.
1 - Before when we found a 'partial install' we would delete ALL local runtimes. I am astounded this exists. There are other extensions that might depend on those other runtimes and this just deletes it from underneath them. A partial install is rare, I think you need to pkill vscode or something since the install script runs out of proc. We consider an install partial if our code never got to mark it as installed after calling the install script. Its rare but does happen, at 2010 occurences out of 470k in recent days.
Renames a function which appears to handle partial installs to clarify it only manages local ones.
Gets rid of some unused functions or moves them to a better place.
Renames IInstallKey to DotnetInstall. The install key is the string representing the directory of the install. It shouldn't have two meanings.
Add logic for ref counting which extensoins own which installs.
* wip - record extension owner for installs
Fix record keeping to properly ref count extensions
Rename types to match standard better
Still need to figure out how to manage uninstalling and 'graveyard' as well as how to remove a 'legacy' install via version when it is a dotnetinstall object.
* Separate out 'graveyard' code or management of old installs to another class
* Dont pull files from dist - bad copilot, bad
* Update some more of the code to use new types
* update mock objects to get the build working
* respond to linter
* Add back in the old telemetry prooperties
I got rid of them because they are duplicated data. But TBH there are a lot of things that rely on the pre-existing property names. It would be much harder to query new and old data together if getting rid of them, so I added them back even though it causes some data duplication.
* Fix bugs with install management
* add missing semicolon lol
* Fix a bug with clearing out the installed list
The logic was missing a !.
* Fix Install Graveyard Logic
* Fix test logic for legacy installs
* Skip unnecessary preinstall local sdk check on global sdk
* Add back code to account for preinstall in sdk status check
* Remove unnecessary string checks
We never give a string to this class anymore so it's not needed
* Add event for when tracking installs messes up somehow
* Refactor classes out to many more files
* Respond to linter
* Isolate install tracking management to a separate class
* Clean up acquisition worker and remove unnecessary logic
* Port the worker to using an external install manager to isolate protected data
* Respond to linter concerns
* Remove potential deadlock with installation management
* Increase timeouts
With the change where you need to hold a lock to edit extension state to protect against timing challenges where an install is being added or deleted at the same time and can cause incorrect behavior, we had to slow down the product.
* Migrate to a singleton for install tracker so no dupliacte lock is created
* Working build for changing 'isRuntime' to mode for 'aspnet' 'sdk' or 'runtime'
* undo the singleton in test as the context needs to be per test
* remove the singleton pattern as it will result in a bug with multiple acquisition requests having a stale context
* Add Tests for Install Tracking with who owns what install
* improve timeout on test
* Revert "improve timeout on test"
This reverts commit a4168f6ec5.
* fix timeout
* Fix Test
* Fix Logic for adding pre existing installs
* Fix tests to be successful
* Code Cleanup & Remove Potential Tech Debt for Modes
remove places where we make assumptions on the existing model (sdk or runtime) to prevent future tech debt
* respond to linter
* Fix bug with context being in a different mode than a legacy installation
Mark old function as deprecated and add a comment explaining why our assumption there works
Fix possible bugs with context management where context could be a different mode than what was previously installed which could cause future bugs down the line that would be hard to track down
* Add back in removed events to prevent breaking old telemetry dashboards
* Final Code Review from Me
* Fix Test
* Remove circular dependency from test artifact
* add missing file header from refactor
* fix test
* delete directory per mode file as there is a git bug where changes in file caps is not being picked up
* Add back into directory file with proper capitilzation per git bug
* Add missing await which might cause test to fail inconsistently
* Make the lock based timeout based on the user setting for timeouts
* Enable Policheck
* Remove the old pipeline
We probably should have done this post 1ES Migration
* azdo is being buggy and not letting me move from 1 to another
so undo this and do it in another pr
Revert "Remove the old pipeline"
This reverts commit 3c4cc97fa0.
* azdo is buggy no more
Revert "azdo is being buggy and not letting me move from 1 to another"
This reverts commit 012133fd34.
* Update axios cache interceptor for bug fix
* Force update axios as well
* Migrate from vsce to @vscode/vsce
This is required for using newer versions of vsce.
* Update Node to 18 from 16 as VSCE now requires 18
* Extend log test timeout since it seems to sometimes take longer
* Make the assert not a question
* Update distro schema file to account for changes in the file.
Break out the shared types into reusable definitions to make it easier to reflect reality.
* fix offset on required properties
* add logic for adding a global shared path to extension
* add tests for global path config
* update comment out code
* update mock comment
* remove commented out code in ExtensionConfigurationWorker
* update ExistingPathResolver
* add tests for global shared path
* remove empty line
* resolve duplicate import
* fix tslint
* resolve format comments
* get rid of set global path to avoid type error
* add logic for checking local paths first in ExistingPathResolver
* create set global path method
* fix test
* Fix Typo
* change the field name for global shared path; add sharedExistingDotnetPath to package.json; change get and set method for shared path for ExtensionConfigurationWorker
* fix typo
* update readme for global shared path
* Add more examples to the path for .NET
* Consolidate read.me section to one page
* Simplify Logic for Failure to set Manual Path
I thought about the direction this should go, and I think that the old logic only makes sense because we'd need to concat the existing settings. In this case we are setting the global setting, so we can just add it without having to do any additional logic.
* Consolidate setting global and local to one function for simplicity
* update sample page to use the string instead of path obj
* Refactor path setting functions to two separate ones
We need to access the configuration with the direct objects the users can use the settings json file, hence this could not be simplified to a shared object.
* update class member names to disambiguate overloaded terms
* Check whether a matching extension id extension exists before calling the global path
* Make read.me more clear
* Update read.me to show that the example is only an example
* .some may not exist in certain versions so dont use it?
* Dont activate the runtime extension under test
* dont rely on test env variable since we cant set one within vscode launch profile
* Fix ability to run extension tests under launch json
We activate the extension in the test to inject custom objects to track 'telemetry' under test and analyze the telemetry events to ensure things are going well.
We cant activate the extension twice.
In the recent past we made a change to auto activate as a necessity to check the operating system before exposing APIs to users that were not available on their OS.
When I made that change I think we actually broke the ability to run the runtime extension tests under the launch profile. This fixes that.
* Remove global setting from test
This setting impacts the return path of all tests when correctly implemented. We dont want to dupe the extension test configuration for a single test to run it with a different configuration either, so we cant test it this way.
* Add tests that dont rely on the extension
* fix typo
* improve warning messages to correctly diagnose what went wrong
* remark that you need to restart vscode for the setting to take effect
* return empty properties array instead of undefined so properties are forwarded via telemetry observer
---------
Co-authored-by: Noah Gilson <noahgilson@microsoft.com>
Co-authored-by: Noah Gilson <nagilson@umich.edu>
* Update the changelog to include details about this version
* add the release date
* Use \ to get a new line
* Use \ to get a new line for better readability
* Use \ to get a new line for better readability
Per our discussion at https://github.com/dotnet/vscode-dotnet-runtime/issues/1741#issuecomment-2051238090, it would be useful to have stdout even if we dont error, because the error condition is only if:
When running the install script:
The process could not be spawned.
The process could not be killed.
Sending a message to the child process failed.
The child process was aborted via the signal option.
* Update Dependencies
* Remove is online
Is-online now requires ES6.
It has not been maintained in over 2 years.
The checks it does are fairly rudimentary. We only rely on this code to try to diagnose the error which has already happened.
I would trust this code rather than rely on external libraries.
* Fix timeout logic for no is online
* Fix Linting Issues
* Protect against no versions of .NET to recommend
The API contract is to return an IDotnetVersion which is an array of versions. If there are no packages of .NET available to install, we should have a specific message for that and return an empty array instead of what we do right now, which is
a - returning a bad version
b - returning undefined
c - erroring out
* respond to linter to accept any error type
* Update Dependencies
* Remove is online
Is-online now requires ES6.
It has not been maintained in over 2 years.
The checks it does are fairly rudimentary. We only rely on this code to try to diagnose the error which has already happened.
I would trust this code rather than rely on external libraries.
* Fix timeout logic for no is online
* Fix Linting Issues
The migration tool and doc examples all use the unofficial one ecks dee
Also create a dupe of the PR pipeline so we can migrate main to that file once it is checked in and then delete the other one to rename it to a more descriptive name.
The migration tool and doc examples all use the unofficial one ecks dee
Also create a dupe of the PR pipeline so we can migrate main to that file once it is checked in and then delete the other one to rename it to a more descriptive name.
* Migrate pipeline to 1es
* Clean up changes made from migration tool
* Separate pipeline out into many yaml files so we can have 2 pipelines without code duplication
due to new internal requirement
* remove unncessary pipeline backup
* Remove pipeline from 1es for public as it is not going to be added to the organization in public
* Update yaml loop logic
Remove devcontainer as it is out of compliance and not used by anyone
* fix mappping by adding body to the loop with indentation:
* add a 1es repository
* Create a separate 1es pipeline
* use self to search in the correct repository
* migrate to self on 1es
* use folder name pipeline-templates over templates
* use yaml over yml
* temporarily move jobs into one os
* split pools
* Move the loop to out of the job
* move the task into steps instead of a job as u cant do a job in a job
* remove template keyword
* dont call steps template in job outside of steps
* use yaml instead of yml
* add sdl
* use unique names for jobs
* use _ instead of space
* Use azure pipelines name to conform to style
* give image to each parameter
* make it 1 os for 1 os job and use different pool
* Call pool explicyk
* try to fix names
* Use different source for mac instead of 1es
* Update get-func-name
* Update public PR pipeline
* use public pool as thats what the name param is for
* give image name instead of vmimage as it snot used
* try to switch to a better pool
* try using internal pool on public as external is not available
* Publish logs to os specific folder
* condition steps so they are skipped if not needed per os
* dont care if it succeeded
* add paran
* remove paran
* use parameter pools instead of agent os
* move to 7.0 instead of 6 bc six is broken in the cache online maybe
* switch to 1es pool
* try using public pool
* See if mac pipeline has a mirror in svc
* use image names from open
* Use vm image in the image
* Use vm image in the image
* update vm image
* try to condition the pool
* code cleanup now that everything was working
* fix whitespace
* Update vscode-test
* Update vscode-test to the new name
* respond to most pr feedback
* rename the file as I cannot change the pipeline name while its in main maybe
Noah Gilson
Chet Husk
JL03-Yue
Noah Gilson