* Update Readme.md to have a Coming Soon
* Move source into src folder (#8)
* Move source into src folder
* Move Tools and SLN
* Update LICENSE.txt reference
* Fix access modifier for ClipTimeToViewportTimeRangePercentColumnGenerator (#12)
* Update version to 0.108.1 (#13)
* Add security disclaimer to IsolationAssemblyLoader (#10)
* Add First Sample (#23)
* Initial Code Commit.
* Update samples directory names.
* Adding documentation.
* Resolve some comments
* Resolve remaining comments
* Add license to CS files and links to README
* Add Nuget Package Reference
Co-authored-by: Jayson Maxson <jmaxson@ntdev.microsoft.com>
Co-authored-by: Luke Bordonaro <lukebo@microsoft.com>
* Create better documentation (#29)
* Create better documentation skeleton
* .attachments and brief overview
* Fix linking
* Fixing Image and edited overview
* Attachments typo
* CDS Model documentation
* No implementation details in CDS
* Updated CDSM image
* Fixed links + wording
* Fixed slight errors
* Add Installing-WPA document and image.
* Update Creating-your-project.md and the related images.
* hidden markdown + Data processing pipeline.
* Fix hidden
* Renewed pictures + cleaned up structure
* Fixed typos
* Initial Pipeline Documentation.
* Document Extended Tables.
* Add a link to real examples.
* Source Parser wording
* Data Extension
* Begin work on the Creating-a-simple-custom-data-source.md documentation.
* Trying to use an emoji to see what happens.
* Add SDK Videos
* Try Add Video
* Add thumbnail and fix md
* Add missing requirement.
* Added RequiresCooker attribute descriptions.
* Update Creating-a-simple-custom-data-source.md
* Update parameter name.
* Tweak to some documentation highlighting
* Tweak warning information.
* Tweaks to simple CDS doc
* collapsible components
* Edit typos
* table of contents
* Added Table of contents + formatting
* Add information on table building to Creating-a-simple-custom-data-source.md
* Reword Cooker description.
* Reword Source Cookers.
Co-authored-by: Angeeras Ramanath <anramanath@microsoft.com>
Co-authored-by: Jayson Maxson <jmaxson@ntdev.microsoft.com>
Co-authored-by: Zack Newman <zachnew@microsoft.com>
Co-authored-by: Tristan Gibeau <41077386+trgibeau@users.noreply.github.com>
* Create PluginsLoader and associated unit tests. (#21)
* Create PluginsLoader and associated unit tests.
Refactor assembly processing logic to indicate if all
assemblies found were processed successfully.
Add a DiscoveryStarted method to the IExtensionTypeObserver
interface, which gets called at the start of a round of
processing assemblies.
* Use deployment items instead of looking at build output directly in tests
* Add assert messages and use TestContext.WriteLine
* Added some concurrent tests and address PR comments
* Add UnitTest attribute
* Fix csproj whitespace
* Update architecture overview (#34)
* Update architecture overview
* Address PR comments
* Address more PR comments
* Add SqlPlugin sample (#31)
* Add SqlPlugin sample
* Fix repeated GUIDs and whitespace in tables
* Use an XmlReader to parse XML file
* Add video links to README
* Remove IKeyedDataType methods from simple SqlEvent
* Use more efficient IsFileSupportedCore implementation
* Address PR comments
* Update data processing pipeline architecture documentation (#35)
* Update data processing pipeline architecture documentation
* Remove old next steps text
* Fix hyperlink
* Address PR comments
* Create UITest attribute (#40)
* Small tweaks to remainder of initial documentation (#37)
* Fix misc documentation formatting/wording
Standardize diagrams across all docs
* Fix engine header
* Reference SqlPlugin sample
* Remove WPA reference to LTTng plugin
* Address PR comments
* Address PR comments
* User/zachnew/folder support (#39)
* Add support for Data Sources beyond files.
__Overview__
The current design of Data Sources in the SDK is veryWindows-centric in that
it assumes that everything is a file with an extension.
Three scenarios are driving this change:
- Plugins that need to process Directories rather than Files
- Plugins that need to process Files that do not have extensions
- Plugins that need to process something other than a File or Directory
This change generalizes away from file to enable support of arbitrary Data Sources,
and in keeping with the existing API, adds two (2) new attributes for
authors to use on their Custom Data Sources:
- ExtensionlessFileDataSourceAttribute
- DirectoryDataSourceAttribute
The _ExtensionlessFileDataSourceAttribute_ is used to denote that Files that do
not have extensions may be routed to the Custom Data Source.
The _DirectoryDataSourceAttribute_ is used to denote that Directories may be routed
to the Custom Data Source.
In order to support these attributes, as well as any arbitrary Data Sources, we
have generalized the ability for Custom Data Sources to state that they support
a Data Source. _IsFileSupported has been changed to IsDataSourceSupported_ and now
takes an _IDataSource_ rather than a string. This allows for any _IDataSource_
implementation to be interrogated, even user defined implemetations.
The existing attributes for Files and Directories are now used simply to enhance
the performance of assigning Data Sources to their appropriate Custom Data Sources
for processing. They are also used by UI components in their Open dialogs. Unrecognized
implementations of _IDataSource_ will simply be routed to all Custom Data Sources and
have their _IsDataSourceSupported_ method called.
The _CustomDataSourceReference_ class has been updated to work in terms of _IDataSource_s
as well, and the myriad of extension methods around this class and the _ICustomDataSource_
interface have been refactored to better reflect the now generalized world. This class
has also been made abstract in order to allow for easier testing of components that utilize
_CustomDataSourceReference_s.
Finally, the _Engine_ has been updated to operate in terms of _IDataSource_ rather than
string paths. The old _AddFile_ methods have been converted to extension methods for
convenience.
__Details__
- Custom Data Source interfaces and base classes updated to use the more general
IDataSource
- New Data Source Attributes added to enable additional common scenarios.
- Support for arbitrary IDataSources.
- DataSourceResolver introduced to eliminate duplication of assigning Data Sources
to the Custom Data Sources that can process said Data Sources.
- Engine updated to operate on IDataSource rather than strings.
- CustomDataSourceReference updated to allow for better mocking in tests.
- Additional Tests added
- Existing tests updated
- NBGV updated due to breaks on build machines.
__Breaking Changes__
This change introduces a number of breaking changes. They are detailed below.
- Microsoft.Performance.SDK.Runtime.CustomDataSourceExtensions
- Removed method TryGetFileExtension
- Removed method TryGetFileDescription
- Removed method Supports
- Microsoft.Performance.SDK.Runtime.CustomDataSourceExtensions
- Removed field EqualityComparer
- Microsoft.Performance.SDK.Runtime.CustomDataSourceReferenceExtensions
- Renamed TryGetFileExtension to TryGetFileExtensions
- Renamed TryGetCanonicalFileExtension to TryGetCanonicalFileExtensions
- TryGetFileDescription now has a required parameter
- Microsoft.Performance.SDK.Processing.CustomDataSourceBase
- Removed method IsFileSupported
- Removed method IsFileSupportedCore
- Added method IsDataSourceSupportedCore
- Microsoft.Performance.SDK.Processing.ICustomDataSource
- Removed method IsFileSupported
- Added method IsDataSourceSupported
- Microsoft.Performance.SDK.ReadOnlyHashSet<T>
- Now implements IReadOnlyCollection<T>
- Microsoft.Performance.Toolkit.Engine.Engine
- Renamed FreeFilesToProcess to FreeDataSourcesToProcess
- Renamed FilesToProcess to DataSourcesToProcess
- Renamed Microsoft.Performance.Toolkit.EngineUnsupportedDataSourceException to UnsupportedCustomDataSourceException
- Renamed Microsoft.Performance.Toolkit.EngineUnsupportedFileException to UnsupportedDataSourceException
* Enforce SDK Version Compatibility.
We introduce the concept of checking the compatability of a Plugin
with the version of the SDK being hosted.
A Plugin is considered to be using a Version of the SDK compatible with
the Host if the following are true:
1) The Plugin is using SDK Version greater than or equal to the lowest supported version (currently 0.109.0)
2) The Major Version of the Plugin's referenced SDK is equal to the Major version of the hosted SDK.
3) If the Major Version is 1 or greater, then the Minor version of the plugin must be less than or equal
to the Minor version of the SDK. If the Major Version is zero, then a simple range check is used to make
sure the Plugin version is greater than or equal to the lowest supported and less than or equal to the
current SDK version.
If either of these checks fails, then the Plugin is rejected as incompatible
with the host.
_IMPLEMENTATION_
We added a class to do the version checks. This class is used deep in the
Discovery mechanism to do the check as close to loading as possible. This
gives us the opportunity to pinpoint the incompatible to the assembly.
We implemented a pass through of the ErrorInfo class in order to propogate
all errors during Discovery up to the PluginLoader. Before, we had a simple
Boolean, and so the nature of errors was getting lost. In order to tell
the user what all has gone wrong, we MUST propogate error information.
All in all, this change checks compatability, and provides for propogating
all errors up the call stack back to the loader so that failures can be
reported conveniently and accurately by the host to the end user.
__Breaking Changes__
This change introduces a number of breaking changes. They are detailed below.
- Microsoft.Performance.SDK.Runtime.NetCoreapp.PluginsLoader
- TryLoadPlugins(IEnumerable<string> directories, out IList<string> failed) changed to TryLoadPlugins(IEnumerable<string> directories, out IDictionary<string, ErrorInfo> failed)
- Task<(bool, IList<string>)> TryLoadPluginsAsync(IEnumerable<string> directories) changed ti Task<(bool, IDictionary<string, ErrorInfo>)> TryLoadPluginsAsync(IEnumerable<string> directories)
- Microsoft.Performance.SDK.Runtime.Discovery.AssemblyExtensionDiscovery
- All ProcessAssemblies methods now have an 'out ErrorInfo' parameter
* Generalize Data Source Attributes.
This change extends the DataSourceAttribute base attribute
to define the Type of Data Source accepted by the decorated
Custom Data Source.
Users may implement a preliminary filtering method in order to
shrink the space of valid Data Sources (for example, the File Data Source
shrinks the acceptable space to files with a certain extension.)
Data Sources declared by a Custom Data Source will be routed to said
Custom Data Source for determining if the Data Source is supported.
Data Sources that are unrecognized do not get routed to any Custom Data
Sources.
__Breaking Changes__
This change introduces a number of breaking changes. They are detailed below.
- Microsoft.Performance.SDK.Runtime.DataSource
- moved to Microsoft.Performance.SDK.Processing.DataSource
- Microsoft.Performance.SDK.Runtime.DataSourceExtensions
- moved to Microsoft.Performance.SDK.Processing.DataSourceExtensions
- Microsoft.Performance.SDK.Runtime.FileDataSource
- moved to Microsoft.Performance.SDK.Processing.FileDataSource
- Microsoft.Performance.SDK.Processing.DataSourceAttribute
- a new parameter has been added to the constructors
* Rev to Version 0.109.0 (#27)
* Add SQL sample file (#48)
* Make SetApplicationEnvironmentCore virtual (#51)
* Fix SampleAddIn's DataSourceInfo and update DataSourceInfo docstrings (#28)
* Fix SampleAddIn's DataSourceInfo and update DataSourceInfo docstrings
* Make firstEventWallClockUtc docstring clearer
* User/zachnew/disposable (#54)
* Add disposable infra to Engine.
* Cleanup engine tests.
* Add disposable infra to PluginsLoader.
* Add disposable infra to DataExtensionRepository.
* Add dispose helper methods.
* Add disposable infra to *Reference classes.
* Dispose all references in repository.
* Add disposable to Plugin Catalog.
* Dispose resources in PluginsLoader.
* Dispose resources in the Engine.
* Add CDSR tests for disposable.
* Add SDCR disposable tests.
* Add CDCR disposable tests.
* Add TER disposable tests.
* Fix logic errors leading to failing tests.
* Add DataExtensionRepository tests.
* Use Dispose helper method.
* Make sure all managed resources are nulled out.
* Added a DAG for the complete reference graph.
* Add ExtensionRoot for a single entry point to all extensions.
* Add DAG documentation.
* Add doc updates.
* Add cycle detection.
* Add disposeProcessor method.
* Call base.Dispose last.
* Track created processors.
* Add ref tracking for source parsers.
* Make references 'releasable.'
* Add release to other references.
* Add unit tests.
* Make CDSR consistent with other references with Instances.
* Remove unnecessary finalizers.
* Remove unneeded release code.
* Document disposable on AssemblyTypeReference.
* Update documentation comment.
* Update doc comments.
* Delete unnecessary class.
* Do not allow an exception out.
* Add disposable docs.
* Update doc comment.
* Remove extra line.
* Seal classes.
* Remove redundant calculation.
* Address PR comments.
* User/zachnew/docs (#57)
* Edits to disposable documentation.
* Fix SqlSourceParser endtime calculation
* Whitespace
* Removed IComparable<TKey> from IKeyedDataType<TKey> (#26)
* Add Table support to Engine (#56)
* Bring changes from feature/AddBuildTable branch in Azure DevOps repo.
* Add new ITableResult, code cleanup
* build break
* Cleanup and add tests
* Clean up comments
* Add doc comments
* Add missing copyright
* Updates based on PR feedback
* Rebase on Develop
* Updates based on PR comments
* Updates based on PR feedback
* Fix typo
Co-authored-by: Jayson Maxson <jmaxson@ntdev.microsoft.com>
* Update nuspecs. (#62)
* Fix engine with processor tables (#63)
* Fix processor tables from not loading in the engine
* Add test for processor table
* add missing header comment
* fix runtime tests
* Update ErrorInfo formatting. (#66)
* Update ErrorInfo printing logic.
Some error properties, such as lists, were not printing properly. Rather than
printing all of their items, they were simply printing the type name, which is
not useful.
Some other internal errors had overridden ToString unnecessarily, which led to
a loss of information.
This change rectifies these issues so that printing is consistent and useful.
* Update version file.
* Revert "Update version file."
This reverts commit 3dd6b42940.
* Add migration document for 0.109 (#61)
* checkpoint
* Add migration document.
* Add suggested changes section.
* Update heading.
* Add links
* Address PR comments.
* Add missed migration.
* Remove broken link.
* Update version.json for preview versioning. (#67)
* Update version.json for preview versioning.
* Remove servicing from the ref spec.
Co-authored-by: Tristan Gibeau <41077386+trgibeau@users.noreply.github.com>
Co-authored-by: Jayson Maxson <jmaxson@ntdev.microsoft.com>
Co-authored-by: Angeeras Ramanath <anramanath@microsoft.com>
Co-authored-by: Zack Newman <zachnew@microsoft.com>
Tristan Gibeau
Zack Newman
Luke Bordonaro