Граф коммитов

2 Коммитов

Автор SHA1 Сообщение Дата
Howard Kapustein e86588b528
MSIX Dynamic Dependencies (#393)
* Added initial skeleton of DynamicDepedencies Flat-C APIs

More expansive skeleton. Fixed some build errors

Updated per API Review

Updated to match API changes

Updated per spec

Restructured DynDep API + Test layout

New dyndep dirs

Added WinRT API. Skeleton, but get past the project-isms. Compiles clean!

Changed /W3 to /W4

Implemented most of the WinRT API

Removed CreateForUser as WinRT has no Windows.System.User to SID conversion :-(

Removed CreateForUser

Added ProjectReunion_BootstrapDLL

Added MIDL generated .h file

Added Detours

Added README.md to track when we forked Detours (since they don't provide a nuget package or like artifact we can cleanly consume)

Fixed build configurations. Tests don't build for ARM64 due to CppTest not supporting ARM64, otherwise should all be good now

Started filling out e.g. Detours initialization for ProjectReunion_DLL. Almost ready to start some testing

Added wil::DLLMain() calls to ProjectReunion_[Boostrap]DLL projects

Forgot to add new dllmain.cpp. Moved ProjectReunion_BootstrapDLL project under dev in ProjectReunion.sln's view (no change to filesystem layout)

Added test Framework packages

Got MakeMSIX working!

An absolute path sneaked into the sln. Fixed

Incorporated feedback

Moved makemsix.cmd to the tools directory

Updated makemsix.cmd to also sign the generated package

Adding tests for Bootstrap API

Added Main.Sidecar test project

Added Main.Sidecar package for testing. Updating Bootstrap API to find the framework via a dependency of the package providing the LifetimeManager implementation (e.g. Main.Sidecar). Minor build-ism/makefile (well, vcxproj) corrections

Added comment about the LifetimeManager's implementation

Fixed absolute-ish relative path that snuck in (must have been inserted via VS' Add dialog)

Fixed absolute-ish path the snuck into the .sln

Simplified nested namespace statements

Incorporated feedback

Update Main.Sidecar's appxmanifest with COM registration info and other minor corrections

Fixed header filename

Simplified namespace'ing syntax

Incorproated feedback

Moved implementation bits into MddCore namespace. Pulled remaining code and pseudocode from spec. Marked work to come with TODO, often pseudocode blocked in #if defined(TODO_Stuff)...#endif so as not to break the build; will be revised shortly as I finish the implementation and light up tests

Added FAILFAST checks in MddBootstrapInitialize() to detect multiple initialization

Added missing end-tag

Fixed Main.Sidecar's manifest and logos. Updated makemsix.cmd to copy Assets\* if exists, and make logo.png optional (should have one or the other, but probably not both)

Started to exceptionalize error handling; more to come. Bootstrap test framing in place. But...Main.Sidecar manifest creation fails because there's no typelib :-( More vcxproj hackery needed...

Added message loop to Main.Sidecar exe to solve the need-to-live-until-told-to-quit problem. Removed typelib from appxmanifest as unnecessary

Removed leftover end-tag in appxmanifest.xml

Manifest tweaks. Added Main.Sidecar2 as a WRL alternative to Main.Sidecar, which is incomplete as you can't make a pure C++/WinRT OOP Server today you need a hybrid of C++/WinRT plus WRL; Main.Sidecar's incomplete; ignoring for now while focusing on getting Sidecar2 working. Build fails because the manifest can't find the proxystub dll. The project needs to be split into 3 -- make exe, make proxystub dll, make msix

Added Main.Sidecar3*. ProxyStub mostly in place (needs dlldata.c?). More to come

Main.Sidecar3 progress

Closer to working - Main.Sidecar3.exe looks almost right

More tweaks. Almost ther

Fiddling with msix makefile. Need more structural changes

It's alive! Aliiiiiive! Cleaned up Main.Sidecar variants.

Added new files missed in previous commit

Bootstrap test almost working! Did some HRESULT->Exception conversion; more to come

Force out test thread to be MTA despite CppUnitTest 'conveniently' initializing us as STA

First blood! ShutdownWithoutInitialize() passes! And there was much rejoicing

Updated Main.Sidecar's PackagedCOM OOP Server. It works (at last)! Minor related improvements. More to come

Add .def file for the Main.Sidecar's exe

Incorporated a stand-in for ProjectReunion's Framework package so we can test the Bootstrap API before the (real) framework package is created

Small tweaks per testing

More code cleanup. Bootstrap tests pass!

Added TODOs for follow up

DUpdated samples to match API

Added DynDep tests against current API implementation (e.g. return E_NOTIMPL); update coming with API implementation. Worked out test environment issues where we need packages for product code but don't have packages with locally built binaries like real product code (chicken/egg problems).

Changed Detours initialization to FAIL_FAST if anything goes wrong. Keeping HRESULT return for future extensibility so callers are prepared to handle it

Implement dynamic package graph

Lit up Create+Delete

Dialed down the chattiness of nmake projects

Fixed up more build errors

Checkingpoint

Fixed compile errors

More checkpointing

Add DataStore

Cleaned up build warnings. Incorporated DataStore into the UT

Implemented GetApplicationData

Fixed and enhanced tests. Fixed runtime. It works. Wow. It's aliiiive....

More verification in tests. Remove leaves an extra ';', need to fix

Fixed UpdatePath to produce the same PATH after Add+Remove (i.e. same as when we started)

Changed Framework.Math.* packages to be trivial Win32 compiled dlls w/no dependencies (save for kernel32.dll) to simplify testing

Added Detour'ing functions when ProjectReunion.dll is loaded (DllMain). Fixed Detour'd GetCurrentPackageInfo* to work as expected (bit fiddling is fun for the whole family!). Expanded verifications to include the package graph (GetCurrentPackageInfo*)

Fixed test to properly #include Math.*.h

Expanded tests to verify Add(ProjectReunion) works as expected

Test refinments

Generate context ids. Extend tests to a 2-fwk scenario

Partial DataStore support

Expanded the test suite. Fixed some warnings

Expanded test suite. Validating

Cleaned up some old style syntax

Fixed warnings. Fixing lifetimeKind=FilePath|RegistryKey expiry. Revised the no-static-package-graph detection

Expanded the test suite. All tests pass!

Remvoed obsolete comment

Updated per rebase

Fixed warnings only visible in Release builds. Fixed some build issues in some flavors; working on the rest

Partial vcxproj/Build fixes - x64=OK. ARM=Fail because MIDL generates #if defined(_ARM_) but VC++ defines _M_ARM_; should we just /D_ARM_ for ARM builds? ARM64=Fail because of some problem in dlldata.c. x86=Fail because 'LNK2001unresolved external symbol _ObjectStublessClient3' in the DynamicDependency.DataStore.ProxyStub project; need to #define something (_MERGE_PROXYSTUB)? Discussing with folks who know more

Removed MinimumTargetSystem=NT100 to fix the ObjectStublessClient problem; defaults to NT60 which 'doesn't need to link with ole32 for ObjectStublessClient'. How cryptic

LifetimeKind=RegistryKey didn't properly detect if a regkey was missing vs error (outofmemory, etc)

Fixed incorrect calling convention on Detour'd functions. x86 tests detected; x64 and compiler didn't

Start of DynamicDependencyLifetimeManager - create DDLM based on Main.Sidecar

Added appextension to DDLM manifest

Fixed ProjectReunion_BootstrapDLL to use MIDL output per-configuration/architecture (not checked in an shared across)

Revised Bootstrap API to support hunt for DDLM. Refactored VS projects to play nice (gah!). Now to update the tests to match

Bootstrap tests pass! TODO: Update other tests to match and cleanup

Fixed Win32 test syntax

Fixed test (fake) Reunion framework version to be >= test LifetimeManager <sheepish grin>. Twewaking tests to play nice with DDLM

Updated Create_Add_Architectures_Current test. Streamlined bootstrap logic

More test update sync for DDLM. Likewise updated some project references

More test sync-up-age for DDLM

Changed some old HRESULT-error based code to exceptions

More exception-ification

Removed the Main.Sidecar test projects/artifacts (superseded/formalized as the DDLM)

Removed some debugging hackery

Revamped WinRT tests for parity with Win32's more extensive tests

Fixed an inverted log message check

Added MddGetIdForPackageDependencyContext() so WinRT API could work, and generally useful for debugging, troubleshooting and testability. Implemented ScopeIsSystem. Cleanup up some leftover cruft

* Preparing to remove obsolete skeletal samples

* VS updated the sln after loading

* Removed dead entry leftover from a prior bad merge

* Added start of UndockedRegFreeWinRT to ProjectReunion_DLL

* Reworked URFW #includes to cull out redudancies and consolidate with ProjectReunion existing plumbing. Switched the few WRL uses to WIL

* Incorproated URFW's DllMain setup logic. Reverted ComPtr usage to minimize forkage. Baseline looks good as a URFW-fork. Now to add AppxManifest.xml support and test

* Reworking WinRT support to better fit with Mdd + URFW architectures

* Parse appxmanifest.xml for WinRT inproc servers

* Added slightly improved error reporting to makemsix.cmd

* Grabbed updated test cert

* WinRT support in progress but running into reentrancy issues. Need to refactor the WinRT data to be independent of the PackageGraph data (can't lock non-reentrant mutex and call GetActivationFactory which in turn does a GetActivationFactory and grab the lock already locked...

* Reworking WinRT support to avoid reentrancy problems

* Checkpointing while revising the WinRT support

* Corrected AppxManifest parsing and cleaned up some error reporting

* Tweaked error logging to avoid false spew to the debug output

* Updated appxmanifest parsing to produce absolute filenames (not package-relative) as needed for use at runtime

* Fix WinRT not passing down lifetimeArtifact

* Fixed PSID user support. Fixed WinRT's GetFromId(). Added WinRT's missing GetFromIdForSystem() to complement GetFromId() to match Create+CreateForSystem. Fixed some glitches in the WinRT test code. WinRT tests are passing!

* Small fix in text code. Something's wrong with the exception throw/catch/something causing process termination in Test::DynamicDependency::Test_WinRT::Create_RegistryLifetime_NoExist and Test::DynamicDependency::Test_WinRT::Create_FileLifetime_NoExist, but I don't see what. Will ping some folks in the morning

* Doh! Catching hresult_error, not hresult! Tests now pass as expected. Huzzah!

* Removed or fixed up some dated comments. Added thread safeto to WinRTModuleManager.

* Updated WinRTPInprocModule.Load handling of m_dll to be threadsafe. Fixed GetActivationLocation to NOT override inbox (Windows.*) namespaced types - aside from the obvious badness, it also results in deadlock during MddBootstrapInitialize() as that needs Windows.Management.Deployment.PackageManager to find the DDLM packages to choose from

* Switched PackageGraphNode to use MDD_PACKAGEDEPENDENCY_CONTEXT instead of wil::unique_package_dependency_context to avoid redundant reentrancy foolishness - when we're inside MddRemovePackageDependency should should be using the WIL smart type where its destructor calls MddRemovePackageDependency... The WIL type is for use OUTSIDE the DynamicDependencies implementation, not inside it. Sheepish grin :P

* Fixed a lock reentrancy error

* Updated spec with DDLM info

* Incorporated feedback

* Incorporated feedback

* TAEF and CppUnitTest can't both be used by a project (incompatible definitions) and CppTest project uses TAEF. Removed the unnecessary CppUnitTest header from CppTest's pch.h

* Fixed build break by removing OutDir in Release|x64 flavor of DynamicDependencyLifetimeManager.ProxyStub project (no other flavor specifies this)

* Fix IDynamicDependencyLifetimeManager's Makefile.mak support for Win32 (aka x86)

* Fixed ARM builds. Partially fixed ARM64 builds (less broken; something still very wrong)

* Added References to Framework.Math.Add/Multiply projects so DynDep Test_Win32/WinRT ensure correct build order (fix failures on github CI/pipeline)

* Removed dead commented code (leftover from earlier debugging)

* Fixed a typo

* Fixed AppLifecycle namespace. Fixed project references/dependencies to not (falsely) need or try to use Microsoft.ProjectReunion.winmd

* Minor correction of test MSIX manifests

* Fixed a regression in Add. Updated Remove to ignore context=null (just like Delete ignores id=null) - NOOP rather than error

* Added #include <appmodel.h> to MddBootstrap.h to make it more self-contained

* Wired up AppLifecycleTests for Dynamic Dependencies! Genericized (and TAEF'd) the DynDep/Bootstrap test plumbing for reuse across TAEF tests; see inc\ProjectReunion.Test.*.h. Most AppLifecycleFunctionalTests pass - GetACtivatedEventArgsIsNull, GetActivatedEventArgsForProtocol_Win32 and GetActivatedEventArgsForFile_Win32 fail because AppLifecycleTestApp.exe can't find the bootstrapper. Supposed to be in the directory next to the exe; probably something wrong in AppLifecycleTestApp.vcxproj

* Updated AppLifecycle tests to use DynamicDependencies. Changed DynamicDependencyLifetimeManager.Msix to use a templated/generated manfiest to match the current build architecture. All tests are passing!

* Added files missing from previous commit. Refactored supporting test APIs to be more shrinkwrapped and easier to use (and misuse :P)

* Refactored Test APIs to be more shrinkwrapped (easier to use, harder to misuse)

* Fixed ARM64 dlldata.c compile error (Hint: -DWIN32 is needed for dlldata.c in ARM64 builds). Removed CppUnitTestFramework-based tests from ConfigurationManager for ARM+ARM64 (CppUnit doesn't support ARM)

* Added 'Install Test Certificate' step to the build pipeline. Fingers crossed it's right

* Experimenting with a fix to the pipeline adding Install Test Certificates. TBD if we use it or an another solution. This is just experimenting with AzureDevOps to determine options. Will be reverted shortly

* Fixed typo

* Moved Boostrap packages Setup/Cleanup from Class- to Method-Setup/Cleanup to fix failure during the CI/Pipeline

* Added SetupPackages to CLASS_SETUP as an experiment to fix AppLifeCycle's GetActivatedEventArgsIsNull in the github CI/Pipeline. Currently fails because tthe tests METHOD_SETUP declares a custom manifest in TEST_METHOD_PROPERTY with a dependency on the Framework package (to get at the AppLifeCycle API). Theory is adding this during ClassSetup will make the framework available on the machine so when TAEF does the pkgmgr.RegisterPackage() Deployment will be able to resolve the <PackageDependency> and the rest will happily proceed

* Shuffled some fixture work and sprinkled in an ugly smattering of printf debuggery to track down how the CI/Pipeline runs tests to figure out what the ultimate fix needs to be. Don't stare at the current code unless you've got a strong stomach <g>

* Yet more debug hackery. And the hunt continues

* More testery

* Expanded printf-debugging logic to provide more execution context (poor man's windbg !token + !peb).

* Can't 'include' a manifest when it's also Copied

* Rejiggered the GetActivatedEventArgsIsNull (UAP) test to behave as expected

* Restored IsolationLevel=Method for UAP tests

* Yet more test hackery

* Narrowing tests

* Yet more narrowing of test to identify solutions

* Modified to test fixes for the Protocol test

* Added DevCheck.cmd/ps1 to check and (if necessary) fix the development environment

* Fixed DevCheck error counting/reporting

* Moved diagnostic aids into ProjectReunion.Test.Diagnostics.h

* Added -Check... options to DevCheck (specify nothing is equivalent to -CheckAll). Improve DevCheck help. Add 'Setup TAEF Service' to build pipeline

* Updated build pipeline command's arguments to setup TAEF service to match the recent change to DevCheck's parameter syntax

* Removed accidental duped task

* Removed some old debugging code

* Disabled the AppLifecycleTests task in the CI/Pipeline as github runs as a built-in Administrator account (Elevated and not a split token) which prevents TAEF RunAs:RestrictedUser from working. Given the CI/Pipeline running as a built-in Administrator account TAEF provides no way for us to work around it thus all AppLifecycleTests are Failed or Blocked (despite passing when run locally), and DynamicDependencies doesn't support Elevation so we can have passing tests that don't verify the actual primary target environment (DynDep, MediumIL) or failing tests that block all checkins. This problem will be solved when test execution (the whole pipeline?) is moved to Helix. When that happens we can reenable these tests.

* Add temp/ to .gitignore for temporary files. Added temp\MSTest.pfx generation for inner-loop development needing signed MSIX. NOTE: This will be updated in a future PR to pull MSTest.pfx from the Azure Key Vault.

* Added temporary file to unblock development. Will be deleted when DevCheck.cmd is updated to pull MSTest.pfx from the Azure Key Vault

* updated projects to use temp\MSTest.pfx to sign packages

* Moved temp .pfx generation to BEFORE building projectreunion.sln (as it's needed to build some of the tests). Removed a debugging statement from DevCheck.ps1

* Removed investigation code from AppLifecycleTests

* Incorporated Scott's feedback
2021-03-09 00:04:43 -08:00
Howard Kapustein beb7ac6c68
Spec for MSIX Dynamic Dependencies (#108)
* Start of the DynamicDependencies spec. More to come

* Design designs how Dynamic Dependencies does its magic

* Added mention of ProjectReunion.Loader.dll

* Spec ready for review

* Corrected some minor errors in the spec

* Added a Q to the append re API(s) to register a 'helper' package to pin a package dependency

* Incorporated feedback

* Fixed smart quotes. Streamlined sopme pseudocode removing unnecessary details

* Fixed typos (curse you depedency, depednency and like wrong variants :P)

* Reformatted per Style Guidelines for Markdown files

* Add a Style Guide section

* Moved the style guide to its own PR

* Incorporated feedback

* Incorporated feedback

* Incorporated feedback

* Changed namespace to Microsoft.ProjectReunion.ApplicationModel

* Incorporated feedback

* Clarified behavior re LoaderSearchPathOverride and AllowExecution

* Renamed Microsoft.ProjectReunion.ApplicationModel namespace to ApplicationModel.DynamicDependencies

Revised namespace per recent conversations not to use ProjectReunion in namespaces or API names

* Fixed a bad path in sample 6

* Incorporated review feedback

* Incorporated more API Review feedback

* Corrected namespace and using statements

* Incorporated feedback

* Incorporated more feedback

* More API Review feedback

* Added information about the static package graph resolution

* Incorporated more feedback and minor typo corrections

* Removed PackageDependency.CreateForUser() as WinRT has no way to convert Windows.System.User to a SID. Can be restored in the future when that's addressed

* Incorporated feedback
2020-08-23 17:18:10 -07:00