* Updating the build scripts to support VS2019
* Updating LangVersion to 8.0
* Adding some notes about VS2019 support to prerequisites-for-building.md
* Setting UseSharedCompilation=true and BuildingNETCoreAppVertical=true
* Use _VSCOMNTOOLS and workaround some BuildTools test.target checks
* Reverting the build-tools version and adding logic to restore the appropriate RoslynToolset
Fixes#6944
This change enables reflection on fields and in doing so fixes some gaps in the `ExternalReferencesTableNode` which fixes reflecting on some types that were not previously caught in the tests.
I found out that one bucket of the remaining CoreCLR CPAOT failures
is caused by missing support for instantiating stubs. We must emit
instantiating stubs whenever we hit LDVIRTFTN on a generic method,
otherwise JIT crashes with an assertion failure.
This case was already supposed to be taken care of by means of the
DynamicHelperCell representing the instantiating stub but
the implementation was incomplete. This change properly wires up
the instantiation stub logic and fixes various transitive issues
it uncovered.
Most notably this involved fixing the calls to DynamicHelperCell
to pass around the exact method, rooting the canonical method for
compilation when emitting an instantiating stub and fixing method
fixup signature formatting to include the instantiating & unboxing
stub flag, otherwise we were hitting a spurious assertion failure
regarding output symbol duplicates in the CPAOT object emitter.
Thanks
Tomas
* Fix catch clauses referring to a generic method argument
Scrutinizing the CoreCLR Pri#1 results I found a large bucket of
failures caused by mismatched catch clauses for generic exception
types. With tons of help from Jan Vorlicek I tracked this down to
a subtle difference in getMethodInfo flags. I have added one of
the previously failing Pri#1 tests to the CPAOT trivial unit test
suite.
* Set up the method flags unconditionally per JanK's PR feedback
I have audited the corresponding Crossgen code and I must admit
Jan Kotas is right as always - the extra method flags should be set
no matter whether the R2R build mode is active.
Callsites that load a virtual method delegate AV in the JIT due to `getCallInfo` returning a stub call kind instead of virtual call kind, which the JIT isn't expecting. Fix `getCallInfo` to behave like CoreCLR's Crossgen.
Tweak the control flow of 'getCallInfo' to more closely resemble what Crossgen does. This makes things easier to reason about since the logic in `getCallInfo` is complicated.
MethodWithToken is now used as a dictionary key and needs implementations for `Equals` and `GetHashCode`
Switch GVMs to use virtual entrypoint to match CoreCLR. With the test case I added to ReadyToRunUnit, the runtime would AV returning the value from the GVM when using regular method entry.
Remove the string constructor special casing in getCallinfo - it was for the CoreRT runtime.
* Basic infra support for running local CPAOT GC stress tests
This change basically introduces a new option /gcstresslevel
in tests\runtest.cmd akin to a similar option in the CoreCLR-specific
script. I have also bumped up the timeouts to 1 hour as CPAOT
Top200 test suite is currently timing out in ILC compilation of
1~2 tests (typically the bigvtbl tests).
Thanks
Tomas
* Improve comment on /gcstresslevel to clarify it's specific to CPAOT
* Cleanup based on Michal's PR feedback
I have modified the change along Michal's suggestions. I have
removed the explicit setting of environment variables in the
tests\runtest.cmd script and instead I just added logic for forwarding
the gcstresslevel flag. For now I have kept the timeout constant
as an internal detail of the inner runtest script for CoreCLR.
At this moment I haven't made any changes to the build_and_run_test
script; for some reason it didn't work in gcstress mode even before
with the environment variables. I'm going to try to investigate whether
we might be able to get rid of this duality and use the same script
for building CoreCLR CPAOT tests in both scenarios.
Thanks
Tomas
I have modified the change along Michal's suggestions. I have
removed the explicit setting of environment variables in the
tests\runtest.cmd script and instead I just added logic for forwarding
the gcstresslevel flag. For now I have kept the timeout constant
as an internal detail of the inner runtest script for CoreCLR.
At this moment I haven't made any changes to the build_and_run_test
script; for some reason it didn't work in gcstress mode even before
with the environment variables. I'm going to try to investigate whether
we might be able to get rid of this duality and use the same script
for building CoreCLR CPAOT tests in both scenarios.
Thanks
Tomas
1) Add new form of generic lookup for generic non-GC static base.
2) Fix previously missing generic lookup for fields.
2) Fix treatment of generic fields according to CoreCLR.
Thanks
Tomas
This change basically introduces a new option /gcstresslevel
in tests\runtest.cmd akin to a similar option in the CoreCLR-specific
script. I have also bumped up the timeouts to 1 hour as CPAOT
Top200 test suite is currently timing out in ILC compilation of
1~2 tests (typically the bigvtbl tests).
Thanks
Tomas
This change adds support for the helper
CORINFO_HELP_TYPEHANDLE_TO_RUNTIMETYPEHANDLE and fixes RVA field
management. In contrast to Crossgen which selectively copies over
parts of the MSIL file (including the RVA field data), we simply
point the RVA field address to the pre-existing data block in the
MSIL file embedded in the R2R image.
To achieve this goal I had to slightly generalize the section
builder by adding new support for "section start nodes" - symbol
nodes representing the beginning of each PE section. In case of
sections copied from the original MSIL executable, the section
start nodes point at the "original" MSIL section content.
Using this construct I was able to introduce a new node RVAFieldNode
which represents a given RVA field as an offset relative to one
of the section start symbols. Upon codegen, the RVAFieldNode
emits a relocatable pointer-sized record in the R/W data section
which points at the beginning of the RVA field data in the MSIL.
I have also ported JanK's RVA unit test to our base CPAOT test suite.
Fixes https://github.com/dotnet/corert/issues/6780:
- `ShouldStopProcessingWorkNow` was checking the count of existing threads and decrementing the count of processing threads. Due to differences from CoreCLR, the condition needs to be different. The effect was that when hill climbing decides to crease the thread count goal, `ShouldStop...` stops all threads that were processing work if there are enough existing threads, and even though there are several thread requests, the now-all-waiting threads are not released to process more work, leading to deadlock. The condition in `ShouldAdjustMaxWorkersActive` was also incorrect, fixed both.
- Fixed a few other small things that I saw
Fixes a compilation failure when compiling ASP.NET with `<RootAllApplicationAssemblies>true</RootAllApplicationAssemblies>`.
The fix is to mirror the dependency that the normal `MethodCodeNode` has for special unboxing thunks.
Previously the support for `[ThreadStatic]` was incomplete. This change adds some more tests (the last 3 of which fail prior to these changes). Fixes#6733 - I've included a call to `Thread.Sleep` to test that the assert is no longer hit. The sleep time is short so as not to affect the CI, but if its changed to something noticeable, like 10s, it can be seen to take effect.
Also included is a small comment typo.
* [CppCodeGen] Emit reflection metadata
* [CppCodeGen] Enable Reflection test
* [CppCodeGen] Add reference to constructed type in ImportLdToken
This patch fixes attribute reflection tests that fails due to
missing type metadata.
* [CppCodeGen] Use double indirection for gc statics as runtime expects
* [CppCodeGen] Emit header for nodes as runtime expects
* [CppCodeGen] Fix reloc emitting
We should take into account delta for reloc node
* [CppCodeGen] Fix boxing/unboxing
As we use double for float and int32_t for sbyte and int16 we should
use exact type for boxing/unboxing otherwise we will get incorrect
results.
This patch fixes float box/unbox issues in reflection tests
* [CppCodeGen] Enable generics reflection tests
* [CppCodeGen] Fix ldftn import
ImportLdFtn should generate code that obtains address of actual method not
unboxing stub. Unboxing stubs are used in case of delegate construction.
* [CppCodeGen] Fix ldftn importing in case of virual method of sealed class
* [CppCodeGen] Fix call importing in case of delegate instantiation
Remove unnecessary update of the first argument (firstParameter) of the
delegate initialization function.
* [CppCodeGen] Enable Delegates tests
1) Clean up generic lookup tests so that the TH->RTTH helper is
only called from a single method (GetTypeName). This is no longer
strictly necessary as Crossgen started to support the helper
but I believe it to be good for general hygiene.
2) Minor improvements in the R2R test harness to display ordered
sets of whitelisted vs. non-whitelisted jitted methods.
3) It seems to me that GenericLookupSignature should always pass
around the instantiating stub flag as the exact purpose of the
signature is to cater for shared methods with the extra
instantiation argument.
4) Based on JanK's clarification I have fixed the CPAOT compiler
to emit separate generic lookup cells for each context method.
Thanks
Tomas
This avoids generation of some of the helpers in favor of RyuJIT generating code that does what the helper would do. While this causes an about 1% regression in size of the generated executable, the benefit is better code density and fewer calls.
To support multi-core testing, associate the test process's PID with the ETW filter so that only assemblies loaded by the runtime the harness started are considered.
Emit the assembly name for methods that are jitted to allow grouping of methods by their assembly.
Add a new parameter, `--include` which configures the ETW filter to include all assemblies in a given folder, instead of having to specify a large number of assemblies individually.
Bug fixes around path casing causing events to be dropped.
* fix nit in StaticLibrary test (#4987)
* add README documentation on building native libraries (#4987)
* Move documentation for building native libraries to samples
* Fix static library build
* Add sources for NativeLibrary sample
Based on Michal's suggestion I have implemented an approximate
port of the two above CoreCLR changes. These changes improve
performance of GetHashCode on Enum types by avoiding boxing
thanks to transforming the calls to the equivalent method calls
on their underlying types.
My proposed compensation for #9138 is in the signature builder
- we just check whether the MemberRef token we received has
the proper owning type and, when it doesn't, we encode it
explicitly in the signature.
I have created a simple unit test that actually doesn't do much
but I verified by submitting it to R2RDump that I indeed see
direct calls to byte.GetHashCode and int.GetHashCode in the
produced machine code.
Thanks
Tomas
Currently there's just one native library that gets picked up
by the above wildcard spec - Microsoft.DiaSymReader.Native.amd64 -
so I have excluded it explicitly in the script as I don't expect
this list of exclusions to fluctuate too wildly. We would need
something smarter if we wanted to be able to automatically
orchestrate builds of complete folders (e.g. a UWP app).
Thanks
Tomas
* Add two xunit packages to runtime.depproj to unblock about 90 tests
This change adds the packages "xunit.core" and "xunit.performance.core"
required by some CoreCLR tests to external test dependencies. I have
copied the clauses from external.depproj in CoreCLR and their versions
from dependency.props. This change reduces the number of failures from
346 to 254.
Thanks
Tomas
Replace use of 2.1 runtime with a recent version from master targetting netcoreapp3.0. This version now includes System.Runtime.Intrinsics, which was previously named System.Runtime.Intrinsics.Experimental (and lived in a separate Nuget package) meaning the test drop we used couldn't bind against it.
runtest.cmd /coreclr all /mode readytorun fail rate improved from 489 -> 346 (2001/2347 or 85% pass rate).
1) Pass the proper configuration and architecture to msbuild when
restoring XUnitTooling so that it gets restored into the proper
folder - this fixes building the CoreCLR tests in release mode.
2) Set the 'Optimize' property for Microsoft.NetCore.Native.targets
in CoreCLR test and framework assembly build to actually make a
difference w.r.t. release codegen by specifying '-O' to ILC.
3) Update comments in the R2R test scripts to indicate we now
support per-configuration output folders.
Thanks
Tomas
P.S. Top200 tests have approximately the same pass rate in
debug and release mode (debug reports 6~8 errors, release 8~9 errors
in my local testing).
Framework assembly compilation in release mode has somewhat worse
pass rate, 110 out of the 152 assemblies build amounting to about
72% pass rate.
* Enable R2R tests in release mode
This change enables R2R tests in release mode. It seems to me that
previously the tests were disabled in release mode because they
were crashing the CPAOT compiler due to a bug I made months ago -
apparently I put the CanInline method in the wrong class so it got
never exercised. In release mode, when inlining is active, this bug
ended up inlining bits of framework into the test app that we
couldn't express with the available ref tokens.
Thanks
Tomas
* Update MSBuild properties
* Revert "Disable building tests in official builds (#6565)"
This reverts commit 8aab18c5a5.
* Revert "Fix Linux pipeline (#6566)"
This reverts commit 23be7aa294.
Early on in CPAOT bring-up I asked Simon to set a short timeout
for test executions as many tests were timing out due to compiler
bugs. This is no longer the case - we only have a handful of
remaining failing tests and some of them are now suffering from
the short timeout. For now I have bumped up the timeout from 30
seconds to 2 minutes and that seems to be fixing all Top200 tests
that used to fail with the -103 exit code (about 5 tests overall).
Thanks
Tomas
* Change the load expression to extend to double when the type is float and the TypeKind is double.
This extend floats to doubles when doing comparisons such as ceq. Previously it was bit stuffing the float into a double.
Casing of method name.
Add test to load double const.
* Initial support for running tests against CPAOT-built framework
The pass rate is not super high right now, I'm seeing 109 failing
tests or about 13% pass rate but the main point is that the pass
rate is not zero, we can build on that.
In the AVAILABLE_TYPES R2R node, I added provisions for emitting
exported types. This was one of the issues we were hitting - the
partial facade System.Runtime wasn't properly exporting its type
forwards.
In the PE builder, I dropped copying of all directories per JanK's
suggestion, the latest impulse being a mismatch between the debug
directory and the CPAOT-compiled executable.
* Modify available types to be emitted based on metadata
I have applied JanK's suggestion to change AVAILABLE_TYPES
to be generated based on metadata rather than based on the results
of dependency analysis.
* Addressed Michal's PR feedback
1) Unified DefinedTypeInfo and ExportedTypeInfo to use a common
generic class TypeInfo<THandle> per Michal's suggestion.
2) Fixed exported type traversal to cater for nested forwards.
3) Added a tiny fix I noticed by comparing CPAOT and Crossgen
code - querying method dictionary in a generic lookup should use
the "METHOD_HANDLE", not "METHOD_DICTIONARY" fixup. This increases
the pass rate of Top200 against CPAOT-built framework to about
82% (22 failures out of 126 tests).
Thanks
Tomas
* Generic lookup support
This change introduces CPAOT support for all three forms of generic
lookup (THISOBJ, CLASSPARAM, METHODPARAM) and the corresponding new
GenericLookupSignature. I have added several unit tests exercising
the new functionality.
* Further fixes for generic lookup support
My initial generic lookup change only dealt with type-based lookup.
During experimentation with framework compilation I hit the need
to also implement method-based generic lookups. This change mostly
expands the initial implementation to also cater for method lookups
by expanding the RuntimeDeterminedTypeHelper and bits of other code
as appropriate.
I have also managed to get a bit more insight into emitting
signatures for instantited methods. CoreCLR runtime doesn't support
direct consumption of mdtMethodSpec tokens in method signatures -
instantiation type arguments for generic methods must always be
explicitly specified in the signature; it however does support
direct consumption of an instantiated OwningType via a mdtTypeSpec.
* Always emit signatures in instance entrypoint table using Def tokens
I found out that method signatures used to encode methods in the
INSTANCE_METHOD_ENTRYPOINTS R2R header table cannot use MemberRef
encoding for methods on instantiated types as it hits an assertion
failure in the CoreCLR runtime in SigMatchesMethodDesc.
This change introduces a boolean parameter the table can use to
specify it forbids ref-based encoding.
* Version resiliency fix - partial resurrection of tokens
During our today meeting JanK, Michal and David finally made me
comprehend the token problem. Propagation of tokens from JIT to
the managed compiler is indeed necessary due to a corner case
regarding version resiliency, however it's thankfully limited to
methods outside of the version bubble.
The problem is as follows: assume we have a "Program" and a
"Library" outside of its version bubble. At IL compilation time
of "Program" using Roslyn, the "Library" looks as follows:
<code>
class Base { void Method(); };
class Derived : Base { void Method(); }
</code>
When "Program" calls "Base.Method()." vs "Derived.Method()",
the appropriate MemberRef tokens representing
"[Library]Base.Method()" and "[Library]Derived.Method()"
get stored into the MSIL.
In the second step, prior to CPAOT compilation, we replace the
"Library" with an updated version where the Method() only exists
in the Base class (this is supposed to be a safe change w.r.t.
version resiliency):
<code>
class Base { void Method(); };
class Derived : Base { }
</code>
When we now run the CPAOT compiler, a lossy transformation takes
place - both IL tokens, "[Library]Base.Method()" and
"[Library]Derived.Method()" get squashed to the same MethodDesc
"[Library]Base.Method()" (because that's where the compiler sees
the method at CPAOT time).
Without explicit token propagation i.e. with just the reverse
synthetic token lookup, we end up encoding one of the methods
(an arbitrary one based on which one we saw last) into both
indirection cell signatures.
This breaks when later, at runtime, someone puts back the original
"Library" version where the Method() sits in both classes
because both R2R signatures contain the same reference so that
the runtime has no way of knowing they should be in fact separate.
As the problem is obviously limited to methods outside of the
version bubble, we can thankfully keep most of the ModuleTokenResolver
and related logic as for types and fields it's completely fine,
we just need to revive a bit of logic propagating tokens from JIT
for methods.
Amusingly enough it turns out that overall it mostly simplifies
the code rather than making it more complex. The gist of my change
regards splitting "ExternalMethodImport" to two classes,
"ExternalMethodImport" and "LocalMethodImport".
This split has several nice properties - for LocalMethodImport,
we always have a _localMethod available but we don't need to
maintain constrained type information - within the version bubble
we can look up all tokens using the ECMA objects so we don't really
need the constrained calls. The various R2R header tables
(method & instance entrypoints, exception info) also mostly need
to only emit local methods.
ExternalMethodImport's newly include the method token information,
propagating it to the method signature encoder - this is necessary
as we no longer record method token mapping in ModuleTokenResolver
which can now only resolve methods within the version bubble.
The change passes the basic R2R unit tests and seems to have a
comparable pass rate w.r.t. CoreCLR Pri#0 tests. I have yet to verify
that it indeed fixes the above described repro case but in general
I believe the change is sound.
The only problem I hit is in the method getFunctionEntryPoint
where the token is not readily available, for now I highlighted it
with a TODO comment; I'm not completely clear on when JIT uses this
method; if it turns out this can be used for cross-bubble calls,
we'll likely need to fix JIT to pass the appropriate token information
to the managed code.
Thanks
Tomas
Cross-module signature fixes, non-compilable method fixes and various fixes for generic methods
This change fixes the build of the trivial ASP.NET app. The problem
is that, once we start decoding method / type signatures involving
multiple metadata readers, the metadata reader by itself is not
sufficient to decode a handle into a typesystem object, we need
the EcmaModule objects for this purpose. In this change I have added
another lookup table to CompilerTypeSystemContext that can
back-resolve metadata readers to their respective modules.
This change contains my previous non-compilable method fix slightly
polished based on Michal's PR feedback and a bunch of fixes for
generic methods, most notably fixing previously missing emission
of the instance entrypoint header table. I have ported CoreCLR
implementation of version-resilient hashcodes so that the instance
methods are properly discoverable at runtime.
In the process I have slightly cleaned up the implementation of
the entrypoint tables: as these are now "summary nodes", no longer
state machines accumulating the methods during compilation, I have
removed the public "state machine" methods (Add). I have also moved
GetFixupBlob to MethodWithGCInfo so that it can be used from both
entrypoint tables.
I have included my recent fix for missing FrameInfos on x86 in a
slightly improved form - instead of forking all the code asking for
FrameInfos, I have modified its initialization so that, on x86,
where CorInfoImpl passes null, I generate a single-element
pseudo-frame info that lets me unify most of the related code.
Based on investigation of Michal's unit test I have also fixed
several places in method signature emission, token harvesting
and canonicalization. I have added Michal's sample as a new R2R
unit test.
I have also fixed a bug that omitted the constrained type and return type
in method signature formatting so that two external method imports
with different constrained types ended up incorrectly reported as
duplicates.
As Michal pointed out, SignatureContext no longer needs to implement
ISignatureTypeProvider so I removed it. I am not yet directly replacing
SignatureContext with ModuleTokenProvider as I believe their split
may be still useful for the envisioned merged multi-file R2R build.
In my mental view ModuleTokenProvider handles the synthetic token logic
and may ultimately become per input module whereas SignatureContext
semantically represents the [output] compilation module. I fully admit
my knowledge and design ideas in this area are still limited.
Based on Michal's PR feedback I have also switched TypesTableNode over
to use the same version-resilient hashcode calculation logic I implemented
while standing up the instance method table entrypoint R2R header table.
* Split the command line out and remove parameters we know aren't needed in R2R mode.
* Wipe the \native folder before compiling to clear out previous compilation attempts.
* Remove some logic that skips native assemblies I was using previously. It's not needed since System.*.dll and Microsoft.*.dll are all managed.
Add /r2rframework switch to runtest.cmd which runs the CPAOT compiler in
ready-to-run mode against the managed framework assemblies in CoreCLR.
I made a couple small bug fixes to unblock compilation of 30 framework
assemblies with code in them. There are also many small facade
assemblies with no code that we put a ready-to-run header in but don't
do anything interesting. Overall, currently, we produce ready-to-run
images for 84 / 214 assemblies.
The produced images are placed in bin\obj\<build>\CoreClrRuntime\native.
This folder also includes the .rsp file to compile specific assemblies
again for investigation.
Generic type markers in the ETW events used by the ready-to-run harness messed up the name mangling. Don't consider any generic arguments as part of the name when formatting the method to remove namespaces so we don't mistake a period in the generic type for part of the method's type's namespace.
In ready-to-run mode, use the CoreCLR shared framework implementation
assemblies to generate test R2R images instead of the AOT versions
currently used in standard RyuJit mode.
System.Object has a pointer field for the EEType in each object header
in the AOT implementation of System.Private.CoreLib. The CoreCLR
System.Object does not have this field, and instead the runtime adds an
implicit pointer field for the MethodTable. Handle this in the R2R
compiler with a custom layout algorithm for System.Object.
Adjust the test scripting to consume the CoreCLR framework.
Assert if we accidentally pass in the AOT System.Private.CoreLib in R2R
mode.
CoreCLR all tests pass rate improved by 21 tests to 1370/2347 passing
(58.4%).
Static placement fixes
Our existing CPAOT implementation of statics
placement is incorrect as there's a difference between CoreCLR
and CoreRT in statics allocation algorithm - basically CoreRT
allocates statics per type whereas CoreCLR does that per module.
This change modifies the field layout algorithm to use a loose
managed rewrite of the CoreCLR logic. I have also included Michal's
addition to the ReadyToRunUnit test demonstrating this
behavior.
Thanks
Tomas
Fix test command-line arguments in the CoreCLR test runner script
Apparently some of the CoreCLR tests were failing with complaints
towards incorrect command-line syntax. I tried to go over the
archived source code for the System.CommandLine package but I have
found no trace of automated splitting of argument values. For this
reason I have adjusted the script to pass multiple --testargs
arguments in such case. I believe that at the end of the day it
simplified the script overall.
Thanks
Tomas
When I was testing #6366 in a mode we don't normally test (debug compiler with IL scanner enabled), I hit an assert around inconsistent reflection metadata for a Module type. Turns out we weren't tracking reflection on unconstructable types. Fixed that and added a test.
* The ready-to-run harness includes time out logic to kill tests that get stuck. Use it to run CoreCLR tests since many tests get stuck in an infinite loop at the moment.
* Add an option to disable ETL tracing when using the harness. Instead, it just runs the app with `Process.Start()` and calls `WaitForExit()`. The CoreCLR tests are too noisy right now to run with ETL on.
* Make some build system changes so that the harness is run from pre-built binaries instead of using `dotnet run`. This is much faster and required for tests running in parallel. Using `dotnet run` on the same project in parallel hits file in use errors.
* Fix harness to allow multiple arguments passed to tests
* Properly kill the test process when it times out
* Reduce the timeout to 30 seconds. None of our tests, when running correctly, approach this timeout so it is still a conservative setting.
* Fix exception handling info in R2R images
1) ReadyToRunObjectWriter support for "nested ObjectData" where a given node
can produce an enumeration of subordinate ObjectData items. This is used
to produce the EH info that comes from CorInfoImpl in form of ObjectData
blobs.
2) One other change in ReadyToRunObjectWriter is the newly added support
for modifying the PE header directories - this is needed to publish
the RUNTIME_FUNCTIONS R2R header table in the Exception Directory,
otherwise RtlLookupFunctionEntry doesn't work.
3) Fix the EH info itself in CorInfoImpl as CoreCLR runtime that we're
targeting with the R2R images uses a different encoding than CoreRT.
4) MethodGCInfoNode is created for each funclet and holds the index
of the corresponding FrameInfo which is used to emit the proper unwind
and GC info for each funclet.
5) RuntimeFunctionsTableNode - emit a separate RUNTIME_FUNCTION entry
for each funclet and emit the appropriate final sentinel entry.
6) ReadyToRunCodegenNodeFactory - new helpers, support for personality
routines, rooting exception lookup table.
* Deleted a bit of previously overlooked dead code from R2R object writer
* Minor extra cleanup in the R2R object writer
With the proposed changes, we no longer need to pass around
the SectionBuilder explicitly as we have it in a field.
* Additional cleanup based on Michal's feedback
I have refactored / simplified emission of the exception handling
info. The crucial observation is that, in CoreCLR, the EH info
is a dumb blob that doesn't contain any relocs. We can leverage
this information to simplify the emitter.
I have removed the logic for emitting subordinate ObjectData
nodes from the object writer and from ExceptionInfoLookupTableNode.
I have deleted SyntheticSymbolNode and instead I introduced
a primitive EHInfoNode that gets populated from the lookup table
in LayoutMethodsWithGCInfo.
In ReadyToRunObjectWriter I kept the previous move of object
emission data to the dedicated method EmitObjectData as I believe
it makes the method EmitPortableExecutable easier to read.
I found out MethodEHInfoNode was not used for anything - it was
a leftover from one of my earlier failed attempts to attack
EH info - so I deleted it.
In CorInfoImpl, I moved the EH info emitters into the forked files
CorInfoImpl.RyuJit / ReadyToRun as the EH info is completely
different for the two targeting runtimes.
* Additional MethodGCInfoNode cleanup
In this additional delta I simplified MethodGCInfoNode management.
I changed the logic so that there's just one MethodGCInfoNode per
method; apart from requiring fewer allocations, this let me
drop the delayed factory logic in MethodWithGCInfo.
* Removed obsolete comment
* Addressing Simon's PR feedback
Codegen fixes required to make Console.WriteLine work on WebAssembly (when combined with #5987 and a matching CoreFX build):
The 'this' pointer for reference types is a byref for constrained virtual calls and needs to be dereferenced.
Enable P/Invoke marshaling. This isn't easy to test directly, but we'll notice the effect as we try to use framework code that P/Invokes (in Console.WriteLine, there's SafeHandle marshaling).
* Run CoreCLR tests that are known to pass in ready-to-run during CI
* Add a mechanism to easily collect the list of passing / failing tests for use in inclusion / exclusion lists.
* Implementation of reverse type reference lookup
Based on JanK's suggestion, I have implemented the basic logic
for reverse type reference lookup. I have demonstrated basic
functionality of this feature by adding two delegate tests involving
Action and Func<char, char>.
I also had to slightly refactor type signature emitter - my
initial implementation was heavily dependent on parsing an existing
metadata signature but in the resolution process we can hit synthetic
types that don't have such a metadata representation, most notably
for generics.
For this reason I believe we need custom recursive descent into the
type tree and this change includes the initial implementations. For
now I'm keeping the generic signature decoder as it's still used by
method signatures but my current thinking is that it will also become
unneeded at some point and we'll be better of with a custom
implementation matching our exact needs.
Thanks
Tomas
* Addressing part of PR feedback
I have refactored SignatureBuilder to use a big switch over the
TypeDesc.Category. This overall simplified things. I fixed several
signatures to stop emitting type signatures when only looking up
references as signature emission may be a relatively costly process.
I tried to add some provisions to increase robustness of the
reference resolution but for now I'm keeping the reference enumeration
as just hooking up resolveToken is not sufficient to provide a
sufficiently complete mapping from types to tokens.
Thanks
Tomas
* Incomplete changes for shared generic support
* Fix bug in X64 emitter
The MOV(register, register) instruction was implemented incorrectly.
The method GetRexPrefix assumes that the register specified as
parameter is the target, however we were using the other opcode
where the register is the source. I have fixed this inconsistency
and I added code comments to the individual REX values to improve
clarity.
Thanks
Tomas
* Improved support for interface dispatch and unboxing stubs
Based on debugging calls to Dispose on classes and valuetypes and
thanks to help from JanK I finally managed to make calls to dispose
work on both. The change mostly involves propagation of the
new unboxingStub flag, switching over to general MethodEntry
fixups that allow encoding the value of the flag and general
infrastructural plumbing in the R2RCG node factory.
Thanks
Tomas
* Some additional minor fixes for interface dispatch
I removed an apparently incorrect check I had put in CorInfoImpl
previously. I fixed a trivial typo in the DelayLoadHelperThunk.
With these changes I'm again passing all the previous tests
plus the new Dispose on class / struct tests demonstrating
initial vestiges of unboxing stub support. Generic enumeration
still doesn't work for some reason, I'll investigate that next.
Thanks
Tomas
* Removal of reverse lookup in favor of eager lookup
Based on Michal's suggestion I have replaced the reverse lookup
in ReadyToRunCodegenNodeFactory with eager lookup called from
CorInfoImpl.resolveToken. The current implementation seems to be
the minimum enough to compile the 22 trivial unit tests.
Thanks
Tomas
* Initial support for constrained type virtual method calls
Thanks to JanK's help I was finally able to put together
initial support for constrained method calls finally fixing
the call to List<string>.Enumerator.Dispose().
Thanks
Tomas
* Add unit tests for features unblocked by my previous commits
This mostly covers my recent contributions in the field of
unboxing stubs, constrained method calls and token lookup
improvements.
Thanks
Tomas
* Fix rebase conflict
* Addressing [most of] Michal's PR feedback
I have trivially applied the mechanical feedback (fixing the typo
lea -> mov in the X64Emitter comment, applying the good suggestion
to simplify register references by using the X64 namespace and
unifying the VALUETYPE / NULLABLE case in signature encoding.
For the remaining feedback, I'm trying to provide my rationale for
further discussion:
1) Why do we need the || READYTORUN part in CorInfoImpl around line
3100? To let us pass around call site names to interface dispatch
cells - as interface dispatch cells are per call site even though
they might reference the same interface method, we need some way
to distinguish their symbols as we validate symbol uniqueness to
maintain output consistency.
If you're fine with the concept and you're just suggesting there's
an easier way to encode this by means of preprocessor directives,
I don't see that, please enlighten me.
2) Stashing token information in CorInfoImpl - I think the big
problem is signature building where in general we need to dig
arbitrarily deep into composite signatures.
Based on how good your suggestion for using resolveToken seems
to be working, I'm starting to think that it might be easiest to
drop the token propagation from CorInfoImpl and just always
synthesize the tokens based on the pre-recorded information; it
might unify parts of the previously forked NodeFactory bits too.
Once we start mucking with multiple assemblies within a versioning
bubble, I think we have several possible ways to go. As I mentioned
in a comment to the previous Simon's PR, I believe that the
R2RCGNF class has already gotten too big and would merit splitting
into multipler parts.
One of these should be exactly this "token manager" and perhaps
separating it from the node factory into a class of its own might
address some of your concerns; I however still believe that in
general, CoreCLR signatures support reference module override
so that we should be able to use arbitrary token sources across
the entire versioning bubble for the purpose of referencing externals.
Thanks
Tomas
* Move token management logic into a separate class ModuleTokenResolver
In this mostly mechanical change I tried to address part of Michal's
PR feedback by moving the token resolution logic away from the
ReadyToRunCodegenNodeFactory into a separate class. While the change
is mostly mechanical, I believe it nicely demonstrates
the relationship between the two classes and possible future steps
regarding migration of the logic from one to the other. For now
the resolver naturally hangs off the factory, that can easily change
by it becoming a part of the CorInfoImpl.ReadyToRun class.
Thanks
Tomas
* Further propagation of ModuleTokenResolver
This change removes direct references to ReadyToRunCodegenNodeFactory
from those signatures that only needed it for token resolution.
Thanks
Tomas
* Remove unnecessary | READYTORUN in CorInfoImpl
Based on Michal's suggestion I'm removing the | READYTORUN clause
in the conditional directive producing call site name for
interface dispatch cells. I have double-checked that SUPPORT_JIT
is not defined in the project so that the condition is true.
Thanks
Tomas
* Support readytorun mode for CoreCLR tests
Currently 226 / 1829 tests pass (12.4%) from the KnownGood subset.
* CR Feedback
Break out the two large command lines into a variable so echo and command run use the exact same command line. This prevents the classical problem where we forget to keep the echo in sync with the actually run command line.
Make comparisons case insensitive.
* Compile single input file specified in the compile option to single output file instead of multiple output files
* Parallelize multi-module with msbuild
* Fix missing binaries error
* Move BuildReadyToRunMultiModule to BuildIntegration folder
* Avoid using additional option moduleFilePaths
* Get multi-module to work without moduleFilePaths
* Fix error by not creating FullVTable
* Don't add EETypes to LibraryRootProvider for ReadyToRun
* Use ecmaTypes from compilationModule instead of eeTypeNode so AvailableTypes table has entries, get correct file extensions
* Disable ReadyToRunMultiModule test for JIT
* Compute AvailableTypes hash for types with namespaces
* Fix merge errors
* Use correct hashcode algorithm, add test for nested type
* Uncomment CheckCanGenerateMethod, remove IlcMultiModule property, remove todos
* Rename ReadyToRunMultiModule
If for some reason (ie a test harness bug or user error) no assembly load events are fired, the harness will see the absence of jitted methods as a sign that all code ran R2R perfectly. Add a smell test that checks we at least saw one test module loaded.
Fix up tabs and some unused variables.
* Build header tables using metadata manager
* Enable MetadataManager to track use of types / methods
* Introduce `AvailableType` which replaces the CoreRT EETypeNode.
Ready-to-run doesn't emit types directly into the image but records
their use and fills a table of available types used for type resolution
between ready-to-run modules by the runtime.
* Add dependency tracking in a few existing places where references to
types are being made. In library compilation mode, the non-private types
will be rooted so they always end up in the available types table.
* Use a single NodeCache for method entrypoints instead of a separate
dictionary for ready-to-run entrypoints. To smooth over the API
differences between the base `NodeFactory` and
`ReadyToRunCodegenNodeFactory`, allow a custom GetOrAdd method that
takes a creator callback.
* Switch over `RuntimeFunctionsTable`, `TypesTable`,
`MethodEntryPointTable` to use metadata manager for the list of types /
methods to emit entries for.
* Add System.Commandline package, fix concurrency error causing test to display as failed
* Add referenced assemblies to testModules, use System.CommandLine
This is a port of dotnet/coreclr#17732. Fixes bug 603305.
Per popular demand from people who would like to serialize ref returning properties or use them in data binding, adding support for this in MethodInfo.Invoke. The choice was to make these dereference the result and return it like that. For the corner case of returning a ref to a null, a `NullReferenceException` will be thrown (this behavior has a very unfortunate side effect of slowing down the general Invoke path because we can't throw this exception from a place where we would end up wrapping this in a TargetInvocationException).
I also made tweaks to byref parameters path to fix byref pointer parameters, but this is horribly broken on the CLR so I didn't bother to test it here. Should work.
Also includes a slight size on disk optimization that merges common tails of the static/instance paths of the invoker thunk.
Testing of the calling convention converter portion was done by disabling invoke thunks generation in NUTC and running UnitTests. I also ran it with GCStresss (RH_GCStressThrottleMode=1, RH_GCStress_Mode=1). Let me know if there's more testing needed. This has a huge test matrix for being such a corner case thing.
[tfs-changeset: 1709919]
Use LLVM arguments and returns where possible (no GC references) instead of passing them on the shadow stack. The argument optimization saves 3% optimized (5% compressed). The return change adds about 0.5% to the uncompressed file, but saves about 6% compressed. They both save size in debug. This also greatly simplifies debugging and reading code. Also includes a fix to the class constructor runner where it was calling cctors with the wrong signature and some test fixes.
Restore CoreCLR and CoreRun
* To test the ready-to-run binaries produced by ILCompiler, restore Microsoft.NetCore.App, which contains the CoreCLR runtime plus the set of framework assemblies defined by netcoreapp2.1 which should be enough to run most tests.
* Also restore CoreRun.exe, the host used to execute test binaries.
ReadyToRun ETW Test Harness
* Add a test harness to validate ready-to-run binaries properly execute the ready-to-run native code instead of silently falling back to Jit.
* To validate this, The Microsoft.Diagnostics.Tracing (EventTrace) package is used to enable CLR ETW events during test execution. The ModuleLoad event is intercepted to find the module IDs of assemblies relevant to test (recall, all CLR ETW events are machine-wide across all CoreCLR / CLR runtimes currently running). The MethodLoadVerbose events are intercepted and their module ID is matched with those filtered in the ModuleLoad event. This allows us to focus solely on the test binaries and not the dependent framework.
* An optional whitelist file can be provided, whose format is a text file with one method name per line. The output of test failures can be pasted into a whitelist file to create a baseline.
Ready-to-run test system integration
* Add a new code generator type to runtest.cmd supporting readytorun mode.
* Ready-to-run tests need to know the location of the CoreCLR runtime (they're executed with CoreRun) as well as the ETW test harness test project.
* Add a ready-to-run specific set of tests (from Tomas' private bringup branch) which will only run in ready-to-run mode. This is controlled by the presence of a file named "readytorun" in the test source folder. This works the same way as the web assembly tests, where we opt in until the compiler support is mature enough to run all tests in this mode.
* The ready-to-run tests will run as part of the larger suite when runtest.cmd is invoked. To run just ready-to-run tests, use runtest.cmd /mode readytorun.
* Handle multidimensional array instantiation (#5421)
newobj instruction on multidimensional array is now processed with
ArrayHelpers.NewObjArray. Enabled instantiating multidimensional arrays.
* Add testing for multidimensional array instantiation.
Only nullity and lengths are checked due to get & set not working on
multidimensional arrays.
Runtime configuration (e.g. whether to use server GC) can currently be provided either through environment variables at runtime, or through a RhConfig.ini file placed next to the executable. This adds another channel where we burn a blob similar to RhConfig.ini format into the executable. The order of precedence is environment variable > RhConfig.ini > embedded config.
The `ServerGarbageCollection` project property that is used to determine whether to link with a runtime that supports server GC will be also used to control whether to generate a configuration that enables server GC at runtime by default.
Fixes#6100.
These tests started failing after RyuJIT update in #5969.
```
at System.Diagnostics.Debug.Assert(Boolean condition, String message, String detailMessage)
at Internal.JitInterface.CorInfoImpl.convertPInvokeCalliToCall(CORINFO_RESOLVED_TOKEN& pResolvedToken, Boolean mustConvert) in D:\j\workspace\debug_windows_nt28ae10f6\src\JitInterface\src\CorInfoImpl.cs:line 3453
at Internal.JitInterface.CorInfoImpl._convertPInvokeCalliToCall(IntPtr thisHandle, IntPtr* ppException, CORINFO_RESOLVED_TOKEN& pResolvedToken, Boolean mustConvert) in D:\j\workspace\debug_windows_nt28ae10f6\src\JitInterface\src\CorInfoBase.cs:line 2488
```
We were not allowing these because we couldn't properly fill out the vtable (can't really make unboxing thunks to put in the vtable). We actually need the vtable, but only because the generic dictionary lives there and reflection might touch it.
[tfs-changeset: 1705047]
#2695 got closed without any effort to validate things work. While this test has changed since the issue got filed, maybe we can still luck out and hit the issue in the CI (in which case I'll reopen the issue). If not, at least we'll get rid of the comment that references the closed issue.
* Separate Issues on a per OS basis
* Add Linux exclusions
* Fix -p flag
* Add OSX exclusions for all-green
* Final Test Definitions
* Enable tests in runtest
* Unify OSX and Linux issues
* Update docs to reflect changes
* Remove Threading.Tasks.Tests and intermittent test failures
* Prune test list to include only System.Collections.Tests
* Add RSP file support
Incremental CI Improvements
Add JSON validation
* Define broad test exclusions
Remove duplicate test names
* Add OSX and Linux URLs
* Revert exit commands
* Change test log output directory and disable test execution printing
* Remove exclusions to test CI failure
* Disable simple and multimodule tests during CoreFX
* Revert "Remove exclusions to test CI failure"
This reverts commit 162a354365.
* Change exit code to test test result reporting
* Add exclusions
* Add removed exclusions
* Minor shell script fixes
- Enable struct Marshal APIs for reference types
- Fix mismatched argument order for native to managed marshaling
- Fix sizes reported by Marshal.SizeOf APIs
- Delete some unnecessary wrappers and fixed a few typos
Fixes#5674 and #5217
Code to call InitializeModules on startup and associated bug fixes and workarounds. Handles mismatched struct calling conventions between LLVM and Clang and works around the dup opcode incorrectly reloading values. Enables StartupCodeMain and fix associated bugs in pinvokes. Implements main return codes
Gets rid of `LINK : warning LNK4098: defaultlib 'LIBCMTD' conflicts with use of other libs; use /NODEFAULTLIB:library [d:\git\rt\tests\src\Simple\StaticLibrary\StaticLibrary.csproj]`.
Since pointer types cannot be boxed, reflection invented a `System.Reflection.Pointer` type that can be used in places where an object reference is required. This implements support for the `Pointer` type in:
* Delegate.DynamicInvoke for Project N and CoreRT
* MethodBase.Invoke for Project N and CoreRT
The MethodBase change is most annoying because it requires us to do extra work due to not being able to instantiate generics over pointer types (the invoke thunks are otherwise generic). We need to unwrap the pointed to types and compensate for that in the thunk. This also means we now create instantiations over void (for void pointers), but meh.
Also fixes two bugs found on the way (so that I get the whole trifecta of IL2IL, NUTC, and binder work):
* Pointer type hashcodes were generated wrong by the binder for pointer types deeper than level 1
* The null to default(pointer) conversion didn't work (this also affected pointer typed field setters)
Fixes bug 457960.
[tfs-changeset: 1692475]
Make the Server GC an optional component by building two flavors of the runtime and selecting a specific one in the build via the `ServerGarbageCollection` MSBuild property. Don't build the Server GC flavor for Web Assembly.
Fixes#5182, Fixes#5306.
This got broken when sorting got introduced to stabilize the executable image. Instead of relying on implicit ordering of nodes, use order specified in the map.
Since we currently allow unnormalized things in the dependency graph (see #5264), we need to ensure scanner produces a dependency graph that can be queried for unnormalized vtable slices.
This fixes a regression in a ASP.NET scenario. Also fixes#5508.
* add preliminary Library test project (#4985)
* update library test and make it pass on Windows (#4985)
* rename Library test project to SharedLibrary (#4985)
* update sharedlib test, make it pass on darwin (#4985)
* update msbuild tag names (#4985)
* move native runner rsp file to native intermediate output path (#4985)
* add test for static library, make it pass on unix (#4985)
* update static library test, make it pass on Windows
* skip shared library test for linux (#4985)
* fix StaticLibrary linux build (#4985)
* update unix linker args when building StaticLibrary native binary (#4985)
* add copyright headers to all newly added source files (#4985)
* fix multimodule build errors for native library builds (#4985)
* add comment explaining owning module of native library startup method (#4985)
* set owning module of native library startup method to gen'd module (#4985)
* remove unneeded System.Linq import (#4985)
* Implement static constructor triggering. Also includes several bug fixes found in the process:
* Don't make the ClassConstructorRunner depend on module intialization
* Moves non-GC statics and thread statics from globals to the type's data regions. GC statics can't be moved until we can call InitializeModules on startup.
* Devirtualizing interface calls to structs in order to be able to compile the class constructor runner.
* Add a prolog block before Block0 to allow branching to Block0, which happens in retail builds
* Correct use of unreachable in traps and at the end of finally blocks to fix more retail build problems
* Stop reusing spill slots when a spilled value is spilled again. This avoids cases where the spills feeding into a block don't use the same slot
* Enable thunks for NativeCallable methods and call RhpReversePInvoke for them
* Fix alignment for cases where a small type is followed by a larger one
* Workaround for Emscripten atomics bug
* Automatically execute WebAssembly tests on Windows by using emrun to launch a headless instance of Firefox (which doesn't interfere with any other Firefox usage). This should cut down on manual testing and help us move toward CI integration.
* Implement intrinsic call to InitializeArray.
RuntimeHelpers.InitializeArray is implemented as a call to LLVM
intrinsic llvm.memcpy.p0i8.p0i8.i32, which copies from generated global
constant to provided target array.
Currently the base size of the array object is hardcoded to 8, since no
EEType is provided in the intrinsic call.
Fix#5345.
Add testing code for InitializeArray in HelloWasm.
Roslyn generates RuntimeHelpers.InitializeArray for arrays of integral
types with initializer of length greater or equal than 3. Here we use
this feature to generate a call to RuntimeHelpers.InitializeArray for
test.
Catagorize array argument types in InitializeArray
Differently handle vectors, multidimensional arrays and object typed
arrays in InitializeArray intrinsic.
Picks up the latest version of Pri0 tests (as of yesterday).
There was a massive cleanup of Pri0 tests on the CoreCLR side so we now run only about 2000 tests. Hopefully, they're more representative now. Most of what we picked as "Top200" was no longer part of the Pri0 suite, so I made a new one.
We might want to consider switching to Pri1 for the rolling builds.
* Implement break opcode as llvm.debugtrap.
In Emscripten, `llvm.debugtrap` is implemented as a round-trip call to
`debugger;` statement JavaScript which will invoke debugger in browsers;
and in LLVM it's implemented as `unreachable` instruction of WASM. This
should be a better match than `llvm.trap` on `break` opcode semantics.
Fix#4511
Set IsDebuggerPresent to TRUE in case of WASM. Although
`Debugger._isDebuggerAttached` is not set, this modify itself will allow
`Debugger.Break` to work.
Add test in HelloWasm for `Debugger.Break`.
* Enable wasm building on OSX.
Upgrade libLLVM to 4.0.0 and LLVMSharp to 5.0.0
LLVM upgrade: Fix 'Use still stuck around after Def is destroyed' of the deleted basic blocks
Add wasm support in runtest.sh
Remove HelloWasm.csproj reference to .ilproj on non-windows OS because of dependency on ilasm
Fix LinkNative target to execute correct commands for Unix wasm builds.
Added support for building on Ubuntu 16.04.3
Update documentation on how to build WebAssembly.
* Implement castclass & isinst for wasm (#4510)
Implement castclass and isinst opcodes in ILToWebAssemblyImporter by
doing related function calls to methods in System.Runtime.TypeCast.
Fix#4510
Add castclass & isinst test in HelloWasm.
This includes three types of casting:
* castclass/isinst to classes.
* castclass/isinst to interfaces.
* castclass/isinst to array types.
For now, the second and third part of test is failing due to runtime
implementation problems which should be further digged into.
* Implements localloc and fixes other issues required to make Int32.ToString work (which relies on stack allocation, Spans and various value type special cases). Includes:
* Allocating localloc buffers on the C++ stack since they're guaranteed not to have GC references and LLVM might be able to optimize them a bit better
* Handling newobj for value types by allocating them in a spill slot
* Implementing the ByReference.get_Value intrinsic
* Fixing various shadow stack management bugs around calls. In particular, return values are now spilled to avoid the next call overwriting them.
* A few new tests
implemented array instructions, implemented internal calling convention, switched to managed runtime calls, Refactored call to not require stack operations, finished work to map pinvokes into the same generated methods/signatures
* Implement cpobj opcode for wasm
Implements the ILToWebAssembly.ImportCpObj method using an LLVM load and
store.
Adds a test for for wasm cpobj to the HelloWasm test using an IL
project.
* Ensure reflection has access to instance methods on byref-like types
Adding the test uncovered a subtle issue. When rooting a virtual method for reflection, we should root the virtual method use of the slot defining method (not e.g. the `ToString` method on the byref-like type - we don't have virtual method use information for that).
* Disable part that doesn't work on Unix
* Changes to hook up the portable runtime and bootstrapper. Includes implementing allocation using RhpNewFast as well as some floating point codegen fixes required to make the new code compile.
Removes usage of buggy dladdr API in WebAssembly, fixes order of conditional branch expressions, which fixes printing when C# optimizations are enabled, adds debug and release flags to emcc, updates WebAssembly documentation to reflect the new build flavor and linking steps. Adds thunks for RuntimeExport methods to fix linker errors for missing runtime exports.
Latest XCode errors on ordered comparison of pointer with integer
Also, fixed some unnecessary always-true comparisons that the C++ compiler emitted warnings for.
* Determinism test harness
Add a /determinism mode to runtest.cmd that will invoke ILC twice with different random determinism seeds. This causes graph expansion to be randomized based on seed (though deterministic for a given seed).
* Add Utf8String.CompareTo
Implementation taken from S.P.CoreLib. All the loop unrolling optimizations were excluded; we re-implement them if this function becomes a perf bottleneck.
* Define the mechanisms for determinism
To ensure deterministic output, all nodes that are emitted to the binary are sorted after compilation when retrieving the final marked nodes list. The overall approach is to sort nodes of the same type together (ie, all EETypeNodes come before NonGcStaticsNodes). Within nodes of the same type, a comparison function uses a CompilerComparer to compare the key identifying data of a node. For example, an EETypeNode is defined by a TypeDesc. The CompilerComparer provides a stable comparison function for comparing various type system primitives. Some nodes need to be emitted in an early and specific order for compiler correctness; this is provided by partitioning all sorted nodes into two phases: the first phase containined specifically ordered nodes, and a second phase for all other nodes whose ordering in the output binary doesn't matter.
Add SortableDependencyNode abstract class which provides the sortability layer on top of a DependencyNodeCore. Both ObjectNode and EmbeddedObjectNode derive from SortableDependencyNode to provide sortability across all nodes that are emitted to the output binary.
Introduce `ISortableSymbolNode` to provide sortability for nodes that are currently referred to in the compiler via the ISymbolNode interface. Such nodes are actually `ObjectNode` or `EmbeddedObjectNode` instances that we've lost type information for. Instances that implement `ISortableSymbolNode` redirect to the matching SortableDependencyNode methods.
Add `EmbeddedDataContainerNode` as a base class of `ArrayOfEmbeddedDataNode` to allow comparison of different instantiations.
Refactor the interface dispatch map index calculation so it's done when the dispatch map is emitted at the end of compilation. Previously it was done when the indirection cell in the dispatch map array got marked. This mechanism is incompatible with generating stable dispatch map indices. The arrays of embedded nodes now stabilize IDs as they emit their final data. This introduces an output ordering dependency - OptionalEETypeNodes must be emitted after the dispatch map since they encode dispatch map indices. Manually enforce this with C++ code generation, since it doesn't emit the real dispatch map structure and builds its own.
Modify InterfaceDispatchMapNode to use the type name in name mangling instead of an index into the dispatch map table.
Modify ObjectDumper to also amit a sha256 hash of each node's data. This dump is used to diff the map files and prove determinism.
* Fill out ordering functions for all nodes
Most of these are not very interesting. Here's the overall approach:
- Every different type of node needs a unique ClassCode. These were generated using Math.Random.
- The various metadata / native layout nodes plus arrays of EmbeddedObjectNodes get placed in the Ordered phase with specific ordering of each.
- All other nodes go in the Unordered phase
- To order nodes of the same type, the data that represents the key for the node in NodeFactory is compared. Ie, for an EEType, that would be its defining TypeDesc, whereas a FrozenStringNode is defined by the string it represents).
- The marked nodes list also contains raw DependencyNodeCore<T> nodes, which aren't emitted. Those are all shuffled after the emitted nodes and not sorted amongst themselves. That hasn't proven to be a problem with determinism and saves a bunch of hopefully unnecessary comparisons.
Added support for ldsfld/stsfld/ldsflda/ldflda including general value type support, throw on invalid branches to the first basicblock in a routine and ensure type sameness for icmp
* Fix instance method parameter management and a couple of codegen issues that showed up when more methods started compiling. Includes implementing ldfld for instance fields (progress toward #4530) and the leave opcode. This is enough to make the String.Length getter work.
* Implement neg (float/integer) and not IL instructions. Fixes#4524 and #4525
* Neg and not basic tests added to HelloWasm.
Name parameter updated in LLVM build methods to reflect the operation being built.
Use PushExpression method instead of direct push to stack.
This PR completes, adds tests for, and enables the CLR Thread Pool on Unix. It also adds Windows implementations for the low level primitives used in the CLR Thread Pool.
The thread pool is enabled by setting the MSBuild property FeaturePortableThreadPool to true, which is the default on Unix.
Supersedes #4039.
Rebased and squashed replacement for original PR #4124.
Changes that allow loading a printing a frozen string. Includes:
* Implement ldnull and ldind
* Fix brtrue and brfalse for pointers
* Fix the polarity of ceq and friends
* Change functions we can't compile into traps so that they don't break LLVM
* Put call arguments in the right order
* Miscellaneous stack canonicalization fixes (we'll have to revisit this in a more comprehensive way soon)
* Minor diagnosability improvements
This change adds support for Server GC.
To enable the Server GC for an application, the RH_UseServerGC environment variable (which is already supported by the runtime) should be set to 1.
On Alpine Linux, the default shell is ash. To build .NET Core, we
explicitly install bash.
If we run the following command:
```sh
echo 0x3F > /proc/self/coredump_filter
```
in bash, it raises the exception:
> echo: write error: Invalid argument
while with ask, it works fine.
Problem is that, when `echo` is run from `bash`, it adds a trailing
linefeed which kernel seems to reject.
Adding `-n` avoids adding the linefeed.
Original exception while building CoreRT:
```sh
chmod +x /corert/Tools/dotnetcli//dotnet
/corert/bin/Linux.x64.Debug/ILCompiler.DependencyAnalysisFramework.Tests/RunTests.sh: line 67: echo: write error: Invalid argument
/corert/Tools/dotnetcli//dotnet xunit.console.netcore.exe ILCompiler.DependencyAnalysisFramework.Tests.dll -xml testResults.xml -notrait Benchmark=true -notrait category=nontests -notrait category=nonlinuxtests -notrait category=OuterLoop -notrait category=failing
```
For static array fields, the real element type which can be a descendent of the declaring type must be added to the preinitilizeded data. The element count should also be there.
[tfs-changeset: 1674701]
* Added a dedicated wait thread to the thread pool following similar design principles as CoreCLR
* Fixed project file
* Fix missing return.
* Use the RuntimeThread APIs instead of the Interop-level APIs to create the WaitThread.
* Moved callback execution and unregistering to the thread pool
* Encapsulated signaling the user provided wait handle to ensure it only happens once.
* Handle the blocking case by not queuing the unregistration of the wait on the threadpool when the handle is invalid and instead unregistering directly.
* Rename WaitThread to ClrThreadPool.WaitThread and remove Unix suffix since (in theory), we will be able to move this to work on Windows as well in the future.
* Cache WaitHandle array and timeout and only recreate when the list of RegisteredWaitHandles is changed
* Fixed ClrThreadPool class modifiers in ClrThreadPool.WaitThread.cs
* Fixed reference to WaitThread class
* Fixed reference to WaitThread class
* Re-architected WaitThread to use a partial section of the array of wait handles to wait on and queue all removals to happen on the wait thread itself.
* Fixed compile error in WaitAny
* Make ClrThreadPool.WaitThread non-static and add scaffolding code to ensure handles can be unregistered by the user from the right WaitThread
* Fixed possible indxing errors, prevented double remove requests, and renamed some methods to be more descriptive
* Add comments
* PR feedback
* PR feedback
* Add a lock around selecting a wait thread to register the wait handle on
* ClrThreadPool is no longer static in master
* Only process removals before calling WaitAny, not after
* Try-Finally in ProcessRemovals
* Remove safeToDisposeHandleEvent as per PR feedback
* Added try-finally in UnregisterWait
* Refactor SetEvent into WaitHandle so as to not have direct dependencies on the Unix Wait Subsystem
* waitHandles.Length -> numWaitHandles in WaitAnyCore
* Added try finally in PerformCallback and added a way to ensure Unregister only returns true once
* Refactored timeout to actually account for previous waits when calculating timeout
* Account for handles that timeout while doing other work than waiting
* Changed removals aglorithm to fill nulls in while removing instead of doing so in a later loop
* Create 1st Wait Thread lazily
* Refactor RegisteredWaitHandle. Made wait threads time out. Start wait
threads outside of global lock.
* Fix wait thread accessibility
* Fix registered wait initialization to match CoreCLR
* Removed the lock from RegisteredWait in favor of a CompareExchange flag
* Replaced CompareExchanges with volatile loads and Exchanges
* Fix asserts in WaitSubsystem
* Fix bounds in safe wait handle release
* Added Wait Thread tests.
* Added more wait thread tests and fixed code style.
* Remove resolved TODO.
* Uninterruptible waits on wait handles where supported. Internalize CanUnregister. Return true on first unregister call even if automatically unregistered eariler
* Fix typos
* Add basic recursive event to handle overlapping callbacks
* Fix repeating timer timeout
* Remove dependency on Environment.TickCount from RegisteredWait
* Fixed test
* Fix race conditions on callbacks + unregister based on PR feedback
* Only timeout on change event when there are no user waits
* Fix overload call
* Remove code used for initial debugging
* Double times used in test to allow for about 50 ms window for test timing instead of 25 ms
* Move RegisteredWait into RegisteredWaitHandle
* Unregister on wait thread before callback
* Add test and code to ensure events are not observed on wait thread after an unregistration, even if it is non-blocking.
* Fixed incorrect timeout class
* Expand time frame for failing test to 100ms from 50ms. Add another assert to ensure it's working correctly
* Use event setting for test instead of timeout.
* Cleaned up leftover variables in test
* Only use sync context when wait is uninterruptible
* Make IsBlocking setter private.
* Fixed blocking unregisters.
* PR Renames
* Remove unneeded _unregisterSignaled member.
* Add debug check of _numRequestedCallbacks.
* Add argument verification for top of range.
* Clean up tests per PR feedback.
* Add CheckedWait helper.
* Added dispose-after-unregister test.
* Added block comment stating the reasoning behind not having a blocking wait on removal in unregister.
* Added finalizer to RegisteredWaitHandle.
* Changed overlapping callback test to use events instead of sleeps in the job.
* Refactor timeout times in tests and add comment to ClrThreadPool constant.
* Move updating the user's supplied unregister event to be in the callback lock.
* Added lots of comments and fixed an accessibility modifier
* Clean up tests and argument validation. Lazily create event for blocking unregistration.
* Only grab sync context if wait is interruptible.
* Fixed comment.
* Unregister event will be signaled even if handle has already been unregistered
* Fix timeout checks.
* Added test and code to make it pass for blocking unregistering a handle after it has been removed from the list but is still executing a callback.
* Add an event to wait for the handle to be removed from the list. User code will not get control back until it is safe to dispose the handle.
* Refactor internal event handles
* Changed sleep times for test
* Implement cache for internally used AutoResetEvent
* Exceptions in unregister rollback RegisteredWaitHandle state.
* Change defaults of UserUnregisterWaitHandle and UserUnregisterWaitHandleValue to null and IntPtr.Zero respectively
Once we pick up test assets after dotnet/coreclr#12537 this test will be
passing - moving the test to the category of tests that are waiting for
test update.
- Pick up latest .NET Core 2.0 CLI and buildtools
- Remove all project.json references and convert everything to msbuild projects
- Stick to vanilla .NET CLI project shape as much as possible. Minimize dependencies on buildtools special behaviors
Without this fix, the CoreRT compiler generated the following code for the
test that I am adding:
call Interfaces!_NewHelper_Interfaces_BringUpTest_SomeClass
mov rcx,rax
call
Interfaces!Interfaces_BringUpTest_SomeAbstractBaseClass__get_SomeValue
As you can see, it ends up calling an abstract method on the base class.
With this fix, the interface call is resolved correctly:
call Interfaces!_NewHelper_Interfaces_BringUpTest_SomeClass
mov rcx,rax
call Interfaces!Interfaces_BringUpTest_SomeClass__get_SomeValue
Tanner Gooding
yowl
Jan Kotas
Tomáš Rylek
Michal Strehovský
Filip Navara
Simon Nattress
Koundinya Veluri
Konstantin Baladurin
Andon Andonov
Michal Strehovský
Morgan Brown
Konstantin Baladurin
Simon Nattress
Amy
Blealtan
Anipik
Adeel Mujahid
Jeff Greene
Jeremy Koritzinsky
Toni Solarin-Sodara
Christian Scheuer
Wayde Reitsma
Juan Antonio Cano
Faizur Rahman
t-jekor
Sergiy Kuryata
dotnet-bot
Morgan Brown
Yi Zhang (CLR)