This change fixes an inconsistency between scalar and vector bools, where the formers were stored as i32s in memory (as opposed to i1s in registers) while the latter remained <n x i1>s. This lead to a number of crashes in more complex bool scenarios where scalar and vector element types were expected to match.
The solution is to extend Clang's strategy for scalar bools to vector of bools. Clang relies on three key functions to deal with differences between register and memory type representations which are special-cased for bools and now also for bool vectors. Several HLSL code additions did not properly leverage those functions and this was addressed wherever possible, in some cases removing the need for special cases for bools.
To deal with matrices, a similar concept of register/memory representation was introduced in the matrix lowering code and the lowering passes were updated accordingly.
It was done to improve the compile time, but it does not seem to be
better. The use of `HEAP_NO_SERIALIZE` turns off the
low-fragmenataion-heap, which is what we want for this application.
See https://github.com/Microsoft/DirectXShaderCompiler/pull/1712 for a
discussion.
The code attempted to cast a QualType to an AttributedType, which would fail if it was in fact a TypedefType. Better use the existing helper function we have.
Added front-end `-flegacy-resource-reservation` to control behavior (added automatically for SM <= 5.0)
Added an error for unbounded resources used with `-flegacy-resource-reservation`
Added a mechanism for `DxilModule` to preserve intermediate options during its passes but not in the final DXIL
Changed `HLModule` to not remove unreferenced `DxilResources` but rather make their symbol UndefValue
Fixed assumptions of `DxilResources` having a valid `GlobalVariable`
Added reserved resource range gather pass before elimination of unused resources (if SM <= 5.0)
Changed resource allocation to account for reserved ranges
DxilModule::GetTessellatorPartitioning didn't follow the model of other methods of returning Undefined when it does not apply to the shader stage and would assert.
Per LLVM rules, globals must be initialized unless they have external linkage. It makes sense to consider constant buffer values as having external linkage, since they are "linked in" by the application, so this change:
* Changes CBuffer globals to have external linkage and no initializer
* Fixes a few cases not handling the lack of initializers properly
* Fixes some tests which were testing for initialization
* Added a test for non-initialization of cbuffer-promoted globals
* An earlier bitcast instruction would create an unused GEP instruction referencing the matrix, which the matrix lowering code couldn't handle. Removed that GEP.
* A special case for single-index matrix access was probably meant to catch `m._11` and assumed a value type of float, but it also applied to the first subscript of `m[0][0]`, which returns `floatn`, hence a crash due to mismatched types. The condition was updated to test for non-vector values.
* Fix subobject raw bytes storage; don't point into RDAT for reflection
DxilSubobject:
- combine string and raw bytes storage in m_BytesStorage
- use StringRef as key, since memcmp is used with explicit length
DxilRuntimeReflection:
- add m_BytesMap for local unique copy of raw bytes in RDAT
to prevent pointing into RDAT blob for root sig
- clean up some insertion patterns
* Rename Get[SubobjectString/RawBytes] to Intern[String/RawBytes]
- re-serialize module when changed by removing either subobjects or
root signature metadata
- improve error detection/handling for loading/creating subobjects
- load subobjects from RDAT if not in DxilModule for RDAT comparison
- fix subobjects missing from desc in DxilRuntimeReflection
- default ReducibilityAnalysis to pass so it doesn't fail when no
functions are present
- report error on subobject if validator (< 1.4) will not accept
them, so they will not be captured
- container validation for required/disallowed parts for libraries
- Replace HitGroup subobject by TriangleHitGroup and ProcedurePrimitiveHitGroup
- Add internal HitGroupType enum that maps to D3D12_HIT_GROUP_TYPE
- Add corresponding HitGroupType field to metadata, RDAT and reflection
- DxilModule was not freed
- PragmaMatrixHandler was freed twice when the compiler execution was aborted
prematurely by an exception
- Improve diagnostic tracing in CompilerTest::CompileWhenNoMemThenOOM
* Add root signature flags
- Add LOCAL_ROOT_SIGNATURE flag in RootFlags
- Add DESCRIPTORS_STATIC_KEEPING_BUFFER_BOUNDS_CHECKS flag in DescriptorTable
- New DxilRootSignatureCompilationFlags argument on Compile/ParseRootSignature
to specify whether the root signature is local or global.
* Fix root signature flag issues, add cmd test for root sig from define
Tristan Labelle
Vishal Sharma
Ehsan
Steven Perron
Minmin Gong
Helena Kotas
Lei Zhang
Young Kim
Tex Riddell
Shoeb Patel
Xiang Li