Originally @lizhengxing's PR. Retargeting main.
This PR pulls 2 upstream changes, Add a new WeakVH value handle; NFC
(f1c0eafd5b)
and Use a 2 bit pointer in ValueHandleBase::PrevPair; NFC
(b297bff1cc),
into DXC.
Here's the summary of the changes:
Add a new WeakVH value handle; NFC
> WeakVH nulls itself out if the value it was tracking gets deleted, but
it does not track RAUW.
>
> Reviewers: dblaikie, davide
>
> Subscribers: mcrosier, llvm-commits
>
> Differential Revision: https://reviews.llvm.org/D32267
Use a 2 bit pointer in ValueHandleBase::PrevPair; NFC
> This was an omission in r301813. I had made the supporting changes to
make this happen, but I forgot to actually update the
>
> PrevPair declaration.
This is part 4 and 5 of the fix for #6659.
This PR pulls the upstream change, Rename WeakVH to WeakTrackingVH; NFC
(e6bca0eecb),
into DXC.
Here's the summary of the change:
> I plan to use WeakVH to mean "nulls itself out on deletion, but does
not track RAUW" in a subsequent commit.
>
> Reviewers: dblaikie, davide
>
> Reviewed By: davide
>
> Subscribers: arsenm, mehdi_amini, mcrosier, mzolotukhin, jfb,
llvm-commits, nhaehnle
>
> Differential Revision: https://reviews.llvm.org/D32266
This is part 3 of the fix for #6659.
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
This PR pulls the upstream change, Emulate TrackingVH using WeakVH
(8a6238201f),
into DXC.
Here's the summary of the change:
> This frees up one slot in the HandleBaseKind enum, which I will use
later to add a new kind of value handle. The size of the HandleBaseKind
enum is important because we store a HandleBaseKind in
> the low two bits of a (in the worst case) 4 byte aligned pointer.
>
> Reviewers: davide, chandlerc
>
> Subscribers: mcrosier, llvm-commits
>
> Differential Revision: https://reviews.llvm.org/D32634
This is part 2 of the fix for #6659.
Vulkan implementation can have different limits on the maximum value
used as an id in a SPIR-V binary. SPIRV-Tools generall assumes this
limit is 0x3FFFFF because all implementations must support at least that
value for an id. Since many implementations can support larger values,
the tools allows an option that will set a different limit. This commit
add an option to DXC to do the same.
Fixes#6636
This PR pulls the upstream change, Use accessors for ValueHandleBase::V;
NFC
(6f08789d30),
into DXC.
Here's the summary of the change:
> This changes code that touches ValueHandleBase::V to go through
getValPtr and (newly added) setValPtr. This functionality will be used
later, but also seemed like a generally good cleanup.
>
> I also renamed the field to Val, but that's just to make it obvious
that I fixed all the uses.
This is part 1 of the fix for #6659.
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Add a pass to run hlsl::RemoveUnstructuredLoopExits in isolation
Example: opt -dxil-r-u-l-e a.ll -S
Add some basic tests.
No functional change to the pass itself.
This PR (https://github.com/microsoft/DirectXShaderCompiler/pull/6598)
pulls the upstream global reassociation algorithm change in DXC and can
reduce redundant calculations obviously.
However, from the testing result of a large offline suite of shaders,
some shaders got worse compilation results and couldn't benefit from
this upstream change.
This PR adds a flag for the upstream global reassociation change. It
would be easier to roll back if a shader get worse compilation result
due to this upstream change.
This is part 2 of the fix for #6593.
The implementation of index_is_valid was incorrect returning false when
the input index was equal to Term->getNumSuccessors(). The end iterator
will have such an index, and it is valid to construct such an iterator.
For example, assume we have a block with two successors:
```
BasicBlock* bb = ...;
SuccIterator b(bb); // Index 0
SuccIterator e(bb, true); // Index 2, for example
SuccIterator v = b;
b += 2; // Without this fix, this asserts
assert(b == e);
```
Note that this was also fixed upstream in
https://reviews.llvm.org/D47467
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Natalie Chouinard <chouinard.nm@gmail.com>
This commit adds the compiler option (-Qsource_in_debug_module) to the
help text.
New help text output below:
```
Utility Options:
-dumpbin Load a binary file rather than compiling
-extractrootsignature Extract root signature from shader bytecode (must be used with /Fo <file>)
-getprivate <file> Save private data from shader blob
-link Link list of libraries provided in <inputs> argument separated by ';'
-P Preprocess to file
-Qembed_debug Embed PDB in shader container (must be used with /Zi)
-Qsource_in_debug_module
Embed source info in PDB
-Qstrip_debug Strip debug information from 4_0+ shader bytecode (must be used with /Fo <file>)
-Qstrip_priv Strip private data from shader bytecode (must be used with /Fo <file>)
-Qstrip_reflect Strip reflection data from shader bytecode (must be used with /Fo <file>)
-Qstrip_rootsignature Strip root signature data from shader bytecode (must be used with /Fo <file>)
-setprivate <file> Private data to add to compiled shader blob
-setrootsignature <file>
Attach root signature to shader bytecode
-verifyrootsignature <file>
Verify shader bytecode with root signature
```
fixes#6028
---------
Co-authored-by: Cooper Partin <coopp@ntdev.microsoft.com>
These flags had some effect on load code generation, but never worked
right and were not tested in the DXC compiler let alone with drivers.
This removes any notion of the flags beyond option processing and
produces a warning message that the flag is not supported and will be
ignored. The flags are both not listed in help anymore either. A simple
OptionsTest and lit test verify that the warning message is produced for either
flag and existing tests that used the flag are updated or removed.
Incidentally fixed the capitalization of an existing options warning message.
Fixes#6306
Context-specific diagnostics are warnings with DefaultError, since in
theory they could be dead code at this stage.
There were some changes to existing Sema diagnostic hooks along the way.
SemaHLSLDiagnoseTU.cpp - HLSLMethodCallDiagnoseVisitor
- extend to non-method calls and pass CallExpr
- instead of skipping when call reached from multiple entry/export
functions, pass locallyVisited flag for skipping of redundant diags not
dependent on entry properties.
- use llvm::SmallPtrSet for DiagnosticCalls
- capture launch type for entry and pass it down
Replace DiagnoseHLSLMethodCall hook with CheckHLSLFunctionCall:
Move hook into Sema::CheckFunctionCall in SemaChecking.cpp to be general
to either a function call or a method call.
Refactor existing code in DiagnoseHLSLMethodCall into
CheckFinishedCrossGroupSharingCall called in the hook for this method.
Add CheckBarrierCall for context-free flag validity check, called by
CheckHLSLFunctionCall.
Refactor CalculateLOD checking into more generic parts, dispatched based
on IntrinsicOp from top-level DiagnoseReachableHLSLCall function.
Switch to using new note: "entry function defined here", instead of
"declared here".
Fix intrinsic checking to use IsBuiltinTable on the intrinsic
attribute's group.
**Preparation Changes**
While these could be broken out, the barrier diagnostic changes are
dependent on them, and testing for some of these is dependent on the
other parts of the barrier diagnostic changes.
Correct errors in tests, check MaxRecords for EmptyNodeOutput in AST
Note that Barrier(..., 3) means SemanticFlags: GroupSync | GroupScope.
These are only valid in a shader with a visible group, and GroupSync on
memory with at least group scope, so they have been changed to 0 in
specific cases in these tests.
Make DiagnoseSVForLaunchType a static function, add FIXME notes.
Use StringRef for IsBuiltinTable, compare strings instead of pointers.
Fix FileCheckerTest for -verify output.
New ShaderModel::HasVisibleGroup uses ShaderKind and NodeLaunchType to
determine whether shader type has a visible group.
This commit fixes the setlocale( ) logic to include the UTF-8 supported
string value for Mariner distros.
It also introduces a new RAII class `ScopedLocale' which ensure that the
locale setting is set/reset during string conversion operations.
Fixes: #6201
---------
Co-authored-by: Cooper Partin <coopp@ntdev.microsoft.com>
Co-authored-by: cooppunix <cooppunix&mariner.com>
According to the documentation: "A hull shader is implemented with an
HLSL function, and has the following properties: [...] The shader output
is between 1 and 32 control points".
https://learn.microsoft.com/en-us/windows/win32/direct3d11/direct3d-11-advanced-stages-tessellation#hull-shader-stage
This change adds a check to verify that the outputcontrolpoints
attribute is set on Hull shaders and has an argument in the valid range.
Fixes#3733 (by making the error consistent between backends)
This commit adds a new optional -ftime-trace-granularity option that is
already implemented in llvm-project. This change is a surgical port of
an existing feature from the upstream llvm-project repo into the DXC
codebase.
The following commits in the llvm-project repo were copied and followed
for this change. Clean cherry picks were not possible due to the
differences in repos like change of file locations and other dependant
changes made in the repo.
*** Adds the granularity configuration setting ***
'Time profiler: small fixes and optimizations'
Commit: 26536728591d5fdac373ef535ae122b873f73292
*** Wires up the commandline option -ftime-trace-granularity to the
TraceProfiler code ***
'[Support] Fix -ftime-trace-granularity option'
Commit: 4fdcabf259c4ab94654e6cd5d95d0e0313159c70
Fixes#6372
---------
Signed-off-by: Cooper Partin <coopp@ntdev.microsoft.com>
Co-authored-by: Cooper Partin <coopp@ntdev.microsoft.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
This change adds NodeRecordType alignment field to RDAT to make it
possible to validate pointer and stride alignment in the runtime.
This includes a change to DXIL metadata to preserve the record alignment
without requiring recovery by looking for GetNodeRecordPtr.
Fixes#6270
This change exposes the clang 3.7 `-fdiagnostics-format=` flag in DXC with supported values `clang`, `msvc`, `mdvc-fallback`, and `vi`.
This option allows having DXC output diagnostic messages in string formats that are recognized natively by Visual Studio.
Resolves#6160 & #1811, both of which had been closed as not planned.
Fix crash if a RayGen shader contains a `while(true)` loop, caused by
dominator tree construction not ignoring `nullptr` successor AST nodes
which occur in this case.
Also add regression test case.
Fixes#5035.
Co-authored-by: Jannik Silvanus <jasilvanus@users.noreply.github.com>
Fix barrier allowed ops and flags by shader kind
New barrier operations lacked validation and for RDAT info: had
incorrect min target and shader stage flags.
- Identify barrier DXIL operations with new `is_barrier` in `hctdb.py`
and generated `OP::IsDxilOpBarrier`.
- Identify when a barrier op requires shader stage with group
(compute-like stage), or when it requires node memory.
- Add new `OptFeatureInfo_RequiresGroup` to identify function only
compatible with a shader stage with a visible group for access to
groupshared memory or use of group sync.
- Translate to original `BarrierMode` when compatible; adds
`BarrierMode::Invalid` to identify invalid cases.
- Account for `DXIL::MemoryTypeFlags::AllMemory` being allowed and
auto-masked by driver.
- Properly set min shader model and compatible shader stage flags.
- Validate barrier for shader stage.
- Added new barriers to counters which were missing.
Adressing parts of: #6256 and #6292Fixes#6266
There was a typo in dxcpdbutils where the normalized file path wasn't
actually being used. This change fixes that and adds thorough testing
for the DxcPdbUtils path normalization. Also fixed another case where
`/dir/my_file` gets normalized to `.\\dir\my_file`.
DXIL operations should use is_const=True for constant arguments. This
allows for convenience methods to retrieve the constant value, and could
(should, but currently doesn't) result in validation that the argument
is constant.
`BarrierByNodeRecordHandle` SemanticFlags argument must be constant.
`MetadataIdx` for both `createNodeOutputHandle` and
`CreateNodeInputRecordHandle` must be constant.
This change switches from instrumentation per instruction to
instrumentation per basic block. Furthermore, not every instruction in a
basic block needs to write debug data to the output UAV- it's enough to
know that the basic block was entered, as long as the calling
application can figure out what instructions were in that block. To
support that knowledge, the pass now emits a text "precis" of each basic
block.
Also, the previous branchless UAV bounds enforcement was replaced with
something similar that emits fewer instructions at the cost of a larger
UAV. This tradeoff is WELL worth it.
Additionally, the debug pass used to add extra blocks in order to
solidify the arguments to phi instructions. This work was unnecessary,
and added a lot of complexity to the resulting instrumented shader. The
debugger application is only interested in the value of the phi itself
and the actual value produced via the actual preceding edge.
Full details are in the comments in the code.
This change reduces driver-side compilation overhead from "overnight" to
2 minutes on a 160k-instruction shader.
This change adds a ground truth path normalization helper:
- All slashes are changed to system native (`\` for windows, `/` for all
others)
- All repeated slashes are removed (except for leading slashes, so
windows UNC paths are not broken)
- All relative paths (including the main file, and ones that begin with
`..`) are prepended with `./` or `.\` if not already
The path normalization is applied in the following places:
- Main file path in dxcompilerobj
- Paths passed into IDxcIncludeHandler
- Paths written into all DI* debug info
- Paths written into dx.content metadata and Pdb SourceInfo
- All paths loaded up by DxcPdbUtils
The reason for this change is to make it easier to tools authors to
implement recompilation (example: PIX edit-and-continue). When the paths
in all the above places match, the files can be matched with a normal
string equal instead of having to path normalization.
Metadata loading for node record properties assumed fixed positions for
values in a (key,value) property list, skipping and ignoring the keys.
This was not correct, fragile, and not the intended implementation of
the key,value property pattern.
There is no functional change, but this makes the loader robust to
additional properties instead of crashing or loading the wrong thing.
Fixes#6276
Upon further discussion, the team has agreed that in certain degenerate
cases, the current diagnostics are insufficient.
In the case that min == max in the wave size range attribute, a
defaultError warning should be emitted. Additionally, there should be an
explicit way to handle the case where 0,0,0 is passed to the wavesize
range attribute.
This PR directly handles and tests both of these cases.
Fixes#6161
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Add ShaderKind::Last_1_8 for shader mask
Add shader model comments before flag groupings in DxilConstants.h and DxilShaderFlags.
Add missing flag checks for min shader model in RDAT function info. Move ShaderCompatInfo computation into DxilModule, propagate callee info.
Move computation of shader model requirements based on flags into DxilModule. Finalize requirements for entry functions in AdjustMinimumShaderModelAndFlags.
Fixes for function level flag tracking:
- DerivativesInMeshAndAmpShaders: use flag to track deriv use, then adjust for entry functions.
- hasUAVs: based on resource use in function instead of global resources.
- WriteableMSAATextures: based on use in function instead of global resources.
- Also catch cases for dynamic res from any use by looking at create/annotate handle, not just the TextureStoreSample op.
- RaytracingTier1_1: move module-level detection to CollectShaderFlagsForModule
- Marked deriv and quad ops as being supported in node.
- Fixed SampleCmpBias to be considered gradient op.
- Update RDAT definitions to dump more useful info for testing
Fixes#6218.
This option is not enabled by default. When enabled, this option adds
the MaximallyReconvergesKHR execution mode to the module, and the
associated extension.
Related to #6222
---------
Signed-off-by: Nathan Gauër <brioche@google.com>
These TODOs were intended for if we were to implement deep RDAT
validation rather than relying on the exact binary comparison.
This isn't happening for SM 6.8, so these are no longer
PRERELEASE-TODOs.
The comment in RDAT_Macros.inl didn't really belong there in the first
place, so I removed it.
Fixes#5699.
Two related sets of changes:
1) Modify the value-to-declare pass to tolerate bitfields, which are
detected by their habit of being smaller than the type in which they are
contained. The pass produces dbg.declare statements that map each HLSL
variable to some storage. With bitfields, that mapping becomes
many-to-one, because many bitfields can be stored in the same basic
type. Most of the changes are to preserve the size of the bitfield as
the code descends the type hierarchy into the underlying basic type,
which is what motivates these new "type overrride" parameters to some
functions. Complications to watch out for are min16 types, which may or
may not be their own purpose-made (16-bit) types depending on whether or
not they are enabled.
2) Modify the PIX debug-info API to return enough data to PIX to allow
it to extract (by bit shifting) a bitfield from the underlying type.
This bit is also about preserving knowledge of the field's size as
opposed to its containing type's size, hence the new dxcapi
interface/method.
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
This PR improves the experience for debugging tools that consume SPIR-V
produced by passing in-memory strings to dxcompiler.dll. When dxc.exe is
used, OpSource and OpDebugSource are populated by reading original
source from the filesystem. dxcompiler.dll users instead find their
input string is preprocessed, which is confusing behavior.
Contrary to the leading comment in the code this change modifies,
debugging tools do not want preprocessed source. Such tools show and
allow editing of include files. If everything is pasted into a single
source string, finding what to edit is difficult. Editing the
preprocessed source has the additional complication that a user would
have to edit also the #line markers, which is hard to get right
manually. In the preprocessed source, all comments are lost, which makes
it hard to read the code. Finally in preprocessed source, all macros and
code under conditional compilation are lost. That will make it
impossible to modify those parts in an edit.
Co-authored-by: Steve Urquhart <stevur01@ntd.nintendo.com>
When asserts are disabled (LLVM_ENABLE_ASSERTIONS=0), DXASSERT* macros
are defined to nothing, resulting in no statements followed by a
semicolon. This results in:
warning C4390: ';': empty controlled statement found; is this the
intent?
Which fails because warnings are treated as errors.
Fix by adding the standard "do {} while(0)". Also opportunistically
remove the extra semicolon for non-WIN32 builds.
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Added DiagnoseUsedHLSLMethodCall which only report error when the
MethodCall is used.
DiagnoseUsedHLSLMethodCall is used in DiagnoseTranslationUnit.
In DiagnoseTranslationUnit, traverse all visited functions in call
graph, diagnose illegal intrinsic call inside these functions.
Shader kind is also checked in DiagnoseUsedHLSLMethodCall.
Fixes https://github.com/microsoft/DirectXShaderCompiler/issues/5855
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
PSV0 MaximumExpectedWaveLaneCount was incorrectly set to 0 for
non-range.
Create struct for WaveSize in DxilFunctionProps.h.
Centralize encoding and validation logic there.
Use validation logic in both SemaHLSL and DxilValidation.
Remove test requiring newer shader model in CodeGenHLSL.
Add comprehensive test for compute and node, cs and lib targets, SM 6.6
and 6.8, testing ast, metadata and RDAT.
Add PSV0 tests to catch incorrect runtime data.
Update validation rules and test for more cases.
Wave Size should be able to take a range of possible wavesizes. This PR
aims to implement what is described in this spec:
https://github.com/microsoft/hlsl-specs/pull/149/files
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
New feature flag ShaderFeatureInfo_SampleCmpGradientOrBias = 0x80000000
and
ShaderFeatureInfo_ExtendedCommandInfo = 0x100000000 were added.
They will be set when SampleCmpBias/Grad startInstance/VertexLocation
exists.
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
In DxcPixLiveVariables.cpp,
UniqueScopeForInlinedFunctions::AscendScopeHierarchy tries to keep track
of a variable's scope and the scope at which its containing function was
inlined, if any.
This scope duo has to be the same in two places for things to work out:
the scopes inferred from dbg.declare, and then again when the scope
hierarchy is ascended in order to discover all variables accessible from
a given point in the program.
AscendScopeHierarchy was forgetting to update the inlined part of the
scope, resulting in, for example, PIX's debugger losing track of
variables outside of, for example, a for loop, even though the code can
see those variables at that point.
Just setting the inlined scope to be the same as the function scope
fixes this problem.
(In addition to these new tests, existing PIX tests exercise this
situation happily)
The IsRay() function defined in the ShaderModel class can be misleading.
It is included in a set of functions that are used to test whether or
not the shader in question was targeted to a specific shader stage.
However, it is impossible for a shader to be targeted to, say, the
"raygeneration" stage, or any other stages associated with the kinds
that the IsRay() function checks for. Like the "node" shader kind, a
shader can only have these shader kinds if the associated attribute is
found on the function declaration. To prevent confusion, and to keep the
Is*() functions inside the ShaderModel class restricted to only those
targetable shader stages, the IsRay() function can be removed. This PR
comes as a response to #6008
The ExtractRecordStructFromArray intrinsic is left over dead code that
should be removed. This PR removes the intrinsic from existence. In the
cases where this intrinsic was used, DefaultSubscript is used instead.
Moved from #5893 to target the main branch instead.
Fixes https://github.com/microsoft/DirectXShaderCompiler/issues/5351
- In `DxilPipelineStateValidation.h`, add `PSVRuntimeInfo3` derived from
`PSVRuntimeInfo2`, containing one `uint32_t EntryFunctionName` member.
- Update `MAX_PSV_VERSION` and `RuntimeInfoSize()` for new version.
- Add pointer to `DxilPipelineStateValidation` and assign in
`ReadOrWrite` along with other versioned pointers.
- Add `DxilPipelineStateValidation::GetEntryFunctionName()` accessor to
look up name in string table.
- Update PSV0 versioning to key off validator version 1.8 in
`DxilContainerAssembler.cpp`
- Add entry name to string table and set `EntryFunctionName` to offset.
- Add initialization to DxilPSVWriter members
- Add asserts in DxilPSVWriter::write() to ensure buffer sizes not
changed after m_PSVInitInfo is set.
- Add test to DxilContainerTest, adding/updating common functions as
needed
Note: Testing the HS passthrough case revealed problems blocking that
scenario.
Filed issue #6001 to track this.
Fixes#5944
Turn off clang formatting for `RDAT_LibraryTypes.inl` and
`RDAT_SubobjectTypes.inl`. Macro indentation makes these files easier to
read.
This PR was created in 2 steps:
1. `RDAT_LibraryTypes.inl` and `RDAT_SubobjectTypes.inl` were reverted
to the version `64348c7e~1` The commit
64348c7eec is the first that started
preparing the code base for clang formatting. Except for that there were
no functional commits done to these files since then.
2. `//clang-format off` and a comment with explanation was added to the
top of these files
- Reorganize part and table IDs to move shader info after ver 1.8
- Move NodeShaderInfo into RuntimeDataFunctionInfo2 shader info union,
since it is mutually exclusive with other shader types
- Remove RuntimeDataFunctionInfo3
- Re-organize RDAT_LibraryTypes into 3 sections by versioning
- Format unformatted section of RDAT_LibraryTypes due to typo
- Add indentation to make definitions easier to read and disable
clang-format for RDAT_LibraryTypes.inl and RDAT_SubobjectTypes.inl
- Include Node in ShaderStageFlags for 1.8+
- Added some explicit scopes due to VS IDE thinking they were ambiguous
- RDATDumper: Don't try to visit or print null records
- Fix reflection tests:
- tests meant to check shader info use: -Vd -validator-version 0.0
- use regex for record strides for versioning robustness
- update ShaderStageFlags to include Node
- use regex for RuntimeDataFunctionInfo name for versioning robustness
- remove shader info CHECKs for cases not meant to test that
Fixes#5900
Adam Yang
Zhengxing li
Steven Perron
David Neto
Antonio Maiorano
Cooper Partin
Greg Roth
Tex Riddell
Natalie Chouinard
Jonas Meyer
Jannik Silvanus
Jeff Noyle
Joshua Batista
Nathan Gauër
Steve Urquhart
Helena Kotas
Xiang Li