This change applies clang-format to all of the sources added in DXC that
were not part of the original LLVM/Clang 3.7 release when DXC forked.
This provides consistent code formatting across the codebase.
Fixes#5591.
In the case of an empty string, DxcCreateBlob does not take ownership of
the input data object, and ends up leaking it.
Example ASAN error:
```
./bin/dxc -E main -T lib_6_3 /home/amaiorano/src/external/DirectXShaderCompiler/tools/clang/test/DXC/show-includes.hlsl
==524344==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 1 byte(s) in 1 object(s) allocated from:
...
```
We don't keep these annotations up to date or rely on them, so to
simplify our platform layering and ease code formatting this change
removes all the SAL annotations from internal APIs. This change also
replaces most `_Analysis_Assme_(...)` annotations with `assert(...)`.
One `_Analysis_Assume_` in ParseDecl.cpp needed to be updated because it
was incorrect. The code was `_Analysis_Assume_(assert(name.size() >
sizeof("space"));`. When converted to an `assert` it fired in our test
suite because the sizeof a string literal includes the null terminator,
but the size of a StringRef does not.
A few of the `_Analysis_Assume_` annotations were removed because they
didn't make sense (like the ones inside the `DXASSERT` implementation,
and a few others were removed to avoid introducing additional header or
linkage dependencies.
This change does not introduce any functional or behavioral changes.
Add dxcutil::ComputeSerializeDxilFlags to share code with linker
Use refactored ComputeSerializeDxilFlags in DxcCompiler::Compile()
---------
Co-authored-by: Tex Riddell <texr@microsoft.com>
One more batch of NFC changes to get DXC ready to clang-format. This
change disables clang-format of some include blocks and breaks other
include blocks into groups. With this change all of the sources in DXC
can be clang-formatted and will continue to build on all supported
platforms.
Refactored -opt-enable and -opt-disable.
- Added `hlsl::option::OptimizationToggles` to manage the toggles
instead of checking a `std::map`.
- All the options are moved into `hlsl::options::TOGGLE_*` constants,
where each constant contains both the option's name and whether it's
default on or off.
+ Previously, every check had to be either `toggles.count("my-flag") &&
toggles.find("my-flag")->second` for default off, and
`!toggles.count("my-flag") || toggles.map.find("my-flag")->second` for
default on.
+ Now, a check is simply `toggles.IsEnabled(TOGGLE_MY_FLAG)`.
Allow new/delete operators to be called before DllMain and allocator
initialization in DxcInitThreadMalloc().
The operators can be called before DllMain from CRT libraries when
static linking is enabled. If that happens, the new/delete operators
will fallback to the standard allocator and use CoTaskMemAlloc/Free
directly instead of CoGetMalloc, Alloc/Free & Release for better perf.
Enables fixing of
[#5163](https://github.com/microsoft/DirectXShaderCompiler/issues/5163)
and
[#5178](https://github.com/microsoft/DirectXShaderCompiler/issues/5178).
Add macro DXC_DISABLE_ALLOCATOR_OVERRIDES which, if defined, will use
the default CRT allocator for new/delete and malloc/realloc/free.
We need this on Chrome for our GN build of DXC, because our component
builds use libc++as a shared library, and we end up with mismatched
new/delete calls between libc++ and these custom overrides.
---------
Co-authored-by: Chris Bieneman <cbieneman@microsoft.com>
A previous PR added this spriv option. The default should have been
false, but in the block that checks for spirv options on a non-spirv
build, the default was set to true. This broke testing on non-spirv
builds.
This is the first commit in a series that will enable array of
RW, consume, and append Structured buffers. This commit removes the
error when one is used on its own.
There will be a failure if a counter variable is needed.
This is the first PR is a series that will slowly enable different
features for arrays of RWStructuredBuffers. To see the general direction
the changes will go see
https://github.com/microsoft/DirectXShaderCompiler/pull/5407.
Add `-fspv-preserve-interface` CLI option to prevent DCE optimization
pass from compiling out interface variables. It happens if these
variables are unused.
The option may be useful when decompiling SPIR-V back to HLSL.
Personally, I need it to convert DX12 shaders to DX11 ones using
SPIRV-Cross as a tool for converting SPIR-V, produced by DXC, to the old
shader model HLSL.
SPIR-V Tools now have a parameter in `RegisterPerformancePasses()` and
`RegisterLegalizationPasses()` for this. This PR creates a new command
line option in DXC and passes it to the `spvtools::Optimizer`.
Closes#4567
The current behavior of the lifetime markers flags is to allow the
`-disable-lifetime-markers` flag to override the enable flag no matter
where it occurs. This means that if you write
dxc -disable-lifetime-markers ... -enable-lifetime-markers
the lifetime markers will be disabled. This works differently from most
clang flags which allow the last version of the enable/disable flag to win.
This PR modifies these flags so that when there are multiple occurences of
enable/disable the last one will win. It is not clear if the original behavior
was intentional or just a mis-understanding of the flag API. There were no
tests enforcing the original behavior.
* [SPIR-V] Add fspv-preserve-bindings option to preserve decoration bindings even when unused
* Set preserve bindings option in SpirvEmitter::spirvToolsLegalize() also
This patch got a bit away from me. It started as me trying to remove
one of the Windows system functions we have a custom implementation of
(StringCchCopyEx). It didn't really make sense to me that we needed a
custom override of a function where we could have just used `strncpy`
or other LLVM utilities to provide the functionality.
After I rewrote the code calling `StringCchCopyEx`, I wanted to make
sure it was tested, which led me to discover that it was never called
in our existing tests (see:
https://microsoft.github.io/DirectXShaderCompiler/coverage/home/runner/w
ork/DirectXShaderCompiler/DirectXShaderCompiler/lib/DxcSupport/WinFuncti
ons.cpp.html#L27).
I then looked into the behavior of
`clang_getCursorSpellingWithFormatting`, which it turns out _does_
return fully qualified names. As a result the parent traversal we were
attempting always terminated immediately with one name segment found.
I wrote an additional test where the cursor name is _not_ fully
qualified in source to verify that the API works as expected in all
cases then started deleting code.
This patch deletes two functions (`StringCchCopyEx` &
`ConcatPartsReversed`), and simplifies the code in
`GetCursorQualifiedName`. It also adds the additional
non-fully-qualified name test case.
This change reworks how we were filtering top-level implicit decls from
the `-ast-dump` output. The change has a couple of key differences, but
the behavior of the `-ast-dump` flag remains unchanged.
The differences in this change are:
* By default Decl::dump _will_ dump implicit top-level decls which
allows attached debuggers to dump top-level decls.
* Top-level implicit decls are determined by the decl's `Implicit` bit
rather than an invalid source location.
* Disabling dumping implicit decls is now a language option, so it can
be toggled.
* A new `-ast-dump-implicit` flag is added to dump the AST including
implicit top-level decls.
The attached test case verifies that the `-ast-dump` flag does not dump
implicit top-level decls, and it is intended to be strict enough that
if we add new top-level decls in the future and fail to mark them as
`implicit` the test will fail.
The attached test also trys to fill some gaps in testing around the
lazy decl initialization by verifying that incomplete implicit decls
are generated, and that a used decl gets completed as expected.
When present in the library search path, linux builds of
libdxcompiler.so and the dxc executable should load libdxil.so and use
it for validation and signing. However, the code to do that was still
searching for dxil.dll even on these platforms.
This change creates appropriate names for the dxcompiler and dxil
libraries for each platform and uses them where appropriate.
In addition, this changes some of the internal interfaces from wide
chars to simple chars as the wide interface wasn't useful here.
* hlsl, spirv: allow use of ffinit-math-only
This options should match GCC's & Clang's behavior:
From GCC documention:
```
Allow optimizations for floating-point arithmetic that assume that arguments and results are not NaNs or +-Infs.
This option is not turned on by any -O option since it can result in incorrect output for programs that depend on an exact implementation of IEEE or ISO rules/specifications for math functions. It may, however, yield faster code for programs that do not require the guarantees of these specifications.
The default is -fno-finite-math-only.
```
This commit allows the flag to be used again, and makes SPIR-V min/max
intrinsics use FMin/FMax instead of NMin/NMax when enabled.
Fixes#4954
Signed-off-by: Nathan Gauër <brioche@google.com>
Co-authored-by: Laura Hermanns <laura.hermanns@epicgames.com>
This flag is _extremely_ experimental. It is the first step in an effort
to clean up and adjust our optimization pipeline ordering. This first
step enables compiling shaders using a more traditional LLVM pass
ordering and the normal inlining behavior. For library shaders it also
skips adding the `alwaysinline` attribute to all function calls, so we
end up only inlining profitable calls based on the inliner heuristics.
There is much more work to do to get this flag to generate valid
shaders, but adding the flag so that we can start working on it is the
first step.
* Backport -ftime-trace from upstream llvm & clang
This change back-ports the -ftime-trace feature implemented by Aras
Pranckevičius (@aras-p) into DXC.
Aras has a great blog post describing the motivaiton for this feature
and how to use it (see:
https://aras-p.info/blog/2019/01/16/time-trace-timeline-flame-chart-prof
iler-for-Clang/).
As shaders are getting more and more complex we need better timing data
for ourselves as compiler writers and our users as HLSL writers to know
how to improve compile times of complex shaders.
This implementation is a starting point to give us data to help drive
our efforts to improve compile-time performance.
Because DXC's LLVM is very far removed from the version of LLVM the
patch was originally written for this change is not strictly a
cherry-pick of
https://github.com/llvm/llvm-project/commit/d880de2d19d46f7cfea7aa593602
458440bc1e8d. Instead this change is a manually applied and
reconstructed patch with significant alterations along the way. The
alaterations are performed by myself, but credit for the patch remains
with @aras-p.
Patch by Aras Pranckevičius (@aras-p).
* Address some low hanging fruit UBSan failures
Undefined Behavior Sanitizer causes hard failures on runtime conditions that exhibit undefined behavior as defined by C and C++. These issues may not cause immediately identifiable software errors, but the code behavior is dependent on the compiler and runtime implementation. As a result they may become difficult to diagnose or detect errors in the future if the compiler or runtime changes.
This change addresses occurrences of the following undefined behaviors.
* Performing arithmetic operations on null pointers.
* Calling memcpy with a source or destination pointer that is null.
* Calling memset with a destination pointer that is null.
* Assigning an enum a value which is not a valid enum for that enum.
* Reading from unaligned pointers.
After this change, there are 23 remaining undefined behavior failures in the current DXC tests which are less trivial to resolve.
This just surfaces and plumbs throug the -ftime-report clang flag in
dxc.exe.
-ftime-report prints timing data for different parts of the compile
to help identify and track down performance issues. When using the
dxcompiler library interface the output for the time report is returned
as a DXC_OUT_TIME_REPORT buffer.
I've set up the -ftime-report tests to run against the DXC command
line. On Windows this usees the hcttestcmds file, on Linux (and windows
if enabled) this will test through a lit shell suite.
* [NFC] Transform all test cases to use `-HV 2021`
This just updates test cases to use `-HV 2021` instead of the
per-feature options.
* Remove HLSL 2021 per-feature flags
This change removes the HLSL 2021 per-feature enabling flags. Now that
HLSL 2021 is feature complete we're removing the ability to opt into
individual features. Any code relying on a single feature of HLSL 2021
should now adopt HLSL 2021 completely.
Co-authored-by: Greg Roth <grroth@microsoft.com>
Exceptions for error handling can have a significant performance impact. The exception-throw codepaths are usually cold, and rely on large in-memory tables that need to be paged in.
This change to instead propagate returned error codes simplifies the code and allows the compiler to more aggressively optimize around both success and failure conditions.
* Cleanup windows build to conform to C++ 17
MSVC's C++ language implementation deviates from the standard in a
number of ways, this patch cleans up several issues that become errors
if you build DXC using clang-cl, which has a more strict
interpreteation of C++.
This change should not have any functional impact on the final program.
Specific changes include:
* static_assert(false...) is removed, this is always ill-formed by the
C++ standard.
* Under template instantiations implicit `this` is not always allowed.
* Character array `<` comparision is... probably not what was intended.
* Implicit conversions are stricter than MSVC enforces.
* Template base types need explicit type specialization when referred
to.
* It is illegal to initialize more than one member of a union.
* Scope nested enums cannot be forward declared.
* Header paths should be case-sensitive.
* Exception specifications of specific types are disallowed in C++ 17.
* Clang is stricter about const-correctness, especially as it relates
to constant c strings.
* Update based on review feedback.
* Converted string construction to copy-constructor calls.
* Removed constexpr method bodys to cause linker error.
* Catching other methods I missed.
* I was a bit too over-zealous deleting methods.
* WinAdapter: Remove virtual dtors from IUnknown to fix vtable ABI
The vtable for `IUnknown` and its subclasses contain two deletion
pointers when compiled on non-Windows systems with `IUnknown` from
`WinAdapter.h`:
vtable for 'DxcLibrary' @ 0x7ffff7cbc5f8 (subobject @ 0x5555556bb9e0):
[0]: 0x7ffff6a56d40 <DxcLibrary::QueryInterface(_GUID const&, void**)>
[1]: 0x7ffff6a56d20 <DxcLibrary::AddRef()>
[2]: 0x7ffff6a56d30 <DxcLibrary::Release()>
[3]: 0x7ffff6b36bc0 <IUnknown::~IUnknown()> // Complete object destructor
[4]: 0x7ffff6a57130 <DxcLibrary::~DxcLibrary()> // Deleting destructor
[5]: 0x7ffff6a56d50 <DxcLibrary::SetMalloc(IMalloc*)>
[6]: 0x7ffff6a56d60 <DxcLibrary::CreateBlobFromBlob(IDxcBlob*, unsigned int, unsigned int, IDxcBlob**)>
... More DxcLibrary virtual functions
This shifts the the pointers for functions for all subclasses, and is
[annoying] to deal with in otherwise cross-platform applications using
DirectXShaderCompiler as library. `dxcompiler.dll` compiled on/for
Windows without `WinAdapter.h` does not suffer this problem, and only
has three function pointers for `IUnknown`.
Fortunately, it is easily solved by removing the virtual destructor from
`IUnknown`. LLVM enables `-Wnon-virtual-dtor` that warns against
classes with virtual methods but no virtual destructor, though this
warning is best not enabled akin to Windows builds where `IUnknown` from
`windows.h` (`unknwn.h`) results in the same warning on MSVC ([1]/[2]).
[annoying]: 1e624792fc/src/unknown.rs
[1]: https://github.com/microsoft/DirectXShaderCompiler/issues/3783#issuecomment-844189358
[2]: https://godbolt.org/z/hKPT6ThEf
* WinAdapter: Make `IUnknown` and `IMalloc` pure-virtual classes
`IUnknown` in Windows' `unknwn.h` and `IMalloc` in `ObjIdl.h` are marked
as pure virtual, and are best marked as such in `WinAdapter` for
non-Windows platforms too [1]. Only the shim for `IMalloc` was relying
on the default refcounting implementation, all other subclasses either
contain pure-virtual methods themselves or provide an implementation for
`AddRef`/`Release` as required. Likewise the default implementation for
`IMalloc` was only instantiated once by `CoGetMalloc`, and has been
moved into a local class implementing the `IMalloc` interface instead.
[1]: https://github.com/microsoft/DirectXShaderCompiler/pull/3793#issuecomment-846459741
* WinAdapter: Add three missing virtual functions to `IMalloc` interface
To prevent unexpected vtable breakage, add the missing functions from
the [documentation]. Note that they are listed in the wrong order, the
right order is retrieved from the `ObjIdl.h` header and implementations
for `IMalloc` in DirectXShaderCompiler. All implementations are now
properly using the `override` keyword too, to enforce virtual method
existence in the base class.
[documentation]: https://docs.microsoft.com/en-us/windows/win32/api/objidl/nn-objidl-imalloc
* Make all WinAdapter destructions explicit
This prevents warnings about non-virtual destructor usage that trip up
the Linux build. It represents status quo on Windows.
Co-authored-by: Greg Roth <grroth@microsoft.com>
This is (admittedly) a little hacky. DXC spends a lot of compile time
updating debug information because we always generate it so that we can
generate diagnostics for late-running validation.
This patch adds a new flag -fdisable-loc-tracking, which disables
generating debug locations if the user is not generating debug info.
This flag results in about a 15% compile time improvement on a pessimistic
test case. YMMV.
Assume that matrices are stored in the column major order in raw
buffers, e.g., `ByteAddressBuffer` and `RWByteAddressBuffer`.
Add a new flag,`-fspv-use-legacy-buffer-matrix-order`, so that shaders
that depend on the previous matrix order (row major) can opt-out of this
change.
Fixes: https://github.com/microsoft/DirectXShaderCompiler/issues/3370
Though we may support -Gis with the SPIR-V backend in future, it is
currently not supported, so this change makes that explicit to the user
with an error message when it is used.
Related to #3331
Some DXC options are not supported in combination with the SPIR-V
backend. Rather than silently ignore them, we should emit a clear error
message. This change adds appropriate error messages for a couple
options that have been mentioned in issues #3111 and #4496, but more
should be added in future.
This debug/development option matches the `--print-all` flag in
spirv-opt or `--print-after-all`/`--print-before-all` in llvm's opt.
The new option can be enabled with `-fspv-print-all`.
In many cases, systems where wchar was represented as 32-bits were being
referred to by types and variables affixed with "utf16" or similar. To
avoid confusion, this renames all such places to "wide".
In a few cases, the conversion were explicitly 16-bit, which wasn't the
intent.
* Refactor HLSLVersion to enum
This change converts HLSLVersion to an enum type `hlsl::LangStd`. Most
of the change is a mechanical appending `hlsl::LangStd::v` to the
integers we previously compared for version checks.
Additionally this change does the following:
* Adds a parseHLSLVersion function to unify parsing code
* Parsing code is converted to an llvm::StringSwitch over the possible
values instead of converting to integer
* Added enum value for `vLatest` so that changing the latest version is
done in one place
* Added enum value for `v202x` which parses from 202x to allow new
language features a place to land
* Updates based on PR feedback
* Adding /setprivate, /setrootsignature, and /Qstrip_priv to compile API calls
* Adding setprivate to initial container serialization, change warning to error
* Removed unused variable
* Adding private data blob last, because no gaurantee of aligned size
* Add error messages for conflicting private options
These were disabled at a point that the DXC CI didn't reject builds for
failing on Linux, now that they do, we can enable these tests sure in
the knowledge that any changes to them that break on Linux will be
detected and fixed quickly
Fixed *nix error code return. The previous caused an infinite loop of retries.
This definition more accurately identifies the error that has occurred as not
benefiting from repeated attempts
Skip tests and filechecker elements that entirely depend on reflection or
linker interfaces that aren't implemented on *nix yet.
Correct capitalization in a few places to satisfy a case-sensitive filesystem
Detect directories from test include handler. *nix virtual filesystem will try to
load the file using the include handler when performing stat() on a directory.
The test include handler is pretty basic and would provide the next file, leaving
it without a file to offer when the real include came along.
By detecting a name that looks like a dir and returning failure, it
correctly indicates that the file isn't found by the handler and saves
the virtual file for when its needed.
Don't mark non-debug builds as such. The DEBUG define determines which
tests get run. Some of which depend on consistencies that can only be relied
upon in debug builds. So DEBUG is disabled for all non-debug builds.
Correct free of new() memory. Incidental regex allocations were changed for
HLSL to use the global new() operator, but were still freed using the standard
free(), but replacing the default free() with regex_free() allows the default new()
operator to be paired with the default delete() operator
Correct misnamed test flag. The *nix flag was mistakenly called InputFile instead
of InputPath. By renaming it, the ManualFile test can target a specific HLSL file.
Fix misused ArrayRef in legacy GV replacement. The replacement of the GV with
the legacy form used in libraries collected the arguments into an arrayref without
an array to ref, which caused garbage in release builds. Also moved to using end
vars for loops where the elements can get deleted during the loop.
Fix *nix conversion to utf8. When an unknown codepage is found, it tries to convert
to utf16, but that isn't' actually helpful for *nix for a shader codepage derived from
the BOM, which is correctly identified as utf32
This commit adds `-M, -MF, -MD` options to print dependencies of
compilation target.
For example, when a HLSL file X includes Y, Z, W files,
$ ./bin/dxc -M -T ps_6_0 -E main X
will print:
X
Y
Z
W
The first line i.e., X is the name of the main compilation target.
Following lines are dependencies for X.
`-MF<file_path>` and `-MD` are the same as the ones supported by Clang.
See https://clang.llvm.org/docs/ClangCommandLineReference.html#dependency-file-generation
Extended current encoding option to also set a default encoding for source files. Includes support for UTF-8 and UTF-16 encoded sources with and without BOMs. The default encoding will be used for the main source file when there is no BOM and the DxcBuffer encoding is not specified. The default encoding will be used for included source files with no BOM.
Chris B
Antonio Maiorano
Xiang Li
Justin Bogner
Helena Kotas
Adam Yang
Natalie Chouinard
Tex Riddell
Steven Perron
Kirill Kozlov
David Peixotto
Rajiv Gupta
Greg Roth
Nathan Gauër
harshpg
Greg Fischer
Marijn Suijten
Jakub Kuderski
JiaoluAMD
Grace Jennings
Adam Yang
Jaebaek Seo