Clang-format determines if a header is the "main" header by looking at
the header name without the last extension and matching it against the
source file (with some extra handling for cuda files).
The automation generating temporary files with a `.tmp` suffix tripped
this up so it wasn't matching the "main" header when sorting includes.
This corrects the issue by writing temporaries to a tmp directory
instead of munging the suffix.
This integrates formatting generated sources with clang-format with the
hctgen generation process.
This change makes a few small adjustments to how the build-time header
generation works. This change, disables automated build-time generation
for C++ sources if you don't have clang-format available on your system.
CMake can automatically detect clang-format installed as part of Visual
Studio, or based on your PATH. You can also explicitly set
`CLANG_FORMAT_EXE` when configuring to manually override. **Developers
on macOS** will need to install clang-format manually and place it on
their path to generate DXC's generated sources.
If clang-format cannot be found and `HLSL_COPY_GENERATED_SOURCES` is On
a fatal error will be reported at configuration time.
This change _does not_ make clang-format a requirement to build DXC, but
it does make it a requirement if you are modifying the generated
sources.
Add -Wno-unused-parameter -Wno-unknown-pragams -Wno-switch which are
enabled for clang in linux build.
Fix
-Wimplicit-fallthrough
-Wconstant-logical-operand (Thanks patch from Chris)
-Wunused-function
-Wunused-variable
-Wtrigraphs
-Wnonportable-include-path
Still more warnings to be fixed.
The standard TAEF install paths use the x64 and x86 paths just like the
TAEF_ARCH. I'm not entirely sure where the alternative TAEF_BIN_ARCH
naming is used, but it seems to be used in the custom tooling for
fetching and building TAEF.
This only alters the parts of the code that I've been able to verify
locally.
* Enable lit by default.
* Update README for git user bin.
* Add DXC_DISABLE_LIT to replace DXC_ENABLE_LIT
* Set -DDXC_DISABLE_LIT=Off for appveyor.
* Keep original name for ClangSPIRVTests
* Set correct path for unit tests binary.
* Remove doc about DXC_DISABLE_LIT
* Remove extra space.
* Remove DxcOnUnix.rst.
This pulls in a couple upstream commits by myself and @bogner.
commit 3adaa8cf4a6df45430b47a62c649c7cb54bb1626
Author: Justin Bogner <mail@justinbogner.com>
Date: Wed Jun 8 22:36:37 2016 +0000
cmake: Simplify add_lit_testsuites
cmake 3.4 introduced LIST_DIRECTORIES to glob recurse, which can be
used to simplify this code greatly.
llvm-svn: 272217
commit 25ef9e34c9de7b1edc327189b78200459337f7e5
Author: Justin Bogner <mail@justinbogner.com>
Date: Mon Jul 25 18:07:14 2016 +0000
cmake: When adding lit testsuites, ignore Output directories
With in-tree builds we can get Output directories scattered among
our
tests. Recursing into those to find tests doesn't make sense.
Thanks to nlewycky for noticing this!
llvm-svn: 276667
commit a44ef999fb5e9243df9c4459df1b17b66aa3fd23
Author: Chris Bieneman <chris.bieneman@me.com>
Date: Mon Jan 3 11:52:44 2022 -0600
[NFC][CMake] Add FOLDER to utility targets
As Visual Studio's CMake support is getting better and better the
line
between IDE generator and non-IDE generators is blurring. Visual
Studio
2019 and later have a very useful UI that can handle all of the
various
targets we create, but if they are unsorted it is wildly unwieldy.
This change sorts the lit testsuite targets and per-component
install
targets into folders, which are not generated for IDE generators but
are generated by default under Visual Studio's CMake + Ninja
integration.
1. added dxc_batch as dependent for clang tests.
2. Not run exec test by default.
3. Strip directory which has dxil.dll from %PATH% for taef test which not need it.
4. Add select_filter and use @Name for TestName.
* Enable mix select filter by use shell=True when open subprocess.
5. Only run execution test when dxil.dll exist or experimental mode enabled.
Having this enabled makes dxc depends on ncurses5. This is required by
LLVM when supporting colored message output, but DXC doesn't supports it
for now.
Fixes#4892
A common source of bugs. Although many of these were harmless, some were
not. By eliminating them and enabling this warning as an error, we won't
add more in the future.
Fixed a real SPIRV bug that required a test update to expect the correct behavior.
SPIRV testing was expecting incorrect results that came from a fallthrough error
The `LLVM_REQUIRES_*` variables are internal variables in LLVM's
configuration used to determine if a build setting is required, they
don't actually control the build setting itself, so we shouldn't be
setting them.
`LLVM_ENABLE_*` variables control the build settings, and we do set
those correctly.
* Update Linux build and test workflows to use lit
This change updates the build and test workflows in our azure pipelines
and the GitHub action for code coverage data to use lit instead of ctest
as the test runner. This adds additional test coverage by running some
of the LLVM & Clang tests.
* Fix missing dependency between LLVMAnalysis & DXIL
The default Linux linker fails because LLVMAnalysis depends on LLVMDXIL
and BFD doesn't cycle libraries.
* Updating pipelines to remove mkdir and cd
This just cleans up the pipeline shell commands to avoid unnecissary
mkdir and cd invocations.
Based on feedback from @Keenuts. Thanks!
* Disable LLVM LIT tests that use garbage collection
DXC disabled support for the LLVM garbage collection intrinsics.
* Enable running tests under LIT
If `CLANG_INCLUDE_TESTS=On`, the googletest unit tests are built as
part of the clang unit test suite. This allows them to run via lit
using the `check-clang-unit` target.
* Configure DXILConv tests to run under check-all
* Remove DXC_BUILD_ARCH variable
This commit simplifies the cross-targeting build support to not require
specifying the build architecture explicitly. Instead it is derived from
the build tools used in the generation.
I've tested this change locally for Win32, x64 and arm64.
* Special handling for arm64ec
* Bump required CMake to 3.10
CMake 3.10 is very old, but has a feature this PR uses. Raising the
requirement should have no impact on our users.
* Fix build issues caused by changes in CMake 3.10
* [CMake] Simplify code coverage CMake configuration
This just addes an option processed by the CMake cache file to configure
code coverage for DXC. With this patch you can generate a coverage
report locally if you build with Clang using the following commnads:
cmake -G Ninja -DDXC_COVERAGE=On <other options> -C
<path to dxc>/cmake/caches/PredefinedParams.cmake <path to dxc>
ninja
ctest
ninja generate-coverage-report
* Adding code comment explaining caches
CMake cache scripts aren't super common, but are very useful. Adding a
comment to explain how they work is probably generally valuable.
This cherry-picks in a bunch of upstream LLVM changes and has a few small modifications for DXC.
* [CMake] Add LLVM_BUILD_INSTRUMENTED option to enable building with
-fprofile-instr-generate
This is the first step in supporting PGO data generation via CMake.
I've marked the option as advanced and experimental until it is fleshed
out further.
llvm-svn: 255298
* Add support for collating profiles for use with code coverage
Differential Revision: http://reviews.llvm.org/D20993
llvm-svn: 272599
Patch by: Vedant Kumar <vsk@apple.com>
* [CMake] Add targets for generating coverage reports
This is a pretty small bit of CMake goop to generate code coverage
reports. I always forget the right script invocation and end up
fumbling around too much.
Wouldn't it be great to have targets that "Just Work"?
Well, I thought so.
At present this only really works correctly for LLVM, but I'll extend
it in subsequent patches to work for subprojects.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D109019
* Update prepare-code-coverage.py from llvm/main
Theres's a swath of changes that went in here. Cherry-picking them all
indivudially is not worth it. Contained commits:
8773822c578a [Zequan Wu] [Utils] Add -compilation-dir flag to
prepare-code-coverage-artifact.py
577d59bc679d [Nico Weber] svn propset svn:executable on
utils/prepare-code-coverage-artifact.py
9a10eac663dd [Chris Bieneman] [Coverage] Apply filtered paths to summary
16776315460e [Vedant Kumar] [utils] coverage: Add help text about the
--restrict flag (NFC)
f4df0edf3ea3 [Vedant Kumar] [utils] Add a '--unified-report' option to
the code coverage prep script
7466987cb7ba [Vedant Kumar] [utils] Use print_function in the code
coverage prep script, NFC.
273b6dc3e2f7 [Vedant Kumar] [utils] Add an '--only-merge' option to the
code coverage prep script
547ebad0b984 [Vedant Kumar] [utils] Teach the code coverage prep script
about --restrict
05ee94f1b573 [Vedant Kumar] [utils] Generate html reports with the code
coverage utility script
d9aed82cf7e2 [Vedant Kumar] Add support for collating profiles for use
with code coverage
* HLSL-specific change to coverage reporting
This works around DXC not having a cmake target to run tests by
exposing a new variable which can configure which target gets run to
produce profiles. A similar change will be useful upstream, so I'll add
a change there too.
* WinAdapter: Remove virtual dtors from IUnknown to fix vtable ABI
The vtable for `IUnknown` and its subclasses contain two deletion
pointers when compiled on non-Windows systems with `IUnknown` from
`WinAdapter.h`:
vtable for 'DxcLibrary' @ 0x7ffff7cbc5f8 (subobject @ 0x5555556bb9e0):
[0]: 0x7ffff6a56d40 <DxcLibrary::QueryInterface(_GUID const&, void**)>
[1]: 0x7ffff6a56d20 <DxcLibrary::AddRef()>
[2]: 0x7ffff6a56d30 <DxcLibrary::Release()>
[3]: 0x7ffff6b36bc0 <IUnknown::~IUnknown()> // Complete object destructor
[4]: 0x7ffff6a57130 <DxcLibrary::~DxcLibrary()> // Deleting destructor
[5]: 0x7ffff6a56d50 <DxcLibrary::SetMalloc(IMalloc*)>
[6]: 0x7ffff6a56d60 <DxcLibrary::CreateBlobFromBlob(IDxcBlob*, unsigned int, unsigned int, IDxcBlob**)>
... More DxcLibrary virtual functions
This shifts the the pointers for functions for all subclasses, and is
[annoying] to deal with in otherwise cross-platform applications using
DirectXShaderCompiler as library. `dxcompiler.dll` compiled on/for
Windows without `WinAdapter.h` does not suffer this problem, and only
has three function pointers for `IUnknown`.
Fortunately, it is easily solved by removing the virtual destructor from
`IUnknown`. LLVM enables `-Wnon-virtual-dtor` that warns against
classes with virtual methods but no virtual destructor, though this
warning is best not enabled akin to Windows builds where `IUnknown` from
`windows.h` (`unknwn.h`) results in the same warning on MSVC ([1]/[2]).
[annoying]: 1e624792fc/src/unknown.rs
[1]: https://github.com/microsoft/DirectXShaderCompiler/issues/3783#issuecomment-844189358
[2]: https://godbolt.org/z/hKPT6ThEf
* WinAdapter: Make `IUnknown` and `IMalloc` pure-virtual classes
`IUnknown` in Windows' `unknwn.h` and `IMalloc` in `ObjIdl.h` are marked
as pure virtual, and are best marked as such in `WinAdapter` for
non-Windows platforms too [1]. Only the shim for `IMalloc` was relying
on the default refcounting implementation, all other subclasses either
contain pure-virtual methods themselves or provide an implementation for
`AddRef`/`Release` as required. Likewise the default implementation for
`IMalloc` was only instantiated once by `CoGetMalloc`, and has been
moved into a local class implementing the `IMalloc` interface instead.
[1]: https://github.com/microsoft/DirectXShaderCompiler/pull/3793#issuecomment-846459741
* WinAdapter: Add three missing virtual functions to `IMalloc` interface
To prevent unexpected vtable breakage, add the missing functions from
the [documentation]. Note that they are listed in the wrong order, the
right order is retrieved from the `ObjIdl.h` header and implementations
for `IMalloc` in DirectXShaderCompiler. All implementations are now
properly using the `override` keyword too, to enforce virtual method
existence in the base class.
[documentation]: https://docs.microsoft.com/en-us/windows/win32/api/objidl/nn-objidl-imalloc
* Make all WinAdapter destructions explicit
This prevents warnings about non-virtual destructor usage that trip up
the Linux build. It represents status quo on Windows.
Co-authored-by: Greg Roth <grroth@microsoft.com>
I added this upstream a few years ago so that you can control passing
`-O1` to the compiler when building debug sanitized builds. Without
being able to disable optimizing sanitizer builds it is really tricky to
debug sanitizer failures, but without optimizing the build ASan builds
take _forever_ to run tests which makes them difficult to use.
This adds a new option to the build to enable link-time optimzied builds
of DXC. On Windows with MSVC this seems to give about a 4% runtime
performance boost on my one locally run test case... and on Linux I'm
seeing about 6% on that same locally run test case.
LTO/LTCG is enabled by default for "Official" builds.
It's free performance so yay :D
Backport an LLVM cmake option to specify the C/C++ linker to use.
This implementation is simplified and unlike the llvm-project one
does not contain any linker-specific tweaks.
On my system, setting `-DLLVM_USE_LINKER=mold` reduces the full build
time from 81s to 43s.
* Clean up debug preprocessor macros
This change removes the `DBG` macro value in favor of `NDEBUG`, and
also converts many uses of `_DEBUG` with `NDEBUG`.
_DEBUG is set by MSVC when debug information generation is enabled.
NDEBUG is set by CMake debug configurations and by LLVM on all
configurations when `LLVM_ENABLE_ASSERTIONS` is set.
Aligning with LLVM's NDEBUG usage allows a consistent pattern
throughout the codebase.
* Address PR feedback
Use environment variable HLSL_AGILITYSDK_DIR which you can point to
extracted contents of an AgilitySDK.
Automatically copy binaries to appropriate location when set.
Look for MinTe binary set from TAEF. If found, set a local TAEF binary
location.
If this is set when running execution tests, copy MinTe there, and if the
HLSL_AGILITYSDK_DIR was set, also copy agility SDK binaries under the
.\D3D12\ subdir there, so the test can automatically pick it up.
Add hctbins.cmd to copy extra bins necessary for running things out of
the ...\bin or ...\test directories such as HLSLHost.exe. This includes
AgilitySDK bins if set.
Update FindTAEF.cmake to set binary dir based on HLSL_TAEF_DIR
Since we don't actually support a single project building to multiple
architectures, we look for te.exe in x86 OR x64 subdir.
Whichever one is set in hctstart.cmd should be the one that will be used.
These were disabled at a point that the DXC CI didn't reject builds for
failing on Linux, now that they do, we can enable these tests sure in
the knowledge that any changes to them that break on Linux will be
detected and fixed quickly
Fixed *nix error code return. The previous caused an infinite loop of retries.
This definition more accurately identifies the error that has occurred as not
benefiting from repeated attempts
Skip tests and filechecker elements that entirely depend on reflection or
linker interfaces that aren't implemented on *nix yet.
Correct capitalization in a few places to satisfy a case-sensitive filesystem
Detect directories from test include handler. *nix virtual filesystem will try to
load the file using the include handler when performing stat() on a directory.
The test include handler is pretty basic and would provide the next file, leaving
it without a file to offer when the real include came along.
By detecting a name that looks like a dir and returning failure, it
correctly indicates that the file isn't found by the handler and saves
the virtual file for when its needed.
Don't mark non-debug builds as such. The DEBUG define determines which
tests get run. Some of which depend on consistencies that can only be relied
upon in debug builds. So DEBUG is disabled for all non-debug builds.
Correct free of new() memory. Incidental regex allocations were changed for
HLSL to use the global new() operator, but were still freed using the standard
free(), but replacing the default free() with regex_free() allows the default new()
operator to be paired with the default delete() operator
Correct misnamed test flag. The *nix flag was mistakenly called InputFile instead
of InputPath. By renaming it, the ManualFile test can target a specific HLSL file.
Fix misused ArrayRef in legacy GV replacement. The replacement of the GV with
the legacy form used in libraries collected the arguments into an arrayref without
an array to ref, which caused garbage in release builds. Also moved to using end
vars for loops where the elements can get deleted during the loop.
Fix *nix conversion to utf8. When an unknown codepage is found, it tries to convert
to utf16, but that isn't' actually helpful for *nix for a shader codepage derived from
the BOM, which is correctly identified as utf32
* Minor fix for dependencies of generated sources
For the `CODE_TAG` sources we need to ensure that they are dependencies
of the generation command. This also requires creating an extra
temporary file so that the dependencies aren't circular.
* Fix typo
We've seen some lingering odd issues with autocrlf. I think this will
resolve them all because our gitattrbutes sets text=auto which is only
overridden if autocrlf=input.
We'll know if this works based on whether or not the AppVeyor build
works.
This change updates the HCTGen code for unix systems to match the line
endings of the checked out file for in-tree generation. It also updates
the SPIR-V whole file tests to strip trailing carriage returns.
* Automate generating of intrinsic code
This change removes generated code for HLSL and DXIL intrinsics and
sets up the build system so that it will generate the code and
automatically regenerate it on changes to any of the generation scripts
or input files.
* Covering the last few generated files
* Move namespace priting into hctgen
I somehow missed that the open namespace was in manual code and the
close was in the generated code...
* Fix pix passes generator
* Work around missing autocrlf on Windows checkout
*shakes fist* AppVeyor...
* Actually pass through --force-lf
* Restore lit and googletest sources from LLVM 3.7.0
This commit just re-adds sources that were removed somewhere along the
way.
* Pull in googletest & googlemock from LLVM 4.0
Googlemock was introduced in LLVM 4.0, and is used by SPIR-V's tests.
This pulls in the LLVM 4.0 version of GoogleTest and GoogleMock to
replace the external submodules.
LLVM's version of GoolgeTest and GoogleMock have some minor extensions
to work better with LIT for error reporting and producing cleaner test
output.
* Remove external googletest
* Fix lit to handle comment in gtest names
This fix came into LLVM with the updated googletest in a977582dead2
* Ignore raw_fd_ostream errors when not closing
There's some odditites with the changes in the filesystem code that
cause this to error sometimes in unit tests. Until we can dedicate time
to looking into the filesystem code, just swallow that error...
* Fix bot failure
Missed an option change.
* Fixing MSVC build failure
* A cleaner build fix
This should address issues with VS 2019 in a cleaner way.
* Fix build
CMake supports passing in CMake scripts via the `-C` command line
option, which can set CMake cache variables to initialize options
before the first CMakeLists file is processed. This is a portable and
shell-agnostic way of supporting what the
`cmake-predefined-config-params` file is used for.
* Use D3D12 headers from the same Windows 10 SDK version as cmake is using (cmake variable CMAKE_VS_WINDOWS_TARGET_PLATFORM_VERSION)
* For ARM64EC builds detect SDK version from VSDevCmd environment variable instead of a fixed version in hctbuild.cmd
* Add option -show-cmake-log to hctbuild and enable it in AppVeyor
Antonio Maiorano
Chris B
Xiang Li
r-a-sattarov
Nathan Gauër
FantasqueX
Greg Roth
Helena Kotas
Paul Dodzweit
Marijn Suijten
Jakub Kuderski
Tex Riddell
Jaebaek Seo
Minmin Gong
Chris B