TraceLoggingDynamic uses SHA1 hashing for non-security purposes.
To avoid potential problems on systems where SHA1 libraries are disabled
for security reasons, use a a local implementation of SHA1. This is
probably much slower than the library version, but it's not used in any
performance-critical code paths.
Also update some type annotations since I was getting warnings that the
`*` syntax is not supported. Seems like `Array` is as specific as you
can get with the annotations.
Fix exception on older builds of python where the algorithm-specific
constructors don't recognize the usedforsecurity parameter. Fix by using
the generic constructor which ignores unrecognized parameters.
**Why**
`tracelogging` crate cannot be used in kernel-mode code because of the reliance on Win32 APIs such as `EventRegister`.
**What Changed**
* Added optional `kernel_mode` feature which replaces Win32 APIs with kernel-mode APIs like `EtwRegister`
**How Tested**
Added it to a kernel-mode driver built using [windows-drivers-rs](https://github.com/microsoft/windows-drivers-rs) and tested tracelogging prints on WinDbg.
**Usage**
Enabling `kernel_mode` feature will require the client to link with `NtosKrnl.lib` for definition of `Etw*` APIs.
* Fixes for consistency with the eventheader crates
* Fixes ported from the LinuxTracePoints repo
* More eventheader consistency
- Accept activity_id as either Guid or array-of-byte.
- Accept time32, time64, and errno fields.
* Doc fixes
* Fix some inaccurate comments
* Use provider_enabled for consistency with eventheader
* rust tracelogging_dynamic - make register non-mut
`tracelogging_dynamic::Provider` is annoying to use in multithreaded
scenarios. The pain point is that the register call takes self by `&mut`.
This is because register is the point at which the metadata is
configured. However, we don't really need to configure metadata at
registration, and in fact it probably makes more sense to configure
metadata at construction and make the object completely immutable. That
makes multithreaded use cases much nicer.
This is a breaking change, but probably worth doing. I don't think there
are a lot of users of TraceLoggingDynamic.
tracelogging\native.rs:
- I don't think there is any value in panic for overlapping calls to
unregister, so just allow those. (We do still want to catch overlapping
register/register or register/unregister, but we can allow overlapping
unregister/unregister.)
tracelogging\provider.rs:
- Update doc comments with the relaxed prerequisites for unregister.
tracelogging_dynamic\provider.rs:
- Move configuration from the register method to the constructor.
- Update doc comments with the relaxed prerequisites for unregister.
- `register` now takes `self: Pin<&Self>`.
- `pin!` is now stabilized.
* Update tracelogging_dynamic version to 1.1.0
* Don't put .vscode files into git
* Comments, docs, changelog
* Use v1.1.0
---------
Co-authored-by: Doug Cook (WINDOWS) <dcook@windows.microsoft.com>
* rust: add filetime_from_systemtime macro
- Add a `filetime_from_systemtime!(systemTime)` macro. Export it from
both tracelogging and from tracelogging_dynamic crates. Note that
we're using a macro so that the tracelogging and tracelogging_dynamic
crates do not become dependent on the `std` crate.
- As an optimization, split the `filetime_from_duration(dur, after)`
function into two functions - one for after == true and one for
after == false.
* Make sure tests cover expression correctly
* Bump version to 1.0.1
---------
Co-authored-by: Doug Cook (WINDOWS) <dcook@windows.microsoft.com>
Implement TraceLogging provider for Rust.
- `tracelogging` crate + `tracelogging_macros` crate implement a TraceLogging system similar to Windows SDK `TraceLoggingProvider.h`, intended for general developer use, requires events to be defined at compile-time, does most of the work at compile-time to minimize runtime overhead and dependencies.
- `tracelogging_dynamic` crate implements a runtime-buffered TraceLogging system, intended for middle-layer use, e.g. when implementing a higher-level dynamic logging system like OpenTelemetry or logging for a scripting language.
I've attached the generated Docs as a zip so you can browse the project's documentation.
[tracelogging.zip](https://github.com/microsoft/tracelogging/files/9559453/tracelogging.zip)
Feedback is welcome everywhere, but here are a few specific areas where I would welcome review to make sure I didn't miss a better solution:
- The design is very similar to `TraceLoggingProvider.h`.
- Fields within events are defined with a syntax similar to a function call: `TYPE("FIELD NAME", VALUE)` or in more complex scenarios, `TYPE("FIELD NAME", VALUE, format(FORMAT), tag(TAG))`.
- I considered WIL-style, where you make a provider class and define all your events in one place, then use the class elsewhere. I didn't like the syntax - I couldn't come up with good syntax for things like event/field names with non-identifier characters, field types, field formats, field tags.
- I considered a field syntax that looks more like Rust constant declarations: `"FIELD NAME": TYPE = VALUE`. I liked it, but other people on my team were confused by having a string literal as a variable name, so I went back to the function call syntax.
- The TYPE names are listed in a table in the documentation comments of the `write_event` macro.
- Counted string types are `str` + character width + format:
- `str8` = UTF-8
- `str8_json` = UTF-8 JSON
- `str8_xml` = UTF-8 XML
- `str8_cp1252` = ANSI
- `str16` = UTF-16LE
- `str16_json` = UTF-16LE JSON
- `str16_xml` = UTF-16LE XML
- Nul-terminated string types are `strz` + character width + format:
- `strz8` = UTF-8
- `strz8_json` = UTF-8 JSON
- `strz8_xml` = UTF-8 XML
- `strz8_cp1252` = ANSI
- `strz16` = UTF-16LE
- `strz16_json` = UTF-16LE JSON
- `strz16_xml` = UTF-16LE XML
- Some of the design decisions were influenced by the possibility of supporting something similar on Linux.
- With ETW, there is no technical reason why we couldn't treat Provider as a normal variable, e.g. `let my_provider = new_provider!(...)`. However, it would be very difficult to support such a syntax with Linux tracing systems, so I only support a `define!(MY_PROVIDER, ...)` syntax that creates a static variable.
- With ETW, there is no reason why you can't pass a reference to a provider variable as a parameter or store a reference to a provider variable as a field. However, it would be very difficult to support this under Linux, so I've documented that the `write_event` macro requires the original provider symbol, not a reference to it. (I don't enforce this in any way, though maybe I should.)
* TraceLogging for Rust
* Updates, fixes
* Add Rust to parent README
* Cleanup and fixes
* Remove enforcement of PROVIDER symbol
* Better doc comments
* Add links to the "Normal field types" table
* Rename StrZ to CStr
* Review the footnotes
* Move tracelogging_macros back to root
* Simplify lifetimes on descriptors
* Guid uses try_into, tests avoid race conditions
Co-authored-by: Doug Cook (WINDOWS) <dcook@windows.microsoft.com>
- Add a GetRawFields method that returns a copy of fields currently in
the builder.
- Add an AppendRawFields method that appends fields extracted via
GetRawFields.
- Rename the Latin1 class to UncheckedASCIIEncoding and implement the
rest of it so it is safe for general use.
- Add some validation of the new GetRawFields and AppendRawFields methods.
- Change how the test code uses SetStructFieldCount so it is more
obvious what is happening.
Co-authored-by: Doug Cook (WINDOWS) <dcook@windows.microsoft.com>
* TraceLoggingDynamic.cs - minor perf improvements
- Add the ability to patch a struct size in case you don't know the
final size until after the fields have been enumerated.
- Minor performance improvement in AddMetadata by minimizing redundant
range checks.
- Minor performance improvement in Reset by minimizing redundant range
checks.
- Moderate improvements in UTF-16 strings by skipping the encoding.
* Add Utf8NameEncoding property for configurable encoder
UTF8 encoder constitutes about 50% of the time in the builder. Allow
customer to provide their own encoder if they want a faster one.
* Add AnsiString overloads that use encoding
Co-authored-by: Doug Cook (WINDOWS) <dcook@windows.microsoft.com>
Update the SAL annotations in TraceLoggingDynamic.h to use the recommended
SAL 2 constructs. Fixes#36
Co-authored-by: Brian Rogers <broger@microsoft.com>
* Fixing the ltng for amd64
* Aliasing the call to *_ring_buffer_align *_ring_buffer_align_ctx based on the lttng version
* fixing a bug here in aliasing and removing the function prototype
It has been brought up that these LTTNG helpers are pretty close to being compilable in C++ mode, if not for a few casts that are the result of language/compiler differences.
These changes aren't very intrusive and let you run this project through g++ now.
* Log strings for IPv4, IPv6, sockaddr
LTTNG does not have native types for IPv4 addresses, IPv6 addresses, or
sockaddr. Add indirect support - these objects will be converted to
strings before being added to the trace.
- Add LttngHelpers APIs to format these types.
- Add TraceLoggingProvider.h support for these types, including new
macros for IPv4 and IPv6.
- Add tests.
* Support for lttng-ust 2.13+
lttng-ust 2.13 is a major breaking change and requires significant
updates to the LttngHelpers and TraceLoggingProvider libraries.
OPEN ISSUE:
- CMake's LTTngUST package is not picking up the new lttng-ust-common
library. For now, I've added it by hand (which breaks 2.10).
Overview of changes in 2.13:
- Many LTTng symbols were renamed.
- Sequence no longer includes the length field. Instead, the sequence
field references the length field by name.
- Most of the structs are now designed to be versionable, i.e. they start
with a size.
- The lttng_type field has been split into separate structs, which are
related via C-style inheritance (shared initial field which is a
discriminator enum value).
- The field descriptor now stores an `lttng_type_common*` by reference
instead of storing an `lttng_type` by value.
- In cases of a 1-to-many relationship, the parent struct now generally has
pointer-to-array-of-pointer-to-struct (`const struct foo* const*`), not
pointer-to-array-of-struct (`const struct foo*`) as before.
Verified that LTTNG 2.10 and 2.13 build and produce the same output.
README.md:
- Fix some typos and some confusing descriptions.
LttngHelpers.h:
- Add an abstraction layer to simplify working with both pre-2.13 and
post-2.13 LTTng.
- Note that if your code consumes LttngHelpers.h and targets 2.08..2.12, your
code should continue to work without any changes. However, if your
code targets 2.07 and directly references lttngh_BoolEnumDesc, it will
need fixes (your code was broken anyway). And if your code wants to
target 2.13, it will need to switch to use the abstraction layer (and
will still have a few ifdefs based on `lttngh_UST_VER`).
TraceLoggingProvider.h:
- Switch to using the new abstraction layer. Mostly this means passing
around "UstType" macro parameters (provided by LttngHelpers.h) instead
of composing the UstTypes ourselves. (We still have to compose the array
types.)
- Add support for 2.13. (Look for `lttngh_UST_VER`.)
- Add support for using a macro for `providerSymbol` in
TraceLoggingWrite.
- Converting char16/char32 into "sequence of utf8" is a bit problematic
since 2.13 considers a sequence to be 2 fields, so instead we now convert
char16/char32 into "nul-terminated utf8".
- A few minor bits of cleanup.
LttngHelpers.c:
- In 2.13, provide predefined UstType constants for the precomposed
types (covers all cases except fixed-length arrays).
- Use adaptation layer where appropriate.
- Add support for 2.13. (Look for `lttngh_UST_VER`.)
- Refactor to extract subroutines so that the version-specific
alternatives are easier to follow.
test\CMakeLists.txt:
- Add more tests, run against both C and C++.
- 2.13 requires linking against new lttng-ust-common library. For some
reason, CMake's LTTngUST doesn't pick this up, so add it directly.
(FIXME)
* CMake improvements (still not fixed)
* PR feedback
Co-authored-by: Doug Cook (WINDOWS) <dcook@windows.microsoft.com>
Bug in TraceLoggingEventEnabled that only occurs if the event has a
keyword.
`TraceLoggingEventEnabled(ProviderSymbol, "EventName")` needs to look up
the requested event. Events are identified by a
"ProviderName:EventName;KeywordSuffix" string.
TraceLoggingEventEnabled is supposed to work by creating a
"ProviderName:EventName" string, doing a prefix match on the event, and
if the prefix match succeeds, we check that the next character is a ';'
(meaning we matched up to the keyword prefix) or a 0 (meaning we matched
the full event string).
Due to a bug, instead it creates a "ProviderName:EventName;k;" string.
This successfully matches as long as the event has no keywords, but it
incorrectly fails to match if the event does have any keywords.
Fix is to cut off the last three characters after creating the
"ProviderName:EventName;k;" string.
Previously, LttngHelpers.h would pass down an alignment characterisc of the input data, rather than the eventual UTF8 output.
Fix this by targeting the required alignment of either the UTF8 string or the uint16_t that will prepend sequences.
Fix non-ASCII character in comments.
Improve length checking in _tlg_EventFullName.
- Include the minimum keyword suffix length in the initial length check,
eliminating the need to check it later.
- Improve the string in the assertions.
- Even though the macros are doing a static_assert for provider name
length, validate it just in case.
- Tweak the validation to always compare with MAXLEN - 5 for consistency.
Doug Cook (WINDOWS)
Kyle Sabo
Doug Cook
Hamza Mustafa
Rose
Doug Cook (WINDOWS)
microsoft-github-policy-service[bot]
Bobby Reynolds
Brian Rogers
Reiley Yang
Arsham Mesbah
Eric Vaughn
Benjamin Thaut
Stephen Hemminger
Nick Bopp
Nick Bopp
Sinan Kaya