* set up release branch for v1.9.0
* Fix unhandled exception in CommonAnnotatedKey.TryCreate
* Add test file.
---------
Co-authored-by: Suvam Mukherjee <sumukherjee@microsoft.com>
Prior to this, if the user running the script had never installed a
dotnet tool before, they would get an error when the script attempted to
use nbgv. Despite the script auto-installing it, if dotnet tool has
never run before, it won't be on the PATH yet since dotnet tool mutates
that PATH but it doesn't get refreshed within the same session.
Starting on the 1st of September, `generate_and_detect_common_annotated_key_test` began failing with:
```
---- end_to_end_tests::generate_and_detect_common_annotated_key_test stdout ----
thread 'end_to_end_tests::generate_and_detect_common_annotated_key_test' panicked at src\end_to_end_tests.rs:23:49:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
`generate_common_annotated_test_key` was producing `Ok("")` since the hard-coded `test_char` combined with this month happened to generate an invalid key.
The code was hitting the second branch of `generate_common_annotated_test_key`:
``` rust
// The HIS v2 standard requires that there be no special characters in the generated key.
if !key.contains('+') && !key.contains('/') {
if !long_form {
key = key.substring(0, key.len() - 4).to_string();
}
return Ok(key);
} else if test_char.is_some() {
// We could not produce a valid test key given the current signature,
// checksum seed, reserved bits and specified test character.
key = String::new();
break;
}
```
This hardens the test to prevent failing as time changes.
Follow up:
* Check in a `Cargo.lock` to ensure deterministic packages. (This is orthogonal but package non-determinism was one of the (incorrect) hypotheses for the root cause of the failure.)
* Consider accepting a timestamp into the API so that it can be made deterministic.
* Consider returning `Err("…")` instead of `Some("")`, since `""` is not a valid test key anyways.
No release note as product code was not changed.
* Generate annotated hash (#84) (#85)
* Add ComputeCommonAnnotatedHash
* Update return value for annotated hash helper that favors byte[].
* Update tests.
Co-authored-by: Michael C. Fanning <mikefan@microsoft.com>
* updating release history (#86)
---------
Co-authored-by: Michael C. Fanning <mikefan@microsoft.com>
When the Scan struct was updated to use a ScanEngine, the scan_defs()
method was missed being added. This caused the FFI crate to break
unknowingly.
Add scan_defs() to the Scan struct, so it behaviors exactly as before.
Signed-off-by: Beau Belgrave <beaub@linux.microsoft.com>
Co-authored-by: Beau Belgrave <beaub@linux.microsoft.com>
* Rust/Marvin: add functions operating solely on slices.
Instead of relying the offset and length field in the checksum
calculation implementation, we can use that information to construct
a slice directly after validating the relevant invariants and use
the slice to perform our calculations. This makes the code simpler,
faster, and more idiomatic.
* Rust/Marvin tests: make test cases easier to modify, and test offsets and different lengths
* Rust: add release note for marvin functions
* fix(cross_company_correlating_id): Decrease allocations & formatting calls during generation of c3id
There are some steps in the generation of the Cross Company Correlation ID
for which we know the amount and type of data that we will be processing at
compile time. Passing this knowledge directly to those functions decreases
the amount of time spent formatting & reallocating at runtime.
* fix(cross_company_correlating_id): remove final allocation
Thanks to a comment by @suvamM I realized that there is no need to
allocate the string data at all. Since we know the length of the data
up front, and the formatting we do is so basic that doing it by hand
works fine, we can remove the allocation and gain another 7% speedup.
* fix(cross_company_correlating_id): borrow only once
* fix(cross_company_correlating_id): document hex encoding
* fix(Rust/His): add tests/compile time constructs to
validate concurrency
* feat!(Rust/His): tighten bounds on validator
This allows `ScanEngine` to be Send + Sync, allowing
for proper concurrent usage
* Rust: update changelog
* Rust/His: Introduce ScanEngine struct
Previously the Scan struct would track it's own state as well as allow
external state operations. To do this though, it required a RefCell
instance for the non-external state. This prevented the Scan struct
from being fully used in scoped thread contexts without a Mutex or Arc.
Introduce the ScanEngine struct which contains all the same code as the
Scan struct (rename) as before except for the internal state tracking.
Re-write the Scan struct to simply host the ScanEngine and a ScanState.
This ensures the logic is the same as before, however, now we can
completely drop the RefCell and make it much clearer for callers what we
expect them to do.
Signed-off-by: Beau Belgrave <beaub@linux.microsoft.com>
* updating release notes
---------
Signed-off-by: Beau Belgrave <beaub@linux.microsoft.com>
Co-authored-by: Beau Belgrave <beaub@linux.microsoft.com>
Co-authored-by: Suvam Mukherjee <sumukherjee@microsoft.com>
Today the existing Scan struct tracks the options to be used as well as
the current state of the stream. This is useful when doing many streams
of data on a per-core basis. It is not as useful if you are using many
threads.
When many threads are being used, it's more useful to have a single
immutable reference that can be passed small mutable states that can
individually be reset or thrown away.
Add ScanState struct that solely keeps track of the state of a given
stream of data.
Add concurrent_parse_bytes() and concurrent_parse_reader() functions
that take a ScanState mutable reference while operating on the immutable
Scan struct. This allows for many threads to share the same Scan struct.
Move the internal scanning state within the Scan struct to a RefCell
ScanState. This allows us to keep our API the same and not break our FFI
layers while allowing for immutable changes to the byte scan.
Move the various byte scan logic methods to immutable self references
that take in a mutable ScanState reference.
Signed-off-by: Beau Belgrave <beaub@linux.microsoft.com>
Co-authored-by: Beau Belgrave <beaub@linux.microsoft.com>
The current benchmark test for the Rust scanner omitted the scanner
getting reset. This causes the scanner to accumulate possible matches
and cause the Vec to resize. This leads to an inaccurate picture of
performance of the scanner.
By properly resetting the scanner, the performance increases by ~20%.
Signed-off-by: Beau Belgrave <beaub@linux.microsoft.com>
Co-authored-by: Beau Belgrave <beaub@linux.microsoft.com>
* adding an end-to-end test
* adding constants for long form and standard
* adding validation
* fixing failing test
* adding long-form support
* fixing generation
* adding support for long-form, normalizing rust and c# common key generation and validation
* fixes and cleanups
* adding common key generator test
* adding validation test
* adding test
* adding test
* adding test
* Fixing SEC101/200 GenerateTruePositiveExamples test case
* updating release notes
* updating test based on PR feedback
* New test>
* Add new test.
* updating well known regexes
* updating tests
* adding comments
---------
Co-authored-by: Michael C. Fanning <mikefan@microsoft.com>
Ross Wollman
Michael C. Fanning
Nick Guerrera
Suvam Mukherjee
jdraaijer-microsoft
Beau Belgrave
dependabot[bot]