* Add Basic Socket and HTTP Listener Rules
Language specific rules for C#, C, C++, Javascript, Java and Python. Generic Socket rule for all other languages.
* Bump dependencies.
* Fixes an issue with setting correct index values when matching a property of an xml tag with an xpath
* Update comments
* Improve robustness of new test cases.
Also check value and index location of matches.
* Fixes JSONPath boundary finding for booleans
Boundary detection for boolean values was off because proper JSON requires booleans to be lower case in the raw text, but JsonElement.ToString() returns capitalized boolean names.
* Adds a test case
* Update XmlAndJsonTests.cs
* Bump dependencies
* Update OAT
* Add detections for some SAAS services
* Improve comment detection for urls with protocol specification in languages with // as the comment format
* Update font awesome reference
* Update tagreportgroups.json
* Update test cases for multiline strings
* Remove unused imports
* Remove legacy files from separate rules repo
* Switch Tag Detection for Features page to Regex
And update all search patterns to be regular expression istead of exact match
* Update Dependencies
* Add rule verifier check to require a description.
GitHub Sarif Upload Action requires the Help.Text property of each Rule object in the sarif to be populated. We populate this in DevSkim with either the Recommendation, if present, or the Description, if not present. However, previously it was possible to have neither. This adds a verification step to require a description to ensure there is a value to use for sarif export.
* Adds test to fail on rules without description
* Description is now a required field in each rule, SemVer relevant
* Description field may not be null
Fix#533
* Adds DependsOnTags field to Rule
After processing of rules DependsOnTags will be checked and then matches whose matching rule have required tags which are not all present in the UniqueTags will be removed before returning results. Adds tests for new functionality and rule verifier will check to make sure all tags which are depended on are present in the rule set.
SemVer changes. Public properties in the Rule object have been changed to IList from a combination of Array and List.
* Fix query for rules with null depends on field
* Formatting: Add Missing Braces Back
* Revert "Formatting: Add Missing Braces Back"
This reverts commit d6dc1fc5a7.
* Set beta flag
* Fix Verification message for missing depends_on_tags
Change a few more List to IList.
Improve edge case in verification where if two rules shared the same id they would both receive the error message about depends_on_tags.
* Improve description for test rules
* Formatting and additional comments on the Rule and AnalyzeCommand objects
* Catch edge case in verification where an overridden rule doesn't have the depends on of its overrider
This verification step prevents a potential issue if a ruleset contains Rule A, which is Overridden by Rule B which depends on TagX.
If RuleA and RuleB match, but TagX is not present, no results would be returned. This is because overrides are performed on a file by file basis, as the last step in processing each file. Depends on tags are checked after all files have been processed, and so the overridden matches are no longer tracked.
* Adds Support for Chained Dependent Rules
And tests for same.
* Typo in test rule description
* Small refactor to deduplicate logic
* Improve variable names
* Use System.Text.Json instead of Newtonsoft.Json. Approximately 2x faster result output serialization with significantly reduced memory usage (for JSON format).
* Adds simple benchmark for json writer.
---------
Co-authored-by: Gabe Stocco <98900+gfs@users.noreply.github.com>
* Refactor to include yamlpath directly in rule engine nupkg
* Fix Yaml Tests
Restore some missing braces mistakenly stripped by previous format run.
Adds locks to constructing structured data query objs (like the YamlDocument) to avoid thread collisions.
Should fix supporting disabling parallelized Analyzer
* Fix#511Fixes#511.
AI was not appropriately adjusting the Index of the pattern match by the index of the yaml element like it did for XML and JSON.
Adds a new test case to ensure the index matched is correct using sample from the linked bug.
* Support multiple Yaml documents in a single file.
Check all documents listed in the stream rather than just the first.
* Remove beta flag in version.json
* Add the new ruleverifier options to the pack command.
* Typos
* Run formatting - no logic changes.
* Update Dependencies
* Refactor tests a bit
* Fix template issue after reformatting. Add missing integrity flags.
* wip checkin
Partial refactor. Requires updated OAT just merged.
* wip checkin
Partial refactor. Requires updated OAT just merged.
* Fix issues with refactored within
* WIP
* Fix inversion in conditions
* Fix regex json rule offset
* Fix off by one
* Validate subclause of within clause.
* Adds Experimental XML path parsing
* Add JSON support for the Structured queries
* Fixes
* Add logging to textcontainer
* Implement XPath and JsonPath for string matching rules.
* Removed unused.
* Fix potential double return.
* Remove unused imports
* Remove redundant and non-cached regex builder
* Null checking
* Nit: Typos in test data
Also move the String clause tests to the correct file.
* Allow specification of both a JsonPath and an XML Path
A user potentially would want to detect a similar pattern at a location inside both an XML and a JSON file, so they can provide separate path specifications for each rather than having two nearly identical rules.
* Adds test case for a rule that targets both Json and XML
* Adds validation for JsonPath and XPath arguments
* Fix#480
Fixes case sensitivity of Enum based arguments
Fixes default Confidence argument not being respected
Fixes severity not being respected
Fixes custom rules not being validated if provided as a directory
Adds new option to skip validation of custom rules
Adds a TextContainer based API for analyze.
* Simplify post validation action
Gabe Stocco
Dmitry
Dmitry