Граф коммитов

368 Коммитов

Автор SHA1 Сообщение Дата
Anders Schack-Mulligen cfd08f23a5 FileSystem: Fix qldoc. 2023-09-28 08:58:55 +02:00
Asger F ba1f8766aa Shared: update PartialPathNode accordingly 2023-09-27 14:38:28 +02:00
Asger F 6792592a15 Shared: add late checks for stateful in/out barriers 2023-09-27 14:38:28 +02:00
Taus ff35f9fb8c Shared: Clean up `NodeInfo` in shared extractor
I was perusing the shared extractor the other day, when I came across
the `NodeInfo` struct. I noticed that the `fields` and `subtypes` fields
on this struct had two seemingly identical ways of expressing the same
thing: `None` and `Some(empty)` (where `empty` is respectively the empty
map and the empty vector). As far as I can tell, there's no semantic
difference in either case, so we can just elide the option type entirely
and use the empty value directly. This has the nice side-effect of
cleaning up some of the other code.
2023-09-27 12:29:07 +00:00
Tom Hvitved e7a3dc83bc Data flow: Performance improvements 2023-09-26 20:42:03 +02:00
Asger F 6e869452b5 Shared: remove pragma[inline] implied by bindingset 2023-09-26 15:42:24 +02:00
Asger F c7e892fa8e Shared: change note 2023-09-26 15:24:29 +02:00
Asger F cfed7e9b6c Shared: add in/out barriers with flow state 2023-09-25 09:58:49 +02:00
Anders Schack-Mulligen aa2c7a79d6 Dataflow: Add bindingset 2023-09-25 08:55:09 +02:00
Anders Schack-Mulligen 66da997b7b Dataflow: Make use of defaults for language-specific hooks. 2023-09-22 14:54:22 +02:00
Anders Schack-Mulligen 7e04ac55b7
Merge pull request #14268 from aschackmull/java/xmlparsers-typetrack
Java/Dataflow: Add new light-weight data flow api and use it in XmlParsers
2023-09-21 13:33:21 +02:00
Anders Schack-Mulligen 13f7daf71e
Merge pull request #13982 from aschackmull/dataflow/typeflow-calledge-pruning
Dataflow: Add type-based call-edge pruning.
2023-09-21 13:33:08 +02:00
Anders Schack-Mulligen 3dadfa2243 Dataflow: review fixes 2023-09-21 11:52:41 +02:00
Anders Schack-Mulligen d285afba08 Typetracking: minor perf fix. 2023-09-20 14:52:49 +02:00
github-actions[bot] 3acf5244b0 Post-release preparation for codeql-cli-2.14.6 2023-09-20 10:25:10 +00:00
Anders Schack-Mulligen d7e965f863 Dataflow: Add lightweight api based on TypeTracking. 2023-09-20 10:21:21 +02:00
Anders Schack-Mulligen d7bd8c7ffd Shared/TypeTracking: Add support for flow from non-LocalSourceNode source and bugfix in smallstep. 2023-09-20 10:19:33 +02:00
Tom Hvitved 7c2df87ea2
Merge pull request #14247 from hvitved/dataflow/fix-consitency-checks
Data flow: Fix two consistency checks
2023-09-19 15:45:21 +02:00
github-actions[bot] 0a3670727f Release preparation for version 2.14.6 2023-09-19 11:40:30 +00:00
Anders Schack-Mulligen 42054539f4 Dataflow: Minor review fixes. 2023-09-19 12:12:15 +02:00
Tom Hvitved daad69bb30 Data flow: Fix two consistency checks 2023-09-18 16:36:11 +02:00
Anders Schack-Mulligen b13d026434 Dataflow: Review fixes. 2023-09-18 13:15:26 +02:00
erik-krogh 9c0682848e
use final class aliases to use `extends` instead of `instanceof` in the shared libraries 2023-09-18 10:25:49 +02:00
Anders Schack-Mulligen 74787bfba8 Dataflow: One more minor perf tweak. 2023-09-13 15:43:46 +02:00
Anders Schack-Mulligen b63d518114 Dataflow: Minor perf tweak. 2023-09-13 15:43:46 +02:00
Anders Schack-Mulligen b456ba217a Dataflow: Improve rev-to-fwd call edge pruning. 2023-09-13 15:43:46 +02:00
Anders Schack-Mulligen f456bf8d57 Dataflow: Add change note. 2023-09-13 15:43:46 +02:00
Anders Schack-Mulligen 13e7e6b983 Dataflow: Improve fwd-taken call edge predicate and improve fwd-to-rev call edge pruning. 2023-09-13 15:43:46 +02:00
Anders Schack-Mulligen 47f68504a8 Dataflow: Postpone typeflow calledge pruning until stage 3. 2023-09-13 15:43:46 +02:00
Anders Schack-Mulligen c8094d34a7 Dataflow: Add type-based call-edge pruning. 2023-09-13 15:43:45 +02:00
Tom Hvitved c13a8e41ad Data flow: Add more consistency checks 2023-09-12 20:05:05 +02:00
github-actions[bot] d699880c86 Post-release preparation for codeql-cli-2.14.4 2023-09-08 21:17:52 +00:00
github-actions[bot] abf2b12b1c Release preparation for version 2.14.4 2023-09-05 16:56:14 +00:00
Tom Hvitved 73370e7282
Merge pull request #14100 from hvitved/dataflow/consistency-pack
Data flow: Add consistency checks to shared ql pack
2023-08-31 11:47:40 +02:00
Asger F 2d5c40db31
Merge pull request #14048 from asgerf/shared/variable-capture-write-source-node
Variable capture: allow arbitrary data-flow nodes to be the source of a write
2023-08-31 10:20:48 +02:00
Tom Hvitved de7c9bdd9b Data flow: Add consistency checks to shared ql pack 2023-08-30 15:29:41 +02:00
Tom Hvitved c32c4bb6d2
Merge pull request #14086 from hvitved/csharp/perf-fixes
C#: Various performance fixes
2023-08-30 12:13:52 +02:00
yoff ae4c76c788
Merge pull request #13975 from yoff/python/parsemodechars-not-chars 2023-08-29 14:05:57 +02:00
Tom Hvitved 1ac9d2ee5b CFG: Compute `splitsToString` using `concat`, and exclude partial split sets 2023-08-29 11:23:56 +02:00
Jeroen Ketema 0d1fd88729
Merge pull request #14050 from jketema/inline-6
Consolidate all `InlineFlowTest` libraries in the dataflow qlpack
2023-08-29 09:30:35 +02:00
Dave Bartolomeo 3343b78015
Merge pull request #14074 from github/post-release-prep/codeql-cli-2.14.3
Post-release preparation for codeql-cli-2.14.3
2023-08-28 13:34:10 -04:00
github-actions[bot] 3eba77421a Post-release preparation for codeql-cli-2.14.3 2023-08-28 15:53:49 +00:00
Tom Hvitved 42fd9f0c54
Merge pull request #14047 from hvitved/dataflow/join-fix
Data flow: Fix a bad join order
2023-08-25 12:18:24 +02:00
Jeroen Ketema 9d573e5544
Consolidate all `InlineFlowTest` libraries in the dataflow qlpack 2023-08-24 21:38:46 +02:00
Rasmus Lerchedahl Petersen e9e6bce80a shared: handle empty groups in delta 2023-08-24 21:21:49 +02:00
Asger F 6c664e93ef
Merge pull request #14035 from asgerf/shared/variable-capture-nested
Variable capture: synchronize with aliases in nested scopes
2023-08-24 15:39:34 +02:00
Asger F cd7c851d64 VariableCapture: add VariableWriteSourceNode 2023-08-24 14:06:44 +02:00
Asger F 1286235773 Address review comments 2023-08-24 13:58:33 +02:00
Erik Krogh Kristensen 59de92ce64
Merge pull request #14027 from erik-krogh/py-reg-app
ReDoS: limit concretize to strings of at most length 100
2023-08-24 12:57:42 +02:00
Tom Hvitved f2eed4d8c4 Data flow: Fix a bad join order
Before
```
Evaluated relational algebra for predicate DataFlowImpl#248dabc3::MakeImpl#DataFlowImplSpecific#21008cd7::RubyDataFlow#::Impl#DataFlow#167ac380::DataFlowMake#DataFlowImplSpecific#21008cd7::RubyDataFlow#::Global#XSS#e59174e9::OrmTracking::Config#::C#::MkStage#Stage2#::Stage#Stage3Param#::flowThroughIntoCall#6#ffffff@0ea4e2mt with tuple counts:
           1065437   ~0%    {4} r1 = SCAN project#DataFlowImpl#248dabc3::MakeImpl#DataFlowImplSpecific#21008cd7::RubyDataFlow#::Impl#DataFlow#167ac380::DataFlowMake#DataFlowImplSpecific#21008cd7::RubyDataFlow#::Global#XSS#e59174e9::OrmTracking::Config#::C#::MkStage#Stage2#::Stage#Stage3Param#::fwdFlow#9#fffffffff#2 OUTPUT In.0, In.3, In.1, In.2
        1158508760   ~0%    {6} r2 = JOIN r1 WITH project#DataFlowImpl#248dabc3::MakeImpl#DataFlowImplSpecific#21008cd7::RubyDataFlow#::Impl#DataFlow#167ac380::DataFlowMake#DataFlowImplSpecific#21008cd7::RubyDataFlow#::Global#XSS#e59174e9::OrmTracking::Config#::C#::MkStage#Stage2#::Stage#Stage3Param#::flowIntoCallApa#6#ffffff_14023#join_rhs ON FIRST 2 OUTPUT Lhs.0, Lhs.2, Lhs.3, Rhs.2, Rhs.3, Rhs.4

                            {6} r3 = SELECT r2 ON In.5 != false
        1158470345   ~4%    {6} r4 = SCAN r3 OUTPUT In.4, In.1, In.2, In.0, In.3, In.5

                            {6} r5 = SELECT r2 ON In.5 = false
             38415   ~0%    {5} r6 = SCAN r5 OUTPUT In.2, In.0, In.1, In.3, In.4
                 4   ~0%    {5} r7 = JOIN r6 WITH DataFlowImplCommon#f7de413b::MakeImplCommon#DataFlowImplSpecific#21008cd7::RubyDataFlow#::Cached::TApproxFrontNil#f ON FIRST 1 OUTPUT Lhs.4, Lhs.2, Lhs.0, Lhs.1, Lhs.3
                 4   ~0%    {6} r8 = SCAN r7 OUTPUT In.0, In.1, In.2, In.3, In.4, false

        1158470349   ~4%    {6} r9 = r4 UNION r8
             44065   ~3%    {6} r10 = JOIN r9 WITH project#DataFlowImpl#248dabc3::MakeImpl#DataFlowImplSpecific#21008cd7::RubyDataFlow#::Impl#DataFlow#167ac380::DataFlowMake#DataFlowImplSpecific#21008cd7::RubyDataFlow#::Global#XSS#e59174e9::OrmTracking::Config#::C#::MkStage#Stage2#::Stage#Stage3Param#::returnFlowsThrough#8#ffffffff ON FIRST 3 OUTPUT Lhs.4, Lhs.3, Lhs.0, Lhs.5, Lhs.2, Rhs.3
                            return r10
```

After
```
Evaluated relational algebra for predicate DataFlowImpl#248dabc3::MakeImpl#DataFlowImplSpecific#21008cd7::RubyDataFlow#::Impl#DataFlow#167ac380::DataFlowMake#DataFlowImplSpecific#21008cd7::RubyDataFlow#::Global#XSS#e59174e9::OrmTracking::Config#::C#::MkStage#Stage2#::Stage#Stage3Param#::flowThroughIntoCall#6#ffffff@979c54q9 with tuple counts:
         11095   ~0%    {4} r1 = SCAN project#DataFlowImpl#248dabc3::MakeImpl#DataFlowImplSpecific#21008cd7::RubyDataFlow#::Impl#DataFlow#167ac380::DataFlowMake#DataFlowImplSpecific#21008cd7::RubyDataFlow#::Global#XSS#e59174e9::OrmTracking::Config#::C#::MkStage#Stage2#::Stage#Stage3Param#::returnFlowsThrough#8#ffffffff OUTPUT In.0, In.3, In.1, In.2
        470154   ~1%    {8} r2 = JOIN r1 WITH project#DataFlowImpl#248dabc3::MakeImpl#DataFlowImplSpecific#21008cd7::RubyDataFlow#::Impl#DataFlow#167ac380::DataFlowMake#DataFlowImplSpecific#21008cd7::RubyDataFlow#::Global#XSS#e59174e9::OrmTracking::Config#::C#::MkStage#Stage2#::Stage#Stage3Param#::flowIntoCallApa#6#ffffff_20134#join_rhs ON FIRST 1 OUTPUT Lhs.1, Lhs.0, Lhs.2, Lhs.3, Rhs.1, Rhs.2, Rhs.3, Rhs.4

                        {8} r3 = SELECT r2 ON In.6 != false
        470152   ~0%    {8} r4 = SCAN r3 OUTPUT In.5, In.2, In.3, In.7, In.0, In.1, In.4, In.6

                        {8} r5 = SELECT r2 ON In.6 = false
             2   ~0%    {7} r6 = SCAN r5 OUTPUT In.3, In.0, In.1, In.2, In.4, In.5, In.7
             0   ~0%    {7} r7 = JOIN r6 WITH DataFlowImplCommon#f7de413b::MakeImplCommon#DataFlowImplSpecific#21008cd7::RubyDataFlow#::Cached::TApproxFrontNil#f ON FIRST 1 OUTPUT Lhs.5, Lhs.3, Lhs.0, Lhs.6, Lhs.1, Lhs.2, Lhs.4
             0   ~0%    {8} r8 = SCAN r7 OUTPUT In.0, In.1, In.2, In.3, In.4, In.5, In.6, false

        470152   ~0%    {8} r9 = r4 UNION r8
         44065   ~3%    {6} r10 = JOIN r9 WITH project#DataFlowImpl#248dabc3::MakeImpl#DataFlowImplSpecific#21008cd7::RubyDataFlow#::Impl#DataFlow#167ac380::DataFlowMake#DataFlowImplSpecific#21008cd7::RubyDataFlow#::Global#XSS#e59174e9::OrmTracking::Config#::C#::MkStage#Stage2#::Stage#Stage3Param#::fwdFlow#9#fffffffff#2 ON FIRST 4 OUTPUT Lhs.6, Lhs.0, Lhs.5, Lhs.7, Lhs.2, Lhs.4
                        return r10
```
2023-08-24 12:08:34 +02:00
Tom Hvitved 7723dbc6d7
Merge pull request #14026 from hvitved/dataflow/stage3-call-ctx
Data flow: Use call contexts in stage 3
2023-08-24 11:52:08 +02:00
Asger F b424f3fe83 Update a comment to be more accurate 2023-08-24 11:12:39 +02:00
erik-krogh db2b8d4bcc
remove some test code I accidentially commited 2023-08-24 07:56:05 +02:00
Harry Maclean b76842ad3d Shared: Fix clippy lint 2023-08-23 16:24:57 +01:00
Harry Maclean 3680613f2d Shared: Restrict extractor file globs to filenames 2023-08-23 16:09:56 +01:00
Harry Maclean cc7ef5dac1 Shared: Fix clippy lint in shared extractor 2023-08-23 14:11:22 +01:00
Harry Maclean ed40d72e4f Shared: Bump extractor version 2023-08-23 14:11:22 +01:00
Harry Maclean 7e2abf20c6 Shared: Support glob patterns in shared extractor
Replace the `file_extensions` field with `file_globs`, which supports
UNIX style glob patterns powered by the `globset` crate.

This allows files with no extension (e.g. Dockerfiles) to be extracted,
by specifying a glob such as `*Dockerfile`.

One surprising aspect of this change is that the globs match against the
whole path, rather than just the file name.

This is a breaking change.
2023-08-23 14:11:21 +01:00
Asger F 8aec87ea57 Update VariableCapture.qll 2023-08-23 14:57:26 +02:00
erik-krogh 25e4f2c3a2
limit concretize to strings of at most length 100 2023-08-23 10:26:29 +02:00
Tom Hvitved 3810b796a0 Data flow: Use call contexts in stage 3 2023-08-23 10:05:57 +02:00
Tom Hvitved 12d1d04592
Merge pull request #13983 from hvitved/dataflow/reduced-dispatch-early-join
Data flow: Earlier call-context based dispatch filtering
2023-08-21 13:20:08 +02:00
Tom Hvitved 1b4520b058 Data flow: Update QL doc 2023-08-21 12:56:37 +02:00
Jeroen Ketema 2d0f73d7c2
Merge pull request #13881 from jketema/shared-taint-tracking
Introduce shared taint tracking library
2023-08-21 12:45:49 +02:00
github-actions[bot] 098dfb4242 Release preparation for version 2.14.3 2023-08-18 14:48:15 +00:00
Tom Hvitved 81ed72c96a Data flow: Revert join order changes 2023-08-18 10:49:33 +02:00
Tom Hvitved 4d951d8df1 Address review comments 2023-08-17 21:04:58 +02:00
Ed Minnix a36c12ff1f Add trust-boundary-violation sink kind 2023-08-17 13:05:37 -04:00
Tom Hvitved 7cc01ea8b5
Merge pull request #13595 from hvitved/csharp/use-shared-cfg-pack
C#: Adopt shared CFG construction library from shared `controlflow` pack
2023-08-17 10:37:09 +02:00
Jeroen Ketema 33e8310625
Merge branch 'main' into shared-taint-tracking 2023-08-17 00:14:25 +02:00
yoff 7f2f6f14e7
Merge pull request #13729 from yoff/python/model-aws-lambdas
Python/JavaScript: Shared module for serverless functions
2023-08-16 15:14:08 +02:00
Tom Hvitved 570654d1f0 Data flow: Earlier call-context based dispatch filtering 2023-08-16 14:24:45 +02:00
yoff 99bc050366
Update shared/yaml/codeql/serverless/ServerLess.qll
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2023-08-15 15:09:30 +02:00
Erik Krogh Kristensen 6a3b9e10eb
Merge pull request #13914 from erik-krogh/escape-unicode
ReDoS: escape unicode chars in the output for the ReDoS queries
2023-08-15 11:21:21 +02:00
Henry Mercer 1213eba630
Merge branch 'main' into post-release-prep/codeql-cli-2.14.2 2023-08-11 13:54:55 +01:00
Tom Hvitved 9b38028e25 Data flow: Fix `localWriteStep` consistency query 2023-08-10 15:31:04 +02:00
github-actions[bot] 432c21d4fb Post-release preparation for codeql-cli-2.14.2 2023-08-09 18:45:18 +00:00
erik-krogh fe542565c3
fix performance 2023-08-09 13:48:07 +02:00
erik-krogh 0bce42410a
support arbitrary codepoints in NfaUtils.qll 2023-08-08 22:14:51 +02:00
erik-krogh 859e1bfabc
add constraint that i should be between 0 and 65535 2023-08-08 21:11:59 +02:00
erik-krogh 0391e063ca
move `to4digitHex` to `Numbers.qll` 2023-08-08 21:10:58 +02:00
Anders Schack-Mulligen 0ca3f3308b
Merge pull request #13478 from aschackmull/java/varcapture
Java: Add proper support for variable capture flow.
2023-08-08 16:22:56 +02:00
Anders Schack-Mulligen 1cd32722be Java: More review fixes. 2023-08-08 14:32:48 +02:00
Anders Schack-Mulligen 9d59f50340 Java: Review fixes. 2023-08-08 13:37:40 +02:00
erik-krogh 03fbd387df
way better hex conversion 2023-08-08 09:08:39 +02:00
erik-krogh 92db7b047c
escape unicode chars in the output for the ReDoS queries 2023-08-08 00:15:54 +02:00
github-actions[bot] 79c90fa36a Release preparation for version 2.14.2 2023-08-07 18:08:52 +00:00
Jeroen Ketema 8b6a7985db
Refactor the traint-tracking library to follow the dataflow library refactoring 2023-08-07 15:23:15 +02:00
Jeroen Ketema 5d2984b7a5
Merge branch 'main' into shared-taint-tracking 2023-08-07 15:22:29 +02:00
Rasmus Lerchedahl Petersen ef1a5cb0fa shared: change-note 2023-08-07 11:50:03 +02:00
Tom Hvitved b926a7ebba Data flow: Update QL doc 2023-08-07 11:35:21 +02:00
Tom Hvitved 4d14311653 Data flow: Rename `DataFlowParameter` to `InputSig` 2023-08-07 11:35:21 +02:00
Tom Hvitved 6208175aa9 Data flow: Move `DataFlowParameter` into `DataFlow.qll` 2023-08-07 11:35:21 +02:00
Tom Hvitved 0d33c32d8e Data flow: Move `DataFlowImpl(Common).qll` into an `internal` folder 2023-08-07 11:35:21 +02:00
Rasmus Lerchedahl Petersen b8717b3046 Python: remove incorrect use of "stem" 2023-08-07 11:25:31 +02:00
Jeroen Ketema c4a65e58bb
Add change note 2023-08-04 22:53:33 +02:00
Jeroen Ketema 20b792545d
Add missing QLDoc 2023-08-04 22:52:05 +02:00
Jeroen Ketema 7ba2f7a22a
Address review comments 2023-08-04 22:52:05 +02:00
Jeroen Ketema bdd64ce86d
Introduce shared taint tracking library 2023-08-04 22:51:55 +02:00
Mathias Vorreiter Pedersen abe3a816ce
Merge pull request #13851 from MathiasVP/sink-without-states
DataFlow: Support stateless `isSink` in `StateConfigSig`s
2023-08-04 18:01:42 +02:00