Граф коммитов

4587 Коммитов

Автор SHA1 Сообщение Дата
Rasmus Wriedt Larsen b2611fe198 Merge branch 'main' into redos-cwe-1333 2021-11-24 10:42:43 +01:00
Erik Krogh Kristensen 87a1ccd428 Merge branch 'main' into getRubyInSync 2021-11-23 20:20:37 +01:00
Taus 8cccee6eba
Merge pull request #6972 from yoff/python/promote-redos
Python: Promote ReDoS queries
2021-11-23 14:02:09 +01:00
Rasmus Wriedt Larsen baafd9f8ba Python: Add an other path injection FP
Along with the root cause, which is the `StringConstCompare`
BarrierGuard, that does only allows `in <iterable literal>` and not
`in <variable referencing iterable literal>`
2021-11-23 12:59:15 +01:00
Rasmus Wriedt Larsen eaed870b31 Python: Fix performance problem in PoorMansFunctionResolution
Before these changes:

[2021-11-22 12:02:50] (8s) Tuple counts for PoorMansFunctionResolution::getSimpleMethodReferenceWithinClass#ff/2@cbddf257 after 8.6s:
                      387565   ~0%     {3} r1 = JOIN Attributes::AttrRead#class#f WITH Attributes::AttrRef::accesses_dispred#bff ON FIRST 1 OUTPUT Rhs.2, Lhs.0 'result', Rhs.1
                      6548632  ~0%     {3} r2 = JOIN r1 WITH Function::Function::getName_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'func', Lhs.1 'result', Lhs.2
                      5640480  ~0%     {4} r3 = JOIN r2 WITH Class::Class::getAMethod_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'result', Lhs.2, Lhs.0 'func'
                      55660458 ~0%     {5} r4 = JOIN r3 WITH Class::Class::getAMethod_dispred#ff ON FIRST 1 OUTPUT Rhs.1, 0, Lhs.1 'result', Lhs.2, Lhs.3 'func'
                      55621412 ~0%     {4} r5 = JOIN r4 WITH AstGenerated::Function_::getArg_dispred#fff ON FIRST 2 OUTPUT Rhs.2, Lhs.2 'result', Lhs.3, Lhs.4 'func'
                      54467144 ~0%     {4} r6 = JOIN r5 WITH DataFlowPublic::ParameterNode::getParameter_dispred#fb_10#join_rhs ON FIRST 1 OUTPUT Lhs.2, Rhs.1, Lhs.1 'result', Lhs.3 'func'
                      20928    ~0%     {2} r7 = JOIN r6 WITH LocalSources::Cached::hasLocalSource#ff ON FIRST 2 OUTPUT Lhs.3 'func', Lhs.2 'result'
                                       return r7

With these changes:

[2021-11-22 11:54:25] (415s) Tuple counts for PoorMansFunctionResolution::getSimpleMethodReferenceWithinClass_helper#fff/3@14db70a8 after 75ms:
                      388306 ~0%     {2} r1 = JOIN Attributes::AttrRead#class#f WITH Attributes::AttrRef::getObject_dispred#bf ON FIRST 1 OUTPUT Rhs.1, Lhs.0 'read'
                      379420 ~4%     {2} r2 = JOIN r1 WITH LocalSources::Cached::hasLocalSource#ff ON FIRST 1 OUTPUT Rhs.1, Lhs.1 'read'
                      175082 ~0%     {2} r3 = JOIN r2 WITH DataFlowPublic::ParameterNode#class#fff ON FIRST 1 OUTPUT Rhs.2, Lhs.1 'read'
                      175082 ~2%     {3} r4 = JOIN r3 WITH Essa::ParameterDefinition::getParameter_dispred#ff ON FIRST 1 OUTPUT 0, Rhs.1, Lhs.1 'read'
                      166798 ~0%     {2} r5 = JOIN r4 WITH AstGenerated::Function_::getArg_dispred#fff_120#join_rhs ON FIRST 2 OUTPUT Rhs.2 'func', Lhs.2 'read'
                      162096 ~0%     {3} r6 = JOIN r5 WITH Class::Class::getAMethod_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.0 'func', Rhs.1 'cls', Lhs.1 'read'
                                     return r6

[2021-11-22 11:54:25] (415s) Tuple counts for PoorMansFunctionResolution::getSimpleMethodReferenceWithinClass_helper2#ffff/4@2b60f0s9 after 63ms:
                      162046 ~0%     {3} r1 = SCAN PoorMansFunctionResolution::getSimpleMethodReferenceWithinClass_helper#fff OUTPUT In.2 'read', In.0 'func', In.1 'cls'
                      162046 ~0%     {3} r2 = JOIN r1 WITH Attributes::AttrRead#class#f ON FIRST 1 OUTPUT Lhs.1 'func', Lhs.2 'cls', Lhs.0 'read'
                      162046 ~1%     {3} r3 = JOIN r2 WITH py_Functions ON FIRST 1 OUTPUT Lhs.1 'cls', Lhs.2 'read', Lhs.0 'func'
                      162046 ~0%     {3} r4 = JOIN r3 WITH py_Classes ON FIRST 1 OUTPUT Lhs.1 'read', Lhs.2 'func', Lhs.0 'cls'
                      161935 ~5%     {4} r5 = JOIN r4 WITH Attributes::AttrRef::getAttributeName_dispred#bf ON FIRST 1 OUTPUT Rhs.1, Lhs.0 'read', Lhs.1 'func', Lhs.2 'cls'
                      688526 ~1%     {4} r6 = JOIN r5 WITH Function::Function::getName_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.2 'func', Lhs.3 'cls', Lhs.1 'read', Rhs.1 'readFunction'
                                     return r6

[2021-11-22 11:54:25] (415s) Tuple counts for PoorMansFunctionResolution::getSimpleMethodReferenceWithinClass#ff/2@f73ae6dq after 58ms:
                      688526 ~0%     {4} r1 = SCAN PoorMansFunctionResolution::getSimpleMethodReferenceWithinClass_helper2#ffff OUTPUT In.1, In.0, In.3 'func', In.2 'result'
                      688526 ~0%     {3} r2 = JOIN r1 WITH Class::Class::getAMethod_dispred#ff ON FIRST 2 OUTPUT Rhs.0, Lhs.2 'func', Lhs.3 'result'
                      20913  ~0%     {2} r3 = JOIN r2 WITH Class::Class::getAMethod_dispred#ff ON FIRST 2 OUTPUT Lhs.1 'func', Lhs.2 'result'
                                     return r3

We need the `pragma[only_bind_into]` in getSimpleMethodReferenceWithinClass_helper2, otherwise the tuple counts would look like, which is needlessly big.

[2021-11-22 17:14:34] (2s) Tuple counts for PoorMansFunctionResolution::getSimpleMethodReferenceWithinClass_helper2#ffff/4@5f0505h7 after 711ms:
                      13570510 ~3%     {2} r1 = JOIN Function::Function::getName_dispred#ff_10#join_rhs WITH Attributes::AttrRef::getAttributeName_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'read', Lhs.1 'readFunction'
                      688526   ~1%     {4} r2 = JOIN r1 WITH PoorMansFunctionResolution::getSimpleMethodReferenceWithinClass_helper#fff_201#join_rhs ON FIRST 1 OUTPUT Rhs.1 'func', Rhs.2 'cls', Lhs.0 'read', Lhs.1 'readFunction'
                                       return r2
2021-11-22 17:22:39 +01:00
Rasmus Wriedt Larsen f09f1c4c50 Python: Minor refactor in PoorMansFunctionResolution 2021-11-22 11:11:29 +01:00
Nick Rolfe df6ba43cca Python: treat \A, \Z, \b, \B as special chars, not escapes 2021-11-19 15:49:53 +00:00
Erik Krogh Kristensen ee858d840e get ReDoSUtil in sync for ruby 2021-11-18 16:49:34 +01:00
Erik Krogh Kristensen 1cca377e7d
Merge pull request #6561 from erik-krogh/htmlReg
JS/Py/Ruby: add a bad-tag-filter query
2021-11-18 09:39:13 +01:00
Taus eed98bd76a
Merge pull request #5588 from jorgectf/jorgectf/python/jwt-queries
Python: Add JWT security-related queries
2021-11-16 15:40:45 +01:00
jorgectf 9ad8a85f4d Delete redundant checks in `verifiesSignature()` 2021-11-16 15:08:18 +01:00
Anders Schack-Mulligen c70d384d28
Merge pull request #7045 from aschackmull/dataflow/hidden-ret-subpaths
Data flow: Support hidden return nodes in subpaths predicate
2021-11-16 15:04:51 +01:00
jorgectf 3fe2a08376 Update `.expected` file 2021-11-16 15:03:49 +01:00
Jorge a722631278
Apply suggestions from code review
Co-authored-by: Taus <tausbn@github.com>
2021-11-16 15:02:06 +01:00
Rasmus Wriedt Larsen 98e6fc8a88 Python/Ruby: Remove owasp tags
These are no longer correct, since the A1 category changed from 2017 to
2021, see https://owasp.org/Top10/#whats-changed-in-the-top-10-for-2021

Since only a very few queries had these tags, I think we're much better
off having them removed.
2021-11-16 12:03:50 +01:00
Rasmus Wriedt Larsen a980f26fda Python: Model `os.stat` (and friends) 2021-11-16 10:45:32 +01:00
Rasmus Wriedt Larsen 9f4107d211 Python: Model `posixpath`, `ntpath`, and `genericpath` modules 2021-11-16 10:45:14 +01:00
Rasmus Wriedt Larsen 6b7abacc5f
Merge pull request #7135 from RasmusWL/b32hexencode
Python: Model `b32hexencode`/`b32hexdecode`
2021-11-15 15:51:46 +01:00
Rasmus Wriedt Larsen 39927fa613 Python: Model `b32hexencode`/`b32hexdecode`
New in Python 3.10

See
- https://devdocs.io/python~3.10/library/base64#base64.b32hexencode
- https://devdocs.io/python~3.10/library/base64#base64.b32hexdecode
2021-11-15 15:23:49 +01:00
Rasmus Wriedt Larsen cfdfcaa3e8 Python: Support `Path.hardlink_to` (new in 3.10)
See https://docs.python.org/3.10/library/pathlib.html#pathlib.Path.hardlink_to
2021-11-15 14:57:59 +01:00
Rasmus Wriedt Larsen 5d60975f65 Python: Support `aiter` and `anext` (new in 3.10)
See
- https://docs.python.org/3/whatsnew/3.10.html#other-language-changes
- https://docs.python.org/3.10/library/functions.html#aiter
- https://docs.python.org/3.10/library/functions.html#anext
2021-11-15 14:55:34 +01:00
Rasmus Wriedt Larsen 7c3b68b7f8
Merge pull request #7091 from RasmusWL/port-request-without-validation
Python: Port `py/request-without-cert-validation` to use API graphs
2021-11-15 13:51:57 +01:00
Rasmus Wriedt Larsen 9e097f5430 Python: Improve PoorMansFunctionResolution 2021-11-15 13:40:19 +01:00
Rasmus Wriedt Larsen 0d4cb1e6ce Python: Add test of PoorMansFunctionResolution 2021-11-15 13:34:39 +01:00
Rasmus Wriedt Larsen 6eb4525ab2 Python: Model `wsgiref.simple_server` applications 2021-11-15 13:34:39 +01:00
Rasmus Wriedt Larsen e812029c03 Python: Add test for `wsgiref.simple_server` 2021-11-15 13:34:38 +01:00
Taus c17560f948
Merge pull request #7096 from tausbn/python-fix-more-bad-joins
Python: Fix a bunch of performance issues
2021-11-15 12:10:27 +01:00
yoff 5beb681580
Merge pull request #7087 from RasmusWL/path-injection-fp
Python: Add interesting path-injection FP
2021-11-12 15:20:19 +01:00
yoff 9f614b1d98
Merge pull request #7016 from RasmusWL/django-rest-framework
Python: Model Django REST framework
2021-11-12 14:27:56 +01:00
Rasmus Wriedt Larsen b11d11c0c9 Python: Add change-note 2021-11-12 14:27:01 +01:00
Rasmus Wriedt Larsen 491f72bb2a Python: Adjust generated code to be more familiar 2021-11-12 13:30:03 +01:00
Rasmus Wriedt Larsen de69e4c645 Python: Expand on SubclassFinder implementation note 2021-11-12 13:29:03 +01:00
Rasmus Wriedt Larsen f7b53321b9 Python: Remove copy-pasted comment 2021-11-12 13:19:20 +01:00
Taus 55ea715ce9
Merge pull request #7033 from RasmusWL/flask-admin 2021-11-12 12:18:56 +01:00
Rasmus Wriedt Larsen 860b1a5cc3 Python: Other minor QLDoc adjustment 2021-11-12 11:46:45 +01:00
Rasmus Wriedt Larsen 99081ea7e0
Python: Minor adjustment in QLDoc 2021-11-12 11:42:36 +01:00
Rasmus Wriedt Larsen 5e4b866f2b Python: Model `rest_framework.exceptions.APIException` 2021-11-12 11:37:54 +01:00
Rasmus Wriedt Larsen 62e58b534c Python: SubclassFinder: reorder + comment 2021-11-12 11:11:13 +01:00
Rasmus Wriedt Larsen f48ecb1dc8
Python: Apply suggestions from code review
Co-authored-by: yoff <lerchedahl@gmail.com>
2021-11-12 10:57:56 +01:00
Rasmus Wriedt Larsen 06cae3dac2
Merge pull request #7104 from yoff/python/model-aiomysql
Python: model aiomysql
2021-11-11 16:58:01 +01:00
Rasmus Lerchedahl Petersen e2a2a42d59 Python: Fix api references 2021-11-11 13:20:57 +01:00
Erik Krogh Kristensen b513033e0f
Merge pull request #7021 from erik-krogh/cwe326
JS: Add insufficient key size query
2021-11-11 12:17:04 +01:00
Anders Schack-Mulligen 7ffd9b4f9e Dataflow: Include read/store steps when finding non-hidden return. 2021-11-11 11:26:21 +01:00
Anders Schack-Mulligen 6d9fb3ca43 Dataflow: Sync. 2021-11-10 15:11:13 +01:00
yoff d23a920ed4
Merge branch 'main' into python/model-aiomysql 2021-11-10 14:32:36 +01:00
Rasmus Lerchedahl Petersen 57e7bfbdba Python: model aiomysql 2021-11-10 14:29:39 +01:00
Rasmus Lerchedahl Petersen 047cff0749 Python: test `aiomysql` 2021-11-10 14:24:45 +01:00
Rasmus Wriedt Larsen de926dc2a1
Merge pull request #7085 from yoff/python/model-aiopg
Python: model aiopg
2021-11-10 13:10:30 +01:00
Rasmus Wriedt Larsen 1f90dcadf1 Python: Use backtracker for verify arg 2021-11-10 12:54:23 +01:00
Rasmus Lerchedahl Petersen 92a7114b72 Python: Add API references 2021-11-10 11:06:58 +01:00
Rasmus Lerchedahl Petersen c6d285dd2a Python: Fix test 2021-11-10 11:06:45 +01:00
yoff a856395d56
Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-11-10 10:51:40 +01:00
Rasmus Wriedt Larsen 77099fe9d0 Python: Always use `@id py/`
The two queries in CWE-020 are used for manual evaluation (is my
understanding), and the two IDE queries should work based on their tags,
and not on the query-id.
2021-11-10 10:40:17 +01:00
Taus 33135e909a Python: Add magic to `named_argument_transfer`
This predicate was materialised as a _big_, _cached_ relation:

```
(169s) Tuple counts for PointsTo::InterProceduralPointsTo::named_argument_transfer#ffff#join_rhs/4@38ce07 after 53.4s:
25212     ~4%     {3} r1 = SCAN Function::Function::getArgByName_dispred#fff OUTPUT In.1, In.0 'arg1', In.2 'arg2'
159751200 ~0%     {4} r2 = JOIN r1 WITH Flow::CallNode::getArgByName_dispred#fff_102#join_rhs ON FIRST 1 OUTPUT Rhs.1 'arg0', Lhs.1 'arg1', Lhs.2 'arg2', Rhs.2 'arg3'
                  return r2
```

... However it's only used in a single place (where it is immediately
joined with the points-to relation to relate the caller and argument),
none of these joins were ever larger than 2000 tuples. This made it
pretty clear that we could gain something by pushing in that points-to
join as a bit of manual magic.

However, doing so didn't actually fix anything, since the join-orderer
then decided to join `func.getArgByName(name)` with
`call.getArgByName(name)` on `name` as the first thing (which caused a
join of the same size as above).

Unbinding didn't work, since `name` would then be an unbound `string`,
so instead I factored out relating the function, parameter, and name
thereof into its own predicate. (I could also have done this with the
call, but I would expect there to be more calls than function
definitions in general.)

Overall, this resulted in going from

```
(709s)
Definitions.ql-7:PointsTo::InterProceduralPointsTo::named_argument_transfer#ffff#join_rhs ......... 53.5s
Definitions.ql-7:Instances::InstanceObject::initializer_dispred#fbf ............................... 35.3s (456 evaluations with max 136ms in Instances::InstanceObject::initializer_dispred#fbf/3@i110#0508e8)
Definitions.ql-10:DefinitionTracking::jump_to_defn_attribute#fbf .................................. 27s (100 evaluations with max 12.8s in DefinitionTracking::jump_to_defn_attribute#fbf/3@i1#fc1f7x)
Definitions.ql-7:PointsTo::PointsToInternal::pointsTo#ffff ........................................ 16.1s (681 evaluations with max 2.5s in PointsTo::PointsToInternal::pointsTo#ffff/4@i4#0508eg)
Definitions.ql-7:Constants::ConstantObjectInternal::attribute#ffff ................................ 13.4s (505 evaluations with max 50ms in Constants::ConstantObjectInternal::attribute#ffff/4@i153#0508e5)
Definitions.ql-10:DefinitionTracking::assignment_jump_to_defn_attribute#fbf ....................... 12.4s (99 evaluations with max 11.8s in DefinitionTracking::assignment_jump_to_defn_attribute#fbf/3@i2#fc1f
7z)
...
```

to

```
(668s)
Definitions.ql-7:Instances::InstanceObject::initializer_dispred#fbf ................... 35.4s (456 evaluations with max 140ms in Instances::InstanceObject::initializer_dispred#fbf/3@i110#bf4328)
Definitions.ql-10:DefinitionTracking::jump_to_defn_attribute#fbf ...................... 27.4s (100 evaluations with max 13.3s in DefinitionTracking::jump_to_defn_attribute#fbf/3@i1#679d7x)
Definitions.ql-7:PointsTo::PointsToInternal::pointsTo#ffff ............................ 16.1s (681 evaluations with max 2.5s in PointsTo::PointsToInternal::pointsTo#ffff/4@i4#bf432g)
Definitions.ql-7:Constants::ConstantObjectInternal::attribute#ffff .................... 14.4s (505 evaluations with max 51ms in Constants::ConstantObjectInternal::attribute#ffff/4@i140#bf4325)
Definitions.ql-10:DefinitionTracking::assignment_jump_to_defn_attribute#fbf ........... 12.3s (99 evaluations with max 11.7s in DefinitionTracking::assignment_jump_to_defn_attribute#fbf/3@i2#679d
7z)
...
```
2021-11-09 21:39:32 +00:00
Taus e2f79d8516 Python: Fix several bad `getScope` joins
It seems the optimiser has started getting the wrong end of the stick
whenever we write `foo.getScope() = bar.getScope()` for some expressions
`foo` and `bar`.

This lead to things like

```
(196s) Tuple counts for Definitions::ModuleVariable::global_variable_callnode#ff/2@5ab278 after 2m33s:
2952757013 ~0%     {2} r1 = JOIN Definitions::ModuleVariable::global_variable_callnode#ff#shared WITH Variables::Variable::getScope_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'this', Lhs.1 'result'
495693     ~0%     {2} r2 = JOIN r1 WITH Variables::GlobalVariable#class#f ON FIRST 1 OUTPUT Lhs.0 'this', Lhs.1 'result'
453589     ~0%     {2} r3 = JOIN r2 WITH Definitions::ModuleVariable#f ON FIRST 1 OUTPUT Lhs.0 'this', Lhs.1 'result'
                   return r3
```

and

```
(315s) Tuple counts for Definitions::SsaSourceVariable::getAUse_dispred#ff/2@a39328 after 1m57s:
...
1785275    ~3%       {2} r24 = Definitions::ModuleVariable::global_variable_callnode#ff#shared UNION Definitions::SsaSourceVariable::getAUse_dispred#ff#shared
3008614987 ~0%       {2} r25 = JOIN r24 WITH Variables::Variable::getScope_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'this', Lhs.1 'result'
127        ~1%       {2} r26 = JOIN r25 WITH Definitions::NonLocalVariable#class#f ON FIRST 1 OUTPUT Lhs.0 'this', Lhs.1 'result'
127        ~1%       {2} r27 = JOIN r26 WITH Variables::LocalVariable#f ON FIRST 1 OUTPUT Lhs.0 'this', Lhs.1 'result'
...
```

(Note the timings: 2m33s and 1m57s.)

Now we have the much more reasonable

```
(38s) Tuple counts for Definitions::ModuleVariable::global_variable_callnode#ff/2@c53031 after 42ms:
453589 ~0%     {2} r1 = JOIN Definitions::ModuleVariable::global_variable_callnode#ff#shared WITH Definitions::ModuleVariable::scope_as_global_variable#ff_10#join_rhs ON FIRST 1 OUTPUT Rhs.1 'this', Lhs.1 'result'
               return r1
```

and

```
(46s) Tuple counts for Definitions::SsaSourceVariable::getAUse_dispred#ff/2@4b19de after 375ms:
...
```
2021-11-09 20:54:41 +00:00
Taus 24000a50e6 Python: Fix bad join in `py/redundant-assignment`
A minor thing, but still best avoided:
```
Tuple counts for RedundantAssignment::pyflakes_commented_line#bb/2@0477c7 after 14.2s:
160826   ~0%     {2} r1 = SCAN py_comments OUTPUT In.2, In.1
160826   ~0%     {3} r2 = JOIN r1 WITH project#Files::Location::hasLocationInfo_dispred#bfffff ON FIRST 1 OUTPUT Rhs.1, Lhs.1, Rhs.2 'result'
160826   ~3%     {3} r3 = JOIN r2 WITH files_10#join_rhs ON FIRST 1 OUTPUT Lhs.2 'result', Lhs.1, Rhs.1 'file'
46672419 ~4%     {4} r4 = JOIN r3 WITH Files::Location::getStartLine_dispred#bf_10#join_rhs ON FIRST 1 OUTPUT Rhs.1, Lhs.2 'file', Lhs.1, Lhs.0 'result'
3951     ~0%     {4} r5 = JOIN r4 WITH Files::Location::getFile_dispred#bf ON FIRST 2 OUTPUT Lhs.2, Lhs.3 'result', Lhs.1 'file', toLower(Lhs.2)
0        ~0%     {4} r6 = SELECT r5 ON In.3 contains "pyflakes"
0        ~0%     {2} r7 = SCAN r6 OUTPUT In.2 'file', In.1 'result'
                return r7
```
2021-11-09 20:31:44 +00:00
Rasmus Wriedt Larsen 985cd1ebdb Python: Port `py/request-without-cert-validation` to use API graphs 2021-11-09 16:37:50 +01:00
Rasmus Wriedt Larsen 59581690fd Python: Add `py/request-without-cert-validation` tests 2021-11-09 16:29:57 +01:00
Rasmus Wriedt Larsen 9710aeecbf Python/C#: Add CWE-1333 to redos queries
As is already done in JS and Ruby.
2021-11-09 16:10:38 +01:00
Rasmus Wriedt Larsen f70e4fea55 Python: Add interesting path-injection FP 2021-11-09 14:53:32 +01:00
Rasmus Wriedt Larsen 1e31416049
Merge pull request #7031 from yoff/python/taint-through-with
Python: Taint through `async with`
2021-11-09 14:08:07 +01:00
Rasmus Lerchedahl Petersen ac5a46f24f Python: split test as suggested in review 2021-11-09 13:04:52 +01:00
yoff 5f4aad40c1
Update python/ql/test/experimental/meta/InlineTaintTest.qll
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-11-09 13:00:35 +01:00
Rasmus Lerchedahl Petersen aa1541a5c3 Python: add changenote 2021-11-09 12:57:36 +01:00
Rasmus Lerchedahl Petersen a58c47b07b Python: model `aiopg.sa` 2021-11-09 12:49:57 +01:00
Rasmus Lerchedahl Petersen f53314019a Python: test `aiopg.sa` 2021-11-09 12:42:03 +01:00
Rasmus Lerchedahl Petersen cd332a75fc Python: model aiopg 2021-11-09 12:32:21 +01:00
Rasmus Lerchedahl Petersen cb8f1b4593 Python: Add tests for aiopg 2021-11-09 11:49:31 +01:00
Erik Krogh Kristensen 8727060ca7
add comment about modes of operation
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-11-09 11:15:12 +01:00
Rasmus Lerchedahl Petersen 3f4c2ba24e Python: Support debugging inline taint tests
The module `Conf` is created so that it can be imported
without importing the query predicates from the same file.
2021-11-08 14:08:11 +01:00
Rasmus Lerchedahl Petersen 624b794980 Python: separate taint sources in `with` 2021-11-04 17:06:36 +01:00
Rasmus Wriedt Larsen 9e2bc41648 Python: Improve `hashlib.new` modeling
By using a backwards type-tracker to find possible hashing algorithm
names.
2021-11-04 15:36:32 +01:00
Rasmus Wriedt Larsen 9e91f3a341 Python: Highlight shortcomings of hashlib.new modeling 2021-11-04 15:29:40 +01:00
Erik Krogh Kristensen a19627c72f optionally ignore everything after a dash 2021-11-04 13:19:44 +01:00
Erik Krogh Kristensen 02f500b9c2 Merge branch 'main' into htmlReg 2021-11-04 12:58:42 +01:00
Erik Krogh Kristensen 523c15cd72 don't include mode-of-operation into the algorithm names 2021-11-03 14:54:50 +01:00
Rasmus Wriedt Larsen 84b38b6c32 Python: Add test with custom django json response (FP) 2021-11-03 14:17:08 +01:00
Rasmus Lerchedahl Petersen 05aa314ac9 Python: Add tests for non-async constructs 2021-11-03 10:54:36 +01:00
Mathias Vorreiter Pedersen 4a2894a707
Merge pull request #7025 from MathiasVP/nomagic-parameterCand
Dataflow: Replace a 'noinline' pragma with a 'nomagic' pragma
2021-11-02 20:40:44 +00:00
Rasmus Wriedt Larsen 8cd9fdebf9 Python: Model `flask_admin` 2021-11-02 15:43:13 +01:00
Rasmus Wriedt Larsen ab88d945e2 Python: Add `flask_admin` tests 2021-11-02 15:41:57 +01:00
Rasmus Wriedt Larsen c2632cff3d Python: Add RequestHandler meta query 2021-11-02 15:41:57 +01:00
Rasmus Lerchedahl Petersen 768932d7b3 Python: Add tainttracking step that was removed
when the correpsonding datadlow step was removed.
2021-11-02 15:01:47 +01:00
Rasmus Lerchedahl Petersen 07d5086b07 Python: support user defined taint source 2021-11-02 15:00:23 +01:00
Erik Krogh Kristensen 5975e19f53 sync identical files 2021-11-02 14:45:33 +01:00
yoff 97625d7c2c
Merge pull request #7023 from RasmusWL/toml
Python: Add modeling of `toml`
2021-11-02 14:42:06 +01:00
Rasmus Wriedt Larsen cb6bcada4c Merge branch 'main' into django-rest-framework 2021-11-02 14:33:16 +01:00
yoff 0240631510
Merge pull request #6782 from RasmusWL/fastapi
Python: Model FastAPI
2021-11-02 14:16:12 +01:00
Rasmus Wriedt Larsen c52e453342 Python: Minor rewrite 2021-11-02 13:37:50 +01:00
Anders Schack-Mulligen 7d0152f3c0
Merge pull request #6932 from aschackmull/dataflow/flow-features
Dataflow: Add support for call context restrictions on sources/sinks.
2021-11-02 13:24:17 +01:00
Mathias Vorreiter Pedersen 6f4107ff23 Dataflow: Replace a 'noinline' pragma with a 'nomagic' pragma. 2021-11-02 11:37:40 +00:00
Rasmus Wriedt Larsen 8ee804a8c2 Python: Add `toml` modeling 2021-11-02 11:57:15 +01:00
Rasmus Wriedt Larsen 14bc297946 Python: Add `toml` encode/decode test 2021-11-02 11:57:06 +01:00
Tom Hvitved 302373d154
Merge pull request #6858 from hvitved/python/type-tracker-changes
Python: Type tracker changes
2021-11-02 11:47:01 +01:00
CodeQL CI d5e2026a26
Merge pull request #6934 from erik-krogh/more-instanceof
Approved by MathiasVP, esbena, yoff
2021-11-02 03:46:23 -07:00
CodeQL CI 5d62aa5b29
Merge pull request #6994 from erik-krogh/redundant-cast
Approved by RasmusWL, aschackmull, esbena, geoffw0, hvitved, nickrolfe
2021-11-02 03:45:48 -07:00
Tom Hvitved 1e64893742 Update python/ql/lib/semmle/python/dataflow/new/internal/TypeTracker.qll
Co-authored-by: Taus <tausbn@github.com>
2021-11-02 11:16:32 +01:00
Tom Hvitved 660398aa78 Python: Introduce `TypeBackTracker::getACompatibleTypeTracker()` 2021-11-02 11:16:32 +01:00
Tom Hvitved 73fd66cfed Python: Cache `TypeBackTracker::prepend` 2021-11-02 11:16:32 +01:00
Rasmus Wriedt Larsen 83389be8e2 Python: Add some missing QLDocs 2021-11-02 11:02:51 +01:00
Rasmus Wriedt Larsen 5c2734c643 Python: Fix experimental Django.qll 2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen fd12b144bc Python: Add change-note 2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen a7e4e5ef83 Python: Add rest_framework Response modeling 2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen 13815fe728 Python: Model known APIView subclasses
Added internal helper `.qll` file as well
2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen 62d30630aa Python: Add rest_framework Request taint modeling 2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen 5d77e62f3a Python: Add basic rest_framework Request modeling 2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen 57e13c6066 Python: `rest_framework.decorators.api_view` handling
Had to expose even more things, and had to make the `DjangoRouteHandler`
modeling more flexible so I could extend the char-pred in a different
file.
2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen 222db37c0d Python: Add initial rest_framework modeling
I had to make the Django and PrivateDjango modeling non-private :O
2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen a64e939d71 Python: Add note about `.method` 2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen 75e2555a8a Python: Add rest_framework taint tests 2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen 095f896f95 Python: Add examples of class/function based views 2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen 9bbf08ddcf Python: Add simple Django REST framework code 2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen 9d843153d4 Python: Set up test for Django REST framework
this is just pure Django project for now, (and very much a copy of the
one in `django-v2-v3`), to make it easier to see the changes needed to
set up Django REST framework.
2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen f1307b772a Python: Add RequestHandler meta query 2021-11-02 10:55:44 +01:00
Rasmus Wriedt Larsen b7b9120724 Python: Better handling of Pydantic models 2021-11-02 10:29:17 +01:00
Rasmus Wriedt Larsen c207580ed9 Python: Add extra FastAPI taint tests 2021-11-02 10:20:09 +01:00
Rasmus Wriedt Larsen 17da28118a Python: Small refactor to use `extends .. instanceof` 2021-11-02 10:06:11 +01:00
Erik Krogh Kristensen 0897b004eb revert removal of redundant inline casts in some python files 2021-10-29 14:40:27 +02:00
Erik Krogh Kristensen d36c66cfca remove redundant inline casts in arguments where the type is inferred by the call target 2021-10-29 14:37:56 +02:00
yoff 1c78c792ff
Merge pull request #6991 from RasmusWL/flask-blueprints
Python: Support `flask.blueprints.Blueprint`
2021-10-29 14:06:43 +02:00
Rasmus Wriedt Larsen 7e7c363e43
Python: Apply suggestions from code review
Co-authored-by: yoff <lerchedahl@gmail.com>
2021-10-29 13:59:36 +02:00
Rasmus Wriedt Larsen 85f00fda19
Merge pull request #6776 from yoff/python/model-asyncpg
Python: Model `asyncpg`
2021-10-29 13:54:44 +02:00
Anders Schack-Mulligen 3a1836c9f6
Merge pull request #7000 from aschackmull/dataflow/interface-refactor
Dataflow: Refactor public references to DataFlowCallable
2021-10-29 12:21:13 +02:00
Anders Schack-Mulligen 5951ae79b9 Dataflow: Add language specific predicates. 2021-10-29 11:11:35 +02:00
Anders Schack-Mulligen 00df6798b1 Dataflow: Sync 2021-10-29 11:00:23 +02:00
Erik Krogh Kristensen 6fffdf6101
Merge pull request #6855 from erik-krogh/secCookie
JS: Move cookie queries out of experimental.
2021-10-29 10:23:48 +02:00
Rasmus Lerchedahl Petersen 0f2f68bcbb Python: rename file 2021-10-28 19:14:02 +02:00
yoff 8f9741ae72
Update python/ql/lib/semmle/python/internal/Awaited.qll
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-10-28 19:13:08 +02:00
Erik Krogh Kristensen 15c90adec5 remove redundant cast where the type is enforced by an equality comparison 2021-10-28 18:08:20 +02:00
jorgectf 47b14f1adc Polish `Concepts.qll` qldocs 2021-10-28 17:55:34 +02:00
jorgectf b3ec82cd36 Merge branch 'jorgectf/python/jwt-queries' of https://github.com/jorgectf/codeql into jorgectf/python/jwt-queries 2021-10-28 17:40:33 +02:00
jorgectf a6c285ad32 Apply `getItem(_)` and extend `verifiesSignature` readability 2021-10-28 17:40:27 +02:00
Jorge f4d63cc5e7
Apply suggestions from code review
Co-authored-by: Taus <tausbn@github.com>
2021-10-28 17:34:11 +02:00
jorgectf ef4a27ff8c Apply code review suggestions 2021-10-28 17:31:52 +02:00
Erik Krogh Kristensen e75448ebb0 remove redundant inline casts 2021-10-28 16:35:53 +02:00
Rasmus Lerchedahl Petersen 8c72cc0cdd Python: update change note 2021-10-28 14:53:46 +02:00
Rasmus Lerchedahl Petersen 7201b3e116 Python: add changenote 2021-10-28 14:48:48 +02:00
Rasmus Lerchedahl Petersen 8536f5f5a2 Python: remember to update refs... 2021-10-28 14:32:53 +02:00
Rasmus Lerchedahl Petersen c92249525b Python: update test expectations 2021-10-28 14:03:09 +02:00
Rasmus Wriedt Larsen a33a8fd518 Python: Support `flask.blueprints.Blueprint`
Thanks to @haby0 who originally proposed this as part of
https://github.com/github/codeql/pull/6977
2021-10-28 14:02:03 +02:00
Rasmus Lerchedahl Petersen 3abe3e43d0 Python: autoformat 2021-10-28 13:58:01 +02:00
Rasmus Wriedt Larsen 0acf6aaec8 Python: Add change-note 2021-10-28 13:45:34 +02:00
Rasmus Wriedt Larsen 8c3349f40f Python: Properly model `flask.send_from_directory`
To not include `filename` as path-injection sink.
2021-10-28 13:41:39 +02:00
Rasmus Wriedt Larsen 228e9e973a Python: Minor flask refactor 2021-10-28 13:36:03 +02:00
Rasmus Wriedt Larsen 6648a695eb Python: Add flask specific path-injection test 2021-10-28 13:34:18 +02:00
Rasmus Lerchedahl Petersen b3ba75a00f Python: Fix tests by managing local sources
`API::Node::getAwaited` is restriced to local sources
2021-10-28 13:22:59 +02:00
jorgectf 3dec222922 Merge remote-tracking branch 'origin/main' into jorgectf/python/jwt-queries 2021-10-28 13:11:46 +02:00
jorgectf 7069f45864 Polish documentation 2021-10-28 13:09:28 +02:00
Rasmus Wriedt Larsen 436152a46d Python: Refactor flask file sending tests 2021-10-28 12:37:07 +02:00
Rasmus Wriedt Larsen 6d09334cba
Merge pull request #6330 from porcupineyhairs/pyPathTraversal
Python : Add Flask sinks for path injection query
2021-10-28 11:39:40 +02:00