github
/
codeql
зеркало из https://github.com/github/codeql.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
71 736 коммитов 1 531 ветка 130 Теги 460 MiB
Граф коммитов

4146 Коммитов

Автор SHA1 Сообщение Дата
Mathias Vorreiter Pedersen ae288332cc
Merge pull request #9511 from MathiasVP/fix-completion-for-patterns 
Swift: Fix completion for "naked" patterns
 2022-06-13 10:56:07 +02:00
Mathias Vorreiter Pedersen 32ef535ad7
Merge pull request #9430 from github/redsun82/swift-unknown-location 
Swift: add `UnknownLocation`
 2022-06-13 10:55:53 +02:00
Paolo Tranquilli 28d2fd7a22
Merge pull request #9416 from github/redsun82/swift-codegen-docs 
Swift: document and partially simplify codegen
 2022-06-13 10:50:16 +02:00
Mathias Vorreiter Pedersen 9d7afab185 Swift: Fix completion for patterns that occur outside 'CaseLabelItems'. Previously we'd add an unnecessary 'no-match' completion to these patterns even though they were always matching. This caused some confusing output in the graph tests in particular. 2022-06-13 09:24:18 +01:00
Robert Marsh 97815bfa61 Swift: fix implicit this usage 2022-06-09 18:50:40 +00:00
Robert Marsh 755c56dafe Swift: autoformat 2022-06-09 18:50:39 +00:00
Robert Marsh fca1afa493 Swift: fix inout parameter conflation at return 2022-06-09 17:09:49 +00:00
Robert Marsh 8d4830cd23 Swift: make dataflow test a path-problem 2022-06-09 17:09:49 +00:00
Robert Marsh a7663adf90 Swift: add flow through inout parameters 2022-06-09 17:09:49 +00:00
Robert Marsh 21ba73138d Swift: add CallExprCfgNode 2022-06-09 17:09:49 +00:00
Robert Marsh cb3da0eedd Swift: add some ParamDecl methods 2022-06-09 17:09:48 +00:00
Robert Marsh 776a2965ca Swift: extract isInOut for parameters 2022-06-09 17:09:48 +00:00
Paolo Tranquilli 8b52bb0c31 Swift: build x86_64 on arm64 macOS 
This is a temporary workaround. At a later stage we will add
* a Swift/LLVM prebuilt package for arm64
* universal binary for the extractor
 2022-06-08 10:25:38 +02:00
Paolo Tranquilli a0acb19b68
Swift: remove redundant import 2022-06-03 15:28:48 +02:00
Paolo Tranquilli 01e1c13c29 Swift: add `UnknownLocation` 
`getLocation()` will now exists for all entities. When there is no
valid location, the location will still not be emitted in the DB, but
on the QL side we will then assign a special `UnknownLocation` with
empty filename and 0 for line/column start/end.

This unknown location is currently emitted (with a unique `@` key) at
the start of every extraction, but we can move it elsewhere (and
possibly in a unique global trap file) at a later stage, possibly after
or when we rework the trap file strategy.

This should solve flakiness that was observed on the control flow tests,
which is probably caused by the `nodes` predicate in the `TestOutput`
class in `ControlFlowGraphImplShared.qll` not able to assign a proper
rank when the node does not have a location.
 2022-06-03 14:59:22 +02:00
AlexDenisov 0a861ff616
Merge pull request #9410 from github/alexdenisov/swift-extract-only-primary-files 
Swift: extract only primary files
 2022-06-03 10:44:18 +02:00
Alex Denisov 14384b7696 Swift: switch back to references 2022-06-03 10:22:25 +02:00
Paolo Tranquilli 87234938ef Swift: correct typo 2022-06-03 08:45:59 +02:00
Robert Marsh 43907b6a30
Merge pull request #9409 from github/alexdenisov/swift-fix-ctor-dtor-tostring 
Swift: fix ctor/dtor names in toString
 2022-06-02 10:34:03 -04:00
Paolo Tranquilli ede6bd8ffe Swift: document and partially simplify codegen 
This adds:
* a base `README.md` file to `codegen`
* module docstrings for the modules in `generators`
* help strings on all command line flags

Moreover some unneeded command line flags (`--namespace`,
`--include-dir` and `--trap-affix`) have been dropped.
 2022-06-02 15:28:57 +02:00
Paolo Tranquilli c9d109ddd3
Revert "Swift: Control-flow for key paths" 2022-06-02 14:55:52 +02:00
Mathias Vorreiter Pedersen 927aceb420
Merge pull request #9373 from MathiasVP/cfg-for-key-paths 2022-06-02 08:32:29 +01:00
Alex Denisov 6419af39f3 Swift: extract only primary files 
While the (removed) comment is correct and the frontend can be called in
different modes, both `swift build` and `xcodebuild` always use
`-primary-files` when compiling Swift projects.
The other mode was present only within our test runner (`qltest.sh`), so
removing it and doing what the official build systems do simplifies our
code base.

Additionally, file archival is now a separate function/operation.
 2022-06-02 08:57:03 +02:00
Alex Denisov 1dab01a31c Swift: fix ctor/dtor names in toString 2022-06-02 08:28:46 +02:00
Mathias Vorreiter Pedersen cde2880392 Swift: Add getEnclosingFunction to AstNode. 2022-06-01 16:40:08 +01:00
Paolo Tranquilli 77f7fe8dbc Swift: merge `codegen` and `cppcodegen` 
Python code was simplified, and now a `--generate` option can be used
to drive what can be generated.

The extractor pack creation now will use an internally generated
dbscheme. This should be the same as the checked in one, but doing so
allows `bazel run create-extractor-pack` and `bazel run codegen` to be
run independently from one another, while previously the former had to
follow the latter in case of a schema change. This is the change that
triggered the above simplification, as in order for the two dbscheme
files to be identical, the first `// generated` line had to state the
same generator script.
 2022-06-01 17:07:52 +02:00
Mathias Vorreiter Pedersen 7d962ac62b
Merge pull request #9397 from MathiasVP/use-autogenerated-parent-in-cfg-library 
Swift: Use the autogenerated `getParent`
 2022-06-01 14:45:43 +01:00
Mathias Vorreiter Pedersen eabb5c7137 Swift: Respond to PR comments. 2022-06-01 14:34:22 +01:00
Paolo Tranquilli 3414028b1b Swift: simplify `GetImmediateParent.qll` 2022-06-01 15:01:49 +02:00
Mathias Vorreiter Pedersen db0498e38c Swift: Use the autogenerated 'getParent' predicate in the CFG library. 2022-06-01 13:49:12 +01:00
Paolo Tranquilli 6b90b2b05f Swift: add children to IsPattern 2022-06-01 14:35:58 +02:00
Paolo Tranquilli a4f97dd67a Swift: add comment about unique in getImmediateParent 2022-06-01 14:32:59 +02:00
Paolo Tranquilli bc0a32c26e Swift: sort import list 
Also fix parent tests with updated `statements.swift` file.
 2022-06-01 14:32:59 +02:00
Paolo Tranquilli 3597efb728 Swift: rename to `getImmediateParent` and use hidden AST 2022-06-01 14:32:58 +02:00
Paolo Tranquilli a894ba64c4 Swift: make test run in Python 3.8 2022-06-01 14:32:58 +02:00
Paolo Tranquilli a86d0fc8a7 Swift: move `getAChild` to a separate module 2022-06-01 14:32:58 +02:00
Paolo Tranquilli 946e1f498a Swift: generate `getParent` implementation 
By explicitly marking children in the `schema.yml` file, an internal
`getAChild` predicate is implemented, that is in turn used in `AstNode`
to implement `getParent`.

This is yet to be used in the control flow library to replace the
hand-rolled implementation.

A further, more complex step is to use the same information to fully
generate the core implementation of `PrintAst` (including the
accessor string). This will be done later.

The `parent` tests use the same swift code as the extractor tests, and
this is currently enforced by `sync-files.py`. Notice that `qltest.sh`
had to be modified to deal with multiple files, which was not working
yet.
 2022-06-01 14:32:58 +02:00
Paolo Tranquilli 4d6c8da74b Swift: no perfect forwarding for label fetchers 
While we need universal references to catch different value categories,
we don't need perfect forwarding as `fetchLabel` does not behave
differently on lvalue and rvalues.
 2022-06-01 11:26:21 +02:00
Anders Schack-Mulligen 4f3751dfea
Merge pull request #9316 from hvitved/dataflow/edges-get-a-successor-consistency 
Data flow: Make `PathGraph::edges/2` and `PathNode::getASuccessor/1` consistent
 2022-06-01 10:38:25 +02:00
Paolo Tranquilli 6e44a12a67 Swift: make `enable_if_t` a non-type parameter 
This disallows completely calling the disabled function (which could
be done by explicitly providing `void` otherwise).
 2022-06-01 08:47:53 +02:00
Paolo Tranquilli 905cc23719 Swift: update comment 2022-06-01 08:41:43 +02:00
Paolo Tranquilli 19f16678ac Swift: change translation signature and detection 
Translation now takes const references to swift entities and return
trap entries (instead of taking apointer as an out parameter).
 2022-06-01 08:41:43 +02:00
Paolo Tranquilli c3cb0d6ad7 Swift: fix name extraction in VarDecl 2022-06-01 08:41:43 +02:00
Paolo Tranquilli 86b4f16b3a Swift: remove `Binding<>` class 
That class was meant to allow aggregate initialization of generated
C++ entries having the label `id` as first argument.

As aggregate initialization turned out to be undesirable (names of
fields are not explicit, and `{}` must be inserted for empty
superclasses), this commit removes it and disallows aggregate
initialization altogether by defining empty constructors for generated
classes.
 2022-06-01 08:41:43 +02:00
Paolo Tranquilli 9231013401 Swift: use C++ entry style visitor in `DeclVisitor` 2022-06-01 08:41:43 +02:00
Robert Marsh 42ec6350eb
Merge pull request #9349 from MathiasVP/fix-inconsistent-cfg 
Swift: Fix three CFG inconsistencies
 2022-05-31 14:38:08 -04:00
Mathias Vorreiter Pedersen e2ddfcd437
Merge pull request #9387 from github/geoffw0-patch-2 
Swift: Update readme
 2022-05-31 16:34:17 +01:00
Robert Marsh bd095abea4
Merge pull request #9388 from MathiasVP/cfg-for-yield 
Swift: CFG for `yield`
 2022-05-31 11:22:21 -04:00
Mathias Vorreiter Pedersen 6386daf44c Merge branch 'main' into fix-inconsistent-cfg 2022-05-31 15:59:53 +01:00
Robert Marsh 78fd0385fc
Merge pull request #9355 from MathiasVP/not-all-functions-throw 
Swift: Only construct exceptional edges for calls that may throw
 2022-05-31 10:56:31 -04:00
Mathias Vorreiter Pedersen 5f9d03f7c6 Swift: CFG for 'yield'. 2022-05-31 15:45:43 +01:00
Geoffrey White f598b26b03
Merge pull request #9384 from MathiasVP/qlpacks-for-swift 
Swift: Add qlpacks
 2022-05-31 15:39:20 +01:00
Mathias Vorreiter Pedersen a175f49759
Merge pull request #3 from geoffw0/swiftsrc 
Swift: Add swift-security-and-quality, swift-security-extended packs.
 2022-05-31 15:02:33 +01:00
Mathias Vorreiter Pedersen b5d229d4d8
Apply suggestions from code review 
Co-authored-by: Geoffrey White <40627776+geoffw0@users.noreply.github.com>
 2022-05-31 15:01:36 +01:00
Geoffrey White 01091ae1b9 Swift: Add codeql-suites. 2022-05-31 14:52:53 +01:00
Mathias Vorreiter Pedersen 1d120486b4 Swift: Extract 'yield' statements. 2022-05-31 14:43:09 +01:00
Mathias Vorreiter Pedersen 9af31bab2a Swift: Add qlpacks. 2022-05-31 13:59:44 +01:00
Geoffrey White ce3488b1b5
Update Swift readme 2022-05-31 10:56:28 +01:00
Geoffrey White 27968b60a5
Added Bazel install step. 2022-05-31 10:25:42 +01:00
Henry Mercer b1faba9880
Merge pull request #9359 from github/henrymercer/migrate-to-codeql-workspace 
Migrate `.codeqlmanifest.json` to `codeql-workspace.yml`
 2022-05-30 18:20:11 +01:00
Mathias Vorreiter Pedersen b88fe1b2b4 Swift: Add test case and accept changes. 2022-05-30 17:05:06 +01:00
Mathias Vorreiter Pedersen eed42a4e14 Swift: Make a new scope for each KeyPath expression. 2022-05-30 17:05:06 +01:00
Henry Mercer ca764576be Swift: Update mention of manifest file in docs 2022-05-30 15:54:52 +01:00
Mathias Vorreiter Pedersen 9175354bbd Swift: Add test and accept changes. 2022-05-30 15:51:49 +01:00
Mathias Vorreiter Pedersen 52f0b0d8d8 Swift: Fix extraction of roots in 'KeyPathExpr'. 2022-05-30 15:46:54 +01:00
Mathias Vorreiter Pedersen 21527f66e1 Swift: Extract KeyPath applications and KeyPathDot expressions. 2022-05-30 15:46:18 +01:00
Mathias Vorreiter Pedersen d8916568b6 Merge branch 'main' into fix-inconsistent-cfg 2022-05-30 14:07:10 +01:00
Mathias Vorreiter Pedersen 9b17493b3b Merge branch 'main' into not-all-functions-throw 2022-05-30 14:03:29 +01:00
Mathias Vorreiter Pedersen ef31aec29e Swift: Autoformat. 2022-05-30 12:58:12 +01:00
Mathias Vorreiter Pedersen 425d66e454
Update swift/ql/lib/codeql/swift/dataflow/internal/DataFlowPrivate.qll 2022-05-30 12:52:48 +01:00
Mathias Vorreiter Pedersen 2106d48785 Swift: Add 'Argument.getIndex()' and use it in 'DataFlowDispatch'. 2022-05-30 12:51:29 +01:00
Mathias Vorreiter Pedersen 0d8a9458c6 Merge branch 'main' into rdmarsh2/swift/dataflow-global-flow 2022-05-30 12:46:06 +01:00
Mathias Vorreiter Pedersen 7c25063f7f Merge branch 'main' into swift-decls-in-cfg 2022-05-27 17:56:58 +01:00
Mathias Vorreiter Pedersen c734646099 Swift: Accept test changes. 2022-05-27 17:42:51 +01:00
Mathias Vorreiter Pedersen 6d5504412a Swift: Only create a 'ThrowCompletion' for functions that actually can throw. 2022-05-27 17:42:51 +01:00
Mathias Vorreiter Pedersen b9809b4219 Swift: CFG for opening existentials. 2022-05-27 17:29:22 +01:00
Mathias Vorreiter Pedersen 02c73d3c1c Swift: Fix implicit-this alert. 2022-05-27 17:25:56 +01:00
Mathias Vorreiter Pedersen 6815e731d2 Swift: Add test and accept output 2022-05-27 11:48:20 +01:00
Mathias Vorreiter Pedersen 45bbd24355 Swift: Extract whether a function type is throwing or async. 2022-05-27 11:48:01 +01:00
Mathias Vorreiter Pedersen 7d36bc1c73 Swift: Fix CFG for 'foreach' statements. 2022-05-27 10:01:12 +01:00
Mathias Vorreiter Pedersen 73c189957d Swift: Fix CFG inconsistencies with StmtConditions. 2022-05-27 09:58:36 +01:00
Mathias Vorreiter Pedersen 795c0110e5 Swift: Accept more test changes. 2022-05-27 09:17:52 +01:00
Mathias Vorreiter Pedersen 52a10c826a Merge branch 'main' into swift-to-string 2022-05-27 09:06:36 +01:00
Robert Marsh 32e4c496f6 Swift: Dataflow through Phi nodes 2022-05-26 17:55:25 +00:00
Robert Marsh 507fdef028 Swift: add more data flow tests 2022-05-26 17:33:12 +00:00
Robert Marsh 81ac648065 Swift: flow out of calls via return statements 2022-05-26 16:58:53 +00:00
Robert Marsh ae6d16a40f Swift: flow into callees via params 2022-05-26 16:53:42 +00:00
Robert Marsh 25c8b8141c Swift: add params to CFG 2022-05-26 16:48:24 +00:00
Robert Marsh 3213549a73
Merge pull request #9329 from MathiasVP/fixes-for-9291 
Swift: Fixups for #9291
 2022-05-26 07:25:24 -04:00
Mathias Vorreiter Pedersen df2c1972e9 Swift: Add CFG trees for local declarations and accept test changes. 2022-05-26 09:09:17 +01:00
Mathias Vorreiter Pedersen b715a6b63b Swift: Add test containing local declarations. 2022-05-26 09:06:13 +01:00
Mathias Vorreiter Pedersen c7cc8d2592 Swift: Fix copy-paste error. 2022-05-25 21:36:24 +01:00
Robert Marsh aa77ea6bef Swift: minimal tests for interprocedural flow 2022-05-25 19:24:34 +00:00
Robert Marsh 9f64622f31 Swift: data flow configurations working 2022-05-25 19:23:43 +00:00
Robert Marsh d326b3a91c Swift: global dataflow WIP 2022-05-25 18:54:47 +00:00
Robert Marsh bba3564187 Swift: adjust for changes in main 2022-05-25 18:52:47 +00:00
Robert Marsh 91b34d5e8f Swift: make LambdaCallKind a TODO 2022-05-25 18:26:38 +00:00
Robert Marsh 765e1e1115 Swift: autoformat 2022-05-25 18:26:38 +00:00
Robert Marsh cf22ade9f3 Swift: initial local data flow implementation 2022-05-25 18:26:37 +00:00
Robert Marsh 117a1ad2f4 Swift: DataFlow expr and parameter nodes 2022-05-25 18:26:37 +00:00
Mathias Vorreiter Pedersen fafdb016fa Swift: Fixup based on review comments in #9291. 2022-05-25 16:10:44 +01:00
Mathias Vorreiter Pedersen f17afa8a11 Swift: Accept test changes. 2022-05-25 16:01:42 +01:00
Mathias Vorreiter Pedersen dc2ba5b410 Swift: Implement better 'toString' overrides for all AST nodes. 2022-05-25 15:59:45 +01:00
Mathias Vorreiter Pedersen 80fad348bb Swift: Implement CFG for property reads, writes, and observers. 2022-05-25 13:46:14 +01:00
Mathias Vorreiter Pedersen 67cc1b503b Swift: Implement step 3 from the previous commit message. 2022-05-25 13:44:59 +01:00
Mathias Vorreiter Pedersen 1f4924f978 Swift: Create a custom "AST" version of the public CFG classes. This is 
necessary because the CFG library doesn't support the following
       two requirements simultaneously:
       1. Traverse AST classes by virtual dispatch
       2. Construct ControlFlowElements from non-AST classes

       Because the CFG trees derive from the a base type that must be a
       subtype of `ControlFlowElement`. So if we make `ControlFlowElement`
       an IPA type, we cannot write:
       ```
       class AssignTree extends PostOrderTree instanceof AssignExpr { ... }
       ```
       because `AssignExpr` is not a subtype of PostOrderTree (since
       PostOrderTree is now a subtype of the new IPA type).

       To fix this, Tom suggested the following (which is implemented in
       this PR):
       1. Create a copy of the CFG tree classes (i.e., Pre/PostOrderTree,
          LeafTree, etc.) and call them AstPreOrderTree/AstPostOrderTree,
          AstLeafTree, etc.
       2. For each tree AstTree from step 1, create a instance of the
          internal CFG library's appropriate class.
       3. In `ControlFlowGraphImpl`, proceed as normal with virtual
          dispatch using `instanceof`, but extend the AstTree classes
          from step 1 instead of the CFG's own tree classes.

       This works because each AstTree implements one of the CFG
       library's tree classes (as per step 2).
       This commit performs step 1 and 2. Step 3 will be the next commit.
 2022-05-25 13:39:48 +01:00
Mathias Vorreiter Pedersen ab268514a1 Swift: Create a custom IPA type for 'ControlFlowElement's and fixup various type annotations. 2022-05-25 13:39:48 +01:00
Tom Hvitved bcdef98392 Data flow: Sync files 2022-05-25 14:39:37 +02:00
AlexDenisov 8b131adeb1
Merge pull request #9283 from github/alexdenisov/swift-integration-tests 
Swift: add integration tests
 2022-05-25 10:04:08 +02:00
Michael Nebel 5f3a039c65 Swift: Sync changes to DataFlowImplCommon from PR #9024. 2022-05-25 08:05:22 +02:00
Robert Marsh 8cc509e5e9
Merge pull request #9275 from MathiasVP/swift-add-dataflow-lib 
Swift: Add shared dataflow library
 2022-05-24 15:11:42 -04:00
Alex Denisov fa09078976 Swift: do not keep trap files for tests 2022-05-24 11:48:45 +02:00
Alex Denisov 8e8da66325 Swift: share .gitignore across all tests 2022-05-24 11:48:06 +02:00
Mathias Vorreiter Pedersen 3e1a6a777e Swift: Accept test changes. 2022-05-24 10:22:06 +01:00
Mathias Vorreiter Pedersen dda60abfef Swift: Add CFG for a couple more expressions. 2022-05-24 10:21:04 +01:00
Alex Denisov 528f6f73c5 Swift: add integration tests 2022-05-24 11:12:35 +02:00
Mathias Vorreiter Pedersen 21641eb1cd Swift: Make the 'semantics.ql' test plantform-independent. 2022-05-24 10:00:14 +01:00
Mathias Vorreiter Pedersen a0659072b5 Swift: Add tests and accept test changes. 2022-05-24 09:00:34 +01:00
Mathias Vorreiter Pedersen cf5f1e593e Swift: Extract new entities. 2022-05-24 08:57:05 +01:00
Mathias Vorreiter Pedersen cdb081eaec Swift: Update schema and update generated files. 2022-05-24 08:57:05 +01:00
Mathias Vorreiter Pedersen f46fc34481 Swift: Add skeleton for shared dataflow library. 2022-05-23 18:03:47 +01:00
Mathias Vorreiter Pedersen b681a10bfe Swift: Add shared SSA library. 2022-05-23 18:01:43 +01:00
Mathias Vorreiter Pedersen 9b0d84c1a3
Merge pull request #9268 from MathiasVP/swift-add-cfg-library 
Swift: Extend AST classes and add control-flow library
 2022-05-23 16:37:51 +01:00
Mathias Vorreiter Pedersen 4ba29845e9 Swift: Fix Code Scanning alerts. 2022-05-23 15:18:36 +01:00
Mathias Vorreiter Pedersen 2882c42698 Swift: Sync identical files. 2022-05-23 13:13:26 +01:00
Mathias Vorreiter Pedersen e98728b788 Swift: Fix casing on import alias. 2022-05-23 13:08:09 +01:00
Mathias Vorreiter Pedersen 83bcb53199 Swift: Add tests accept test changes. 2022-05-23 13:05:55 +01:00
Mathias Vorreiter Pedersen 9f8fbd7aa7 Swift: Add control-flow library. 2022-05-23 12:59:06 +01:00
Mathias Vorreiter Pedersen 26f0d3ac43 Swift: Add helper predicates on AST classes 2022-05-23 12:51:51 +01:00
Paolo Tranquilli 06a8cf6f1e
Merge pull request #9198 from github/redsun82/swift-self-contained-cpp-code-gen 
Swift: make C++ code generation more self-contained
 2022-05-23 13:45:58 +02:00
Paolo Tranquilli 1e9fcfb338
Merge pull request #9265 from github/redsun82/swift-rm-codeqlmanifest 
Swift: remove `.codeqlmanifest`
 2022-05-23 13:00:58 +02:00
Paolo Tranquilli a3f6682bbb Swift: remove `.codeqlmanifest` 
The extractor pack entry in there has been moved to the root manifest.
 2022-05-23 12:49:08 +02:00
Paolo Tranquilli ea6a249fee Swift: fix extractor built with `NDEBUG` 
There was a call with side effects in an `assert`, that was therefore
not being called with `NDEBUG` turned on, changing extractor results.
 2022-05-23 12:35:54 +02:00
Paolo Tranquilli fc165c1975 Swift: remove IDE generated comment header 2022-05-20 16:25:33 +02:00
Paolo Tranquilli f52a849ab8
Merge branch 'main' into redsun82/swift-self-contained-cpp-code-gen 2022-05-20 16:16:35 +02:00
Paolo Tranquilli e6f2ab003c Swift: remove empty DB-CHECK.expected files 2022-05-20 16:01:56 +02:00
Paolo Tranquilli b0668ee6c2 Swift: remove unused `decl` properties 2022-05-20 15:42:28 +02:00
Paolo Tranquilli 553930d9e3 Swift: type visitor 
This transfers the current state of `TypeVisitor` from the
proof-of-concept.
 2022-05-20 15:42:20 +02:00
Paolo Tranquilli 922608c65a Swift: expression visitor 
This transfers the current status of `ExprVisitor` from the
proof-of-concept, together with some changes required for swift 5.6.
 2022-05-20 15:41:27 +02:00
Paolo Tranquilli 3f45b73d62 Swift: pattern visitor 
This transfers the current state of `PatternVisitor` from the
proof-of-concept.
 2022-05-20 15:41:27 +02:00
Paolo Tranquilli 19506dae74 Swift: statement visitor 
This transfers the current state of `StmtVisitor` in the PoC, plus some
changes required for the update to swift 5.6.

Also `getLabel` in `SwiftDispatcher` got renamed to `createLabel`, and
is now correctly outputting the label assignment to the trap file.
 2022-05-20 15:41:27 +02:00
Paolo Tranquilli d6ced16aa8 Swift: declaration visitor 
This transfers the current state of `DeclVisitor` from the
proof-of-concept.

TODO: make the `declarations` tests in `extractor-tests` more
comprehensive.
 2022-05-20 15:41:22 +02:00
Paolo Tranquilli 09967bfd42 Swift: add comment about CRTP 2022-05-20 12:35:58 +02:00
Paolo Tranquilli f5b2c31a3c Swift: rename `DispatcherWrapper` to `VisitorBase` 2022-05-20 12:25:45 +02:00
Paolo Tranquilli da00bf99a1 Swift: move TBD code to ql 
This allows to avoid bypassing label type correcness in the extractor,
and allows to independently resolve TBD extractions, as with this
approach TBD nodes do have the correctly typed trap label. The TBD
status is now a predicate on the QL side.

This requires:
* a default visit using the correct type, which is achieved via macro
  metaprogramming in `VisitorBase.h`, following the way
  `swift::ASTVisitor` is programmed
* a mapping from labels to corresponding binding trap entries. The
  functor is defined in `TrapTagTraits.h` and instantiated in generated
  `TrapEntries.h`
* Binding trap entries for TBD unknown entities must not have any other
  field than the `id` (after all, we are supposed to not extract them
  yet). This is why all unextracted fields in `schema.yml` have been
  commented out, and will be uncommentend when visitors are added
 2022-05-20 09:52:27 +02:00
Paolo Tranquilli b66f1b27b0
remove `pip install` mention from README.md 
It is not needed any more since pip requirements were coded in bazel.
 2022-05-19 12:47:20 +02:00
Paolo Tranquilli 3a46db3f81 Swift: make C++ code generation more self-contained 
This is solving a papercut, where the C++ build was relying on the
local dbscheme file to be up-to-date, even if all the information for
building is actually in `schema.yml`. This made a pure C++ development
cycle with changes to `schema.yml` clumsy, as it required a further
dbscheme generation step.

Now for C++ the dbscheme is generated internally in the build files, and
thus a change in `schema.yml` is reflected immediately in the C++ build.

A `swift/codegen` step for checked in generated code (including the
dbscheme) is still required, but a developer can do it just before
running QL tests or committing, instead of during each C++
recompilation.

Some directory reorganization was also carried out, moving specific
generator modules to a new `generators` python package, and only leaving
the two drivers at the top level.
 2022-05-17 17:05:16 +02:00
Paolo Tranquilli fbe7c5be81 Swift: move TBD code to ql 
This allows to avoid bypassing label type correcness in the extractor,
and allows to independently resolve TBD extractions, as with this
approach TBD nodes do have the correctly typed trap label. The TBD
status is now a predicate on the QL side.

This requires:
* a default visit using the correct type, which is achieved via macro
  metaprogramming in `VisitorBase.h`, following the way
  `swift::ASTVisitor` is programmed
* a mapping from labels to corresponding binding trap entries. The
  functor is defined in `TrapTagTraits.h` and instantiated in generated
  `TrapEntries.h`
* Binding trap entries for TBD unknown entities must not have any other
  field than the `id` (after all, we are supposed to not extract them
  yet). This is why all unextracted fields in `schema.yml` have been
  commented out, and will be uncommentend when visitors are added
 2022-05-17 16:31:10 +02:00
Cornelius Riemenschneider 3836d1550a Update Lua tracing configs. 2022-05-17 13:18:28 +00:00
Paolo Tranquilli 16e3b5bfc4 Swift: make `monostate` explicit 2022-05-16 15:51:43 +02:00
Paolo Tranquilli 1b9dcac2dd Swift: replace `getCanonicalPointer` with `std::variant` 
This turned out easier than expected previously. `llvm::PointerUnion`
was also considered, which would have less memory footprint, but it
would require more effort as it is lacking the same implicit conversions
and operators that `std::variant` provides.

Also renamed `ToTag<E>` to `TrapTagOf<E>` and introduced a derived
convenience functor `TrapLabelOf<E>`.
 2022-05-16 09:59:36 +02:00
Alex Denisov 1b75034634 Swift: simplify CRTP monkey-patching 2022-05-13 16:54:15 +02:00
Alex Denisov f857cd11c4 Swift: add comments about SwiftDispatcher lifetime 2022-05-13 16:47:45 +02:00
Alex Denisov acbe9ff9f9 Swift: introduce visitors 2022-05-13 16:26:41 +02:00
Alex Denisov 43199fa723 Swift: clarify getCanonicalPointer 2022-05-13 16:14:16 +02:00
Alex Denisov 35467bc252 Swift: rely on llvm::sys::fs::real_path to get absolute path 2022-05-13 16:13:30 +02:00
Alex Denisov 043b1b9c4a Swift: resolve symlinks 2022-05-13 15:20:30 +02:00
Alex Denisov d7f4c6fb0b Swift: add a comment about lifetime 2022-05-13 15:20:30 +02:00
Alex Denisov c92576690f Swift: change the return types of getCanonicalPointer 2022-05-13 15:20:30 +02:00
Alex Denisov 2f00945a23 Swift: change the return types of getCanonicalPointer 2022-05-13 15:20:30 +02:00
Alex Denisov 039aaec6b7 Swift: make TrapLabelStore store untyped label internally 2022-05-13 15:20:30 +02:00
Alex Denisov e584afb895 Swift: fix format 2022-05-13 15:20:30 +02:00
Alex Denisov 7b9f88637e Swift: describe TrapTagTraits API and implementation 2022-05-13 15:20:30 +02:00
Alex Denisov efa4565af2 Swift: move generated code to `generated` directory 2022-05-13 15:20:29 +02:00
Paolo Tranquilli f52119dc81
Merge branch 'main' into alexdenisov/introduce-dispatcher 2022-05-13 13:44:01 +02:00
Paolo Tranquilli 8cb9fd7eec Swift: publish C++ generated code as artifacts 2022-05-13 11:48:27 +02:00
Alex Denisov d0e2e2bec8 Swift: introduce SwiftDispatcher 2022-05-12 14:09:44 +02:00
Alex Denisov 8f8ece63e7 Swift: add extractor test for declarations 2022-05-12 14:09:44 +02:00
Alex Denisov cfd242e489 Swift: add human readable string representation for Location and UnkownAstNode 2022-05-12 14:09:44 +02:00
Paolo Tranquilli ddb567b639 Swift: remove `Tag` nested alias in `TrapLabel` 2022-05-11 17:44:00 +02:00
Paolo Tranquilli f1413f29c6 Swift: move back file opening code 2022-05-11 16:53:51 +02:00
Paolo Tranquilli a46582d7d5 Swift: replace `friend` in `TrapLabel` with `unsafeCreateFromExplicitId` 2022-05-11 14:42:55 +02:00
Paolo Tranquilli e679612a5a Swift: move most of TrapArena to TrapFile 2022-05-11 12:32:14 +02:00
Paolo Tranquilli e63d079322 Swift: transfer `TrapArena` 2022-05-11 11:28:38 +02:00
Paolo Tranquilli bf71e4c500 Swift: getPrimaryQlClass -> getAPrimaryQlClass 2022-05-10 12:42:18 +02:00
Paolo Tranquilli 0b9dc9703f Swift: changes required for TBD node rework 
These changes are required to allow a new type-safe approach to TBD
nodes, that will come in a separate commit.

This introduces:
* the possibility to add properties to the root `Element`
* a functor taking tags to the corresponding binding trap entry
* `hasProp()` methods for optional properties in QL
* `getPrimaryQlClass()` method
 2022-05-10 11:59:25 +02:00
Paolo Tranquilli c08e6fdc1e Swift codegen: add predicate properties 
Properties marked with `predicate` in the schema are now accepted.

* in the dbscheme, they will translate to a table with a single `id`
  column (and the table name will not be pluralized)
* in C++ classes, they will translate to `bool` fields
* in QL classes, they will translate to predicates

Closes https://github.com/github/codeql-c-team/issues/1016
 2022-05-09 17:50:49 +02:00
Paolo Tranquilli effa9ee207
Merge pull request #9034 from redsun82/swift-cpp-gen 
Swift: add structured C++ generated classes
 2022-05-09 17:49:23 +02:00
Paolo Tranquilli 93f8b6b29d Swift: add missing `trap_affix` 2022-05-09 12:20:22 +02:00
Paolo Tranquilli 20317a280b Swift: make `width` fields `unsigned` 2022-05-09 12:19:52 +02:00
Paolo Tranquilli 9c5b2d7e9d Swift: tweaks for use in the PoC branch 2022-05-09 09:46:47 +02:00
Paolo Tranquilli 918ba1b1fc Swift: make generator.run accept options 2022-05-09 09:34:49 +02:00
AlexDenisov c21849bb2e
Merge pull request #9015 from redsun82/swift-enable-dynamic-library 
Swift: enable dynamic mode
 2022-05-09 09:15:37 +02:00
Paolo Tranquilli 6cbfb5a10c Swift cppgen: emit final trap before bases 2022-05-09 09:02:20 +02:00
Paolo Tranquilli a7129c1f4c Swift: add `--ql-format`/`--no-ql-format` to `codegen` 2022-05-05 18:33:05 +02:00
Paolo Tranquilli b2b5fd281f Swift: add more parametrization 
This enables codegen to run on the swift PoC branch.
 2022-05-05 17:34:00 +02:00
Paolo Tranquilli ac3cceab19 Swift: turn some generated paths to relative 2022-05-05 16:15:16 +02:00
Paolo Tranquilli 7bcc5db4a6 Swift: parametrize namespace and other things in codegen 
This is so that we can use this in the PoC branch.
 2022-05-05 16:01:54 +02:00
Paolo Tranquilli c87fb4df53 Swift: remove now unused `ql.Property.params` 2022-05-05 12:01:13 +02:00
Paolo Tranquilli 9798d8ba26 Swift: add `?*` modifier to schema specification 
This indicates a list of optional entries. This is different than
simply repeatind entries because of the indexing.
 2022-05-05 11:50:12 +02:00
Paolo Tranquilli c2d3aac349 Swift: fix no functools.cache in python 3.8 2022-05-05 09:48:07 +02:00
Paolo Tranquilli d5d1eb717d Swift: add structured C++ generated classes 
This adds `cppgen`, creating structured C++ classes mirroring QL classes
out of `schema.yml`.

An example of generated code at the time of this commit can be found
[in this gist][1].

[1]: https://gist.github.com/redsun82/57304ddb487a8aa40eaa0caa695048fa

Closes https://github.com/github/codeql-c-team/issues/863
 2022-05-04 18:20:25 +02:00
Paolo Tranquilli 10c5c8e71f Swift: add `trapgen` unit tests 
Closes: https://github.com/github/codeql-c-team/issues/981
 2022-05-04 18:20:06 +02:00
Paolo Tranquilli c2be267feb Swift: enable dynamic mode 
Providing `--dynamic_mode=fully` (for example setting it in
`local.bazelrc`) will now work.

All runfiles are now copied in the extractor pack: in dynamic mode,
those will be the executable and the dynamic libraries, while in static
mode only the executable will be part of the runfiles.

Setting the correct `LD_LIBRARY_PATH` in `qltest.sh` then allows to
run tests with this pakcage. If we need something more, we can switch to
a wrapper script in place of `extractor` in the future.

Notice that `LD_LIBRARY_PATH` is also set in static mode, but that has
no consequence.
 2022-05-03 12:33:24 +02:00
AlexDenisov 5c6e5173ad
Merge pull request #8959 from AlexDenisov/alexdenisov/pip-install-from-bazel 
Swift: teach bazel to install python dependencies
 2022-04-29 14:31:37 +02:00
Paolo Tranquilli 2fe38c2bbb Swift: cc wrapper rules 2022-04-29 14:18:36 +02:00
Alex Denisov 7332460268 Swift: teach bazel to install python dependencies 2022-04-29 14:05:36 +02:00
AlexDenisov f6769735e5
Merge pull request #8939 from AlexDenisov/alexdenisov/swift-tracer-integration 
Swift: tracer integration
 2022-04-28 19:20:55 +02:00
AlexDenisov a59d7f6a85
Update swift/extractor/main.cpp 2022-04-28 16:52:34 +02:00
Paolo Tranquilli c4fae0806f Swift: use `#pragma once` 2022-04-28 16:39:27 +02:00
AlexDenisov 84bcc2e64a
Merge branch 'main' into alexdenisov/swift-tracer-integration 2022-04-28 16:28:48 +02:00
Alex Denisov 85918173a6 Swift: ensure the folder for trap files exists 2022-04-28 13:19:25 +02:00
Alex Denisov 9c73ae5a97 Swift: teach extractor to not produce artifacts 2022-04-28 13:18:20 +02:00
Alex Denisov 5b75b4db79 Swift: add tracer config 2022-04-28 13:17:35 +02:00
Alex Denisov 4a03976a15 Swift: set compiler flags explicitly 2022-04-28 13:17:05 +02:00
Paolo Tranquilli 773ef62406 Swift: added trapgen 
This checks in the trapgen script generating trap entries in C++.

The codegen suite has been slightly reorganized, moving the templates
directory up one level and chopping everything into smaller bazel
packages. Running tests is now done via
```
bazel run //swift/codegen/test
```

With respect to the PoC, the nested `codeql::trap` namespace has been
dropped in favour of a `Trap` prefix (or suffix in case of entries)
within the `codeql` namespace. Also, generated C++ code is not checked
in in git any more, and generated during build. Finally, labels get
printed in hex in the trap file.

`TrapLabel` is for the moment only default-constructible, so only one
single label is possible. `TrapArena`, that is responsible for creating
disjoint labels will come in a later commit.
 2022-04-28 12:01:59 +02:00
Paolo Tranquilli f95b5853c1
Merge pull request #8788 from AlexDenisov/alexdenisov/swift-first-extractor-test 
Swift: file extraction
 2022-04-27 17:47:17 +02:00
Mathias Vorreiter Pedersen abbb7f861f
Merge pull request #8904 from MathiasVP/sync-swift-schema 
Swift: Sync schema after extractor changes
 2022-04-27 16:14:46 +01:00
Alex Denisov 272aa594cc Swift: compiler options moved to .bazelrc 2022-04-27 17:11:16 +02:00
Mathias Vorreiter Pedersen 141e8fcd5b Swift: Sync schema. 2022-04-27 14:39:13 +01:00
Paolo Tranquilli 0100c7171d Swift: testing non-trivial dataclass properties 2022-04-27 10:17:49 +02:00
Paolo Tranquilli 7f0476049f Swift: removed spurious `mock` import 2022-04-27 09:11:14 +02:00
Paolo Tranquilli f171ce6341 Swift: add unit tests to code generation 
Tests can be run with
```
bazel test //swift/codegen:tests
```

Coverage can be checked installing `pytest-cov` and running
```
pytest --cov=swift/codegen swift/codegen/test
```
 2022-04-27 08:24:11 +02:00
Alex Denisov 5db18bb845 Swift: add a comment clarifying swift::FrontendObserver 2022-04-26 13:35:10 +02:00
Alex Denisov e2332fc5ec Swift: Replace SwiftExtractor class with a function 2022-04-26 13:32:14 +02:00
Alex Denisov ebd2ff4fc0 Swift: rename classes to reflect they belong to Swift 2022-04-26 11:33:35 +02:00
Alex Denisov 81e4f9165e Swift: remove -frontend option as we don't need it yet 2022-04-26 11:27:41 +02:00
Alex Denisov 5fc4fab38e Swift: add missing 'overrides' 2022-04-26 10:07:41 +02:00
Mathias Vorreiter Pedersen 8869038b4f Swift: Update schema.yml and regenerate files. 2022-04-25 16:15:37 +01:00
Alex Denisov 906ce34e2f Swift: generate QL files properly 2022-04-25 17:09:29 +02:00
Alex Denisov 355504a86a Swift: use File from QL library 2022-04-25 16:58:30 +02:00
Alex Denisov 6f0ddaa431 Merge branch 'main' into alexdenisov/swift-first-extractor-test 2022-04-25 16:53:22 +02:00
Paolo Tranquilli de0fa9e456 Swift: QL generation script 
Also added code generation to the swift checks.
 2022-04-25 13:23:36 +02:00
Alex Denisov 5b20d580be Swift: use parenthesis instead of curly braces 2022-04-25 12:26:34 +02:00
Alex Denisov 8bcdfb2e4f Swift: initialize LLVM 
No need to shutdown LLVM, it's done by the PROGRAM_START macro
 2022-04-25 11:49:21 +02:00
Alex Denisov 462133e0f0 Swift: add more comments 2022-04-25 11:48:51 +02:00
Alex Denisov aa13891667 Swift: regenerate dbscheme 2022-04-22 09:42:22 +02:00
Alex Denisov 2ce46a9b60 Swift: remove test table from dbscheme 2022-04-22 08:48:44 +02:00
Alex Denisov 62d36a29f7 Swift: Extract files 2022-04-22 08:48:38 +02:00
Alex Denisov e85cdf2ec3 Swift: rename certain dbscheme columns 2022-04-22 08:31:09 +02:00
AlexDenisov a5189eae9f
Merge pull request #8735 from redsun82/swift-dbscheme-gen 
Swift: dbscheme generator
 2022-04-22 08:26:59 +02:00
Alex Denisov 682c910d49 Swift: package test SDK 2022-04-20 12:35:19 +02:00
Paolo Tranquilli 24697feebc Swift: integrated template name in dataclass 2022-04-14 15:53:15 +02:00
Paolo Tranquilli 197ea5b8f3 Swift: use more @property in codegen 2022-04-14 12:28:52 +02:00
Paolo Tranquilli 71f9b25500 Swift: uses classes instead of Enum for Properties 2022-04-14 11:35:11 +02:00
Paolo Tranquilli 64496b4c97 Swift: cleanup and some docstrings for codegen 
Also added code generation and clang formatting to the pre-commit
configuration.
 2022-04-14 11:27:41 +02:00
Paolo Tranquilli 91fd83a554 Swift: dbscheme generator 
This patch introduces the basic infrastructure of the code generation
suite and the `dbscheme` generator.

Notice that the checked in `schema.yml` should reflect swift 5.6 but
might need some tweaking.

Closes https://github.com/github/codeql-c-team/issues/979
 2022-04-14 11:27:41 +02:00
Paolo Tranquilli aaf9e7da2f turn off universal_binaries for now 2022-04-13 16:45:23 +02:00
Paolo Tranquilli 73d5691d91 update swift package 2022-04-13 16:22:27 +02:00
Paolo Tranquilli e68172f4b0 Swift: fetch prebuilt swift and link against it 
This is known to break linux integration in sembuild.
 2022-04-13 16:22:27 +02:00
Paolo Tranquilli 8ef28787b6 Swift: do not fail pack creation if dir does not exist 2022-04-12 17:05:26 +02:00
Paolo Tranquilli 6440242268 Swift+Bazel: apply review comments 2022-04-12 16:03:20 +02:00
Paolo Tranquilli a205b465ba Bazel: reorganization 
* fixed 5.0.0 as bazel version
* made dependencies better loadable
* moved `//swift/install` to `//swift:create-extractor-pack` (following
  the clearer ruby naming)
* renamed `extractor_pack` to `extractor-pack` for consistency with Ruby
 2022-04-12 12:40:59 +02:00
Paolo Tranquilli 13b2442fed Bazel: code reorganization 2022-04-12 12:40:59 +02:00
Paolo Tranquilli 664d5ba0a9 Swift: moved install to a separate package 
When importing the workspace from semmle-code, we do not need nor want
to instantiate `@util`, so that must be in a separate bazel package.
 2022-04-12 12:40:59 +02:00
Paolo Tranquilli 95dbf2d666 Swift: first skeleton extractor 
This adds a first dummy extractor for swift.

Running `bazel run //swift:install` will create an `extractor_pack`
directory in `swift`. From that moment providing `--search-path=swift`
will pick up the extractor.
 2022-04-12 12:40:59 +02:00