microsoft
/
tolerant-php-parser
зеркало из https://github.com/microsoft/tolerant-php-parser.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

update api and add helper utilities to dogfood api

This commit is contained in:
Sara Itani 2017-01-16 11:44:39 -08:00
Родитель aca7aee178
Коммит 5b32227b9e
45 изменённых файлов: 550 добавлений и 56 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

207
ApiDocumentation.md Normal file
Просмотреть файл

@ -0,0 +1,207 @@
# API Documentation
> Note: This documentation was auto-generated using this parser to help dogfood the API. Please contribute fixes to
`tools/PrintApiDocumentation.php` and suggest API improvements.
<hr>
## Node
### Node::__construct
> TODO: add doc comment
```php
public function __construct ( int $kind )
```
### Node::getStart
Gets start position of Node, not including leading comments and whitespace.
```php
public function getStart ( ) : int
```
### Node::getFullStart
Gets start position of Node, including leading comments and whitespace
```php
public function getFullStart ( ) : int
```
### Node::getParent
Gets parent of current node (returns null if has no parent)
```php
public function getParent ( )
```
### Node::getAncestor
> TODO: add doc comment
```php
public function getAncestor ( $className )
```
### Node::getRoot
Gets root of the syntax tree (returns self if has no parents)
```php
public function & getRoot ( ) : Node
```
### Node::getDescendantNodesAndTokens
Gets generator containing all descendant Nodes and Tokens.
```php
public function getDescendantNodesAndTokens ( callable $shouldDescendIntoChildrenFn = null )
```
### Node::getDescendantNodes
Gets a generator containing all descendant Nodes.
```php
public function getDescendantNodes ( callable $shouldDescendIntoChildrenFn = null )
```
### Node::getDescendantTokens
Gets generator containing all descendant Tokens.
```php
public function & getDescendantTokens ( callable $shouldDescendIntoChildrenFn = null )
```
### Node::getChildNodesAndTokens
Gets generator containing all child Nodes and Tokens (direct descendants)
```php
public function getChildNodesAndTokens ( ) : \Generator
```
### Node::getChildNodes
Gets generator containing all child Nodes (direct descendants)
```php
public function & getChildNodes ( ) : \Generator
```
### Node::getChildTokens
Gets generator containing all child Tokens (direct descendants)
```php
public function getChildTokens ( )
```
### Node::getWidth
Gets width of a Node (not including comment / whitespace trivia)
```php
public function getWidth ( ) : int
```
### Node::getFullWidth
Gets width of a Node (including comment / whitespace trivia)
```php
public function getFullWidth ( ) : int
```
### Node::getText
Gets string representing Node text (not including leading comment + whitespace trivia)
```php
public function getText ( ) : string
```
### Node::getFullText
Gets full text of Node (including leading comment + whitespace trivia)
```php
public function getFullText ( ) : string
```
### Node::getLeadingCommentAndWhitespaceText
Gets string representing Node's leading comment and whitespace text.
```php
public function getLeadingCommentAndWhitespaceText ( ) : string
```
### Node::jsonSerialize
> TODO: add doc comment
```php
public function jsonSerialize ( )
```
### Node::getNodeKindNameFromValue
Gets name of a Node from its raw kind value.
```php
public static function getNodeKindNameFromValue ( int $value ) : string
```
### Node::getNodeKindName
Gets the name of a Node kind.
```php
public function getNodeKindName ( ) : string
```
### Node::getEndPosition
Get the end index of a Node.
```php
public function getEndPosition ( )
```
### Node::getFileContents
> TODO: add doc comment
```php
public function & getFileContents ( ) : string
```
### Node::getDescendantNodeAtPosition
Searches descendants to find a Node at the given position.
```php
public function getDescendantNodeAtPosition ( int $pos )
```
### Node::__toString
> TODO: add doc comment
```php
public function __toString ( )
```
## Token
### Token::__construct
> TODO: add doc comment
```php
public function __construct ( $kind, $fullStart, $start, $length )
```
### Token::getLeadingCommentsAndWhitespaceText
> TODO: add doc comment
```php
public function getLeadingCommentsAndWhitespaceText ( string $document ) : string
```
### Token::getText
> TODO: add doc comment
```php
public function getText ( string $document ) : string
```
### Token::getFullText
> TODO: add doc comment
```php
public function getFullText ( string & $document ) : string
```
### Token::getStartPosition
> TODO: add doc comment
```php
public function getStartPosition ( )
```
### Token::getFullStartPosition
> TODO: add doc comment
```php
public function getFullStartPosition ( )
```
### Token::getWidth
> TODO: add doc comment
```php
public function getWidth ( )
```
### Token::getFullWidth
> TODO: add doc comment
```php
public function getFullWidth ( )
```
### Token::getEndPosition
> TODO: add doc comment
```php
public function getEndPosition ( )
```
### Token::getTokenKindNameFromValue
> TODO: add doc comment
```php
public static function getTokenKindNameFromValue ( $kindName )
```
## Parser
### Parser::__construct
> TODO: add doc comment
```php
public function __construct ( )
```
### Parser::parseSourceFile
> TODO: add doc comment
```php
public function parseSourceFile ( $fileContents ) : Script
```
## Associativity
## ParseContext

36
Invariants.md Normal file
Просмотреть файл

@ -0,0 +1,36 @@
# Invariants
> This documentation was auto-generated using this parser to help dogfood the API. Please contribute
fixes to `tools/PrintInvariants.php` and suggest API improvements.
We define and test both parser and lexer against a set of invariants (characteristics
about the produced token set or tree that always hold true, no matter what the input). This set of invariants provides
a consistent foundation that makes it easier to ensure the tree is "structurally sound", and confidently
reason about the tree as we continue to build up our understanding.
## Token Invariants
- Sum of the lengths of all the tokens should be equivalent to the length of the document.
- A token's Start is always >= FullStart.
- A token's content exactly matches the range of the file its span specifies
- FullText of each token matches Trivia plus Text
- Concatenating FullText of each token returns the document
- a token's FullText length is equivalent to Length
- a token's FullText length is equivalent to Length - (Start - FullStart)
- a token's Trivia length is equivalent to (Start - FullStart)
- End-of-file token text should have zero length
- Tokens array should always end with end of file token
- Tokens array should contain exactly one EOF token
- Token FullStart should begin immediately after previous token end
- SkippedToken length should be greater than 0
- MissingToken length should be equal to 0
## Node Invariants
- All invariants of Tokens
- The tree length exactly matches the file length.
- All Nodes have at least one child. $encode
- Span of any Node is span of child nodes and tokens.
- Parent of Node contains same child node.
- each child has exactly one parent.
- Every child is Node or Token type
- Root node of tree has no parent.
- root node of tree is never a child.
- Every Node has a Kind

21
src/Node.php
Просмотреть файл

@ -58,11 +58,21 @@ class Node implements \JsonSerializable {
return $this->parent;
}
public function getAncestor($className) {
$ancestor = $this;
while (true) {
$ancestor = $ancestor->parent;
if ($ancestor == null || $ancestor instanceof $className) {
return $ancestor;
}
}
}
/**
* Gets root of the syntax tree (returns self if has no parents)
* @return Node
*/
public function getRoot() : Node {
public function & getRoot() : Node {
$node = $this;
while ($node->parent !== null) {
$node = $node->parent;
@ -143,11 +153,11 @@ class Node implements \JsonSerializable {
}
if (\is_array($val)) {
foreach ($val as $child) {
$child === null ?: yield $child;
$child === null ?: yield $i=>$child;
}
continue;
}
$val === null ?: yield $val;
$val === null ?: yield $i=>$val;
}
}
@ -253,6 +263,7 @@ class Node implements \JsonSerializable {
* @return string
*/
public function getLeadingCommentAndWhitespaceText() : string {
// TODO re-tokenize comments and whitespace
$fileContents = $this->getFileContents();
foreach ($this->getDescendantTokens() as $token) {
return $token->getLeadingCommentsAndWhitespaceText($fileContents);
@ -344,4 +355,8 @@ class Node implements \JsonSerializable {
}
return null;
}
public function __toString() {
return $this->getText();
}
}

1
src/Node/FunctionBody.php
Просмотреть файл

@ -7,6 +7,7 @@
namespace PhpParser\Node;
use PhpParser\Node;
use PhpParser\Node\Statement\CompoundStatementNode;
use PhpParser\NodeKind;
use PhpParser\Token;

1
src/Node/MethodDeclaration.php
Просмотреть файл

@ -9,6 +9,7 @@ namespace PhpParser\Node;
use PhpParser\Node;
use PhpParser\NodeKind;
use PhpParser\Token;
use PhpParser\TokenKind;
class MethodDeclaration extends Node {
/** @var Token[] */

3
src/Node/Statement/CompoundStatementNode.php
Просмотреть файл

@ -5,6 +5,7 @@
*--------------------------------------------------------------------------------------------*/
namespace PhpParser\Node\Statement;
use PhpParser\Node;
use PhpParser\Node\StatementNode;
use PhpParser\NodeKind;
use PhpParser\Token;
@ -13,7 +14,7 @@ class CompoundStatementNode extends StatementNode {
/** @var Token */
public $openBrace;
/** @var array */
/** @var array | Node[] */
public $statements;
/** @var Token */

22
src/Node/StringLiteral.php
Просмотреть файл

@ -11,10 +11,30 @@ use PhpParser\NodeKind;
use PhpParser\Token;
class StringLiteral extends Expression {
/** @var Token[] */
/** @var Token */
public $startQuote;
/** @var Token[] | Node[] */
public $children;
/** @var Token */
public $endQuote;
public function __construct() {
parent::__construct(NodeKind::StringLiteral);
}
public function getStringContentsText() {
// TODO add tests
$stringContents = "";
if (isset($this->startQuote)) {
foreach ($this->children as $child) {
$stringContents .= $child->getFullText($this->getFileContents());
}
} else {
// TODO ensure string consistency (all strings should have start / end quote)
$stringContents = trim($this->children->getText($this->getFileContents()), '"\'');
}
return $stringContents;
}
}

13
src/Parser.php
Просмотреть файл

@ -978,7 +978,7 @@ class Parser {
// TODO validate input token
$expression = new StringLiteral();
$expression->parent = $parentNode;
$expression->children = $this->getCurrentToken();
$expression->children = $this->getCurrentToken(); // TODO - merge string types
$this->advanceToken();
return $expression;
}
@ -987,9 +987,8 @@ class Parser {
// TODO validate input token
$expression = new StringLiteral();
$expression->parent = $parentNode;
$quote = $this->eat(TokenKind::SingleQuoteToken, TokenKind::DoubleQuoteToken, TokenKind::HeredocStart, TokenKind::BacktickToken);
$expression->startQuote = $this->eat(TokenKind::SingleQuoteToken, TokenKind::DoubleQuoteToken, TokenKind::HeredocStart, TokenKind::BacktickToken);
$expression->children = array();
$expression->children[] = $quote;
while (true) {
switch($this->getCurrentToken()->kind) {
@ -999,10 +998,10 @@ class Parser {
$expression->children[] = $this->parseExpression($expression);
$expression->children[] = $this->eat(TokenKind::CloseBraceToken);
continue;
case $quote->kind:
case $startQuoteKind = $expression->startQuote->kind:
case TokenKind::EndOfFileToken:
case TokenKind::HeredocEnd:
$expression->children[] = $this->eat($quote->kind, TokenKind::HeredocEnd);
$expression->endQuote = $this->eat($startQuoteKind, TokenKind::HeredocEnd);
return $expression;
default:
$expression->children[] = $this->getCurrentToken();
@ -2239,7 +2238,7 @@ class Parser {
$token = $this->getCurrentToken();
switch ($token->kind) {
case TokenKind::Name:
$this->advanceToken();
$this->advanceToken(); // TODO all names should be Nodes
return $token;
case TokenKind::VariableName:
case TokenKind::DollarToken:
@ -2345,7 +2344,7 @@ class Parser {
$scopedPropertyAccessExpression->parent = $expression->parent;
$expression->parent = $scopedPropertyAccessExpression;
$scopedPropertyAccessExpression->scopeResolutionQualifier = $expression;
$scopedPropertyAccessExpression->scopeResolutionQualifier = $expression; // TODO ensure always a Node
$scopedPropertyAccessExpression->doubleColon = $this->eat(TokenKind::ColonColonToken);
$scopedPropertyAccessExpression->memberName = $this->parseMemberName($scopedPropertyAccessExpression);

2
src/Token.php
Просмотреть файл

@ -28,7 +28,7 @@ class Token implements \JsonSerializable {
return substr($document, $this->start, $this->length - ($this->start - $this->fullStart));
}
public function getFullText(string $document) : string {
public function getFullText(string & $document) : string {
return substr($document, $this->fullStart, $this->length);
}

2
src/Utilities.php
Просмотреть файл

@ -76,7 +76,7 @@ class Utilities {
return new Range ($start, $end);
}
public static function getLineCharacterPositionFromPosition($pos, $text) {
public static function getLineCharacterPositionFromPosition($pos, $text) : LineCharacterPosition {
$newlinePositions = [];
$newlinePos = -1;
while ($newlinePos = strpos($text, "\n", $newlinePos + 1)) {

18
tests/ParserInvariantsTest.php
Просмотреть файл

@ -11,6 +11,7 @@ require_once(__DIR__ . "/../src/Token.php");
require_once(__DIR__ . "/LexerInvariantsTest.php");
use PhpParser\Node;
use PhpParser\Token;
use PHPUnit\Framework\TestCase;
use PhpParser\TokenKind;
@ -132,6 +133,23 @@ class ParserInvariantsTest extends LexerInvariantsTest {
}
}
/**
* @dataProvider sourceFileNodeProvider
*/
public function testEveryChildIsNodeOrTokenType($filename, Node $sourceFileNode) {
$treeElements = iterator_to_array($sourceFileNode->getDescendantNodesAndTokens());
array_push($treeElements, $sourceFileNode);
foreach ($sourceFileNode->getDescendantNodes() as $descendant) {
foreach ($descendant->getChildNodesAndTokens() as $child) {
if ($child instanceof Node || $child instanceof Token) {
continue;
}
$this->fail("Invariant: Every child is Node or Token type");
}
}
}
/**
* @dataProvider sourceFileNodeProvider
*/

4
tests/cases/parser/anonymousFunctionCreationExpression1.php.tree
Просмотреть файл

@ -60,10 +60,12 @@
"children": [
{
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 7
}
},
"endQuote": null
}
}
]

4
tests/cases/parser/arrayCreationBracketedExpression20.php.tree
Просмотреть файл

@ -27,10 +27,12 @@
"ArrayElement": {
"elementKey": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 2
}
},
"endQuote": null
}
},
"arrowToken": {

4
tests/cases/parser/arrayCreationExpression20.php.tree
Просмотреть файл

@ -30,10 +30,12 @@
"ArrayElement": {
"elementKey": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 2
}
},
"endQuote": null
}
},
"arrowToken": {

4
tests/cases/parser/classConstDeclaration4.php.tree
Просмотреть файл

@ -58,10 +58,12 @@
},
"assignment": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 7
}
},
"endQuote": null
}
}
}

4
tests/cases/parser/classConstDeclaration5.php.tree
Просмотреть файл

@ -62,10 +62,12 @@
},
"assignment": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 7
}
},
"endQuote": null
}
}
}

4
tests/cases/parser/classConstDeclaration7.php.tree
Просмотреть файл

@ -77,10 +77,12 @@
},
"assignment": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 7
}
},
"endQuote": null
}
}
}

4
tests/cases/parser/classConstDeclaration9.php.tree
Просмотреть файл

@ -81,10 +81,12 @@
"byRef": null,
"rightOperand": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 7
}
},
"endQuote": null
}
}
}

4
tests/cases/parser/conditionalExpressions3.php.tree
Просмотреть файл

@ -71,10 +71,12 @@
},
"rightOperand": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 9
}
},
"endQuote": null
}
}
}

4
tests/cases/parser/echoExpression1.php.tree
Просмотреть файл

@ -24,10 +24,12 @@
"children": [
{
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 7
}
},
"endQuote": null
}
},
{

4
tests/cases/parser/echoExpression5.php.tree
Просмотреть файл

@ -50,10 +50,12 @@
},
{
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 2
}
},
"endQuote": null
}
}
]

21
tests/cases/parser/echoExpression6.php.tree
Просмотреть файл

@ -50,17 +50,16 @@
},
{
"StringLiteral": {
"children": [
{
"kind": "DoubleQuoteToken",
"textLength": 1
},
{
"error": "MissingToken",
"kind": "DoubleQuoteToken",
"textLength": 0
}
]
"startQuote": {
"kind": "DoubleQuoteToken",
"textLength": 1
},
"children": [],
"endQuote": {
"error": "MissingToken",
"kind": "DoubleQuoteToken",
"textLength": 0
}
}
}
]

4
tests/cases/parser/echoExpression8.php.tree
Просмотреть файл

@ -24,10 +24,12 @@
"children": [
{
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 4
}
},
"endQuote": null
}
}
]

4
tests/cases/parser/inclusionExpression5.php.tree
Просмотреть файл

@ -43,10 +43,12 @@
},
"rightOperand": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 4
}
},
"endQuote": null
}
}
}

4
tests/cases/parser/listExpression20.php.tree
Просмотреть файл

@ -32,10 +32,12 @@
"ArrayElement": {
"elementKey": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 2
}
},
"endQuote": null
}
},
"arrowToken": {

4
tests/cases/parser/memberAccessExpression9.php.tree
Просмотреть файл

@ -36,10 +36,12 @@
},
"expression": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 3
}
},
"endQuote": null
}
},
"closeBrace": {

4
tests/cases/parser/namespaceUseDeclaration27.php.tree
Просмотреть файл

@ -31,10 +31,12 @@
"byRef": null,
"rightOperand": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 7
}
},
"endQuote": null
}
}
}

4
tests/cases/parser/objectCreationExpression4.php.tree
Просмотреть файл

@ -42,10 +42,12 @@
"dotDotDotToken": null,
"expression": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 3
}
},
"endQuote": null
}
}
}

4
tests/cases/parser/objectCreationExpression6.php.tree
Просмотреть файл

@ -33,10 +33,12 @@
},
"expression": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 3
}
},
"endQuote": null
}
},
"closeBrace": {

4
tests/cases/parser/programStructure10.php.tree
Просмотреть файл

@ -24,10 +24,12 @@
"children": [
{
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 13
}
},
"endQuote": null
}
}
]

4
tests/cases/parser/programStructure11.php.tree
Просмотреть файл

@ -24,10 +24,12 @@
"children": [
{
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 13
}
},
"endQuote": null
}
}
]

4
tests/cases/parser/programStructure22.php.tree
Просмотреть файл

@ -15,10 +15,12 @@
"ExpressionStatement": {
"expression": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 7
}
},
"endQuote": null
}
},
"semicolon": null

4
tests/cases/parser/programStructure29.php.tree
Просмотреть файл

@ -51,10 +51,12 @@
"children": [
{
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 7
}
},
"endQuote": null
}
}
]

4
tests/cases/parser/programStructure31.php.tree
Просмотреть файл

@ -54,10 +54,12 @@
"children": [
{
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 7
}
},
"endQuote": null
}
}
]

4
tests/cases/parser/scopedPropertyAccessExpression11.php.tree
Просмотреть файл

@ -17,10 +17,12 @@
"ScopedPropertyAccessExpression": {
"scopeResolutionQualifier": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 11
}
},
"endQuote": null
}
},
"doubleColon": {

4
tests/cases/parser/scopedPropertyAccessExpression14.php.tree
Просмотреть файл

@ -52,10 +52,12 @@
},
"expression": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 4
}
},
"endQuote": null
}
},
"closeBrace": {

4
tests/cases/parser/scopedPropertyAccessExpression15.php.tree
Просмотреть файл

@ -54,10 +54,12 @@
},
"expression": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 4
}
},
"endQuote": null
}
},
"closeBrace": {

4
tests/cases/parser/scopedPropertyAccessExpression5.php.tree
Просмотреть файл

@ -45,10 +45,12 @@
},
"expression": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 3
}
},
"endQuote": null
}
},
"closeBrace": {

4
tests/cases/parser/templateStringLiteral3.php.tree
Просмотреть файл

@ -15,10 +15,12 @@
"ExpressionStatement": {
"expression": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 7
}
},
"endQuote": null
}
},
"semicolon": {

4
tests/cases/parser/templateStringLiteral4.php.tree
Просмотреть файл

@ -15,10 +15,12 @@
"ExpressionStatement": {
"expression": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 2
}
},
"endQuote": null
}
},
"semicolon": {

4
tests/cases/parser/unsetIntrinsicExpression5.php.tree
Просмотреть файл

@ -54,10 +54,12 @@
},
{
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 2
}
},
"endQuote": null
}
}
]

4
tests/cases/parser/variables4.php.tree
Просмотреть файл

@ -27,10 +27,12 @@
},
"expression": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 7
}
},
"endQuote": null
}
},
"closeBrace": {

4
tests/cases/parser/variables5.php.tree
Просмотреть файл

@ -27,10 +27,12 @@
},
"expression": {
"StringLiteral": {
"startQuote": null,
"children": {
"kind": "StringLiteralToken",
"textLength": 7
}
},
"endQuote": null
}
},
"closeBrace": {

83
tools/PrintApiDocumentation.php Normal file
Просмотреть файл

@ -0,0 +1,83 @@
<?php
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
use PhpParser\Node\MethodDeclaration;
use PhpParser\Node\PropertyDeclaration;
use PhpParser\Node\Statement\ClassDeclaration;
use PhpParser\Parser;
use PhpParser\Token;
use PhpParser\TokenKind;
require_once __DIR__ . "/../src/bootstrap.php";
$files = [
__DIR__ . "/../src/Node.php",
__DIR__ . "/../src/Token.php",
__DIR__ . "/../src/Parser.php"
];
$parser = new Parser();
echo "# API Documentation" . PHP_EOL;
echo "> Note: This documentation was auto-generated using this parser to help dogfood the API. Please contribute fixes to
`tools/PrintApiDocumentation.php` and suggest API improvements.\n<hr>\n\n";
foreach ($files as $file) {
$ast = $parser->parseSourceFile(file_get_contents($file));
foreach ($ast->getDescendantNodes() as $descendant) {
if ($descendant instanceof ClassDeclaration) {
$className = $descendant->name->getText($descendant->getFileContents());
echo "## " . $className . PHP_EOL;
// TODO consider not having a separate classMemberDeclarations node
foreach ($descendant->classMembers->classMemberDeclarations as $member) {
if ($member instanceof MethodDeclaration) {
// TODO this should be a helper function on any modified types
foreach ($member->modifiers as $modifier) {
if ($modifier->kind === TokenKind::PublicKeyword) {
$fileContents = $member->getFileContents();
$signature = implode(" ", getSignatureParts($member));
$comment = trim($member->getLeadingCommentAndWhitespaceText(), "\r\n");
$commentParts = explode("\n", $comment);
$description = [];
foreach ($commentParts as $i=>$part) {
$part = trim($part, "*\r\t /");
if (isset($part[0])) {
if ($part[0] === "@") {
break;
}
$description[] = $part;
}
}
$comment = implode(" ", $description);
if (strlen(trim($comment, " \t")) === 0) {
$comment = "> TODO: add doc comment\n";
}
echo "### " . $className . "::" . $member->name->getText($member->getFileContents()) . PHP_EOL;
echo $comment . PHP_EOL;
echo "```php\n$signature\n```" . PHP_EOL;
}
}
}
}
}
}
}
function getSignatureParts(MethodDeclaration $methodDeclaration) : array {
// TODO - something like this in API?
$parts = [];
foreach ($methodDeclaration->getChildNodesAndTokens() as $i=>$child) {
if ($i === "compoundStatementOrSemicolon") {
return $parts;
}
$parts[] = $child instanceof Token
? $child->getText($methodDeclaration->getFileContents())
: $child->getText();
};
return $parts;
}

52
tools/PrintInvariants.php Normal file
Просмотреть файл

@ -0,0 +1,52 @@
<?php
/*---------------------------------------------------------------------------------------------
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/
use PhpParser\Node;
use PhpParser\Node\StringLiteral;
use PhpParser\Parser;
require_once __DIR__ . "/../src/bootstrap.php";
$lexerInvariants = __DIR__ . "/../tests/LexerInvariantsTest.php";
$parserInvariants = __DIR__ . "/../tests/ParserInvariantsTest.php";
$parser = new Parser();
$lexerInvariantsAst = $parser->parseSourceFile(file_get_contents($lexerInvariants));
$parserInvariantsAst = $parser->parseSourceFile(file_get_contents($parserInvariants));
echo "# Invariants" . PHP_EOL;
echo "> This documentation was auto-generated using this parser to help dogfood the API. Please contribute
fixes to `tools/PrintInvariants.php` and suggest API improvements.\n\n";
echo "We define and test both parser and lexer against a set of invariants (characteristics
about the produced token set or tree that always hold true, no matter what the input). This set of invariants provides
a consistent foundation that makes it easier to ensure the tree is \"structurally sound\", and confidently
reason about the tree as we continue to build up our understanding.\n\n";
echo "## Token Invariants" . PHP_EOL;
printInvariants($lexerInvariantsAst);
echo PHP_EOL;
echo "## Node Invariants" . PHP_EOL;
printInvariants($parserInvariantsAst);
function printInvariants(Node $ast) {
foreach ($ast->getDescendantNodes() as $descendant) {
if ($descendant instanceof StringLiteral) {
$stringContents = $descendant->getStringContentsText();
if (($index = strpos(strtolower($stringContents), $invariantText = "invariant: ")) !== false) {
echo "- " . substr($stringContents, $index + strlen($invariantText)) . PHP_EOL;
}
}
if ($descendant instanceof Node\Statement\ClassDeclaration) {
$baseClass = $descendant->classBaseClause->baseClass->getText();
if (strpos($baseClass, "LexerInvariants") !== false) {
echo "- " . "All invariants of Tokens" . PHP_EOL;
}
}
}
}