We need to declare catch variables as `any` explicitly now (or turn it off
with a config, but I think it's fine like this). I noticed this because VS
Code defaults to its own version of TS and was reporting this error.
Also, some tests were expecting diagnostics to be exceptions still. Fixed,
including one test that was passing when it shouldn't have been as it had
not been updated to react to ADL -> Cadl rename or $ prefix of decorators.
* add option `--diagnostic-level` to specify which level of messages should be reported
* add docs for built-in types
* improve & export semantic-walk lib
Co-authored-by: jianye xi <jianyxi@microsoft.com>
We now run decorators and onBuild handlers in the language server and
thereby show their diagnostics to the IDE user as they code.
This put pressure on the issue of getting mismatched compiler versions since
it becomes a common case where the Cadl being edited pulls in a local
version that is different from the global version used by the global
cadl-server.
We now implement a check that resolving @cadl-lang/compiler from the entry
point cadl file either doesn't resolve or resolves to the same location as
the current compiler. If this check fails, compilation is aborted early with
a diagnostic error. The error message discusses two common causes: running
on the command line from a working directory away from the source or running
in the IDE where the root workspace dir is not the one with the node_modules
that have the cadl compiler package.
In the second error case, the recommended remedy in the message is to
configure the cadl-server path in IDE settings. We actually have this
case (which an end user can also have) in our own repo and this change also
configures our VS Code settings to use the local cadl that we are developing
on.
A mistake during a refactoring caused us to start running decorators and
onBuild in language server. We do intend to do this soon, but doing it
prematurely revealed a version problem where we can't safely run decorators
that load a local Cadl compiler while using the globally installed language
server.
So, for now, revert back to not running decorators, and we will look at how
to deal with the version mismatch as part of doing it again deliberately.
Code used in go-to definition depends on the invariant that a node's `[pos,
end)` range must be inside that of its parent. This invariant may prove
useful for other purposes too. However, there were some places where this
invariant did not hold and this change fixes that.
1. Statements did not include decoroators (if any) inside their `[pos, end)`
range. Incosistently, parameters, model members, and enum members did
already include decorators in their position. This change includes
decorators in positioning always.
2. Member expression positioning was incorrect and started at the position
of the last identifier rather than the start of the full member expression.
3. Namespaces using the `namespace A.B.C {}` syntax that expanded to
`namespace A { namespace B { namespace C {}}}` started the inner namespaces
at their identifiers. The only way to keep this invariant is to have all the
namespaces defined have the same `[pos, end)` range of the entire statement,
so this change does that.
* Refactor decorator binding out of program and into binder
* Refactor decorator execution out of program and into checker
* Decorators can have a .decorator string property with a namespace like A.B on it.
* js files can export a namespace property which has a namespace like A.B in it.
* Decorators and types occupy separate namespaces so can have the same name.
Previously, the language server only parsed files and only reported
diagnostics from parse errors. Now it runs a compilation and reports
semantic errors such as unknown identifiers in open files.
For now, decorators are not invoked nor is onBuild called. There are some
unresolved design issues there still to be discussed...
The language server will try to infer the appropriate entry
point (a.k.a. "main file") for a given document. This is necessary because
you can get different diagnostics in the same files for different entry
points.
The entry point inference is not perfect and there can be unexpected
diagnostics if the imports are done sufficiently creatively to outsmart
it. It works by walking the directory tree from the changed document through
its parents and stopping when a package.json with adlMain or main.adl is
found or we reach a workspace root. If we reach a workspace root without
finding an entry point, then the document is used as its own entry
point. Untitled documents are always analyzed as their own entry points
since they do not exist in any directory where another entry point could be
found.
At some point, we may need to add some configuration to be explicit about
which entry points to use at some point, but the inference should also be
sufficient for common cases. For example, it is sufficient enough to handle
all of our samples.
Additional minor changes:
1. Fix an issue where spreading a property that had an unbound decorator
would report duplicate diagnostics at the same source location indicating
that the decorator could not be found.
2. Route config file I/O through compiler host interface.
3. Fix an issue with debugger config where VS Code was sometimes failing to
attach to language server.
This removes the need for a custom output window pane in Visual Studio.
Also, start tracking workspace folders which will be needed soon, but just
log them for now.
Previously, there was no feedback from language server unless a document
had was written somewhere to disk at least once. Feedback happened live,
not on save, but if the document was "untitled", then there would be no
feedback at all.
We no longer glob files when a directory is passed to `adl
compile`. Instead, we will look for `adlMain` specified by package.json or
main.adl in the directory. This eliminates any dependency on file system
enumeration order and will make it easier for tooling to implement semantic
features.
Also:
* Fix various issues with reporting I/O errors as diagnostics.
* Fix issue where regen-samples would silently pass PR validation if nothing
at all was emitted.
* Make regen-samples have opt-out exclude list instead of opt-in include
list.
* Report correct location for syntax errors in decorators, and fix issue
with copying petstore sample to another folder without type: module in
package.json
* Allow .mjs extension for decorator imports.
* Make sure unahndled promise rejection becomes internal compiler error.
* Fix issue with dumping config with diagnostics from `adl info`
* Avoid repeating config filename in `adl info`
We now continue to semantic analysis when there are parse errors, and we do
not abort the compilation when a semantic error is encountered, but instead
recover and look for more errors.
DiagnosticError is no longer a thing as we never throw diagnostics as
exceptions anymore. throwDiagnostic is now Program.reportDiagnostic.
It ended up being tricky to merge my pending semantic error recovery change
with the config file loading change. This change extracts out some changes
to config file loading from my error recovery branch as it will be easier to
review this independently from that.
* Don't bother making a diagnostic for incorrect file extension, as compiler
should never attempt that. It's OK to throw a regular error in this case.
* Fix issue with returning or spreading default config where mutation of
loaded config could mutate defaults.
* Avoid fileExists + readFile race condition by not using fileExists and
catching readFile ENOENT error instead.
* Move extra json file for multi-file test to correct folder.
* Various small refactorings to make these changes easier.
Added a new package prettier-plugin-adl providing adl support to prettier. Just installing the plugin next to prettier will automatically enable it for .adl files.
Added new adl format command.
Usage:
adl format **/*.adl
This will format all the files matching the pattern.
1. Move to per-language prettier formatter throughout. Global wasn't working
for some folks for TypeScript, so I suspect it's best to use the
per-language setting throughout.
2. Merge the adl-vs .editorconfig into the root .editorconfig, giving better
defaults for editors other than VS Code that support .editorconfig.
3. Add ADL, C# and XML to VS Code settings.json
* Don't allocate substrings to match keywords.
* Make at most one pass over string to get its unquoted/unescaped/unindented
value. (**)
* Add test coverage for impacted code paths.
* Fix issue where string value of an unterminated string was missing final
character(s) in string value.
(**) Actually, we still make two passes in the case of a non-triple-quoted,
multi-line string with \r\n, but that is about to be removed by the next
commit which will disallow non-triple-quoted, multi-line strings
altogether.
1. `compile` had lost its success message. I inadvertently deleted it
a while ago: restored.
2. `generate` was printing undefined as the output path: fixed.
3. `generate` did nothing without --client, required --client for now.
4. `generate` took a long time to do nothing without --language,
required --language.
5. `generate` a client inside the adl tree, and tsc would to compile
it: excluded 'adl-output' in tsconfig.
Also routed running of autorest through run helper so you can see the
autorest args we pass using `adl --debug`
Fixes issues with watch build when a change impacts dependent
projects.
Also fix issues preventing a watch build in VS Code from being
successful due to dist/ or dist-dev/ not being created yet.
With both of these, it is now actually possible to do only rush
update on pristine enlistment, then Ctrl+Shift+B in VS Code as
CONTRIBUTING.md claims.
* Bug fixes
* tokenValue() was wrong if an identifier ever came right after a
string literal.
* There was an incorrect and confusing end-of-file error if file
ends with numeric literal. Generally hardened and cleaned up EOF
checking throughout.
* Digits weren't allowed in identifier with non-ascii characters.
* Perf
* Add more ASCII fast paths
* Inline scanUntil everywhere, and simplify each use to what it
actually needed
* Avoid eager substring allocation and map lookup for keywords in
more cases
* Put empty statements and invalid statements in the tree. We now have
an invariant that if there is a parse error, there must be a node in
the tree with an error, and even empty statements can have errors if
you try to decorate one. All parse tests now check this invariant.
* Refine dumpAST to sort in a more readable order and to print the
substring of source code with each node.
The regex I used in the last round was flawed and missed these.
Also remove some casts where possible, and change TypeInstantiationMap
set function's return type from string to void.
With this change, the parser no longer throws when it encounters an
error, but continues on to report subsequent errors to the user as
well.
For now, however, evalADLScript still throws if there are any parse
errors. More work is needed in the parser to represent which nodes
have the errors and so forth before we can meaningfully analyze a
syntax tree for source that had errors.
Our default response to a token that doesn't match our expectation is
to insert a matching token before the offending token. This is
effectively what is happening wherever we have parseExpected() without
checking the return value. Also when we expect an identifier and do
not find one, we insert an identifier with a unique yet unspeakable
name. In fact, anywhere we hit an invalid expression, we insert one of
these identifiers.
Statements are easier to correct than expressions, and our approach
there is different. Every statement in the language begins with a
reserved word, an at-sign, or a semicolon. If the leading token for
statement is none of these, we report an invalid statement starting at
that token and ending immediately before the next token that is one of
these.
There are also case-by-case refinements to this insertion
strategy. For example, we replace errant semicolons with commas in
comma-only delimited lists rather than inserting semicolons in front
of the comma.
In other cases, we take the approach of parsing a grammar that is a
superset of the language specification, augmented with known common
errors. For example, we parse decorators in many more places than
actually allowed (and signal an error still, of course). We do the
same for import statements inside namespaces.
Over time, I expect that we'll need to do more of these more
deliberate and one-off corrections, but this change still performs
relatively well and it provides the foundation for such improvements.
A major challenge with the approach of correction by inserting tokens
is that it can hang the parser such that it keeps inserting tokens
without making forward progress. To mitigate the risk of such bugs,
all list constructs that are susceptible to this are driven by the
same parseList() routine. This routine has an escape hatch in the loop
where it bails and assumes we've hit a bad representation for the end
of the list if any loop iteration fails to make progress.
A trivial example of a construct that would hit this without this
check is `model M { ]`. The parser proceeds as follows in that case:
1. Parse model keyword: OK.
2. Parse model name (M): OK.
3. Parse open brace: OK.
4. Expect property name, see close bracket: ERROR, insert synthetic
identifier for property name.
5. Expect colon, see close bracket: ERROR, insert colon.
6. Expect property type, see close bracket: ERROR, insert synthetic
identifier for property type.
7. Expect semicolon, see close bracket: ERROR, insert semicolon.
8. Observe that the position has not advanced after a full loop
iteration of parsing properties: ERROR, replace with close brace,
exit loop.
(Note that everywhere I'm saying "insert" or "replace" here, there's
no literal array of tokens that we're mutating, we are just taking the
code paths we would take if those edits were made to the source. There
is rather an implicit "behave as though" replacing/inserting, but that
is really an implementation detail and not part of the logical
algorithm.)
Without the vital step 8, we could convince ourselves that we've
parsed a real property and try to move on to the next one, and do this
over and over again, creating infinitely many synthetically named
properties of synthetically named types!
This change also adjusts many of our error messages, borrowing from
the TypeScript compiler's terse tone for mundane errors.
It also fixes various issues with imprecise or sub-optimal squiggly
locations for various errors.
There are also some ADL team developer productivity improvements in
this change...
The per-test output in Mocha Test Explorer in VS Code now shows the
input source code, the resulting syntax tree, and all parse
diagnostics formatted nicely as the CLI compiler would. The syntax
tree JSON also has boilerplate default-empty things elided and the
start and end positions augmented with line and column number.
Negative parse test cases must now provide regex(es) to match against
the reported diagnostics.
Stack traces in Mocha Test Explorer will now be reported up to 50
frames rather than 10, making it easier to diagnose a stack overflow
in the parser's recursive descent.
A new compilerAssert function is added for asserting something that
should never happen in the compiler. It takes a condition, message,
and optional source node. If the condition is not met, it throws an
AssertionError with the message and if a source node is provided, the
message will be augmented with "occurred while compiling (file) near
line (X) and column (Y)". This is used in only a couple of places
right now. I did not yet scrub the existing throws that could benefit
from this.
If a DiagnosticError or AggregateError occurs in a test, the formatted
diagnostics or inner stack traces are included in the Mocha Test
Explorer per-test output. This is done because tests don't have the
CLI catch handler that has to take special steps for these special
errors.
Note that for all Mocha Test Explorer output improvements above, if
you prefer to run tests on the command line, you can also set
environment variable ADL_VERBOSE_TEST_OUTPUT=true and get all of the
output spewed to the console.
An issue with the typing of `messages` allowed typos when used is
fixed, and the fix makes `Message.X` a `Message` and if you hover over
X in the IDE, you will see the message code, severity and text.
Finally, there's also a minor correction in the tutorial to account
for parenless decorators having been removed from the language.
* Update to latest mocha
* Use source-map-support to get .ts in stack traces.
This is supposed to also allow mocha explorer to navigate to .ts,
but it isn't working. :(
* Enable verbose test logging when run through mocha explorer where
the output is associated with individual tests and spam is not a
concern. Prevents needing to uncomment code to investigate failures.
On the command line, you can also set ADL_VERBOSE_TEST_OUTPUT=true
in the environment to get all of this logging to the console.
* Set things up to allow more than one project in mocha explorer
- Add new @armResource, @armResourceOperations, and @armStandard* decorators
- Add new base types for ARM resources: TrackedResource, ExtensionResource, ProxyResource
- Trim base service namespace from parameter definition names
- Fix issue where dynamic namespaces were not evaluated
- Add _____ sample
- Add _____ sample
- Update _____ and _____ samples
* Warn if VS has older than minimum required version, and skip local build
* Ensure more descriptive warnings and errors go to `rush` summary output
* Add escape hatch ADL_SKIP_VS_BUILD env var to force VS build to skip
* Add LICENSE file to root and all packages
* Standardize tsconfig.json files, invert default to esnext
* Standardize package.json files, putting metadata first, impl last
* Reduce number of stub package.json files for type=module by putting
type=module one-level up where possible
* Standardize dependency version ranges: ~ to external, exact for
adl-* to adl-*. The latter is a prerelease policy to avoid having to
debug mismatches while things are churning and breaking frequently.
* `rush update --full` to pick up patches of our dependencies
* Use consistent casing for CONTRIBUTING.md and README.md
* Fix squigglies on some rush json files with comments in VS Code
* Remove some unused dependencies
* Remove some stale/unused "files" entries from package.json files
* Remove unnecessary placeholder test scripts to keep some noise out
of `rush test`
* Log output from ADL language server to VS output window pane
* Use nullable
* Turn up warnings; treat warnings as errors
* Launch petstore sample on F5 to reduce iteration time
* Fix null ref bug that broke spec generation
* Move processing in-proc to speed things up further
* Reduce polling interval to 200 ms
* Log diagnostics with appropriate source file
* Make output prettier
* Fix some breakage in launch.json after some things moved around
* Initial commit of generated docs/spec.html with just grammar for now
* Add spec generation on source change to VS Code watch tasks
* Move tmlanguage generation from pre-lauch to watch task
* Cleanup .gitattributes
* Automatic text/binary distinction
* Get GitHub to collapse generated files by default
Use a global command that checks the whole repo to avoid having to repeat the
complex commands in each project, and to prevent some files from escaping the
check.
NOTE: This removes per-project `npm run format` and `npm run check-format`.
Use `rush format` and `rush check-format`.
* Fix `rush dogfood` without prior build
* Fix not being able to run other commands while `rush watch` is running
* Switch all build scripts to esm
* Reuse "run" helper throughout for spawning processes with logging + error handling
* Remove `rush compile` and `rush pack`, not really needed
* Remove `rush clean`, didn't do anything as our projects don't have clean scripts
* Clean up `rush --help` to show which commands are custom with better descriptions
This was preventing commenting and indenting operations in VS Code from
working as expected when run from VSIX (as opposed to F5) since the file
was not actually put in the package.
This applies to both namespaces and operations. The tags for a
operation is the union of the oeration and namespace tags.
Empty tags sections are added to an operation if no tags are specified.
Sample usage:
```
@tag("foo")
@tag("bar")
namespace test {
@tag("my_ops")
op create(@path id:int32): null;
}
```
* Setup prettier with config borrowed from azure-sdk-for-js
* Run prettier to format existing code
* Turn format on save on so VS Code with prettier will auto-format
* Add rush check-format / rush format to check formatting / run prettier on whole repo
* Add check-format to PR validation
1. Make throwDiagnostic take the target directly and figure out the
location rather than having the programmer call getSourceLocation.
2. If getting location or formatting message fails, include as much of
the diagnostic as possible in the internal compiler error.
Everywhere we throw an error now reports a location. Use
throwDiagnostic plus getSourceLocationOfXxx helpers going forward
for new errors. (We still need to do error recovery and not just
throw at the first sign of trouble, but that isn't part of this
initial pass.)
Fix duplicate name reporting in binder to report duplicates in
the same file as an error and to report locations of all
duplicates up to the first bound source file that introduced at
least one.
Fix issue where failed compilation was saying "Compilation
completed successfully" and ensure process exits with non-zero
code on failure.
The scanner was doing a lot of work per character. While this may have
been a performance problem, the motivation here is maintainability. This
change makes it so that advancing by one character is simply Scanner.position++.
It also does more work lazily: allocating substrings on demand.
Highlights:
* Remove unused tokens
* Rename Kind to Token
* Remove token position map
* Remove character size bookkeeping
* Remove (ch, chNext, chNext) state
* Convert class to closure
Note: Line and column tracking was temporarily removed to facilitate
refactoring and will be re-added in the following change.
Also:
* Document template parameters in grammar
* Document empty statement in grammar
* Tighten MemberExpression (conservatively for now) to just apply to identifiers
* Use OrHigher convention in grammar and parser
Update language grammar to reflect current state of the language.
Grammarkdown is now used and run during the build. It will error
out if the grammar is invalid in ways such as a reference to
something that isn't defined.
Other minor changes:
* Fix up some VS Code build and launch settings
* Remove stale references to VS Code extension projections
Nick Guerrera
Timothee Guerin
Mike Kistler
JianyeXi
David Wilson
markcowl
Brian Terlson
David Wilson
Mike Kistler
Nick Guerrera
James Casey