## Description
Tag asserts for 2.1.
This is a bit early so will likely have to be done again just before the
release, but this gets the number of changes and thus reviewer groups
required for that more time sensitive change lower.
## Description
This updates the last use of a `2.0.0-internal.x.x.x` package in our
repo, mocha-test-setup used by tinylicious.
With server using client packages (and client using server packages),
it's easy for server to fall behind on the versions of client it
consumes: this updates one particularly out of date case.
The general idea here is that the "header" describes mechanical aspects
of installing / using the library. Where the "footer" enumerates
relevant Fluid Framework policies. The API docs belong in the former
category.
## Description
Implement browser and NodeJS specific debug formatters so TreeNodes show
their actual type and content instead of just "Proxy" and some junk.
This should be considered an early MVP: the experience isn't great but
its an improvement and a starting place to build something nicer.
## Description
In #21785, we inadvertently removed the following two function calls
from the incoming op processing code in the case of
`!modernRuntimeMessage`:
* `ensureContentsDeserialized`
* `unpackRuntimeMessage`
The second is obviously fine, we early-return on the same condition as
`modernRuntimeMessage`.
The first is _probably_ fine, but there's some ambiguity. It's possible
that there are some legacy runtime messages or system messages with an
unexpected format where we need to do this deserialization, and we must
not break those cases.
So this PR adds back that call (moving it to `ContainerRuntime.process`,
and also instruments these two functions so we can collect logs to see
if we hit certain codepaths here at scale.
---------
Co-authored-by: Daniel Madrid <105010181+dannimad@users.noreply.github.com>
## Description
This adds an abstraction for holding internal state and APIs for
TreeNodes. This will let us migrate functionality away from the flex
layer and into the simple tree layer incrementally.
- keeps track of latest relevant revisions in the detached field index
- disposes repair data from the forest when a branch's ancestry is
trimmed
- does not automatically garbage collect repair data if there is a valid
revertible or branch that needs the data
# BREAKING CHANGES
ST versions with versions with this change are at risk of failing to
apply some changes when collaborating with older ST versions (including
loading older documents) because some older ST versions (prior to
https://github.com/microsoft/FluidFramework/pull/21749) did not provide
adequate refreshers. This should not prevent new code from opening old
documents because the old documents include all the repair data in the
summaries they save.
---------
Co-authored-by: Yann Achard <achardy@microsoft.com>
Bumps the client dependencies on build-tools to 0.41.0.
## Automated changes
- The type test generator was updated in this build-tools release, so
the output of all type tests was regenerated.
## Manual changes
- The type test generator uses new names in some cases, which means
existing test disables in package.json had to be renamed. This extra
step could have been avoided if I'd done the bump after the next
release, when the type tests are all reset and the disables are all
reset. But this release contains useful changes and is worth getting
into the release branch as well.
- The azure-client disables needed an additional entry,
"RemovedClassStatics_AzureFunctionTokenProvider" which is expected.
## Description
This PR adds another enum option `ForestType.Expensive`, which is passed
in to set a flag in `ObjectForest` used to have more expensive
asserts/validation for debugging.
## Description
This removes `treeStatus()` from the flex API and does various related
cleanup. This is part of an ongoing effort to reduce the code in
SharedTree's middle API layer.
## Description
This PR refactors ModularChangeFamily to use algorithms for rebasing and
composition which do not require inspecting irrelevant portions of the
changeset trees. Field kinds are now only required to call `rebaseChild`
or `composeChild` when both changesets have edits for that child node.
Further work is still necessary to make the cost of
ModularChangeFamily's compose and rebase scale proportional to the size
of the intersection between the input changesets, instead of scaling
proportional to the size of both changesets. Tasks not addressed in this
PR include incremental pruning of changesets, keeping aliasing paths
short in the new node alias table, and converting various maps and
arrays in ModularChangeset to data structures which can be efficiently
composed.
---------
Co-authored-by: yann-achard-MS <97201204+yann-achard-MS@users.noreply.github.com>
## Bug and root-cause
A GC test that validates deleted stats is flaky. The test uses sweep
timeout of 200 ms. It does the following in sequence:
1. Runs GC wtih every thing referenced and validates that there
unreferenced stats are 0.
2. Unreferences one data store and one blob, runs GC and validates they
show as unreferenced.
3. Unreferences another data store and another blob, runs GC and
validates they show as unreferenced.
4. Wait for sweep timeout, runs GC so that GC delete op is sent.
5. Run GC again and validate that the unreferenced data stores and blobs
show up as deleted.
If 200 ms have passed before step 3 is run, the unreferenced data store
and blob will become sweep ready and will show up as deleted in GC
stats. There was a change few months back that changed the logic to add
sweep ready objects as deleted even when sweep is enabled
(https://github.com/microsoft/FluidFramework/pull/19524) and the test
probably became flaky then.
## Fix
Broke the test above into 2:
1. The first one only validates unreferenced stats (steps 1 to 3 above).
2. The second one that only validates sweep ready and deleted stats. It
unreferences objects, waits for sweep timeout so that nodes are sweep
ready and validates that they show as deleted. It then runs GC again so
that the GC sweep op has round tripped and these nodes are deleted.
Since the test goes from unreferenced -> wait for sweep timeout ->
validation, there is no intermediate GC run like bfore where it can
unexpectedly become sweep ready.
[AB#8350](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/8350)
## Description
#20914 disabled a few Azure E2E tests because they were timing out in
T9s due to a T9s bug that was fixed in #20312. That bug has been patched
into T9s v4 and was also released in T9s v5. We can enable those tests
now.
Validated locally that these pass by running `pnpm test`
## Description
This PR fixes tests in `composeVsIndividual.fuzz.spec.ts`, where the
schema ops were being processed on the main tree rather than its fork.
Direct usages of SchemaBuilder are being minimized, and this example is
no longer necessary since there are many existing examples of how to use
SchemaBuilder in the codebase.
## Description
Before this change, many incorrect usages of schema would slip through
validation and produce asserts when used.
Several of these cases now throw detailed usage errors explaining the
issue.
## Breaking Changes
Some malformed Schema using multiple classes extending the same schema
factory generated class may have partially worked before, and will not
robustly fail.
The "Client Requirements" section is specifically intended to provide
end-users with usage contract information. Non-public packages don't
need this information. Additionally, such information is (generally) not
applicable to "tools" packages, many of which are intended to run in
Node, rather than a browser.
## Description
Make a few changes to type test generation:
- Internal refactoring to compute things like the test name once
- Add type test for class statics
- Remove redundant "Declaration" from name of all type tests
- Treat `typeof` cases like sealed: removing checks for removal of
members of variables, functions and class statics.
This means that going from:
```typescript
function foo(): 1 | 2 { return 1;}
const thing = {a: "a"} as const;
```
To
```typescript
function foo(optional?: string): 1 { return 1;}
const thing = {a: "a", b: "b" } as const;
```
Used to be breaking, but now should not be.
Making the types of variables/functions less specific is still a
breaking change (they are still type tested, just treated like sealed
types).
Note that:
```typescrtipt
type Foo = () => 1|2;
```
To
```typescrtipt
type Foo = () => 1;
```
Is still breaking: that's a type definition not the type of a function
definition, so no changes are made to the policy there.
Changes for client can be viewed here:
https://github.com/microsoft/FluidFramework/pull/21875
## Breaking Changes
Now catches changes to class statics, and broken annotation in
package.json need to have the "Declaration" removed from he names.
We always want to prefer a headless install and require explicit action
update the lockfiles. pnpm supports CLI arguments set in npmrc files, so
we set [frozen-lockfile](https://pnpm.io/cli/install#--frozen-lockfile)
to true, which will make the default install experience error out when a
lockfile update is needed. Users can pass --no-frozen-lockfile to
override and update the lockfile.
## Description
Extract serialized blobs from the ISnapshot which contains blob contents
instead of fetching from driver which could make network calls. This
affects the cases in Data virtualization where initial blob contents for
some trees are missing in the snapshot and therefore driver does not
have the contents cached, so it ends up making a bunch of network calls.
---------
Co-authored-by: Jatin Garg <jatingarg@Jatins-MacBook-Pro-2.local>
As part of making it so I can locally test
[AB#8548](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/8548)
I haven't tested this for remote builds, but this sped up my local runs
of loader back-compat downloads significantly from minutes to a few
seconds.
Even though the remote builds will need to download all the builds, this
speeds up the process of reading whether or not a build was installed.
My guess is that there likely will be pipeline speed reductions.
Locally, running back compat full tests will be significantly faster as
indicated above because we will not need to read the file for every
single version that we have installed, instead we will read the file
once for all installed versions.
## Description
We've run into several scenarios in the past where we want to have
benchmark tests which report their own measurements of something (other
than the execution time and memory usage that benchmark-tool currently
supports), and have those measurements go to Kusto so we can track them
through time.
POC: https://github.com/microsoft/FluidFramework/pull/20772
## Sample
~~~
Op Size
spec.js:54
Insert Nodes
spec.js:54
Many Transactions
spec.js:54
✔ 100 small nodes in 100 transactions @CustomBenchmark (134ms)
spec.js:83
✔ 100 medium nodes in 100 transactions @CustomBenchmark (89ms)
spec.js:83
✔ 100 large nodes in 100 transactions @CustomBenchmark (78ms)
spec.js:83
Single Transaction
spec.js:54
✔ 100 small nodes in 1 transaction @CustomBenchmark (66ms)
spec.js:83
✔ 100 medium nodes in 1 transaction @CustomBenchmark (54ms)
spec.js:83
✔ 100 large nodes in 1 transaction @CustomBenchmark (53ms)
spec.js:83
Remove Nodes
spec.js:54
Many Transactions
spec.js:54
✔ 100 small nodes in 100 transactions (63ms)
spec.js:83
✔ 100 medium nodes in 100 transactions (62ms)
spec.js:83
✔ 100 large nodes in 100 transactions (62ms)
spec.js:83
Single Transaction
spec.js:54
✔ 100 small nodes in 1 transactions containing 1 removal of 100 nodes
spec.js:76
✔ 100 medium nodes in 1 transactions containing 1 removal of 100 nodes
spec.js:76
~~~
## Unit test
~~~
Writing test results relative to package to nyc/junit-report.xml and
nyc/junit-report.json
.mocharc.cjs:11
spec.js:54
`benchmarkCustom` function
spec.js:54
uses `before` and `after`
spec.js:54
✔ test @CustomBenchmark
spec.js:76
✔ run BenchmarkCustom
spec.js:76
2 passing (8ms)
~~~
---------
Co-authored-by: Alex Villarreal <716334+alexvy86@users.noreply.github.com>
Co-authored-by: Craig Macomber (Microsoft) <42876482+CraigMacomber@users.noreply.github.com>
Co-authored-by: Joshua Smithrud <54606601+Josmithr@users.noreply.github.com>
## Description
Adds a VSCode workspace set up for e2e test debugging. It has the
following advantages over the primary (root) workspace:
- Access to legacy module installation from the file explorer
- Dedicated launch configurations for each driver we test against
- Inputs selected at launch time for:
- driver-specific config (e.g. odsp endpoint)
- console output
- whether break points can be set in legacy modules (this is a fairly
large performance hit, so default is 'no')
- Tinylicious launch target is a compound target which allows
simultaneous server + client debugging
- FF code under `.legacy` (i.e. the modules installed for compat
testing) isn't under skipFiles by default, so break points in this code
works (either in the js if source map loading is turned off, or in the
ts if it is turned on, which is configurable at launch time)
The r11s tests are difficult to run on Windows, so I didn't test that
launch target. Other targets all work well as far as I can tell (which
includes setting breakpoints in various places in dependent packages)
Sample view of launch window when using this workspace:
---------
Co-authored-by: Abram Sanderson <absander@microsoft.com>
## Description
Count trees with GroupId separately in telemetry in ODSP driver when
parsing the snapshot as we didn't put its parsing into above path
deliberately to reduce number of special cases for parsing.
---------
Co-authored-by: Jatin Garg <jatingarg@Jatins-MacBook-Pro-2.local>
## Description
With the removal of concrete DDS classes, one pattern that's come up
repeatedly is customer code which previously checked `instanceof
MySharedObject` (usually when the code supported multiple shared object
types for whatever reason), which no longer works. This change adds a
drop-in replacement to the public API surface.
## Breaking Changes
As `SharedObjectKind` was marked sealed, this is non-breaking.
## Alternatives Considered
We could expose free functions in each package easily e.g. by using a
helper like this:
```typescript
export function createSharedObjectTypeguard<TSharedObject>(
kind: ISharedObjectKind<TSharedObject>,
): (loadable: IFluidLoadable) => loadable is IFluidLoadable & TSharedObject {
const factoryType = kind.getFactory().type;
return (loadable: IFluidLoadable): loadable is IFluidLoadable & TSharedObject => {
return isChannel(loadable) && loadable.attributes.type === factoryType;
};
}
```
Ultimately this will be more code though and arguably less discoverable.
We could also add back support for `instanceof` using
`Symbol.hasInstance` (and the same implementation as `.is`), but due to
https://github.com/microsoft/TypeScript/issues/56536, this won't work
for customers using TS below 5.5, so we'll need something else anyway at
least for now.
---------
Co-authored-by: Abram Sanderson <absander@microsoft.com>
Co-authored-by: Craig Macomber (Microsoft) <42876482+CraigMacomber@users.noreply.github.com>
Follow up to #21878, which removed `@deprecated` tags from some of the
properties helpers. There were many `import/no-deprecated` directives
still in the code, so I'm removing them now.
This PR is to log retry param from service call made by the driver in
server side. There will be another PR to logs retry for downstream
request.
---------
Co-authored-by: Lin Lin <linlin1@microsoft.com>
Ensures we don't generate API docs sections for packages that are not
published / whose docs are not published.
Also removes now-unneeded param overrides in private/internal package
READMEs.
Craig Macomber (Microsoft)
Joshua Smithrud
Mark Fields
Alex Villarreal
Noah Encke
daesunp
Tyler Butler
Jenn
alex-pardes
Navin Agarwal
tyler-cai-microsoft
Zach Newton
Jatin Garg
Tong Chen
Abram Sanderson
seanimam
jzaffiro
Lin Lin
Tony Murphy