This change fixes issue covered in PR
https://github.com/microsoft/FluidFramework/pull/22981
After several simplifications, I realized we could reuse
pendingStashedBlobs map to identify the bug scenario, which is
reuploading a stashed blob that was already processed. If while
onUploadResolve we don't see the blob in the pending list but it's on
the stashed one, then just ignore the new upload.
trackPendingStashedUploads is replaced with waitForStashedBlobs which
also implies there are further simplifications to the blob manager
logic, mainly we could also remove the stashedUpload property on the
PendingBlob type since now we keep track of stashed blobs separately .
## Description
This used:
1. `pnpm exec flub bump deps client -g -t latest`
2. Manually update version in root package.json
3. pnpm install --no-frozen-lockfile
4. Remove no longer needed type test breaks from tree (this update had
some fixes to handling of never and system types in type tests)
5. `pnpm run build:fast`
For various call signature API items, we frequently IDs like
`_call_-call`. There are a number of reasons for this, but mostly it
boils down to trying to avoid creating artificial name conflicts while
also ensuring generated files don't contain invalid characters. When we
generate heading IDs from these, we previously weren't escaping the
contents, which was invalid. Hugo previously handled this in most cases,
but Docusaurus (the tooling we're working to adopt in place of Hugo for
generating our website) does not.
This PR updates heading rendering to ensure ID contents are escaped for
Markdown.
[AB#23444](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/23444)
## Description
Modifies `getAllowedContentDiscrepancies` to use a generator, which
enables more efficient code paths on early exit. Also refactors the
strategy it takes a bit to reduce code duplication.
## Reviewer Guidance
This PR should not change semantics of any codepaths.
---------
Co-authored-by: Abram Sanderson <absander@microsoft.com>
- Use audience more directly in PresenceManager
- Clean up Quorum+Audience coordination
- Revise tests and add more cases:
```log
Presence
PresenceManager
when connected
attendee
✔ is not announced via `attendeeDisconnected` when unknown connection is removed
that is joining
✔ first time is announced via `attendeeJoined` with status "Connected"
✔ second time is announced once via `attendeeJoined` with status "Connected" when prior is unknown
- second time is announced once via `attendeeJoined` with status "Connected" when prior is still connected
- first time is announced via `attendeeJoined` with status "Connected" even if unknown to audience
✔ second time is announced once via `attendeeJoined` with status "Connected" even if most recent unknown to audience
✔ as collateral and disconnected is NOT announced via `attendeeJoined`
that is already known
✔ is NOT announced when "rejoined" with same connection (duplicate signal)
- is not announced via `attendeeJoined` when already "Connected"
✔ with status "Connected" is available from `getAttendee` by connection id
✔ with status "Connected" is available from `getAttendee` by session id
✔ with status "Connected" is available from `getAttendees`
✔ with status "Disconnected" is available from `getAttendee` by connection id
✔ with status "Disconnected" is available from `getAttendee` by session id
✔ with status "Disconnected" is available from `getAttendees`
and has their connection removed
✔ is announced via `attendeeDisconnected`
✔ is not announced via `attendeeDisconnected` when already "Disconnected"
that is rejoining
✔ is NOT announced when rejoined with same connection (duplicate signal)
✔ is announced when rejoined with different connection and current information is updated
```
Note: some tests are disabled as they are failing
## Description
This PR updates SystemWorkspace to use audience for session client
connection status.
We also add audience support to Mock Ephemeral Runtime w/ additional
reworking of unit tests
Presence + Audience test cases to add:
- Unknown attendee joins that is in audience (=> emits attendeeJoined)
- Unknown attendee joins that is not in audience (=> does not emit
attendeeJoined)
- Known attendee rejoins that is in audience w/ same connection
(duplicate signal) (=> does not emit attendeeJoined)
- Known attendee rejoins that is in audience w/ different connection (=>
emits attendeeJoined)
- Known attendee rejoins that is not in audience w/ same connection (=>
does not emit attendeeJoined)
- Known attendee rejoins that is not in audience w/ different connection
(=> does not emit attendeeJoined)
## Description
Updates several functions and types in `discrepancies.ts` to use the
term `Discrepancy` over `Incompatibility`. The general trend I have in
mind which is staged for this code is to move policy decisions about
schema *compatibility* to the simple-tree layer, whereas this function
is a helper that enumerates ways in which two stored schemas are
different. The updated naming should help reflect this.
---------
Co-authored-by: Abram Sanderson <absander@microsoft.com>
Events for presence notifications have been implemented. Some
type-related workarounds were needed, but all are documented inline.
Note that the emit.unicast function, used to send notifications to
specific clients, is not tested.
[AB#19706](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/19706)
---------
Co-authored-by: Jason Hartman <jasonha@microsoft.com>
## Description
The alpha enum APIs had some typing issues where they were unioning
things at the wrong levels.
Additionally since their tests predated unhydrated node support, they
didn't make as many nodes as they probably should have.
The tests have been simplified to use unhydrated nodes in a few places
and extended to cover more cases.
More explicit type testing has been added.
This fixes an issue reported in
https://github.com/microsoft/FluidFramework/pull/23055
See changeset for details.
## Breaking Changes
See changeset.
Fixes:
[AB#2309](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/2309)
## Description
In light of [Remove ability to reject reentrant ops (#20621) ·
microsoft/FluidFramework@c9d1562
(github.com)](c9d156264b),
updating this to be about finishing the removal of
ensureNoDataModelChanges.
The following uses of ensureNoDataModelChanges will be removed:
- packages\runtime\container-runtime\src\channelCollection.ts:
- ensureNoDataModelChanges: (...args) => {
- packages\runtime\container-runtime\src\dataStoreContext.ts:
- public ensureNoDataModelChanges(callback: () => T): T {
- packages\runtime\runtime-definitions\src\dataStoreContext.ts:
- ensureNoDataModelChanges<T>(callback: () => T): T;
- packages\runtime\test-runtime-utils\src\mocksDataStoreContext.ts:
- public ensureNoDataModelChanges(callback: () => T): T {
This change splits out the changes the PropertiesManger class from
#22751. The goal is to supply a smaller set of changes that are easier
to review, and allow #22751 to be checked in in two parts. After this
all that will be left in #22751 is plumbing the feature through the
layers.
For the broader context of this change, please read the descriptions
from: https://github.com/microsoft/FluidFramework/pull/22751
---------
Co-authored-by: Tyler Butler <tylerbu@microsoft.com>
Co-authored-by: Abram Sanderson <Abram.sanderson@gmail.com>
## Description
This moves `innerNode` into `#hydrationState`, which makes it clearer
what the possible states and transitions are. A side effect of this is
that the `innerNode` property is now `undefined` in the case where a
node has been hydrated by its corresponding flex node has not yet been
generated. Previously, the `innerNode` remained an
`UnhydratedFlexTreeNode` even after hydration, which while not incorrect
was still confusing to see in the debugger. This PR also does some
related cleanups. However, it does not attempt to clean up
TreeNodeKernel entirely - there is still plenty more possible cleanup,
but this PR would like to avoid trying to do everything at once.
Reverts microsoft/FluidFramework#23060
We're seeing a huge increase in errors in our odsp tests against both
prod and dogfood starting with the first run after this PR was merged,
so reverting as the probable culprit until we can investigate further.
## Description
It turns out the reason I couldn't allow constructing recursive maps
from objects previously could be overcome my inlining the types. I don't
have a complete understanding of how inlineing type sometimes allows
recursive types to work better, but in this case it worked.
Previous output allowed undefined in FieldKindIncompatibility instances
where either the stored or view schema did not have some field
implicitly. However, it is also possible to declare a field as not
present explicitly using forbidden, which should be treated identically
when considering compatibility. This updates the output representation
as well as logic to normalize implicit cases to use forbidden.
---------
Co-authored-by: Abram Sanderson <absander@microsoft.com>
## Description
[AB#22192](https://dev.azure.com/fluidframework/internal/_workitems/edit/22192)
This fixes a bug reported by Loop where the reconnecting banner did not
go away after user returned after being dormant for some time.
Reason for the bug:
Due to socket reuse(multiplexing), we can get "disconnect" event from
other clients in the socket reference. So, a race condition could
happen, where this client is establishing connection and listening for
"connect_document_success" on the socket among other events, but we get
"disconnect" event on the socket reference from other clients, in which
case, we dispose connection object and stop listening to further events
on the socket. Due to this we get stuck as the connection is not yet
established and so we don't return any connection object to the client
(connection manager). So, we remain stuck.
In order to handle this, listen for the "disconnect" event and reject
the promise with the error so that the caller can know and handle the
error.
---------
Co-authored-by: Jatin Garg <jatingarg@Jatins-MacBook-Pro-2.local>
## Description
This exposes the `off()` method on the `Listenable` event interface,
providing an additional way to unsubscribe from events. The current way
of unsubscribing, via the deregistration function returned by `on()`, is
still supported. Having both options available is meant to give
customers options without being opinionated about the best pattern, as
it depends on preference and use case.
To prevent the two strategies from interfering with each other, it has
also been made illegal to register the same listener more than once for
the same event. It is now documented as throwing an error rather than as
undefined behavior.
This also adds a test for registering events with keys that are JS
symbols and does some minor doc cleanup.
picocolors is a smaller alternative to chalk, and it is also ESM and
CJS, while chalk went ESM-only and we were stuck on old versions in the
core build-tools project.
build-tools will eventually be ESM, and using a dual-module-format
package will actually make that transition easier because we won't have
to do dependency switches while also doing ESM conversion.
This PR fixes how we save the obliterate sequence number on the
associated `pendingSegmentGroup`. Previously, the sequence number would
never be set for zero-length obliterates, so we would get stuck
considering the obliterate as unsequenced.
This PR also surfaces zero-length obliterates in
`mergeTreeDeltaCallback` as long as the obliterate's endpoints are not
equal.
This change takes the existing logic to load the createNewModule and
encapsulates it in a function to make it more reusable, and less error
prone.
The primary motivation for this change is that in an upcoming change
I'll want to use the createNewModule and don't want to duplicate the
existing logic
By moving the files of the createNewModule under a folder, and not
exporting them directly, it makes it clearer that they should not be
directly imported.
This is a test repro of
[AB#4630](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/4630).
To reproduce such issue, I had to fake the localId generation using
uuidOverride constructor method in the blob manager so we could aim at a
unique pending blob: such pending blob was: 1 stashed, 2 reapplied to a
different blob manager simulating rehydration, 3 processed and 4
reuploaded by stashing workflow, which ends up in the issue described in
the ADO item.
In order to wait for the reupload I had to create a different blob
manager method waitForStashedBlobs() since the one we have
hasPendingStashedUploads() uses pendingBlobs array as reference, which
blob entry would be removed by the processing step.
A fix will follow up this PR.
## Description
This changes all internal SharedTree objects to have an `events:
Listenable` property rather than implementing Listenable directly. Using
the `.events` pattern is preferable over the alternatives because it
does not employ inheritance (like extending `EventEmitter`) and does not
require any method implementation boilerplate (like implementing
`Listenable`). It also means that any changes to `EventEmitter` or
`Listenable` will not require changes to the object emitting the events
- this is the practical motivation for this change, as the `Listenable`
interface will soon be updated to have a new method. Without this
preparatory change, all the implementations of `Listenable` would need
to be updated at that time.
## Description
Fixes several bugs/inconsistencies in `allowsRepoSuperset` and
`isRepoSuperset` and adds unit tests for this behavior.
## Bugs
1. Forbidden fields should not be convertible to required fields, but
`isRepoSuperset` allowed this conversion.
2. The production codepath did not correctly handle queries between
value schema nodes and other types of nodes (without the added code in
`comparison.ts`, one could hit `0x893` directly below)
3. `isRepoSuperset` did not previously permit addition of new node kinds
## Context
`allowsRepoSuperset` is the current implementation used by the
production codepath to check compatibility between a document's stored
and view schema. There is ongoing work on the schema evolution front to
transition this codepath to one which is more flexible / easily
debugged, which is the logic in `discrepancies.ts`. The general
philosophy behind `discrepancies.ts` is to split the compatibility check
into two parts: one bit of code which detects all differences between
view and stored schema, and another bit of code which decides if any of
those differences should prohibit document operations (read, write,
upgrade schema, etc). This PR helps bring the newer implementation to
parity with what we have today.
---------
Co-authored-by: Abram Sanderson <absander@microsoft.com>
## Description
Removes the `MockLogger` class from the `@alpha+@legacy` API surface. It
has been deprecated for external use since 2024-06-18.
## Breaking Changes
- `MockLogger` is now removed from the `@alpha+@legacy` API surface,
becoming only `@internal`.
Fixes:
[AB#6959](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/6959)
## Description
Yang Gao to detail the specifics and data/incident information in the
Feature Description
Optimize getDelta / internal service calls to reduce load on Alfred &
Riddler and minimize impact on customer traffic
FRS has observed some unexpected getDeltas calls. If the client can send
FRS the reasons why the calls were made, it can greatly help the
investigations. One example is that we have seen some getDeltas calls
trying to read very old ops, which were generated tens of days ago, and
it's hard for us to understand what triggered these calls.
## Reviewer Guidance
Let me know if there's a better way to send the reason to AFR.
The changelog for build-tools is very outdated, so I manually
regenerated the entries for releases 0.7.0 - .0.50.0.
Steps:
```shell
git checkout build-tools_v0.48.0
pnpm changelog
```
I then stashed the changes and repeated the steps for each minor
release.
WillieHabi
Daniel Madrid
Joshua Smithrud
Craig Macomber (Microsoft)
Abram Sanderson
Jason Hartman
yann-achard-MS
Tyler Butler
Sonali Deshpande
MarioJGMsoft
Tony Murphy
Noah Encke
Rishhi Balakrishnan
Alex Villarreal
seanimam
Jatin Garg
tyler-cai-microsoft
Timothy Trindle
