microsoft
/
FluidFramework
зеркало из https://github.com/microsoft/FluidFramework.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

feat(migration-tools): Minimize model loading pattern presence in favor of entryPoint (#23119)

This commit is contained in:
Matt Rakow 2024-11-20 15:01:49 -08:00 коммит произвёл GitHub
Родитель 224f59adf4
Коммит d3bf90ca08
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: B5690EEEBB952194
33 изменённых файлов: 901 добавлений и 791 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

4
examples/utils/example-utils/src/modelLoader/modelContainerRuntimeFactory.ts
Просмотреть файл

@ -19,8 +19,8 @@ import { NamedFluidDataStoreRegistryEntries } from "@fluidframework/runtime-defi
/**
* @internal
*/
export interface IModelContainerRuntimeEntryPoint<T> {
getModel(container: IContainer): Promise<T>;
export interface IModelContainerRuntimeEntryPoint<ModelType> {
getModel(container: IContainer): Promise<ModelType>;
}
/**

89
examples/utils/migration-tools/README.md
Просмотреть файл

@ -2,7 +2,94 @@
This package contains tools for migrating data from one version to another, used by Fluid examples. They are not currently intended for use in production scenarios.
See [GitHub](https://github.com/microsoft/FluidFramework) for more details on the Fluid Framework and packages within.
Use of the migration tools imposes several requirements on the container code and application, detailed here.
## Implementing `IMigratableModel`
Your data model must implement `IMigratableModel` to be migrated using the migration tools.
This includes:
1. A `version` string to identify the model version.
1. Methods to export and import data, and to detect if the model supports a given data format:
1. `importData: (initialData: ImportType) => Promise<void>`
1. `exportData: () => Promise<ExportType>`
1. `supportsDataFormat: (initialData: unknown) => initialData is ImportType`
1. A `dispose` method to clean up the container - most likely calling `IContainer.dispose`.
## Implementing the composite runtime pattern
See documentation for the composite runtime pattern [here](./src/compositeRuntime/README.md).
The migration tools expect to find an `IMigratableModel` by accessing and calling a `getModel()` function provided on the `entryPoint`. They also expect to find an `IMigrationTool` by accessing a `migrationTool` member of the `entryPoint`. These requirements are most easily satisfied by using the composite runtime pattern.
`getModel()` is a function that takes an `IContainer` to aid in producing the `IMigratableModel`. This is because the contract of `IMigratableModel` likely requires functionality from `IContainer` (especially `IContainer.dispose()`).
### Defining the entry point piece
```ts
const rootDatastoreAlias = "my-root-datastore";
export const getModelEntryPointPiece: IEntryPointPiece = {
name: "getModel",
registryEntries: [MyRootDatastoreFactory.registryEntry],
onCreate: async (runtime: IContainerRuntime): Promise<void> => {
const rootDatastore = await runtime.createDataStore(MyRootDatastoreFactory.type);
await rootDatastore.trySetAlias(rootDatastoreAlias);
},
onLoad: async (runtime: IContainerRuntime): Promise<void> => {},
createPiece: async (runtime: IContainerRuntime): Promise<(container: IContainer) => Promise<FluidObject>> => {
const entryPointHandle = await containerRuntime.getAliasedDataStoreEntryPoint(rootDatastoreAlias);
if (entryPointHandle === undefined) {
throw new Error(`Default dataStore [${rootDatastoreAlias}] must exist`);
}
// Entry points are typed as FluidObject and must be cast. Here we know it's a MyRootDatastore since
// we created it just above. Type validation can be added here if desired.
const rootDatastore = entryPointHandle.get() as Promise<MyRootDatastore>;
// MigratableAppModel (defined by the container code author) must implement IMigratableModel.
// Note that we're returning a function of type (container: IContainer) => Promise<FluidObject>,
// where the FluidObject is expected to be an IMigratableModel.
return async (container: IContainer) => new MigratableAppModel(rootDatastore, container);
},
};
```
```ts
// In the IRuntimeFactory
public async instantiateRuntime(
context: IContainerContext,
existing: boolean,
): Promise<IRuntime> {
const compositeEntryPoint = new CompositeEntryPoint();
compositeEntryPoint.addEntryPointPiece(getModelEntryPointPiece);
// migrationToolEntryPointPiece is provided by the migration-tools package
compositeEntryPoint.addEntryPointPiece(migrationToolEntryPointPiece);
return loadCompositeRuntime(context, existing, compositeEntryPoint, this.runtimeOptions);
}
```
### `migrationToolEntryPointPiece`
This package additionally provides a `migrationToolEntryPointPiece` which is an off-the-shelf implementation of the piece to provide the `IMigrationTool`. With these provided pieces, you're only responsible for implementing the `IMigratableModel` piece with your data model.
## `Migrator`
Finally, to actually execute the migration we provide the `Migrator` class. This takes a `SimpleLoader` (see below), the initially loaded model, migration tool, and container ID (TODO: can we simplify this handoff), as well as an optional `DataTransformationCallback` (see below). The migrator provides a collection of APIs to observe the state of the migration, as well as to acquire the new container after migration completes. (TODO: should the migrate() API also live here?)
TODO: Detail usage of the Migrator
### `SimpleLoader`
See documentation for `SimpleLoader` [here](./src/simpleLoader/README.md). `SimpleLoader` is used in place of a `Loader` and is used by the `Migrator`.
### Code loader
To migrate between two different code versions, you must also provide a code loader to the `SimpleLoader` that is capable of loading those two respective code versions. This uses the usual `ICodeDetailsLoader` interface.
### `DataTransformationCallback`
If your old and new code share an import/export format, you don't need a `DataTransformationCallback`. But if the import/export format has changed between versions, you can provide this callback to the `Migrator` and it will be called with the old exported data. This callback is responsible for transforming the data to the new format and returning the transformed data.
<!-- AUTO-GENERATED-CONTENT:START (README_FOOTER) -->

143
examples/utils/migration-tools/api-report/migration-tools.alpha.api.md
Просмотреть файл

@ -4,8 +4,19 @@
```ts
// @alpha
export type CreateModelCallback<ModelType> = (runtime: IContainerRuntime, container: IContainer) => Promise<ModelType>;
// @alpha (undocumented)
export class CompositeEntryPoint {
// (undocumented)
readonly addEntryPointPiece: (entryPointPiece: IEntryPointPiece) => void;
// (undocumented)
readonly onCreate: (runtime: IContainerRuntime) => Promise<void>;
// (undocumented)
readonly onLoad: (runtime: IContainerRuntime) => Promise<void>;
// (undocumented)
readonly provideEntryPoint: (runtime: IContainerRuntime) => Promise<Record<string, FluidObject>>;
// (undocumented)
get registryEntries(): NamedFluidDataStoreRegistryEntries;
}
// @alpha
export type DataTransformationCallback = (exportedData: unknown, modelVersion: string) => Promise<unknown>;
@ -16,17 +27,18 @@ export interface IAcceptedMigrationDetails {
newVersion: string;
}
// @alpha
export interface IAttachedMigratableModel<ModelType> {
migrationTool: IMigrationTool;
model: ModelType;
}
// @alpha
export interface IDetachedMigratableModel<ModelType> {
attach: () => Promise<string>;
migrationTool: IMigrationTool;
model: ModelType;
// @alpha (undocumented)
export interface IEntryPointPiece {
// (undocumented)
readonly createPiece: (runtime: IContainerRuntime) => Promise<FluidObject>;
// (undocumented)
readonly name: string;
// (undocumented)
readonly onCreate: (runtime: IContainerRuntime) => Promise<void>;
// (undocumented)
readonly onLoad: (runtime: IContainerRuntime) => Promise<void>;
// (undocumented)
readonly registryEntries: NamedFluidDataStoreRegistryEntries;
}
// @alpha
@ -41,23 +53,6 @@ export interface IMigratableModel extends IVersionedModel, IImportExportModel<un
dispose(): void;
}
// @alpha (undocumented)
export interface IMigratableModelContainerRuntimeEntryPoint<T> {
// (undocumented)
getModelAndMigrationTool(container: IContainer): Promise<{
model: T;
migrationTool: IMigrationTool;
}>;
}
// @alpha (undocumented)
export interface IMigratableModelLoader<ModelType> {
createDetached(version: string): Promise<IDetachedMigratableModel<ModelType>>;
loadExisting(id: string): Promise<IAttachedMigratableModel<ModelType>>;
loadExistingToSequenceNumber(id: string, sequenceNumber: number): Promise<IAttachedMigratableModel<ModelType>>;
supportsVersion(version: string): Promise<boolean>;
}
// @alpha (undocumented)
export interface IMigrationTool {
readonly acceptedMigration: IAcceptedMigrationDetails | undefined;
@ -102,58 +97,30 @@ export interface IMigratorEvents extends IEvent {
(event: "migrationNotSupported", listener: (version: string) => void): any;
}
// @alpha
export const instantiateMigratableRuntime: <ModelType>(context: IContainerContext, existing: boolean, registryEntries: NamedFluidDataStoreRegistryEntries, createModel: CreateModelCallback<ModelType>, runtimeOptions?: IContainerRuntimeOptions) => Promise<IContainerRuntime & IRuntime>;
// @alpha (undocumented)
export interface ISimpleLoader {
createDetached(version: string): Promise<{
container: IContainer;
attach: () => Promise<string>;
}>;
loadExisting(id: string): Promise<IContainer>;
supportsVersion(version: string): Promise<boolean>;
}
// @alpha
export interface IVersionedModel {
readonly version: string;
}
// @alpha (undocumented)
export class MigratableModelLoader<ModelType> implements IMigratableModelLoader<ModelType> {
constructor(props: Pick<ILoaderProps, "urlResolver" | "documentServiceFactory" | "codeLoader" | "logger"> & {
generateCreateNewRequest: () => IRequest;
});
// (undocumented)
createDetached(version: string): Promise<IDetachedMigratableModel<ModelType>>;
// (undocumented)
loadExisting(id: string): Promise<IAttachedMigratableModel<ModelType>>;
// (undocumented)
loadExistingToSequenceNumber(id: string, sequenceNumber: number): Promise<IAttachedMigratableModel<ModelType>>;
// (undocumented)
supportsVersion(version: string): Promise<boolean>;
}
// @alpha (undocumented)
export class MigratableSessionStorageModelLoader<ModelType> implements IMigratableModelLoader<ModelType> {
constructor(codeLoader: ICodeDetailsLoader, logger?: ITelemetryBaseLogger | undefined);
// (undocumented)
createDetached(version: string): Promise<IDetachedMigratableModel<ModelType>>;
// (undocumented)
loadExisting(id: string): Promise<IAttachedMigratableModel<ModelType>>;
// (undocumented)
loadExistingToSequenceNumber(id: string, sequenceNumber: number): Promise<IAttachedMigratableModel<ModelType>>;
// (undocumented)
supportsVersion(version: string): Promise<boolean>;
}
// @alpha (undocumented)
export class MigratableTinyliciousModelLoader<ModelType> implements IMigratableModelLoader<ModelType> {
constructor(codeLoader: ICodeDetailsLoader);
// (undocumented)
createDetached(version: string): Promise<IDetachedMigratableModel<ModelType>>;
// (undocumented)
loadExisting(id: string): Promise<IAttachedMigratableModel<ModelType>>;
// (undocumented)
loadExistingToSequenceNumber(id: string, sequenceNumber: number): Promise<IAttachedMigratableModel<ModelType>>;
// (undocumented)
supportsVersion(version: string): Promise<boolean>;
}
// @alpha
export const loadCompositeRuntime: (context: IContainerContext, existing: boolean, compositeEntryPoint: CompositeEntryPoint, runtimeOptions?: IContainerRuntimeOptions) => Promise<IContainerRuntime & IRuntime>;
// @alpha
export type MigrationState = "collaborating" | "stopping" | "migrating" | "migrated";
// @alpha (undocumented)
export const migrationToolEntryPointPiece: IEntryPointPiece;
// @alpha (undocumented)
export class MigrationToolFactory implements IFluidDataStoreFactory {
// (undocumented)
@ -166,7 +133,7 @@ export class MigrationToolFactory implements IFluidDataStoreFactory {
// @alpha
export class Migrator implements IMigrator {
constructor(modelLoader: IMigratableModelLoader<IMigratableModel>, initialMigratable: IMigratableModel, initialMigrationTool: IMigrationTool, initialId: string, dataTransformationCallback?: DataTransformationCallback | undefined);
constructor(simpleLoader: ISimpleLoader, initialMigratable: IMigratableModel, initialMigrationTool: IMigrationTool, initialId: string, dataTransformationCallback?: DataTransformationCallback | undefined);
// (undocumented)
get connected(): boolean;
// (undocumented)
@ -181,4 +148,34 @@ export class Migrator implements IMigrator {
get migrationState(): MigrationState;
}
// @alpha (undocumented)
export class SessionStorageSimpleLoader implements ISimpleLoader {
constructor(codeLoader: ICodeDetailsLoader, logger?: ITelemetryBaseLogger | undefined);
// (undocumented)
createDetached(version: string): Promise<{
container: IContainer;
attach: () => Promise<string>;
}>;
// (undocumented)
loadExisting(id: string): Promise<IContainer>;
// (undocumented)
supportsVersion(version: string): Promise<boolean>;
}
// @alpha (undocumented)
export class SimpleLoader implements ISimpleLoader {
constructor(props: Pick<ILoaderProps, "urlResolver" | "documentServiceFactory" | "codeLoader" | "logger"> & {
generateCreateNewRequest: () => IRequest;
});
// (undocumented)
createDetached(version: string): Promise<{
container: IContainer;
attach: () => Promise<string>;
}>;
// (undocumented)
loadExisting(id: string): Promise<IContainer>;
// (undocumented)
supportsVersion(version: string): Promise<boolean>;
}
```

68
examples/utils/migration-tools/src/compositeRuntime/README.md Normal file
Просмотреть файл

@ -0,0 +1,68 @@
## The composite runtime pattern
Fluid containers provide an `entryPoint`, which is how apps access the contents of the container. This `entryPoint` is specified by the author of the "container code", also known as the "container runtime factory" or `IRuntimeFactory`.
Traditionally the container code author creates a single root datastore, and accessing the container `entryPoint` simply returns that datastore. However, the `entryPoint` can actually be any arbitrary object (it is typed as a `FluidObject`).
The composite runtime pattern explores returning an object that is composed of multiple members, each independent from one another. This facilitates mixin patterns, such as adding a data migration tool to a container without impacting the root datastore.
This package provides a `CompositeEntryPoint`, which collects entry point "pieces" that are defined by the container code author (`IEntryPointPiece`). `CompositeEntryPoint` can subsequently be used with `loadCompositeRuntime()` in place of `ContainerRuntime.loadRuntime()` to produce a runtime with the desired `entryPoint`.
Each `IEntryPointPiece` consists of:
* `name`: The name that the entry point piece will be given in the resulting composite entryPoint.
* `registryEntries`: The registry entries that should be added to the container runtime.
* `onCreate`: Actions to be taken upon container creation, e.g. creating and aliasing a datastore.
* `onLoad`: Actions to be taken upon every container load.
* `createPiece`: A function to produce the entry point piece object that the app developer will access.
### Defining the entry point piece
```ts
const rootDatastoreAlias = "my-root-datastore";
export const rootDatastoreEntryPointPiece: IEntryPointPiece = {
name: "rootDatastore",
registryEntries: [MyRootDatastoreFactory.registryEntry],
onCreate: async (runtime: IContainerRuntime): Promise<void> => {
const rootDatastore = await runtime.createDataStore(MyRootDatastoreFactory.type);
await rootDatastore.trySetAlias(rootDatastoreAlias);
},
onLoad: async (runtime: IContainerRuntime): Promise<void> => {},
createPiece: async (runtime: IContainerRuntime): Promise<FluidObject> => {
const entryPointHandle = await containerRuntime.getAliasedDataStoreEntryPoint(rootDatastoreAlias);
if (entryPointHandle === undefined) {
throw new Error(`Default dataStore [${rootDatastoreAlias}] must exist`);
}
return entryPointHandle.get();
},
};
```
### Composing and loading the runtime
```ts
// In the IRuntimeFactory
public async instantiateRuntime(
context: IContainerContext,
existing: boolean,
): Promise<IRuntime> {
const compositeEntryPoint = new CompositeEntryPoint();
compositeEntryPoint.addEntryPointPiece(rootDatastoreEntryPointPiece);
// migrationToolEntryPointPiece is provided by the migration-tools package
compositeEntryPoint.addEntryPointPiece(migrationToolEntryPointPiece);
return loadCompositeRuntime(context, existing, compositeEntryPoint, this.runtimeOptions);
}
```
### Accessing the composite entryPoint from the app
```ts
// Entry points are typed as FluidObject and must be cast. Type validation can be added here if desired.
const { rootDatastore, migrationTool } = (await container.getEntryPoint()) as {
rootDatastore: MyRootDatastore;
migrationTool: IMigrationTool;
};
```

10
examples/utils/migration-tools/src/compositeRuntime/index.ts Normal file
Просмотреть файл

@ -0,0 +1,10 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/
export { IEntryPointPiece } from "./interfaces.js";
export {
CompositeEntryPoint,
loadCompositeRuntime,
} from "./loadCompositeRuntime.js";

19
examples/utils/migration-tools/src/compositeRuntime/interfaces.ts Normal file
Просмотреть файл

@ -0,0 +1,19 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/
import type { IContainerRuntime } from "@fluidframework/container-runtime-definitions/internal";
import type { FluidObject } from "@fluidframework/core-interfaces";
import type { NamedFluidDataStoreRegistryEntries } from "@fluidframework/runtime-definitions/internal";
/**
* @alpha
*/
export interface IEntryPointPiece {
readonly name: string;
readonly registryEntries: NamedFluidDataStoreRegistryEntries;
readonly onCreate: (runtime: IContainerRuntime) => Promise<void>;
readonly onLoad: (runtime: IContainerRuntime) => Promise<void>;
readonly createPiece: (runtime: IContainerRuntime) => Promise<FluidObject>;
}

89
examples/utils/migration-tools/src/compositeRuntime/loadCompositeRuntime.ts Normal file
Просмотреть файл

@ -0,0 +1,89 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/
import { IContainerContext, IRuntime } from "@fluidframework/container-definitions/internal";
import {
ContainerRuntime,
IContainerRuntimeOptions,
} from "@fluidframework/container-runtime/internal";
import { IContainerRuntime } from "@fluidframework/container-runtime-definitions/internal";
import type { FluidObject } from "@fluidframework/core-interfaces";
import type {
NamedFluidDataStoreRegistryEntries,
NamedFluidDataStoreRegistryEntry2,
} from "@fluidframework/runtime-definitions/internal";
import type { IEntryPointPiece } from "./interfaces.js";
// TODO: CompositeEntryPoint isn't really the right name - this is more like CompositeContainerContents
// or CompositeContainerCode?
/**
* @alpha
*/
export class CompositeEntryPoint {
private readonly _entryPointPieces: IEntryPointPiece[] = [];
// TODO: Consider taking a "name" argument here, and don't include "name" on the IEntryPointPiece
// Or maybe allow a default name from the piece but allow override here?
public readonly addEntryPointPiece = (entryPointPiece: IEntryPointPiece): void => {
// TODO: Consider validating no conflicts (e.g. name already exists, registry entry collision)
this._entryPointPieces.push(entryPointPiece);
};
public get registryEntries(): NamedFluidDataStoreRegistryEntries {
const registryEntries: NamedFluidDataStoreRegistryEntry2[] = [];
for (const entryPointPiece of this._entryPointPieces) {
registryEntries.push(...entryPointPiece.registryEntries);
}
return registryEntries;
}
public readonly onCreate = async (runtime: IContainerRuntime): Promise<void> => {
for (const entryPointPiece of this._entryPointPieces) {
await entryPointPiece.onCreate(runtime);
}
};
public readonly onLoad = async (runtime: IContainerRuntime): Promise<void> => {
for (const entryPointPiece of this._entryPointPieces) {
await entryPointPiece.onLoad(runtime);
}
};
public readonly provideEntryPoint = async (
runtime: IContainerRuntime,
): Promise<Record<string, FluidObject>> => {
const entryPoint: Record<string, FluidObject> = {};
for (const entryPointPiece of this._entryPointPieces) {
entryPoint[entryPointPiece.name] = await entryPointPiece.createPiece(runtime);
}
return entryPoint;
};
}
/**
* TODO: Make lint happy
* @alpha
*/
export const loadCompositeRuntime = async (
context: IContainerContext,
existing: boolean,
compositeEntryPoint: CompositeEntryPoint,
runtimeOptions?: IContainerRuntimeOptions,
): Promise<IContainerRuntime & IRuntime> => {
const runtime = await ContainerRuntime.loadRuntime({
context,
registryEntries: compositeEntryPoint.registryEntries,
provideEntryPoint: compositeEntryPoint.provideEntryPoint,
runtimeOptions,
existing,
});
if (!existing) {
await compositeEntryPoint.onCreate(runtime);
}
await compositeEntryPoint.onLoad(runtime);
return runtime;
};

39
examples/utils/migration-tools/src/index.ts
Просмотреть файл

@ -9,28 +9,31 @@
* They are not currently intended for use in production scenarios.
*/
export type {
DataTransformationCallback,
export {
CompositeEntryPoint,
IEntryPointPiece,
loadCompositeRuntime,
} from "./compositeRuntime/index.js";
export {
IAcceptedMigrationDetails,
IImportExportModel,
IMigratableModel,
IMigrationTool,
IMigrationToolEvents,
MigrationState,
migrationToolEntryPointPiece,
MigrationToolFactory,
} from "./migrationTool/index.js";
export {
DataTransformationCallback,
getModelAndMigrationToolFromContainer,
IImportExportModel,
IMigratableModel,
IMigrator,
IMigratorEvents,
IVersionedModel,
MigrationState,
} from "./interfaces/index.js";
export { MigrationToolFactory } from "./migrationTool.js";
export { Migrator } from "./migrator.js";
Migrator,
} from "./migrator/index.js";
export {
CreateModelCallback,
IAttachedMigratableModel,
IDetachedMigratableModel,
IMigratableModelContainerRuntimeEntryPoint,
IMigratableModelLoader,
instantiateMigratableRuntime,
MigratableModelLoader,
MigratableSessionStorageModelLoader,
MigratableTinyliciousModelLoader,
} from "./modelLoader/index.js";
ISimpleLoader,
SessionStorageSimpleLoader,
SimpleLoader,
} from "./simpleLoader/index.js";

52
examples/utils/migration-tools/src/interfaces/migrator.ts
Просмотреть файл

@ -1,52 +0,0 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/
import type { IEvent, IEventProvider } from "@fluidframework/core-interfaces";
import type { IMigratableModel } from "./migratableModel.js";
import type { MigrationState } from "./migrationTool.js";
/**
* The DataTransformationCallback gives an opportunity to modify the exported data before attempting an import
* to the new model. The modelVersion is also provided to inform the appropriate transformation to perform.
* It is async to permit network calls or lazy-loading the transform logic within the function.
* @alpha
*/
export type DataTransformationCallback = (
exportedData: unknown,
modelVersion: string,
) => Promise<unknown>;
/**
* @alpha
*/
export interface IMigratorEvents extends IEvent {
(event: "migrated" | "migrating", listener: () => void);
(event: "migrationNotSupported", listener: (version: string) => void);
}
/**
* @alpha
*/
export interface IMigrator {
readonly events: IEventProvider<IMigratorEvents>;
/**
* The currently monitored migratable model. As the Migrator completes a migration, it will swap in the new
* migrated model and emit a "migrated" event.
*/
readonly currentModel: IMigratableModel;
/**
* The container id of the current model.
*/
readonly currentModelId: string;
/**
* The migration state of the current model. Note that since we swap out for the new model as soon as migration
* completes, we'll only ever see this as collaborating or migrating, never migrated.
*/
readonly migrationState: MigrationState;
}

13
examples/utils/migration-tools/src/migrationTool/index.ts Normal file
Просмотреть файл

@ -0,0 +1,13 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/
export {
IAcceptedMigrationDetails,
IMigrationTool,
IMigrationToolEvents,
MigrationState,
} from "./interfaces.js";
export { MigrationToolFactory } from "./migrationTool.js";
export { migrationToolEntryPointPiece } from "./migrationToolEntryPointPiece.js";

0
examples/utils/migration-tools/src/interfaces/migrationTool.ts → examples/utils/migration-tools/src/migrationTool/interfaces.ts
Просмотреть файл

7
examples/utils/migration-tools/src/migrationTool.ts → examples/utils/migration-tools/src/migrationTool/migrationTool.ts
Просмотреть файл

@ -32,7 +32,7 @@ import type {
IMigrationTool,
IMigrationToolEvents,
MigrationState,
} from "./interfaces/index.js";
} from "./interfaces.js";
const consensusRegisterCollectionId = "consensus-register-collection";
const pactMapId = "pact-map";
@ -101,9 +101,6 @@ class MigrationTool implements IMigrationTool {
});
this.pactMap.on("pending", (key: string) => {
if (key === newVersionKey) {
// TODO: Here take some action to prevent collaboration - the host might not provide a Migrator,
// and we need to ensure we still don't break expectations in that case. Note that during this
// event firing the pactMap has not yet sent its accept op though.
this._events.emit("stopping");
}
});
@ -111,8 +108,6 @@ class MigrationTool implements IMigrationTool {
this.pactMap.on("accepted", (key: string) => {
if (key === newVersionKey) {
this._events.emit("migrating");
// TODO: Here we should stop submitting new summaries since it will complicate loading from the
// accepted sequence number
}
});

52
examples/utils/migration-tools/src/migrationTool/migrationToolEntryPointPiece.ts Normal file
Просмотреть файл

@ -0,0 +1,52 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/
import type { IContainerRuntime } from "@fluidframework/container-runtime-definitions/internal";
import type { FluidObject } from "@fluidframework/core-interfaces";
import type { IEntryPointPiece } from "../compositeRuntime/index.js";
import { MigrationToolFactory } from "./migrationTool.js";
const migrationToolEntryPointPieceName = "migrationTool";
const migrationToolRegistryKey = "migration-tool";
const migrationToolFactory = new MigrationToolFactory();
const migrationToolId = "migration-tool";
async function getDataStoreEntryPoint(
containerRuntime: IContainerRuntime,
alias: string,
): Promise<FluidObject> {
const entryPointHandle = await containerRuntime.getAliasedDataStoreEntryPoint(alias);
if (entryPointHandle === undefined) {
throw new Error(`Default dataStore [${alias}] must exist`);
}
return entryPointHandle.get();
}
/**
* @alpha
*/
export const migrationToolEntryPointPiece: IEntryPointPiece = {
name: migrationToolEntryPointPieceName,
registryEntries: [[migrationToolRegistryKey, Promise.resolve(migrationToolFactory)]],
onCreate: async (runtime: IContainerRuntime): Promise<void> => {
const migrationTool = await runtime.createDataStore(migrationToolRegistryKey);
await migrationTool.trySetAlias(migrationToolId);
},
onLoad: async (runtime: IContainerRuntime): Promise<void> => {
// Force the MigrationTool to instantiate in all cases. The PactMap it uses must be loaded and running in
// order to respond with accept ops, and without this call the MigrationTool won't be instantiated on the
// summarizer client.
await getDataStoreEntryPoint(runtime, migrationToolId);
},
createPiece: async (runtime: IContainerRuntime): Promise<FluidObject> => {
return getDataStoreEntryPoint(runtime, migrationToolId);
},
};

16
examples/utils/migration-tools/src/interfaces/index.ts → examples/utils/migration-tools/src/migrator/index.ts
Просмотреть файл

@ -4,18 +4,14 @@
*/
export {
DataTransformationCallback,
IImportExportModel,
IMigratableModel,
IVersionedModel,
} from "./migratableModel.js";
export {
IAcceptedMigrationDetails,
IMigrationTool,
IMigrationToolEvents,
MigrationState,
} from "./migrationTool.js";
export {
DataTransformationCallback,
IMigrator,
IMigratorEvents,
IVersionedModel,
} from "./interfaces.js";
export {
getModelAndMigrationToolFromContainer,
Migrator,
} from "./migrator.js";

63
examples/utils/migration-tools/src/interfaces/migratableModel.ts → examples/utils/migration-tools/src/migrator/interfaces.ts
Просмотреть файл

@ -3,6 +3,12 @@
* Licensed under the MIT License.
*/
import type { IEvent, IEventProvider } from "@fluidframework/core-interfaces";
import type { MigrationState } from "../migrationTool/index.js";
// #region IMigratableModel
/**
* A model with a detectable version.
*
@ -22,12 +28,6 @@ export interface IVersionedModel {
* @alpha
*/
export interface IImportExportModel<ImportType, ExportType> {
/**
* Permit format checking in a generic manner - without knowing the type of our data or the type of the model,
* we can still check whether the model supports that data.
*/
supportsDataFormat: (initialData: unknown) => initialData is ImportType;
/**
* importData must be called after initialization but before modifying or attaching the model (i.e. can only
* be called on an unaltered, detached model).
@ -38,6 +38,12 @@ export interface IImportExportModel<ImportType, ExportType> {
* Export the data from the model. Can be passed into importData() for a new container to replicate the data.
*/
exportData: () => Promise<ExportType>;
/**
* Permit format checking in a generic manner - without knowing the type of our data or the type of the model,
* we can still check whether the model supports that data.
*/
supportsDataFormat: (initialData: unknown) => initialData is ImportType;
}
// TODO: Is there a better way to express the unknown format here? I think I'd prefer to put the burden of calling
@ -70,3 +76,48 @@ export interface IMigratableModel
*/
dispose(): void;
}
// #region IMigrator
/**
* The DataTransformationCallback gives an opportunity to modify the exported data before attempting an import
* to the new model. The modelVersion is also provided to inform the appropriate transformation to perform.
* It is async to permit network calls or lazy-loading the transform logic within the function.
* @alpha
*/
export type DataTransformationCallback = (
exportedData: unknown,
modelVersion: string,
) => Promise<unknown>;
/**
* @alpha
*/
export interface IMigratorEvents extends IEvent {
(event: "migrated" | "migrating", listener: () => void);
(event: "migrationNotSupported", listener: (version: string) => void);
}
/**
* @alpha
*/
export interface IMigrator {
readonly events: IEventProvider<IMigratorEvents>;
/**
* The currently monitored migratable model. As the Migrator completes a migration, it will swap in the new
* migrated model and emit a "migrated" event.
*/
readonly currentModel: IMigratableModel;
/**
* The container id of the current model.
*/
readonly currentModelId: string;
/**
* The migration state of the current model. Note that since we swap out for the new model as soon as migration
* completes, we'll only ever see this as collaborating or migrating, never migrated.
*/
readonly migrationState: MigrationState;
}

75
examples/utils/migration-tools/src/migrator.ts → examples/utils/migration-tools/src/migrator/migrator.ts
Просмотреть файл

@ -4,18 +4,52 @@
*/
import { TypedEventEmitter } from "@fluid-internal/client-utils";
import type { IContainer } from "@fluidframework/container-definitions/internal";
import type { IEventProvider } from "@fluidframework/core-interfaces";
import { assert } from "@fluidframework/core-utils/internal";
import type { IMigrationTool, MigrationState } from "../migrationTool/index.js";
import { type ISimpleLoader, waitForAtLeastSequenceNumber } from "../simpleLoader/index.js";
import type {
DataTransformationCallback,
IMigratableModel,
IMigrationTool,
IMigrator,
IMigratorEvents,
MigrationState,
} from "./interfaces/index.js";
import type { IDetachedMigratableModel, IMigratableModelLoader } from "./modelLoader/index.js";
} from "./interfaces.js";
// TODO: This probably shouldn't be exported, consider having the migrator get its own model/tool out.
/**
* The purpose of the model pattern and the model loader is to wrap the IContainer in a more useful object and
* interface. This demo uses a convention of the entrypoint providing a getModelAndMigrationTool method to do so.
* It does this with the expectation that the model has been bundled with the container code.
*
* Other strategies to obtain the wrapping model could also work fine here - for example a standalone model code
* loader that separately fetches model code and wraps the container from the outside.
* @internal
*/
export const getModelAndMigrationToolFromContainer = async <ModelType>(
container: IContainer,
): Promise<{ model: ModelType; migrationTool: IMigrationTool }> => {
// TODO: Fix typing here
const entryPoint = (await container.getEntryPoint()) as {
getModel: (container: IContainer) => Promise<ModelType>;
migrationTool: IMigrationTool;
};
// If the user tries to use this model loader with an incompatible container runtime, we want to give them
// a comprehensible error message. So distrust the type by default and do some basic type checking.
if (typeof entryPoint.getModel !== "function") {
throw new TypeError("Incompatible container runtime: doesn't provide getModel");
}
const model = await entryPoint.getModel(container);
if (typeof model !== "object") {
throw new TypeError("Incompatible container runtime: doesn't provide model");
}
if (typeof entryPoint.migrationTool !== "object") {
throw new TypeError("Incompatible container runtime: doesn't provide migrationTool");
}
return { model, migrationTool: entryPoint.migrationTool };
};
/**
* As the Migrator migrates, it updates its reference to the current version of the model.
@ -72,10 +106,13 @@ export class Migrator implements IMigrator {
*/
private _migratedLoadP: Promise<void> | undefined;
// TODO: Better typing, decide if we can just retain attach()
/**
* Detached model that is ready to attach. This is stored for retry scenarios.
*/
private _preparedDetachedModel: IDetachedMigratableModel<IMigratableModel> | undefined;
private _preparedDetachedModel:
| { container: IContainer; attach: () => Promise<string> }
| undefined;
/**
* After attaching the prepared model, but before we have written its ID into the current model, we'll store the ID
@ -84,7 +121,7 @@ export class Migrator implements IMigrator {
private _preparedModelId: string | undefined;
public constructor(
private readonly modelLoader: IMigratableModelLoader<IMigratableModel>,
private readonly simpleLoader: ISimpleLoader,
initialMigratable: IMigratableModel,
initialMigrationTool: IMigrationTool,
initialId: string,
@ -159,7 +196,8 @@ export class Migrator implements IMigrator {
// forward, or at least advise the end user to refresh the page or something.
// TODO: Does the app developer have everything they need to dispose gracefully when recovering with
// a new MigratableModelLoader?
const migrationSupported = await this.modelLoader.supportsVersion(
// TODO: Does the above TODO still matter now that this uses SimpleLoader?
const migrationSupported = await this.simpleLoader.supportsVersion(
acceptedMigration.newVersion,
);
if (!migrationSupported) {
@ -168,19 +206,27 @@ export class Migrator implements IMigrator {
return;
}
const detachedModel = await this.modelLoader.createDetached(
const detachedContainer = await this.simpleLoader.createDetached(
acceptedMigration.newVersion,
);
const migratedModel = detachedModel.model;
const { model: detachedModel } =
await getModelAndMigrationToolFromContainer<IMigratableModel>(
detachedContainer.container,
);
const migratedModel = detachedModel;
// Here we load the model to at least the acceptance sequence number and export. We do this with a
// separately loaded model to ensure we don't include any local un-ack'd changes. Late-arriving messages
// may or may not make it into the migrated data, there is no guarantee either way.
// TODO: Consider making this a read-only client
const { model: exportModel } = await this.modelLoader.loadExistingToSequenceNumber(
this.currentModelId,
const container = await this.simpleLoader.loadExisting(this.currentModelId);
await waitForAtLeastSequenceNumber(
container,
acceptedMigration.migrationSequenceNumber,
);
// TODO: verify IMigratableModel
const { model: exportModel } =
await getModelAndMigrationToolFromContainer<IMigratableModel>(container);
const exportedData = await exportModel.exportData();
exportModel.dispose();
@ -217,7 +263,7 @@ export class Migrator implements IMigrator {
await migratedModel.importData(transformedData);
// Store the detached model for later use and retry scenarios
this._preparedDetachedModel = detachedModel;
this._preparedDetachedModel = detachedContainer;
};
const completeTheMigration = async (): Promise<void> => {
@ -319,7 +365,7 @@ export class Migrator implements IMigrator {
const doTheLoad = async (): Promise<void> => {
// doTheLoad() should only be called once. It will resolve once we complete loading.
const migrationSupported = await this.modelLoader.supportsVersion(
const migrationSupported = await this.simpleLoader.supportsVersion(
acceptedMigration.newVersion,
);
if (!migrationSupported) {
@ -327,8 +373,9 @@ export class Migrator implements IMigrator {
this._migratedLoadP = undefined;
return;
}
const migratedContainer = await this.simpleLoader.loadExisting(migratedId);
const { model: migratedModel, migrationTool: migratedMigrationTool } =
await this.modelLoader.loadExisting(migratedId);
await getModelAndMigrationToolFromContainer<IMigratableModel>(migratedContainer);
// Note: I'm choosing not to dispose the old migratable here, and instead allow the lifecycle management
// of the migratable to be the responsibility of whoever created the Migrator (and handed it its first
// migratable). It could also be fine to dispose here, just need to have an explicit contract to clarify

18
examples/utils/migration-tools/src/modelLoader/index.ts
Просмотреть файл

@ -1,18 +0,0 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/
export {
CreateModelCallback,
IMigratableModelContainerRuntimeEntryPoint,
instantiateMigratableRuntime,
} from "./instantiateMigratableRuntime.js";
export {
IAttachedMigratableModel,
IDetachedMigratableModel,
IMigratableModelLoader,
} from "./interfaces.js";
export { MigratableModelLoader } from "./migratableModelLoader.js";
export { MigratableSessionStorageModelLoader } from "./migratableSessionStorageModelLoader.js";
export { MigratableTinyliciousModelLoader } from "./migratableTinyliciousModelLoader.js";

110
examples/utils/migration-tools/src/modelLoader/instantiateMigratableRuntime.ts
Просмотреть файл

@ -1,110 +0,0 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/
import {
IContainer,
IContainerContext,
IRuntime,
} from "@fluidframework/container-definitions/internal";
import {
ContainerRuntime,
IContainerRuntimeOptions,
} from "@fluidframework/container-runtime/internal";
import { IContainerRuntime } from "@fluidframework/container-runtime-definitions/internal";
import type { IFluidHandle } from "@fluidframework/core-interfaces";
import { NamedFluidDataStoreRegistryEntries } from "@fluidframework/runtime-definitions/internal";
import type { IMigrationTool } from "../interfaces/index.js";
import { MigrationToolFactory } from "../migrationTool.js";
async function getDataStoreEntryPoint<T>(
containerRuntime: IContainerRuntime,
alias: string,
): Promise<T> {
const entryPointHandle = (await containerRuntime.getAliasedDataStoreEntryPoint(alias)) as
| IFluidHandle<T>
| undefined;
if (entryPointHandle === undefined) {
throw new Error(`Default dataStore [${alias}] must exist`);
}
return entryPointHandle.get();
}
/**
* The CreateModelCallback should use the passed runtime and container to construct the model that the
* host app will interact with.
* @alpha
*/
export type CreateModelCallback<ModelType> = (
runtime: IContainerRuntime,
container: IContainer,
) => Promise<ModelType>;
/**
* @privateRemarks
* The MigratableModelLoader expects to work with container runtimes whose entry point conforms to
* this interface.
* @alpha
*/
export interface IMigratableModelContainerRuntimeEntryPoint<T> {
getModelAndMigrationTool(
container: IContainer,
): Promise<{ model: T; migrationTool: IMigrationTool }>;
}
const migrationToolId = "migration-tool";
const migrationToolRegistryKey = "migration-tool";
const migrationToolFactory = new MigrationToolFactory();
/**
* This helper should be used as a stand-in for ContainerRuntime.loadRuntime when using Migrator and MigratableModelLoader.
*
* @privateRemarks
* In addition to what ContainerRuntime.loadRuntime does, this adds in and correctly initializes the migration tools that
* Migrator expects to interact with, and exposes an entrypoint that MigratableModelLoader expects to find.
* TODO: Consider switching to a property bag for parameters.
* @alpha
*/
export const instantiateMigratableRuntime = async <ModelType>(
context: IContainerContext,
existing: boolean,
registryEntries: NamedFluidDataStoreRegistryEntries,
createModel: CreateModelCallback<ModelType>,
runtimeOptions?: IContainerRuntimeOptions,
): Promise<IContainerRuntime & IRuntime> => {
const combinedRegistryEntries: NamedFluidDataStoreRegistryEntries = [
...registryEntries,
[migrationToolRegistryKey, Promise.resolve(migrationToolFactory)],
];
const runtime = await ContainerRuntime.loadRuntime({
context,
registryEntries: combinedRegistryEntries,
provideEntryPoint: async (
containerRuntime: IContainerRuntime,
): Promise<IMigratableModelContainerRuntimeEntryPoint<ModelType>> => ({
getModelAndMigrationTool: async (container: IContainer) => ({
// TODO: Think about the timing and order of the awaits
model: await createModel(containerRuntime, container),
migrationTool: await getDataStoreEntryPoint(containerRuntime, migrationToolId),
}),
}),
runtimeOptions,
existing,
});
if (!existing) {
const migrationTool = await runtime.createDataStore(migrationToolRegistryKey);
await migrationTool.trySetAlias(migrationToolId);
}
// Force the MigrationTool to instantiate in all cases. The PactMap it uses must be loaded and running in
// order to respond with accept ops, and without this call the MigrationTool won't be instantiated on the
// summarizer client.
await getDataStoreEntryPoint(runtime, migrationToolId);
return runtime;
};

80
examples/utils/migration-tools/src/modelLoader/interfaces.ts
Просмотреть файл

@ -1,80 +0,0 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/
import type { IMigrationTool } from "../interfaces/index.js";
// TODO: Consider just extending IAttachedMigratableModel
/**
* Object returned from calling IModelLoader.createDetached().
* @alpha
*/
export interface IDetachedMigratableModel<ModelType> {
/**
* The newly created, detached model object.
*/
model: ModelType;
/**
* The migration tool that will be used to migrate away from this model.
*/
migrationTool: IMigrationTool;
/**
* A function that will attach the model object to the service when called.
* @returns a Promise that will resolve after attach completes with the container ID of the newly attached
* container.
*/
attach: () => Promise<string>;
}
/**
* Object returned from calling IModelLoader.createDetached().
* @alpha
*/
export interface IAttachedMigratableModel<ModelType> {
/**
* The newly created, detached model object.
*/
model: ModelType;
/**
* The migration tool that will be used to migrate away from this model.
*/
migrationTool: IMigrationTool;
}
/**
* @alpha
*/
export interface IMigratableModelLoader<ModelType> {
/**
* Check if the IMigratableModelLoader knows how to instantiate an appropriate model for the provided container code version.
* It is async to permit dynamic model loading - e.g. referring to a remote service to determine if the requested
* model is available.
* @param version - the container code version to check
*/
supportsVersion(version: string): Promise<boolean>;
/**
* Create a detached model using the specified version of container code.
* Returns an object containing the detached model plus an attach callback. When invoked, the attach callback
* returns a promise that will resolve after attach has completed with the id of the container.
* @param version - the container code version to create a model for
*/
createDetached(version: string): Promise<IDetachedMigratableModel<ModelType>>;
/**
* Load a model for the container with the given id.
* @param id - the id of the container to load
*/
loadExisting(id: string): Promise<IAttachedMigratableModel<ModelType>>;
/**
* Load a model for the container with the given id to at least the specified sequence number.
* @param id - the id of the container to load
* @param sequenceNumber - the sequence number we want to load to at minimum
*/
loadExistingToSequenceNumber(
id: string,
sequenceNumber: number,
): Promise<IAttachedMigratableModel<ModelType>>;
}

93
examples/utils/migration-tools/src/modelLoader/migratableSessionStorageModelLoader.ts
Просмотреть файл

@ -1,93 +0,0 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/
import { ICodeDetailsLoader } from "@fluidframework/container-definitions/internal";
import { ITelemetryBaseLogger } from "@fluidframework/core-interfaces";
import { IDocumentServiceFactory } from "@fluidframework/driver-definitions/internal";
import {
LocalDocumentServiceFactory,
LocalResolver,
LocalSessionStorageDbFactory,
createLocalResolverCreateNewRequest,
} from "@fluidframework/local-driver/internal";
import {
ILocalDeltaConnectionServer,
LocalDeltaConnectionServer,
} from "@fluidframework/server-local-server";
import { v4 as uuid } from "uuid";
import type {
IAttachedMigratableModel,
IDetachedMigratableModel,
IMigratableModelLoader,
} from "./interfaces.js";
import { MigratableModelLoader } from "./migratableModelLoader.js";
const urlResolver = new LocalResolver();
const deltaConnectionServerMap = new Map<string, ILocalDeltaConnectionServer>();
const getDocumentServiceFactory = (documentId: string): IDocumentServiceFactory => {
let deltaConnection = deltaConnectionServerMap.get(documentId);
if (deltaConnection === undefined) {
deltaConnection = LocalDeltaConnectionServer.create(new LocalSessionStorageDbFactory());
deltaConnectionServerMap.set(documentId, deltaConnection);
}
return new LocalDocumentServiceFactory(deltaConnection);
};
/**
* @alpha
*/
export class MigratableSessionStorageModelLoader<ModelType>
implements IMigratableModelLoader<ModelType>
{
public constructor(
private readonly codeLoader: ICodeDetailsLoader,
private readonly logger?: ITelemetryBaseLogger,
) {}
public async supportsVersion(version: string): Promise<boolean> {
return true;
}
public async createDetached(version: string): Promise<IDetachedMigratableModel<ModelType>> {
const documentId = uuid();
const modelLoader = new MigratableModelLoader<ModelType>({
urlResolver,
documentServiceFactory: getDocumentServiceFactory(documentId),
codeLoader: this.codeLoader,
logger: this.logger,
generateCreateNewRequest: () => createLocalResolverCreateNewRequest(documentId),
});
return modelLoader.createDetached(version);
}
public async loadExisting(id: string): Promise<IAttachedMigratableModel<ModelType>> {
const documentId = id;
const modelLoader = new MigratableModelLoader<ModelType>({
urlResolver,
documentServiceFactory: getDocumentServiceFactory(documentId),
codeLoader: this.codeLoader,
logger: this.logger,
generateCreateNewRequest: () => createLocalResolverCreateNewRequest(documentId),
});
return modelLoader.loadExisting(`${window.location.origin}/${id}`);
}
public async loadExistingToSequenceNumber(
id: string,
sequenceNumber: number,
): Promise<IAttachedMigratableModel<ModelType>> {
const modelLoader = new MigratableModelLoader<ModelType>({
urlResolver,
documentServiceFactory: getDocumentServiceFactory(id),
codeLoader: this.codeLoader,
generateCreateNewRequest: () => createLocalResolverCreateNewRequest(id),
});
return modelLoader.loadExistingToSequenceNumber(
`${window.location.origin}/${id}`,
sequenceNumber,
);
}
}

70
examples/utils/migration-tools/src/modelLoader/migratableTinyliciousModelLoader.ts
Просмотреть файл

@ -1,70 +0,0 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/
import { ICodeDetailsLoader } from "@fluidframework/container-definitions/internal";
import type {
IDocumentServiceFactory,
IUrlResolver,
} from "@fluidframework/driver-definitions/internal";
import { RouterliciousDocumentServiceFactory } from "@fluidframework/routerlicious-driver/internal";
import {
InsecureTinyliciousTokenProvider,
InsecureTinyliciousUrlResolver,
createTinyliciousCreateNewRequest,
} from "@fluidframework/tinylicious-driver/internal";
import type {
IAttachedMigratableModel,
IDetachedMigratableModel,
IMigratableModelLoader,
} from "./interfaces.js";
import { MigratableModelLoader } from "./migratableModelLoader.js";
class TinyliciousService {
public readonly documentServiceFactory: IDocumentServiceFactory;
public readonly urlResolver: IUrlResolver;
constructor(tinyliciousPort?: number) {
const tokenProvider = new InsecureTinyliciousTokenProvider();
this.urlResolver = new InsecureTinyliciousUrlResolver(tinyliciousPort);
this.documentServiceFactory = new RouterliciousDocumentServiceFactory(tokenProvider);
}
}
/**
* @alpha
*/
export class MigratableTinyliciousModelLoader<ModelType>
implements IMigratableModelLoader<ModelType>
{
private readonly tinyliciousService = new TinyliciousService();
private readonly modelLoader: MigratableModelLoader<ModelType>;
public constructor(codeLoader: ICodeDetailsLoader) {
this.modelLoader = new MigratableModelLoader<ModelType>({
urlResolver: this.tinyliciousService.urlResolver,
documentServiceFactory: this.tinyliciousService.documentServiceFactory,
codeLoader,
generateCreateNewRequest: createTinyliciousCreateNewRequest,
});
}
public async supportsVersion(version: string): Promise<boolean> {
return this.modelLoader.supportsVersion(version);
}
public async createDetached(version: string): Promise<IDetachedMigratableModel<ModelType>> {
return this.modelLoader.createDetached(version);
}
public async loadExisting(id: string): Promise<IAttachedMigratableModel<ModelType>> {
return this.modelLoader.loadExisting(id);
}
public async loadExistingToSequenceNumber(
id: string,
sequenceNumber: number,
): Promise<IAttachedMigratableModel<ModelType>> {
return this.modelLoader.loadExistingToSequenceNumber(id, sequenceNumber);
}
}

24
examples/utils/migration-tools/src/simpleLoader/README.md Normal file
Просмотреть файл

@ -0,0 +1,24 @@
# `SimpleLoader`
This package provides a `SimpleLoader` class, which wraps a `Loader` with a simpler interface. This simpler interface is used by the `Migrator` during migration.
```ts
// Creating the SimpleLoader using Tinylicious
const loader = new SimpleLoader({
urlResolver: new InsecureTinyliciousUrlResolver(),
documentServiceFactory: new RouterliciousDocumentServiceFactory(
new InsecureTinyliciousTokenProvider(),
),
codeLoader: new DemoCodeLoader(),
generateCreateNewRequest: createTinyliciousCreateNewRequest,
});
// Creating and attaching a new container
const { container, attach } = await loader.createDetached("one");
id = await attach();
// Loading an existing container
const container = await loader.loadExisting(id);
```
TODO: Can the `Migrator` take a normal `Loader` and wrap it itself to avoid teaching a new concept here?

8
examples/utils/migration-tools/src/simpleLoader/index.ts Normal file
Просмотреть файл

@ -0,0 +1,8 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/
export type { ISimpleLoader } from "./interfaces.js";
export { SessionStorageSimpleLoader } from "./sessionStorageSimpleLoader.js";
export { SimpleLoader, waitForAtLeastSequenceNumber } from "./simpleLoader.js";

35
examples/utils/migration-tools/src/simpleLoader/interfaces.ts Normal file
Просмотреть файл

@ -0,0 +1,35 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/
import type { IContainer } from "@fluidframework/container-definitions/internal";
/**
* @alpha
*/
export interface ISimpleLoader {
/**
* Check if the ISimpleLoader knows how to instantiate the provided container code version.
* It is async to permit dynamic code loading - e.g. referring to a remote service to determine if the requested
* version is available.
* @param version - the container code version to check
*/
supportsVersion(version: string): Promise<boolean>;
/**
* Create a detached container using the specified version of container code.
* Returns an object containing the detached container plus an attach callback. When invoked, the attach callback
* returns a promise that will resolve after attach has completed with the id of the container.
* @param version - the container code version to create a container for
*/
createDetached(
version: string,
): Promise<{ container: IContainer; attach: () => Promise<string> }>;
/**
* Load the container with the given id.
* @param id - the id of the container to load
*/
loadExisting(id: string): Promise<IContainer>;
}

64
examples/utils/migration-tools/src/simpleLoader/sessionStorageSimpleLoader.ts Normal file
Просмотреть файл

@ -0,0 +1,64 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/
import {
type ICodeDetailsLoader,
type IContainer,
} from "@fluidframework/container-definitions/internal";
import type { ITelemetryBaseLogger } from "@fluidframework/core-interfaces";
import {
createLocalResolverCreateNewRequest,
LocalDocumentServiceFactory,
LocalResolver,
LocalSessionStorageDbFactory,
} from "@fluidframework/local-driver/internal";
import { LocalDeltaConnectionServer } from "@fluidframework/server-local-server";
import { v4 as uuid } from "uuid";
import type { ISimpleLoader } from "./interfaces.js";
import { SimpleLoader } from "./simpleLoader.js";
const urlResolver = new LocalResolver();
const localServer = LocalDeltaConnectionServer.create(new LocalSessionStorageDbFactory());
/**
* @alpha
*/
export class SessionStorageSimpleLoader implements ISimpleLoader {
public constructor(
private readonly codeLoader: ICodeDetailsLoader,
private readonly logger?: ITelemetryBaseLogger,
) {}
public async supportsVersion(version: string): Promise<boolean> {
return true;
}
public async createDetached(
version: string,
): Promise<{ container: IContainer; attach: () => Promise<string> }> {
const documentId = uuid();
const loader = new SimpleLoader({
urlResolver,
documentServiceFactory: new LocalDocumentServiceFactory(localServer),
codeLoader: this.codeLoader,
logger: this.logger,
generateCreateNewRequest: () => createLocalResolverCreateNewRequest(documentId),
});
return loader.createDetached(version);
}
public async loadExisting(id: string): Promise<IContainer> {
const documentId = id;
const loader = new SimpleLoader({
urlResolver,
documentServiceFactory: new LocalDocumentServiceFactory(localServer),
codeLoader: this.codeLoader,
logger: this.logger,
generateCreateNewRequest: () => createLocalResolverCreateNewRequest(documentId),
});
return loader.loadExisting(`${window.location.origin}/${id}`);
}
}

106
examples/utils/migration-tools/src/modelLoader/migratableModelLoader.ts → examples/utils/migration-tools/src/simpleLoader/simpleLoader.ts
Просмотреть файл

@ -12,17 +12,34 @@ import { ILoaderProps, Loader } from "@fluidframework/container-loader/internal"
import type { IRequest } from "@fluidframework/core-interfaces";
import type { ISequencedDocumentMessage } from "@fluidframework/driver-definitions/internal";
import { type IMigratableModelContainerRuntimeEntryPoint } from "./instantiateMigratableRuntime.js";
import type {
IAttachedMigratableModel,
IDetachedMigratableModel,
IMigratableModelLoader,
} from "./interfaces.js";
import type { ISimpleLoader } from "./interfaces.js";
/**
* Get a promise that will resolve once the container has advanced to at least the given sequence number
* @param container - the container to observe
* @param sequenceNumber - the sequence number we want to load to at minimum
*/
export const waitForAtLeastSequenceNumber = async (
container: IContainer,
sequenceNumber: number,
): Promise<void> =>
new Promise<void>((resolve) => {
if (sequenceNumber <= container.deltaManager.lastSequenceNumber) {
resolve();
}
const callbackOps = (message: ISequencedDocumentMessage): void => {
if (sequenceNumber <= message.sequenceNumber) {
resolve();
container.deltaManager.off("op", callbackOps);
}
};
container.deltaManager.on("op", callbackOps);
});
/**
* @alpha
*/
export class MigratableModelLoader<ModelType> implements IMigratableModelLoader<ModelType> {
export class SimpleLoader implements ISimpleLoader {
private readonly loader: IHostLoader;
private readonly generateCreateNewRequest: () => IRequest;
@ -54,47 +71,17 @@ export class MigratableModelLoader<ModelType> implements IMigratableModelLoader<
return true;
}
/**
* The purpose of the model pattern and the model loader is to wrap the IContainer in a more useful object and
* interface. This demo uses a convention of the entrypoint providing a getModelAndMigrationTool method to do so.
* It does this with the expectation that the model has been bundled with the container code.
*
* Other strategies to obtain the wrapping model could also work fine here - for example a standalone model code
* loader that separately fetches model code and wraps the container from the outside.
*/
private async getModelAndMigrationToolFromContainer(
container: IContainer,
): Promise<IAttachedMigratableModel<ModelType>> {
const entryPoint =
(await container.getEntryPoint()) as IMigratableModelContainerRuntimeEntryPoint<ModelType>;
// If the user tries to use this model loader with an incompatible container runtime, we want to give them
// a comprehensible error message. So distrust the type by default and do some basic type checking.
if (typeof entryPoint.getModelAndMigrationTool !== "function") {
throw new TypeError(
"Incompatible container runtime: doesn't provide getModelAndMigrationTool",
);
}
const modelAndMigrationTool = await entryPoint.getModelAndMigrationTool(container);
if (typeof modelAndMigrationTool.model !== "object") {
throw new TypeError("Incompatible container runtime: doesn't provide model");
}
if (typeof modelAndMigrationTool.migrationTool !== "object") {
throw new TypeError("Incompatible container runtime: doesn't provide migrationTool");
}
return modelAndMigrationTool;
}
// It would be preferable for attaching to look more like service.attach(model) rather than returning an attach
// callback here, but this callback at least allows us to keep the method off the model interface.
// It would be preferable for attaching to look more like service.attach(container) rather than returning an attach
// callback here, but this callback at least allows us to keep the method off the container interface.
// TODO: Consider making the version param optional, and in that case having a mechanism to query the codeLoader
// for the latest/default version to use?
public async createDetached(version: string): Promise<IDetachedMigratableModel<ModelType>> {
public async createDetached(
version: string,
): Promise<{ container: IContainer; attach: () => Promise<string> }> {
const container = await this.loader.createDetachedContainer({ package: version });
const { model, migrationTool } =
await this.getModelAndMigrationToolFromContainer(container);
// The attach callback lets us defer the attach so the caller can do whatever initialization pre-attach,
// without leaking out the loader, service, etc. We also return the container ID here so we don't have
// to stamp it on something that would rather not know it (e.g. the model).
// to stamp it on something that would rather not know it (e.g. the container).
const attach = async (): Promise<string> => {
await container.attach(this.generateCreateNewRequest());
if (container.resolvedUrl === undefined) {
@ -102,10 +89,10 @@ export class MigratableModelLoader<ModelType> implements IMigratableModelLoader<
}
return container.resolvedUrl.id;
};
return { model, migrationTool, attach };
return { container, attach };
}
private async loadContainer(id: string): Promise<IContainer> {
public async loadExisting(id: string): Promise<IContainer> {
return this.loader.resolve({
url: id,
headers: {
@ -119,33 +106,4 @@ export class MigratableModelLoader<ModelType> implements IMigratableModelLoader<
},
});
}
public async loadExisting(id: string): Promise<IAttachedMigratableModel<ModelType>> {
const container = await this.loadContainer(id);
const { model, migrationTool } =
await this.getModelAndMigrationToolFromContainer(container);
return { model, migrationTool };
}
public async loadExistingToSequenceNumber(
id: string,
sequenceNumber: number,
): Promise<IAttachedMigratableModel<ModelType>> {
const container = await this.loadContainer(id);
await new Promise<void>((resolve) => {
if (sequenceNumber <= container.deltaManager.lastSequenceNumber) {
resolve();
}
const callbackOps = (message: ISequencedDocumentMessage): void => {
if (sequenceNumber <= message.sequenceNumber) {
resolve();
container.deltaManager.off("op", callbackOps);
}
};
container.deltaManager.on("op", callbackOps);
});
const { model, migrationTool } =
await this.getModelAndMigrationToolFromContainer(container);
return { model, migrationTool };
}
}

36
examples/version-migration/separate-container/README.md
Просмотреть файл

@ -1,12 +1,12 @@
# @fluid-example/version-migration-separate-container
This example experiments with an approach for migrating data from an existing Fluid container into a new Fluid container which may have a different schema or code running on it.
This example experiments with an approach for migrating data from an existing Fluid container into a new Fluid container which may have a different schema and code running on it.
Please note that the ideas explored here are experimental and under development. They are not yet recommended for broad use in production.
## Scenario
Once a Fluid container has been created, it will contain some set of persisted data in the form of the summary as well as any unsummarized ops. This persisted data can only be correctly interpreted by a compatible container code (typically the same one that created it, or a newer backwards-compatible one). This container code knows the appropriate data stores to load to interpret the summary and process the outstanding ops, as well as provides public access to those data stores for use.
Once a Fluid container has been created, it will contain some set of persisted data in the form of the summary as well as any unsummarized ops. This persisted data can only be correctly interpreted by a compatible container code (typically the same one that created it, or a newer but backwards-compatible one). This container code knows the appropriate data stores to load to interpret the summary and process the outstanding ops, as well as provides public access to those data stores for use.
However, suppose you want to change your application's schema in a manner that is not backwards compatible. Examples of this might include:
@ -20,49 +20,29 @@ This example explores one technique to permit these types of changes. It employs
1. Reach consensus amongst connected clients to perform the migration
1. Extract the data from the existing container
1. Transform the data as needed
1. Transform the data as needed (optional)
1. Create a new container with the new code and import the transformed data
1. Redirect clients to the new container
### Reach consensus amongst connected clients to perform the migration
At any given moment, connected clients may have data in flight - ops that are unsequenced or that not all other clients are aware of. To avoid losing this data during the migration, we use a Quorum DDS to partition the op stream and establish the version we are migrating to. Ops sent before the Quorum value acceptance will be included, and clients are expected to stop generating ops after observing the proposal. After the Quorum value is accepted, we know there are no more ops in flight that should be included in the migration, and that the version we are migrating to is the one specified in the Quorum.
At any given moment, connected clients may have data in flight - ops that are unsequenced or that not all other clients are aware of. To avoid losing this data during the migration, we use a PactMap DDS to partition the op stream and establish the version we are migrating to. Ops sent before the PactMap value acceptance will be included, and clients are expected to stop generating ops after observing the proposal. After the PactMap value is accepted, late-arriving ops are not guaranteed to be included in the migration. Applications are recommended to block further edits to the data at this point to avoid the risk of losing those edits.
### Extract the data from the existing container
The container model is expected to provide a mechanism to extract the data from within for migration purposes. The format of the extracted data is up to the model - it could be a string, JSON, some well known file format like .csv, etc. Complex Javascript objects could even be used (since we will be performing the data import locally), but some serializable format is probably the most durable option.
The container model is expected to provide a mechanism to export the data from within for migration purposes. The format of the exported data is up to the model - it could be a string, JSON, some well known file format like .csv, etc. Complex Javascript objects could even be used (since we will be performing the data import locally), but some serializable format is probably the most durable option.
### Transform the data as needed
### Transform the data as needed (optional)
If the new model is incapable of importing the export format of the old model, the format should be transformed accordingly. This can be skipped if the exported format is directly consumable by the new model.
### Create a new container with the new code and import the transformed data
With the exported and transformed data in hand, we can create a new container using the new container code and import the data. We ideally only upload (attach) a single migrated container, since duplicative containers are wasted storage. We use a TaskManager to select a single volunteer for this purpose. Once the container is attached, we write the new container's id into the old container (using a ConsensusRegisterCollection) to finalize the migration - all clients can now know the migration is complete and the data has been migrated to the specified container.
With the exported and transformed data in hand, we can create a new container using the new container code and import the data. We ideally only upload (attach) a single migrated container, since duplicative containers are wasted storage. To minimize duplication, we use a TaskManager to select a single volunteer. Once the container is attached, we write the new container's id into the old container (using a ConsensusRegisterCollection) to finalize the migration. This write lets other clients know the migration is complete and the data has been migrated to the specified container.
### Redirect clients to the new container
As clients observe the migration complete, they load the new container and swap it in for the old one. This includes loading in the approporate new container code, model code, and view code if necessary. Once complete, the client can begin collaborating on the new container.
## Other concepts
This example also explores other concepts that are new but not core to the migration process.
### Container model
In many other examples, we use a "root/default data object" concept (Spaces is a good example, pretty much all of the /examples/data-objects examples as well). The root data object exposes the API that the container wants to expose to the app (host). However, accessing this API is indirect, as the app must first retrieve this data object from the IContainer using `container.request()`.
The container model concept introduced in this example serves a similar purpose of exposing an API for the app, but does so by wrapping the IContainer rather than living inside it as a data object. This removes a layer of indirection for the app, who can load this model directly (see next section). The app can then start using the API surface immediately without the extra step of going through the request pattern.
When the container API surface has been externalized from the container, this can also open up new options for how the data might be represented and organized. There's no longer a need to craft a data object that holds references to all the container's contents if it's not required for the scenario. In this example, the model code knows how to access both the inventory list as well as the killbit, but these two objects remain completely separate from each other in the data schema.
### Model loading
As mentioned above, the `ModelLoader` is able to load directly to a container model. To do this, it wraps a `Loader` to load containers, and uses an `IModelCodeLoader` (similar to the `ICodeDetailsLoader` used in the Container) to match the model code against the container code. This extra code loader is required because the model code must be compatible with the container code within. The model code loader is also the opportunity to run any further async initialization steps that are needed to present the correct API surface on the model (e.g. retrieving handles to have important data available synchronously).
### View loading
Similarly, the view used on a model must be compatible with that model. A view loader can inspect the model and load the appropriate view. This portion is still under development, but will likely be similar to the model loading flow.
As clients observe the migration complete, they load the new container and swap it in for the old one. This includes loading in the approporate new container code. Once complete, the client can begin collaborating on the new container.
<!-- AUTO-GENERATED-CONTENT:START (EXAMPLE_APP_README_HEADER:usesTinylicious=TRUE) -->

54
examples/version-migration/separate-container/src/modelVersion1/containerCode.ts
Просмотреть файл

@ -3,26 +3,19 @@
* Licensed under the MIT License.
*/
import { getDataStoreEntryPoint } from "@fluid-example/example-utils";
import {
type IMigratableModel,
instantiateMigratableRuntime,
CompositeEntryPoint,
loadCompositeRuntime,
migrationToolEntryPointPiece,
} from "@fluid-example/migration-tools/internal";
import type {
IContainer,
IContainerContext,
IRuntime,
IRuntimeFactory,
} from "@fluidframework/container-definitions/internal";
import type { IContainerRuntimeOptions } from "@fluidframework/container-runtime/internal";
import type { IContainerRuntime } from "@fluidframework/container-runtime-definitions/internal";
import type { IInventoryList, IInventoryListAppModel } from "../modelInterfaces.js";
import { InventoryListAppModel } from "./appModel.js";
import { InventoryListInstantiationFactory } from "./inventoryList.js";
const inventoryListId = "default-inventory-list";
import { modelEntryPointPiece } from "./modelEntryPointPiece.js";
/**
* @internal
@ -32,9 +25,6 @@ export class InventoryListContainerRuntimeFactory implements IRuntimeFactory {
return this;
}
private readonly registryEntries = new Map([
InventoryListInstantiationFactory.registryEntry,
]);
private readonly runtimeOptions: IContainerRuntimeOptions | undefined;
/**
* Constructor for the factory. Supports a test mode which spawns the summarizer instantly.
@ -54,37 +44,9 @@ export class InventoryListContainerRuntimeFactory implements IRuntimeFactory {
context: IContainerContext,
existing: boolean,
): Promise<IRuntime> {
const runtime = await instantiateMigratableRuntime(
context,
existing,
this.registryEntries,
this.createModel,
this.runtimeOptions,
);
if (!existing) {
await this.containerInitializingFirstTime(runtime);
}
return runtime;
const compositeEntryPoint = new CompositeEntryPoint();
compositeEntryPoint.addEntryPointPiece(modelEntryPointPiece);
compositeEntryPoint.addEntryPointPiece(migrationToolEntryPointPiece);
return loadCompositeRuntime(context, existing, compositeEntryPoint, this.runtimeOptions);
}
private readonly containerInitializingFirstTime = async (
runtime: IContainerRuntime,
): Promise<void> => {
const inventoryList = await runtime.createDataStore(
InventoryListInstantiationFactory.type,
);
await inventoryList.trySetAlias(inventoryListId);
};
private readonly createModel = async (
runtime: IContainerRuntime,
container: IContainer,
): Promise<IInventoryListAppModel & IMigratableModel> => {
return new InventoryListAppModel(
await getDataStoreEntryPoint<IInventoryList>(runtime, inventoryListId),
container,
);
};
}

56
examples/version-migration/separate-container/src/modelVersion1/modelEntryPointPiece.ts Normal file
Просмотреть файл

@ -0,0 +1,56 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/
// TODO: Note that this would theoretically come from some model loading package, not migration-tools.
// Maybe move back into example-utils for the short-term
import type { IEntryPointPiece } from "@fluid-example/migration-tools/internal";
import type { IContainer } from "@fluidframework/container-definitions/internal";
import type { IContainerRuntime } from "@fluidframework/container-runtime-definitions/internal";
import type { FluidObject } from "@fluidframework/core-interfaces";
import type { IInventoryList } from "../modelInterfaces.js";
import { InventoryListAppModel } from "./appModel.js";
import { InventoryListInstantiationFactory } from "./inventoryList.js";
const modelEntryPointPieceName = "getModel";
const inventoryListAlias = "default-inventory-list";
async function getDataStoreEntryPoint(
runtime: IContainerRuntime,
alias: string,
): Promise<FluidObject> {
const entryPointHandle = await runtime.getAliasedDataStoreEntryPoint(alias);
if (entryPointHandle === undefined) {
throw new Error(`Default dataStore [${alias}] must exist`);
}
return entryPointHandle.get();
}
const createPiece = async (
runtime: IContainerRuntime,
): Promise<(container: IContainer) => Promise<FluidObject>> => {
return async (container: IContainer) =>
new InventoryListAppModel(
(await getDataStoreEntryPoint(runtime, inventoryListAlias)) as IInventoryList,
container,
);
};
export const modelEntryPointPiece: IEntryPointPiece = {
name: modelEntryPointPieceName,
registryEntries: [InventoryListInstantiationFactory.registryEntry],
onCreate: async (runtime: IContainerRuntime): Promise<void> => {
const inventoryList = await runtime.createDataStore(
InventoryListInstantiationFactory.type,
);
await inventoryList.trySetAlias(inventoryListAlias);
},
onLoad: async (runtime: IContainerRuntime): Promise<void> => {},
createPiece,
};

54
examples/version-migration/separate-container/src/modelVersion2/containerCode.ts
Просмотреть файл

@ -3,26 +3,19 @@
* Licensed under the MIT License.
*/
import { getDataStoreEntryPoint } from "@fluid-example/example-utils";
import {
type IMigratableModel,
instantiateMigratableRuntime,
CompositeEntryPoint,
loadCompositeRuntime,
migrationToolEntryPointPiece,
} from "@fluid-example/migration-tools/internal";
import type {
IContainer,
IContainerContext,
IRuntime,
IRuntimeFactory,
} from "@fluidframework/container-definitions/internal";
import type { IContainerRuntimeOptions } from "@fluidframework/container-runtime/internal";
import type { IContainerRuntime } from "@fluidframework/container-runtime-definitions/internal";
import type { IInventoryList, IInventoryListAppModel } from "../modelInterfaces.js";
import { InventoryListAppModel } from "./appModel.js";
import { InventoryListInstantiationFactory } from "./inventoryList.js";
const inventoryListId = "default-inventory-list";
import { modelEntryPointPiece } from "./modelEntryPointPiece.js";
/**
* @internal
@ -32,9 +25,6 @@ export class InventoryListContainerRuntimeFactory implements IRuntimeFactory {
return this;
}
private readonly registryEntries = new Map([
InventoryListInstantiationFactory.registryEntry,
]);
private readonly runtimeOptions: IContainerRuntimeOptions | undefined;
/**
* Constructor for the factory. Supports a test mode which spawns the summarizer instantly.
@ -54,37 +44,9 @@ export class InventoryListContainerRuntimeFactory implements IRuntimeFactory {
context: IContainerContext,
existing: boolean,
): Promise<IRuntime> {
const runtime = await instantiateMigratableRuntime(
context,
existing,
this.registryEntries,
this.createModel,
this.runtimeOptions,
);
if (!existing) {
await this.containerInitializingFirstTime(runtime);
}
return runtime;
const compositeEntryPoint = new CompositeEntryPoint();
compositeEntryPoint.addEntryPointPiece(modelEntryPointPiece);
compositeEntryPoint.addEntryPointPiece(migrationToolEntryPointPiece);
return loadCompositeRuntime(context, existing, compositeEntryPoint, this.runtimeOptions);
}
private readonly containerInitializingFirstTime = async (
runtime: IContainerRuntime,
): Promise<void> => {
const inventoryList = await runtime.createDataStore(
InventoryListInstantiationFactory.type,
);
await inventoryList.trySetAlias(inventoryListId);
};
private readonly createModel = async (
runtime: IContainerRuntime,
container: IContainer,
): Promise<IInventoryListAppModel & IMigratableModel> => {
return new InventoryListAppModel(
await getDataStoreEntryPoint<IInventoryList>(runtime, inventoryListId),
container,
);
};
}

56
examples/version-migration/separate-container/src/modelVersion2/modelEntryPointPiece.ts Normal file
Просмотреть файл

@ -0,0 +1,56 @@
/*!
* Copyright (c) Microsoft Corporation and contributors. All rights reserved.
* Licensed under the MIT License.
*/
// TODO: Note that this would theoretically come from some model loading package, not migration-tools.
// Maybe move back into example-utils for the short-term
import type { IEntryPointPiece } from "@fluid-example/migration-tools/internal";
import type { IContainer } from "@fluidframework/container-definitions/internal";
import type { IContainerRuntime } from "@fluidframework/container-runtime-definitions/internal";
import type { FluidObject } from "@fluidframework/core-interfaces";
import type { IInventoryList, IInventoryListAppModel } from "../modelInterfaces.js";
import { InventoryListAppModel } from "./appModel.js";
import { InventoryListInstantiationFactory } from "./inventoryList.js";
const modelEntryPointPieceName = "getModel";
const inventoryListAlias = "default-inventory-list";
async function getDataStoreEntryPoint(
runtime: IContainerRuntime,
alias: string,
): Promise<FluidObject> {
const entryPointHandle = await runtime.getAliasedDataStoreEntryPoint(alias);
if (entryPointHandle === undefined) {
throw new Error(`Default dataStore [${alias}] must exist`);
}
return entryPointHandle.get();
}
const createPiece = async (
runtime: IContainerRuntime,
): Promise<(container: IContainer) => Promise<IInventoryListAppModel>> => {
return async (container: IContainer) =>
new InventoryListAppModel(
(await getDataStoreEntryPoint(runtime, inventoryListAlias)) as IInventoryList,
container,
);
};
export const modelEntryPointPiece: IEntryPointPiece = {
name: modelEntryPointPieceName,
registryEntries: [InventoryListInstantiationFactory.registryEntry],
onCreate: async (runtime: IContainerRuntime): Promise<void> => {
const inventoryList = await runtime.createDataStore(
InventoryListInstantiationFactory.type,
);
await inventoryList.trySetAlias(inventoryListAlias);
},
onLoad: async (runtime: IContainerRuntime): Promise<void> => {},
createPiece,
};

54
examples/version-migration/separate-container/src/start.ts
Просмотреть файл

@ -8,7 +8,11 @@ import type {
IMigrationTool,
IVersionedModel,
} from "@fluid-example/migration-tools/internal";
import { MigratableModelLoader, Migrator } from "@fluid-example/migration-tools/internal";
import {
getModelAndMigrationToolFromContainer,
Migrator,
SimpleLoader,
} from "@fluid-example/migration-tools/internal";
import { RouterliciousDocumentServiceFactory } from "@fluidframework/routerlicious-driver/internal";
import {
InsecureTinyliciousTokenProvider,
@ -74,13 +78,7 @@ const renderModel = (model: IVersionedModel, migrationTool: IMigrationTool): voi
};
async function start(): Promise<void> {
// If we assumed the container code could consistently present a model to us, we could bake that assumption
// in here as well as in the Migrator -- both places just need a reliable way to get a model regardless of the
// (unknown) container version. So the ModelLoader would be replaced by e.g. container.getEntryPoint() or
// container.getEntryPoint().model if we knew that was the model.
// TODO: This is really loading an IInventoryListAppModel & IMigratableModel (we know this because of what the
// DemoCodeLoader supports). Should we just use that more-specific type in the typing here?
const modelLoader = new MigratableModelLoader<IMigratableModel>({
const loader = new SimpleLoader({
urlResolver: new InsecureTinyliciousUrlResolver(),
documentServiceFactory: new RouterliciousDocumentServiceFactory(
new InsecureTinyliciousTokenProvider(),
@ -96,25 +94,27 @@ async function start(): Promise<void> {
if (location.hash.length === 0) {
// Choosing to create with the "old" version for demo purposes, so we can demo the upgrade flow.
// Normally we would create with the most-recent version.
const createResponse = await modelLoader.createDetached("one");
model = createResponse.model;
migrationTool = createResponse.migrationTool;
id = await createResponse.attach();
const { container, attach } = await loader.createDetached("one");
const modelAndMigrationTool =
await getModelAndMigrationToolFromContainer<IMigratableModel>(container);
model = modelAndMigrationTool.model;
migrationTool = modelAndMigrationTool.migrationTool;
id = await attach();
} else {
id = location.hash.slice(1);
const loadResponse = await modelLoader.loadExisting(id);
model = loadResponse.model;
migrationTool = loadResponse.migrationTool;
const container = await loader.loadExisting(id);
const modelAndMigrationTool =
await getModelAndMigrationToolFromContainer<IMigratableModel>(container);
model = modelAndMigrationTool.model;
migrationTool = modelAndMigrationTool.migrationTool;
}
// The Migrator takes the starting state (model and id) and watches for a migration proposal. It encapsulates
// the migration logic and just lets us know when a new model is loaded and available (with the "migrated" event).
// It also takes a dataTransformationCallback to help in transforming data export format to be compatible for
// import with newly created models.
// TODO: Consider just passing the ModelLoader (or even the model loader construction args?) and kind of wrapping it.
// Then this becomes something like a MigratingModelLoader. Then the model can have a migrationTool but sort of hide it.
const migrator = new Migrator(
modelLoader,
loader,
model,
migrationTool,
id,
@ -127,17 +127,17 @@ async function start(): Promise<void> {
renderModel(model, migrationTool);
updateTabForId(migrator.currentModelId);
});
// If the ModelLoader doesn't know how to load the model required for migration, it emits "migrationNotSupported".
// For example, this might be hit if another client has a newer ModelLoader and proposes a version our
// ModelLoader doesn't know about.
// However, this will never be hit in this demo since we have a finite set of models to support. If the model
// code loader pulls in the appropriate model dynamically, this might also never be hit since all clients
// are theoretically referencing the same model library.
// If the loader doesn't know how to load the container code required for migration, it emits "migrationNotSupported".
// For example, this might be hit if another client has a newer loader and proposes a version our
// loader doesn't know about.
// However, this will never be hit in this demo since we have a finite set of container codes to support. If the
// code loader pulls in the appropriate code dynamically, this might also never be hit since all clients
// are theoretically referencing the same code library.
migrator.events.on("migrationNotSupported", (version: string) => {
// To move forward, we would need to acquire a model loader capable of loading the given model, retry the
// load, and set up a new Migrator with the new model loader.
// To move forward, we would need to acquire a loader capable of loading the given code, retry the
// load, and set up a new Migrator with the new loader.
console.error(
`Tried to migrate to version ${version} which is not supported by the current ModelLoader`,
`Tried to migrate to version ${version} which is not supported by the current loader`,
);
});

35
examples/version-migration/separate-container/tests/index.tsx
Просмотреть файл

@ -4,11 +4,12 @@
*/
import {
getModelAndMigrationToolFromContainer,
IMigratableModel,
IMigrationTool,
IVersionedModel,
MigratableSessionStorageModelLoader,
Migrator,
SessionStorageSimpleLoader,
} from "@fluid-example/migration-tools/internal";
import { createElement } from "react";
@ -46,27 +47,27 @@ window["migrators"] = [];
export async function createContainerAndRenderInElement(element: HTMLDivElement) {
const searchParams = new URLSearchParams(location.search);
const testMode = searchParams.get("testMode") !== null;
const modelLoader = new MigratableSessionStorageModelLoader<
IInventoryListAppModel & IMigratableModel
>(new DemoCodeLoader(testMode));
const loader = new SessionStorageSimpleLoader(new DemoCodeLoader(testMode));
let id: string;
let model: IMigratableModel;
let migrationTool: IMigrationTool;
if (location.hash.length === 0) {
// Normally our code loader is expected to match up with the version passed here.
// But since we're using a StaticCodeLoader that always loads the same runtime factory regardless,
// the version doesn't actually matter.
const createResponse = await modelLoader.createDetached("one");
model = createResponse.model;
migrationTool = createResponse.migrationTool;
// Should be the same as the uuid we generated above.
id = await createResponse.attach();
// Choosing to create with the "old" version for demo purposes, so we can demo the upgrade flow.
// Normally we would create with the most-recent version.
const { container, attach } = await loader.createDetached("one");
const modelAndMigrationTool =
await getModelAndMigrationToolFromContainer<IMigratableModel>(container);
model = modelAndMigrationTool.model;
migrationTool = modelAndMigrationTool.migrationTool;
id = await attach();
} else {
id = location.hash.substring(1);
const loadResponse = await modelLoader.loadExisting(id);
model = loadResponse.model;
migrationTool = loadResponse.migrationTool;
id = location.hash.slice(1);
const container = await loader.loadExisting(id);
const modelAndMigrationTool =
await getModelAndMigrationToolFromContainer<IMigratableModel>(container);
model = modelAndMigrationTool.model;
migrationTool = modelAndMigrationTool.migrationTool;
}
const appDiv = document.createElement("div");
@ -104,7 +105,7 @@ export async function createContainerAndRenderInElement(element: HTMLDivElement)
};
const migrator = new Migrator(
modelLoader,
loader,
model,
migrationTool,
id,