Update to deployment and docs generation: adjustments to deploy v2
artifact instead of main artifacts. Note: while the update aims to
deploy artifacts from the latest release branches instead of main, the
actual website build is still done on the main branch. This will prevent
the pipeline from deploying docs which were generated with an outdated
version of the website.
Notable changes:
deploy-docs:
- shifted most of the stages from build-docs to deploy-docs
- build-docs now runs check_branch_version -> upload-json -> trigger
deploy
build-docs:
- shifted check_branch_version to the top since its output is required
to properly in the upload json step
upload-json-step:
- added a new upload step which uploads artifacts with the name
"latest-v*.tar.gz"
- takes in new parameters from check_branch_version to ensure it only
uploads with this name when the pipeline is running on the latest
release branch
latestVersions:
- since upload-json-step requires majorVersion, I figured it was best to
include it as an output variable for the function as it seems more
robust compared to relying on a string parsing script in the pipeline
itself
download-apis:
- update such that it only downloads "latest-v*.tar.gz" instead of the
main branch artifacts
picocolors is a smaller alternative to chalk, and it is also ESM and
CJS, while chalk went ESM-only and we were stuck on old versions in the
core build-tools project.
build-tools will eventually be ESM, and using a dual-module-format
package will actually make that transition easier because we won't have
to do dependency switches while also doing ESM conversion.
The changelog for build-tools is very outdated, so I manually
regenerated the entries for releases 0.7.0 - .0.50.0.
Steps:
```shell
git checkout build-tools_v0.48.0
pnpm changelog
```
I then stashed the changes and repeated the steps for each minor
release.
## Description
generic types with an extends clause of `never` currently generate type
tests which do not compile due to any not extending never. This is fixed
by using `never` instead of `any`.
`@system` types which do not promise user facing stability are now
omitted from type testing.
The currently unused tags to opt into alternative type testing for a
given type have been converted from `-` separated to camel case since it
seems `-` does not work well in tags.
There is nothing particularly Fluid-centric about the top-level API in
build-infrastructure; I just couldn't think of a better name at the
time. After discussion, we're going to call the top-level object a
BuildProject. The full hierarchy is now:
BuildProjects contain Workspaces contain ReleaseGroups contain Packages
This change required a lot of documentation updates and renaming other
related properties and functions beyond the BuildProject itself.
The package is private so these changes are safe.
Updates the configuration for build-tools packages to use fluid-build to
build.
The changes were modest since it was already using fluid-build for most
of the build. Only the root-level tasks needed to be hooked up, and the
individual package scripts needed to be updated.
[AB#17062](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/17062)
Adds steps to the release tool to check that release notes have been
generated and that the per-package changelogs have been updated. Also
prompts the user for a bump type earlier in the process if needed.
The changelog step is not trivial to check, so I just prompt the user to
select whether they did it. Not ideal, but at least the tool will remind
the release driver that this step is needed. We can add a more complete
check in the future if needed.
---------
Co-authored-by: Joshua Smithrud <54606601+Josmithr@users.noreply.github.com>
This change adds default implementations of the `IFluidRepo`,
`IWorkspace`, `IReleaseGroup`, and `IPackage` interfaces. These
implementations are sufficient for most uses in build-tools and
build-cli. They may be further extended for specific scenarios in future
changes.
I disabled the TypeDoc generation in this PR to make the code changes
easier to review. Once this series of PRs lands, we can enable TypeDoc
or api-markdown-documenter and regen the docs.
---------
Co-authored-by: Alex Villarreal <716334+alexvy86@users.noreply.github.com>
Inquirer has been rewritten and released as `@inquirer/prompts`, so this
PR upgrades the few inquirer uses we have to the new version.
We also use another prompt library in build-cli, `prompts`, and I think
we can likely replace it with the new inquirer, but I left that for
another change.
The latest release of jssm includes the fixes in
https://github.com/StoneCypher/jssm/pull/569, so our patch is no longer
needed and has been removed. However, the jssm-viz package has a similar
problem (open PR: https://github.com/StoneCypher/jssm-viz/pull/54) so I
patched it with type changes similar to what was done for jssm
originally.
If or when the jssm-viz PR is accepted or the issue is otherwise fixed,
we can remove the patch.
oclif-test underwent some major changes in its latest release. Tests in
build-tools have been updated with new patterns to account for the
changes. The changes were informed by [the oclif-test migration
guide](https://github.com/oclif/test/blob/main/MIGRATION.md).
Most of the conversion was straightforward except for tests that use env
variables. For those, I used
[mocked-env](https://www.npmjs.com/package/mocked-env). The tests still
use mocha/chai.
Upgrades @octokit/core to v5, which should have no effect on our uses.
The breaking changes are mostly related to dropping node support:
https://github.com/octokit/core.js/releases/tag/v5.0.0
I could have gone to v6 but there are some other deps in the tree with
peer deps on v5, and the main change in v6 is that it went ESM-only.
# @fluid-tools/build-infrastructure
This change introduces a new package containing types and helper
functions that will be used across multiple build-tools packages,
including `@fluidframework/build-tools` and `@fluid-tools/build-cli`.
The primary purpose of this package is to provide a common way to
enumerate Packages, Release Groups, and Workspaces
across a Fluid repo.
**The package is private and nothing depends on it yet.**
## Background
Today both fluid-build and build-cli (flub) need to understand the
layout of the repo - what packages belong to what release groups, what
folders are workspace roots vs. independent packages, etc. We have been
using the types and classes from fluid-build in the flub code, but the
relationship has always been strange because flub has a concept of
release groups while fluid-build uses a MonoRepo class which is
analogous to a Workspace.
Conceptually, Release Groups and Workspaces are distinct, but in code
there is not a strong representation of the "release group" concept. In
flub we use the MonoRepo class and a `type ReleaseGroup = string`
together as a stand-in for the ReleaseGroup concept which has meant that
release groups and workspaces (the MonoRepo class) are more or less the
same thing in practice.
That has always been a very weird situation that has gotten complicated
as the repo has grown, and now we find that we want the benefits of
interdependencies in between packages provided by workspaces but the
flexibility of releasing and versioning groups of packages within those
workspaces. These changes are a step towards that future.
## The goal
The eventual goal state - far beyond this change - is to have all the shared
types and data about the repo layout and other shared code in the
build-infrastructure package, and then have flub and fluid-build be in
individual packages. They shouldn't depend on each other - all the
shared logic - especially the types - should be in the infra package.
## How the API and interfaces were developed
The interfaces exposed by this package were extracted from the uses of
packages and release groups in build-tools and build-cli. The minimal
set of properties and functions necessary by both of those packages was
used. In some cases, duplicative properties or functions were removed.
For example, the Package type had `dependencies` and
`combinedDependencies`; now it just has the latter and the caller can
filter out what it doesn't want or extend the shared interface.
## Description
Updates the `get-func-name` transitive dependency to address
[CVE-2023-43646](https://nvd.nist.gov/vuln/detail/CVE-2023-43646). It
came through `chai@4.3.7`; updating to `chai@4.5.0` was enough to get a
fixed version of `get-func-name`. Also did a few updates of transitive
dependency `loupe` from 2.3.6 to 2.3.7 (even though `chai` doesn't
strictly needed it), where they guarantee that their dependency ranges
now point to fixed versions of their dependencies.
## Description
Updates the `latest-version` dependency to the (drumroll...) latest
version 😄. No significant changes in [any of the
releases](https://github.com/sindresorhus/latest-version/releases)
(mostly updating Node version support), except the fact that upgrading
their dependency on `package-json` now filters deprecated versions by
default. @tylerbutler do you think that affects us? Do we need to
provide the option explicitly to maintain previous behavior?
The motivation is to get rid of our dependency on `got@9.6.0`, which (in
build-tools) only comes as a transitive dependency of
`latest-version@5.1`.
fluid-build was failing to load in repos without a config file. This
behavior was inadvertently broken when the `getFluidBuildConfig`
function was changed to throw an error when a config was not found. This
prevented existing default config fallback logic to be used.
This change clarifies what the default config is, returns it
consistently from `getFluidBuildConfig` when no config is found, and
updates the FluidRepo loading logic to account for the change.
We recently updated the pnpm config in the repo to do a lockfile-only
install by default. This updates the build tools to pass the
`--no-frozen-lockfile` flag when installing, so the lockfile will be
updated if needed.
We run `changeset version` during changelog generation, which causes
changesets to be consumed and the package.json versions to be bumped. We
need to calculate the same "target version" that the changesets tools
calculate so we can correctly replace the output of the changelogs to
the correct versions and do our other cleanup.
This change fixes the calculation to take into account the bump ranges
in the changesets.
## Description
This PR includes small improvements in the code coverage module.
1.) Add ability to skip coverage check in code coverage module in case
of regression.
2.) Fix glyph and use unicode characters for arrows.
3.) Improve code coverage readme.
4.) Add more details to the code coverage summary.
---------
Co-authored-by: Jatin Garg <jatingarg@Jatins-MacBook-Pro-2.local>
Co-authored-by: Alex Villarreal <716334+alexvy86@users.noreply.github.com>
## Description
Updates transitive dependencies on `braces` from 3.0.2 to 3.0.3 to
address [CVE-2024-4068](https://nvd.nist.gov/vuln/detail/CVE-2024-4068).
A couple of applications of `flub modify lockfile --dependency braces
--version 3.0.3 --releaseGroup <release group>`, and some manual updates
in packages/release groups that we can't target with `flub`, basically
doing the same thing but manually (add an override in package.json,
install dependencies, remove override, install dependencies again to
clean up override from the lockfile).
In a few cases I got unrelated updates, mostly about node types, which I
reverted manually.
Server packages also got semver update from 7.6.0 to 7.6.3 which seems
fine.
## Description
Modify comment for code coverage.
Add the line coverage summary to the summary.
Co-authored-by: Jatin Garg <jatingarg@Jatins-MacBook-Pro-2.local>
Adds support for creating fluid-build tasks from config only, by
declaring the command corresponding to the task and providing globs for
input and output files.
- Adds two new settings to the fluid-build config.
- `declarativeTasks?: DeclarativeTasks;` - a mapping of
executable/command to input/output globs and a gitignore setting that
controls whether gitignore rules are applied to the input/output globs.
- `multiCommandExecutables?: string[]` - a list of executables that have
subcommands, like `flub` or `oclif`. This is needed to ensure that the
commands in the declarative tasks are parsed correctly.
- Added types to represent "task handlers" - functions that create
LeafTask objects to handle tasks.
- Added support for free functions that return instances of LeafTasks -
this is needed so that task handlers can be dynamically created. Task
subclasses with constructors still work as before.
- Added a new DeclarativeTaskHandler class that handles the declarative
tasks. It is a typical Task class, but the primary way it is used is via
a factory function. This allows us to pass in the configuration settings
for each task dynamically but re-use the DeclarativeTask class for all
the core functionality.
- Defined new declarative tasks for oclif, jssm-viz, and syncpack in the
root fluid-build config. These won't take effect until the new version
of build-tools is integrated into client.
- New dependency: globby - globby is helpful because it can auto-apply
gitignore rules which we need to do for some tasks' input/output globs.
[AB#13203](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/13203)
The mocha-json-output-reporter is failing which I was able to determine
via trial and error by adjusting the reporters, it also appears to be
essentially unsupported. It seems we only use it for inline logs in our
ci build output, which shouldn't be necessary as we have the test
results separately in the test tab. I think the best solution is to just
remove this.
Moves the tests for build-cli inside the source folder, so that it
matches the patterns used elsewhere in the repo and release group. Also
updates the tsconfigs to use the shared config in build-common.
Required several lint disables in the tests, but they were all
reasonable IMO. If needed those can be cleaned up in future changes.
## Description
Policy check should probably be invalidated when files are deleted.
`getModifiedFiles` contained some (unstaged) but not all (staged)
deleted files. From its description, its unclear what exactly it should
include, but its only used by FlubCheckPolicyTask which behaves
incorrectly if there are deleted files, regardless of if
`getModifiedFiles` includes them or not (crashes if they are included,
has an incremental build issue if not).
This change avoids it processing a list of modified files entirely, and
just hashes the result of `git diff HEAD` and the head revision, which
is a simple git powered summary of the current working copy state.
I also added some comments since I ran into an entire class inheritance
chain with no doc comments and method overload chains with no doc
comments, which made this code hard to understand. I'm not confident
these comments are correct so they should be reviewed carefully.
This is inspired by
https://github.com/microsoft/FluidFramework/pull/22586 which made me
struggle to determine if `getModifiedFiles` is correct and made me want
to delete it or clarify its specification.
Michael Zhen
Craig Macomber (Microsoft)
Tyler Butler
Sonali Deshpande
Alex Villarreal
Jatin Garg
Tony Murphy