From f0bb0d3aa67f7b71c808446e1657a1f361cfbc9a Mon Sep 17 00:00:00 2001 From: Derrick Stolee Date: Fri, 18 Jun 2021 14:41:15 -0400 Subject: [PATCH] docs: truncate after redirect Signed-off-by: Derrick Stolee --- docs/faq.md | 103 ----------------------------- docs/getting-started.md | 110 ------------------------------- docs/index.md | 142 ---------------------------------------- docs/philosophy.md | 63 ------------------ docs/troubleshooting.md | 57 ---------------- 5 files changed, 475 deletions(-) diff --git a/docs/faq.md b/docs/faq.md index 319e95e8..deadd061 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -6,106 +6,3 @@ Frequently Asked Questions [microsoft-git]: https://github.com/microsoft/git [git-faq]: https://github.com/microsoft/git/blob/HEAD/contrib/scalar/docs/faq.md - -Using Scalar ------------- - -### I don't want a sparse clone, I want every file after I clone! - -Run `scalar clone --full-clone ` to initialize your repo to include -every file. You can switch to a sparse-checkout later by running -`git sparse-checkout init --cone`. - -### I already cloned without `--full-clone`. How do I get everything? - -Run `git sparse-checkout disable`. - -Scalar Design Decisions ------------------------ - -There may be many design decisions within Scalar that are confusing at first -glance. Some of them may cause friction when you use Scalar with your existing -repos and existing habits. - -> Scalar has the most benefit when users design repositories -> with efficient patterns. - -For example: Scalar uses the sparse-checkout feature to limit the size of the -working directory within a large monorepo. It is designed to work efficiently -with monorepos that are highly componentized, allowing most developers to -need many fewer files in their daily work. - -### Why does `scalar clone` create a `/src` folder? - -Scalar uses a file system watcher to keep track of changes under this `src` folder. -Any activity in this folder is assumed to be important to Git operations. By -creating the `src` folder, we are making it easy for your build system to -create output folders outside the `src` directory. We commonly see systems -create folders for build outputs and package downloads. Scalar itself creates -these folders during its builds. - -Your build system may create build artifacts such as `.obj` or `.lib` files -next to your source code. These are commonly "hidden" from Git using -`.gitignore` files. Having such artifacts into your source tree creates -additional work for Git because it needs to look at these files and match them -against the `.gitignore` patterns. - -By following the pattern Scalar tries to establish and placing your build -intermediates and outputs parallel with the `src` folder and not inside it, -you can help optimize Git command performance for developers in the repository -by limiting the number of files Git needs to consider for many common -operations. - -### Why is Scalar a C# tool instead of built directly into Git? - -Currently, Scalar does a few things that Git does not do and a few -things that Git will never do. - -Git does not have a concept of running background jobs, which is critical to -keeping our repositories running smoothly. Git does not have a concept of -cache servers, so we need to use the GVFS protocol during `scalar clone` to -set up the cache server URL. Scalar also creates the `.scalarCache` directory -so multiple enlistments share objects from the remote, allowing a second -`scalar clone` operation to be much faster than the first. All of these concepts -are on our backlog for contributing to Git. Once we do, we will transition -from the Scalar implementation to the Git implementation. - -Scalar uses the GVFS protocol for some repositories. This protocol was created -by Azure Repos to support large repositories before the Git protocol supported -partial clone and the Git client supported missing objects. In a way, the GVFS -protocol proved that there was value in creating the partial clone feature. -This shows precedent for how our C# tools show value that leads to Git client -features. However, Git will never adopt the GVFS protocol, so we need to use -an tool other than the Git client to support teams that use it. - -For now, Scalar requires a -[custom version of Git](https://github.com/microsoft/git), but we hope to relax -that restriction eventually. -We try to keep our fork of Git as close to [upstream](https://github.com/git-for-windows/git) -as we can, but some patches are necessary _for now_. Still, maintaining those -patches as we take updates from upstream has a significant cost. It is better -to create features in C# that use Git as a black box, then remove those C# -features as we contribute replacements to Git upstream. - -Finally, as Git gains the features that we need in Scalar, Scalar will need to -support backwards compatibility with repositories cloned with older versions of -Scalar. Using the C# layer, we can update the Git version and upgrade old -repositories to whatever new features are added to Git. - -### Why doesn't Scalar support case-sensitive directories (or case-insensitive ones on Linux)? - -Although Scalar is agnostic about the contents of your repository and -largely leaves file handling within your working tree to Git, it does check -some file paths and manage some cache files, and to do so Scalar assumes -your filesystem has the filename case-matching semantics typical for your -operating system. On Windows and macOS this means being case-insensitive -when matching filenames, and on Linux it means matching case-sensitively. - -Because all three operating systems support exceptions to their case-matching -conventions (e.g., [case-sensitive NTFS directories][ntfs-case] or -[case-insensitive ext4 directories][ext4-case]), a possible future -enhancement for Scalar would be to support these alternatives using -a per-repository custom Git configuration value. - -[ext4-case]: https://github.com/torvalds/linux/blob/171d4ff79f965c1f164705ef0aaea102a6ad238b/Documentation/admin-guide/ext4.rst#case-insensitive-file-name-lookups -[ntfs-case]: https://devblogs.microsoft.com/commandline/per-directory-case-sensitivity-and-wsl/#per-directory-case-sensitivity diff --git a/docs/getting-started.md b/docs/getting-started.md index da3df422..073ecc85 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -6,113 +6,3 @@ Getting Started [microsoft-git]: https://github.com/microsoft/git [git-getting-started]: https://github.com/microsoft/git/blob/HEAD/contrib/scalar/docs/getting-started.md - -Registering existing Git repos ------------------------------- - -To add a repository to the list of registered repos, run `scalar register []`. -If `` is not provided, then the "current repository" is discovered from -the working directory by scanning the parent paths for a path containing a `.git` -folder, possibly inside a `src` folder. - -To see which repositories are currently tracked by the service, run -`scalar list`. - -Run `scalar unregister []` to remove the repo from this list. - -Creating a new Scalar clone using the GVFS Protocol ---------------------------------------------------- - -The `clone` verb creates a local enlistment of a remote repository using the -[GVFS protocol](https://github.com/microsoft/VFSForGit/blob/HEAD/Protocol.md). - -``` -scalar clone [options] [] -``` - -Create a local copy of the repository at ``. If specified, create the `` -directory and place the repository there. Otherwise, the last section of the `` -will be used for ``. - -At the end, the repo is located at `/src`. By default, the sparse-checkout -feature is enabled and the only files present are those in the root of your -Git repository. Use `git sparse-checkout set` to expand the set of directories -you want to see, or `git sparse-checkout disable` to expand to all files. You -can explore the subdirectories outside your sparse-checkout specification using -`git ls-tree HEAD`. - -### Sparse Repo Mode - -By default, Scalar reduces your working directory to the only the files at the -root of the repository. You need to add the folders you care about to build up -to your working set. - -* `scalar clone ` - * Please choose the **Clone with HTTPS** option in the `Clone Repository` dialog in Azure Repos, not **Clone with SSH**. -* `cd \src` -* At this point, your `src` directory only contains files that appear in your root - tree. No folders are populated. -* Set the directory list for your sparse-checkout using: - 1. `git sparse-checkout set ...` - 2. `git sparse-checkout set --stdin ` to discover the directories -in ``. - -### Options - -These options allow a user to customize their initial enlistment. - -* `--full-clone`: If specified, do not initialize the sparse-checkout feature. - All files will be present in your `src` directory. This behaves very similar - to a Git partial clone in that blobs are downloaded on demand. However, it - will use the GVFS protocol to download all Git objects. - -* `--cache-server-url=`: If specified, set the intended cache server to - the specified ``. All object queries will use the GVFS protocol to this - `` instead of the origin remote. If the remote supplies a list of - cache servers via the `/gvfs/config` endpoint, then the `clone` command - will select a nearby cache server from that list. - -* `--branch=`: Specify the branch to checkout after clone. - -* `--local-cache-path=`: Use this option to override the path for the - local Scalar cache. If not specified, then Scalar will select a default - path to share objects with your other enlistments. On Windows, this path - is a subdirectory of `:\.scalarCache\`. On Mac, this path is a - subdirectory of `~/.scalarCache/`. The default cache path is recommended so - multiple enlistments of the same remote repository share objects on the - same device. - -### Advanced Options - -The options below are not intended for use by a typical user. These are -usually used by build machines to create a temporary enlistment that -operates on a single commit. - -* `--single-branch`: Use this option to only download metadata for the branch - that will be checked out. This is helpful for build machines that target - a remote with many branches. Any `git fetch` commands after the clone will - still ask for all branches. - -* `--no-prefetch`: Use this option to not prefetch commits after clone. This - is not recommended for anyone planning to use their clone for history - traversal. Use of this option will make commands like `git log` or - `git pull` extremely slow and is therefore not recommended. - -Removing a Scalar Clone ------------------------ - -Since the `scalar clone` command sets up a file-system watcher (when available), -that watcher could prevent deleting the enlistment. Run `scalar delete ` -from outside of your enlistment to unregister the enlistment from the filesystem -watcher and delete the enlistment at ``. diff --git a/docs/index.md b/docs/index.md index efe4ee64..c6644340 100644 --- a/docs/index.md +++ b/docs/index.md @@ -3,145 +3,3 @@ Scalar: Enabling Git at Scale > **Note: Scalar has moved!** It is now part of [the `microsoft/git` fork][microsoft-git] > and thus this document has [a new version in that repository][git-index]. - -[microsoft-git]: https://github.com/microsoft/git -[git-index]: https://github.com/microsoft/git/blob/HEAD/contrib/scalar/docs/index.md - -Scalar is a tool that helps Git scale to some of the largest Git repositories. -It achieves this by enabling some advanced Git features, such as: - -* *Sparse-checkout:* limits the size of your working directory. - -* *File system monitor:* tracks the recently modified files and eliminates - the need for Git to scan the entire worktree. - -* *Commit-graph:* accelerates commit walks and reachability calculations, - speeding up commands like `git log`. - -* *Multi-pack-index:* enables fast object lookups across many pack-files. - -* *Incremental repack:* Repacks the packed Git data into fewer pack-file - without disrupting concurrent commands by using the multi-pack-index. - -By running `scalar register` in any Git repo, Scalar will automatically enable -these features for that repo and start running suggested maintenance in the -background. - -Repos cloned with the `scalar clone` command use the -[GVFS protocol](https://github.com/microsoft/VFSForGit/blob/HEAD/Protocol.md) -to significantly reduce the amount of data required to get started -using a repository. By delaying all blob downloads until they are required, -Scalar allows you to work with very large repositories quickly. This protocol -allows a network of _cache servers_ to serve objects with lower latency and -higher throughput. The cache servers also reduce load on the central server. - -Installing on macOS ------------------- - -Scalar is installed and updated on macOS via [Homebrew](https://brew.sh/). - -```sh -brew tap microsoft/git -brew install --cask scalar -``` - -If you wish to use the [GVFS Protocol][gvfs-protocol], then you will -instead need the cask that uses [our custom fork of Git][microsoft-git]: - -```sh -brew tap microsoft/git -brew install --cask scalar-azrepos -``` - -When new versions of Scalar are available, you can upgrade in a few -different ways. First, you can use `brew`: - -```sh -brew update - -# Use only one of the following, depending on which you have installed: -brew upgrade --cask scalar -brew upgrade --cask scalar-azrepos -``` - -Alternatively, you can run `scalar upgrade` and it will run the necessary -`brew` commands on your behalf. - -If your repository has many files in the working directory, then you might -want to install [Watchman](https://github.com/facebook/watchman), which -Scalar will detect and configure with Git's File System Monitor feature. - -```sh -brew install watchman -``` - -Installing on Windows --------------------- - -To install Scalar on Windows, -[download the `Installers_Windows_Release.zip` from the releases page](https://github.com/microsoft/scalar/releases). -Extract the `Installers_Windows_Release` folder, open it in a command prompt, and -run `InstallScalar.bat`. This will install the following components: - -* [Git for Windows](https://github.com/microsoft/git) (with custom patches) -* Scalar -* [Watchman](https://github.com/facebook/watchman), if you use the `--watchman` argument. - -Installing on Linux -------------------- - -Currently, we package a custom version of Git and Scalar as `.deb` packages -that can be installed after downloading from the GitHub releases pages. -At this point, you must install our custom version of Git in order to get -background maintenance as part of `scalar clone` or `scalar register`. As -that feature is accepted and merged into the core Git client, then you can -take advantage of the feature without the custom fork. - -The latest releases can be downloaded and installed as follows: - -> Note: If you don't have `wget`, then try `sudo apt-get install wget` first. - -```sh -# Install git-vfs, a custom fork of Git -wget https://github.com/microsoft/git/releases/download/v2.29.0.vfs.0.0/git-vfs_2.29.0.vfs.0.0.deb -sudo dpkg -i git-vfs_2.29.0.vfs.0.0.deb - -# Install GCM Core -wget https://github.com/microsoft/Git-Credential-Manager-Core/releases/download/v2.0.252-beta/gcmcore-linux_amd64.2.0.252.766.deb -sudo dpkg -i gcmcore-linux_amd64.2.0.252.766.deb -git-credential-manager-core configure - -# Install Scalar -wget https://github.com/microsoft/scalar/releases/download/v20.10.178.6/scalar-azrepos-linux_amd64.20.10.178.0.deb -sudo dpkg -i scalar-azrepos-linux_amd64.20.10.178.0.deb - -# Test installation -git version -git-credential-manager-core version -scalar version -``` - -We are working to provide a package via `apt-get` to make this installation -easier and better for automatically upgrading. - -The current installation via `.deb` package only works on Debian-based -Linux distributions. The software has only been compiled and tested with x86_64/amd64 -architectures. Motivated users can install [`microsoft/git`](https://github.com/microsoft/git) -and Scalar from source. See [the `InstallFromSource.sh` script](https://github.com/microsoft/scalar/blob/2dc48243c50763024b048c5f36d5f50835943dda/Scripts/Linux/InstallFromSource.sh#L62-L76) -for assistance installing Scalar from source. - -Documentation -------------- - -* [Getting Started](getting-started.md): Get started with Scalar. - Includes `scalar register`, `scalar unregister`, `scalar clone`, and - `scalar delete`. - -* [Troubleshooting](troubleshooting.md): - Collect diagnostic information or update custom settings. Includes - `scalar diagnose`, `scalar config`, `scalar upgrade`, and `scalar cache-server`. - -* [The Philosophy of Scalar](philosophy.md): Why does Scalar work the way - it does, and how do we make decisions about its future? - -* [Frequently Asked Questions](faq.md) diff --git a/docs/philosophy.md b/docs/philosophy.md index 357e0dc6..c1976582 100644 --- a/docs/philosophy.md +++ b/docs/philosophy.md @@ -6,66 +6,3 @@ The Philosophy of Scalar [microsoft-git]: https://github.com/microsoft/git [git-philosophy]: https://github.com/microsoft/git/blob/HEAD/contrib/scalar/docs/philosophy.md - -The team building Scalar has **opinions** about Git performance. Scalar -takes out the guesswork by automatically configuring your Git repositories -to take advantage of the latest and greatest features. It is difficult to -say that these are the absolute best settings for every repository, but -these settings do work for some of the largest repositories in the world. - -Scalar intends to do very little more than the standard Git client. We -actively implement new features into Git instead of Scalar, then update -Scalar only to configure those new settings. In particular, we are porting -features like background maintenance to Git to make Scalar simpler and -make Git more powerful. - -Scalar ships with [a custom version of Git][microsoft-git], -but performs most of its benefits without that version. The only feature -that is not intended to ever reach the standard Git client is Scalar's use -of [the GVFS Protocol][gvfs-protocol]. -The GVFS Protocol is a way to reduce object transfer between client and -server, named for its use in [VFS for Git](https://github.com/microsoft/vfsforgit). - -If you don't use the GVFS Protocol, then most of the value of Scalar can -be found in the core Git client. However, most of the advanced features -that really optimize Git's performance are off by default for compatibility -reasons. To really take advantage of Git's latest and greatest features, -you either need to study the [`git config` documentation](https://git-scm.com/docs/git-config) -and regularly read [the Git release notes](https://github.com/git/git/tree/master/Documentation/RelNotes). -Even if you do all that work and customize your Git settings on your machines, -you likely will want to share those settings with other team members. -Or, you can just use Scalar! - -Using `scalar register` on an existing Git repository will give you these -benefits: - -* Additional compression of your `.git/index` file. -* Hourly background `git fetch` operations, keeping you in-sync with your - remotes. -* Advanced data structures, such as the `commit-graph` and `multi-pack-index` - are updated automatically in the background. -* If Watchman is installed, then the FileSystem Monitor hook is configured - to use Watchman's change-tracking, providing faster commands such as - `git status` or `git add`. - -Additionally, if you use `scalar clone` to create a new repository, then -you will automatically get these benefits: - -* Use Git's partial clone feature to only download the files you need for - your current checkout. -* Use Git's [sparse-checkout feature][sparse-checkout] to minimize the - number of files required in your working directory. - [Read more about sparse-checkout here.][sparse-checkout-blog] -* Create the Git repository inside `/src` to make it easy to - place build artifacts outside of the Git repository, such as in - `/bin` or `/packages`. - -We also admit that these **opinions** can always be improved! If you have -an idea of how to improve our setup, consider [creating an issue](https://github.com/microsoft/scalar/issues/new) or contributing a pull request! Some [existing](https://github.com/microsoft/scalar/issues/382) -[issues](https://github.com/microsoft/scalar/issues/388) have already -improved our configuration settings and roadmap! - -[gvfs-protocol]: https://github.com/microsoft/VFSForGit/blob/HEAD/Protocol.md -[microsoft-git]: https://github.com/microsoft/git -[sparse-checkout]: https://git-scm.com/docs/git-sparse-checkout -[sparse-checkout-blog]: https://github.blog/2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout/ diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 3006cbcb..7e34e485 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -6,60 +6,3 @@ Troubleshooting [microsoft-git]: https://github.com/microsoft/git [git-troubleshooting]: https://github.com/microsoft/git/blob/HEAD/contrib/scalar/docs/troubleshooting.md - -Upgrade -------- - -The Scalar service checks for new versions of Scalar daily and will prompt you -for upgrade using a notification. To check yourself, run `scalar upgrade` to -see if an upgrade is available. Run `scalar upgrade --confirm` to actually -perform the upgrade, if you wish. - -Diagnosing Issues ------------------ - -The `scalar diagnose` command collects logs and config details for the current -repository. The resulting zip file helps root-cause issues. - -When run inside your repository, creates a zip file containing several important -files for that repository. This includes: - -* All log files from `scalar` commands run in the enlistment, including - maintenance steps. - -* Log files from the Scalar service. - -* Configuration files from your `.git` folder, such as the `config` file, - `index`, `hooks`, and `refs`. - -* A summary of your Git object database, including the number of loose objects - and the names and sizes of pack-files. - -As the `diagnose` command completes, it provides the path of the resulting -zip file. This zip can be sent to [the support team](mailto:git-client@github.com) for investigation. - -Modifying Configuration Values ------------------------------- - -The Scalar-specific configuration is only available for repos using the -GVFS protocol. - -### Cache Server URL - -When using an enlistment cloned with `scalar clone` and the GVFS protocol, -you will have a value called the cache server URL. Cache servers are a feature -of the GVFS protocol to provide low-latency access to the on-demand object -requests. This modifies the `gvfs.cache-server` setting in your local Git config -file. - -Run `scalar cache-server --get` to see the current cache server. - -Run `scalar cache-server --list` to see the available cache server URLs. - -Run `scalar cache-server --set=` to set your cache server to ``. - -### Scalar Config - -The `scalar config` command is used for customizing the feed used for -Scalar upgrades. This is so large teams can bundle a custom installer -or other tools along with Scalar upgrades.