зеркало из https://github.com/microsoft/git.git
704 строки
30 KiB
Plaintext
704 строки
30 KiB
Plaintext
git-filter-branch(1)
|
|
====================
|
|
|
|
NAME
|
|
----
|
|
git-filter-branch - Rewrite branches
|
|
|
|
SYNOPSIS
|
|
--------
|
|
[verse]
|
|
'git filter-branch' [--setup <command>] [--subdirectory-filter <directory>]
|
|
[--env-filter <command>] [--tree-filter <command>]
|
|
[--index-filter <command>] [--parent-filter <command>]
|
|
[--msg-filter <command>] [--commit-filter <command>]
|
|
[--tag-name-filter <command>] [--prune-empty]
|
|
[--original <namespace>] [-d <directory>] [-f | --force]
|
|
[--state-branch <branch>] [--] [<rev-list options>...]
|
|
|
|
WARNING
|
|
-------
|
|
'git filter-branch' has a plethora of pitfalls that can produce non-obvious
|
|
manglings of the intended history rewrite (and can leave you with little
|
|
time to investigate such problems since it has such abysmal performance).
|
|
These safety and performance issues cannot be backward compatibly fixed and
|
|
as such, its use is not recommended. Please use an alternative history
|
|
filtering tool such as https://github.com/newren/git-filter-repo/[git
|
|
filter-repo]. If you still need to use 'git filter-branch', please
|
|
carefully read <<SAFETY>> (and <<PERFORMANCE>>) to learn about the land
|
|
mines of filter-branch, and then vigilantly avoid as many of the hazards
|
|
listed there as reasonably possible.
|
|
|
|
DESCRIPTION
|
|
-----------
|
|
Lets you rewrite Git revision history by rewriting the branches mentioned
|
|
in the <rev-list options>, applying custom filters on each revision.
|
|
Those filters can modify each tree (e.g. removing a file or running
|
|
a perl rewrite on all files) or information about each commit.
|
|
Otherwise, all information (including original commit times or merge
|
|
information) will be preserved.
|
|
|
|
The command will only rewrite the _positive_ refs mentioned in the
|
|
command line (e.g. if you pass 'a..b', only 'b' will be rewritten).
|
|
If you specify no filters, the commits will be recommitted without any
|
|
changes, which would normally have no effect. Nevertheless, this may be
|
|
useful in the future for compensating for some Git bugs or such,
|
|
therefore such a usage is permitted.
|
|
|
|
*NOTE*: This command honors `.git/info/grafts` file and refs in
|
|
the `refs/replace/` namespace.
|
|
If you have any grafts or replacement refs defined, running this command
|
|
will make them permanent.
|
|
|
|
*WARNING*! The rewritten history will have different object names for all
|
|
the objects and will not converge with the original branch. You will not
|
|
be able to easily push and distribute the rewritten branch on top of the
|
|
original branch. Please do not use this command if you do not know the
|
|
full implications, and avoid using it anyway, if a simple single commit
|
|
would suffice to fix your problem. (See the "RECOVERING FROM UPSTREAM
|
|
REBASE" section in linkgit:git-rebase[1] for further information about
|
|
rewriting published history.)
|
|
|
|
Always verify that the rewritten version is correct: The original refs,
|
|
if different from the rewritten ones, will be stored in the namespace
|
|
'refs/original/'.
|
|
|
|
Note that since this operation is very I/O expensive, it might
|
|
be a good idea to redirect the temporary directory off-disk with the
|
|
`-d` option, e.g. on tmpfs. Reportedly the speedup is very noticeable.
|
|
|
|
|
|
Filters
|
|
~~~~~~~
|
|
|
|
The filters are applied in the order as listed below. The <command>
|
|
argument is always evaluated in the shell context using the 'eval' command
|
|
(with the notable exception of the commit filter, for technical reasons).
|
|
Prior to that, the `$GIT_COMMIT` environment variable will be set to contain
|
|
the id of the commit being rewritten. Also, GIT_AUTHOR_NAME,
|
|
GIT_AUTHOR_EMAIL, GIT_AUTHOR_DATE, GIT_COMMITTER_NAME, GIT_COMMITTER_EMAIL,
|
|
and GIT_COMMITTER_DATE are taken from the current commit and exported to
|
|
the environment, in order to affect the author and committer identities of
|
|
the replacement commit created by linkgit:git-commit-tree[1] after the
|
|
filters have run.
|
|
|
|
If any evaluation of <command> returns a non-zero exit status, the whole
|
|
operation will be aborted.
|
|
|
|
A 'map' function is available that takes an "original sha1 id" argument
|
|
and outputs a "rewritten sha1 id" if the commit has been already
|
|
rewritten, and "original sha1 id" otherwise; the 'map' function can
|
|
return several ids on separate lines if your commit filter emitted
|
|
multiple commits.
|
|
|
|
|
|
OPTIONS
|
|
-------
|
|
|
|
--setup <command>::
|
|
This is not a real filter executed for each commit but a one
|
|
time setup just before the loop. Therefore no commit-specific
|
|
variables are defined yet. Functions or variables defined here
|
|
can be used or modified in the following filter steps except
|
|
the commit filter, for technical reasons.
|
|
|
|
--subdirectory-filter <directory>::
|
|
Only look at the history which touches the given subdirectory.
|
|
The result will contain that directory (and only that) as its
|
|
project root. Implies <<Remap_to_ancestor>>.
|
|
|
|
--env-filter <command>::
|
|
This filter may be used if you only need to modify the environment
|
|
in which the commit will be performed. Specifically, you might
|
|
want to rewrite the author/committer name/email/time environment
|
|
variables (see linkgit:git-commit-tree[1] for details).
|
|
|
|
--tree-filter <command>::
|
|
This is the filter for rewriting the tree and its contents.
|
|
The argument is evaluated in shell with the working
|
|
directory set to the root of the checked out tree. The new tree
|
|
is then used as-is (new files are auto-added, disappeared files
|
|
are auto-removed - neither .gitignore files nor any other ignore
|
|
rules *HAVE ANY EFFECT*!).
|
|
|
|
--index-filter <command>::
|
|
This is the filter for rewriting the index. It is similar to the
|
|
tree filter but does not check out the tree, which makes it much
|
|
faster. Frequently used with `git rm --cached
|
|
--ignore-unmatch ...`, see EXAMPLES below. For hairy
|
|
cases, see linkgit:git-update-index[1].
|
|
|
|
--parent-filter <command>::
|
|
This is the filter for rewriting the commit's parent list.
|
|
It will receive the parent string on stdin and shall output
|
|
the new parent string on stdout. The parent string is in
|
|
the format described in linkgit:git-commit-tree[1]: empty for
|
|
the initial commit, "-p parent" for a normal commit and
|
|
"-p parent1 -p parent2 -p parent3 ..." for a merge commit.
|
|
|
|
--msg-filter <command>::
|
|
This is the filter for rewriting the commit messages.
|
|
The argument is evaluated in the shell with the original
|
|
commit message on standard input; its standard output is
|
|
used as the new commit message.
|
|
|
|
--commit-filter <command>::
|
|
This is the filter for performing the commit.
|
|
If this filter is specified, it will be called instead of the
|
|
'git commit-tree' command, with arguments of the form
|
|
"<TREE_ID> [(-p <PARENT_COMMIT_ID>)...]" and the log message on
|
|
stdin. The commit id is expected on stdout.
|
|
+
|
|
As a special extension, the commit filter may emit multiple
|
|
commit ids; in that case, the rewritten children of the original commit will
|
|
have all of them as parents.
|
|
+
|
|
You can use the 'map' convenience function in this filter, and other
|
|
convenience functions, too. For example, calling 'skip_commit "$@"'
|
|
will leave out the current commit (but not its changes! If you want
|
|
that, use 'git rebase' instead).
|
|
+
|
|
You can also use the `git_commit_non_empty_tree "$@"` instead of
|
|
`git commit-tree "$@"` if you don't wish to keep commits with a single parent
|
|
and that makes no change to the tree.
|
|
|
|
--tag-name-filter <command>::
|
|
This is the filter for rewriting tag names. When passed,
|
|
it will be called for every tag ref that points to a rewritten
|
|
object (or to a tag object which points to a rewritten object).
|
|
The original tag name is passed via standard input, and the new
|
|
tag name is expected on standard output.
|
|
+
|
|
The original tags are not deleted, but can be overwritten;
|
|
use "--tag-name-filter cat" to simply update the tags. In this
|
|
case, be very careful and make sure you have the old tags
|
|
backed up in case the conversion has run afoul.
|
|
+
|
|
Nearly proper rewriting of tag objects is supported. If the tag has
|
|
a message attached, a new tag object will be created with the same message,
|
|
author, and timestamp. If the tag has a signature attached, the
|
|
signature will be stripped. It is by definition impossible to preserve
|
|
signatures. The reason this is "nearly" proper, is because ideally if
|
|
the tag did not change (points to the same object, has the same name, etc.)
|
|
it should retain any signature. That is not the case, signatures will always
|
|
be removed, buyer beware. There is also no support for changing the
|
|
author or timestamp (or the tag message for that matter). Tags which point
|
|
to other tags will be rewritten to point to the underlying commit.
|
|
|
|
--prune-empty::
|
|
Some filters will generate empty commits that leave the tree untouched.
|
|
This option instructs git-filter-branch to remove such commits if they
|
|
have exactly one or zero non-pruned parents; merge commits will
|
|
therefore remain intact. This option cannot be used together with
|
|
`--commit-filter`, though the same effect can be achieved by using the
|
|
provided `git_commit_non_empty_tree` function in a commit filter.
|
|
|
|
--original <namespace>::
|
|
Use this option to set the namespace where the original commits
|
|
will be stored. The default value is 'refs/original'.
|
|
|
|
-d <directory>::
|
|
Use this option to set the path to the temporary directory used for
|
|
rewriting. When applying a tree filter, the command needs to
|
|
temporarily check out the tree to some directory, which may consume
|
|
considerable space in case of large projects. By default it
|
|
does this in the `.git-rewrite/` directory but you can override
|
|
that choice by this parameter.
|
|
|
|
-f::
|
|
--force::
|
|
'git filter-branch' refuses to start with an existing temporary
|
|
directory or when there are already refs starting with
|
|
'refs/original/', unless forced.
|
|
|
|
--state-branch <branch>::
|
|
This option will cause the mapping from old to new objects to
|
|
be loaded from named branch upon startup and saved as a new
|
|
commit to that branch upon exit, enabling incremental of large
|
|
trees. If '<branch>' does not exist it will be created.
|
|
|
|
<rev-list options>...::
|
|
Arguments for 'git rev-list'. All positive refs included by
|
|
these options are rewritten. You may also specify options
|
|
such as `--all`, but you must use `--` to separate them from
|
|
the 'git filter-branch' options. Implies <<Remap_to_ancestor>>.
|
|
|
|
|
|
[[Remap_to_ancestor]]
|
|
Remap to ancestor
|
|
~~~~~~~~~~~~~~~~~
|
|
|
|
By using linkgit:git-rev-list[1] arguments, e.g., path limiters, you can limit the
|
|
set of revisions which get rewritten. However, positive refs on the command
|
|
line are distinguished: we don't let them be excluded by such limiters. For
|
|
this purpose, they are instead rewritten to point at the nearest ancestor that
|
|
was not excluded.
|
|
|
|
|
|
EXIT STATUS
|
|
-----------
|
|
|
|
On success, the exit status is `0`. If the filter can't find any commits to
|
|
rewrite, the exit status is `2`. On any other error, the exit status may be
|
|
any other non-zero value.
|
|
|
|
|
|
EXAMPLES
|
|
--------
|
|
|
|
Suppose you want to remove a file (containing confidential information
|
|
or copyright violation) from all commits:
|
|
|
|
-------------------------------------------------------
|
|
git filter-branch --tree-filter 'rm filename' HEAD
|
|
-------------------------------------------------------
|
|
|
|
However, if the file is absent from the tree of some commit,
|
|
a simple `rm filename` will fail for that tree and commit.
|
|
Thus you may instead want to use `rm -f filename` as the script.
|
|
|
|
Using `--index-filter` with 'git rm' yields a significantly faster
|
|
version. Like with using `rm filename`, `git rm --cached filename`
|
|
will fail if the file is absent from the tree of a commit. If you
|
|
want to "completely forget" a file, it does not matter when it entered
|
|
history, so we also add `--ignore-unmatch`:
|
|
|
|
--------------------------------------------------------------------------
|
|
git filter-branch --index-filter 'git rm --cached --ignore-unmatch filename' HEAD
|
|
--------------------------------------------------------------------------
|
|
|
|
Now, you will get the rewritten history saved in HEAD.
|
|
|
|
To rewrite the repository to look as if `foodir/` had been its project
|
|
root, and discard all other history:
|
|
|
|
-------------------------------------------------------
|
|
git filter-branch --subdirectory-filter foodir -- --all
|
|
-------------------------------------------------------
|
|
|
|
Thus you can, e.g., turn a library subdirectory into a repository of
|
|
its own. Note the `--` that separates 'filter-branch' options from
|
|
revision options, and the `--all` to rewrite all branches and tags.
|
|
|
|
To set a commit (which typically is at the tip of another
|
|
history) to be the parent of the current initial commit, in
|
|
order to paste the other history behind the current history:
|
|
|
|
-------------------------------------------------------------------
|
|
git filter-branch --parent-filter 'sed "s/^\$/-p <graft-id>/"' HEAD
|
|
-------------------------------------------------------------------
|
|
|
|
(if the parent string is empty - which happens when we are dealing with
|
|
the initial commit - add graftcommit as a parent). Note that this assumes
|
|
history with a single root (that is, no merge without common ancestors
|
|
happened). If this is not the case, use:
|
|
|
|
--------------------------------------------------------------------------
|
|
git filter-branch --parent-filter \
|
|
'test $GIT_COMMIT = <commit-id> && echo "-p <graft-id>" || cat' HEAD
|
|
--------------------------------------------------------------------------
|
|
|
|
or even simpler:
|
|
|
|
-----------------------------------------------
|
|
git replace --graft $commit-id $graft-id
|
|
git filter-branch $graft-id..HEAD
|
|
-----------------------------------------------
|
|
|
|
To remove commits authored by "Darl McBribe" from the history:
|
|
|
|
------------------------------------------------------------------------------
|
|
git filter-branch --commit-filter '
|
|
if [ "$GIT_AUTHOR_NAME" = "Darl McBribe" ];
|
|
then
|
|
skip_commit "$@";
|
|
else
|
|
git commit-tree "$@";
|
|
fi' HEAD
|
|
------------------------------------------------------------------------------
|
|
|
|
The function 'skip_commit' is defined as follows:
|
|
|
|
--------------------------
|
|
skip_commit()
|
|
{
|
|
shift;
|
|
while [ -n "$1" ];
|
|
do
|
|
shift;
|
|
map "$1";
|
|
shift;
|
|
done;
|
|
}
|
|
--------------------------
|
|
|
|
The shift magic first throws away the tree id and then the -p
|
|
parameters. Note that this handles merges properly! In case Darl
|
|
committed a merge between P1 and P2, it will be propagated properly
|
|
and all children of the merge will become merge commits with P1,P2
|
|
as their parents instead of the merge commit.
|
|
|
|
*NOTE* the changes introduced by the commits, and which are not reverted
|
|
by subsequent commits, will still be in the rewritten branch. If you want
|
|
to throw out _changes_ together with the commits, you should use the
|
|
interactive mode of 'git rebase'.
|
|
|
|
You can rewrite the commit log messages using `--msg-filter`. For
|
|
example, 'git svn-id' strings in a repository created by 'git svn' can
|
|
be removed this way:
|
|
|
|
-------------------------------------------------------
|
|
git filter-branch --msg-filter '
|
|
sed -e "/^git-svn-id:/d"
|
|
'
|
|
-------------------------------------------------------
|
|
|
|
If you need to add 'Acked-by' lines to, say, the last 10 commits (none
|
|
of which is a merge), use this command:
|
|
|
|
--------------------------------------------------------
|
|
git filter-branch --msg-filter '
|
|
cat &&
|
|
echo "Acked-by: Bugs Bunny <bunny@bugzilla.org>"
|
|
' HEAD~10..HEAD
|
|
--------------------------------------------------------
|
|
|
|
The `--env-filter` option can be used to modify committer and/or author
|
|
identity. For example, if you found out that your commits have the wrong
|
|
identity due to a misconfigured user.email, you can make a correction,
|
|
before publishing the project, like this:
|
|
|
|
--------------------------------------------------------
|
|
git filter-branch --env-filter '
|
|
if test "$GIT_AUTHOR_EMAIL" = "root@localhost"
|
|
then
|
|
GIT_AUTHOR_EMAIL=john@example.com
|
|
fi
|
|
if test "$GIT_COMMITTER_EMAIL" = "root@localhost"
|
|
then
|
|
GIT_COMMITTER_EMAIL=john@example.com
|
|
fi
|
|
' -- --all
|
|
--------------------------------------------------------
|
|
|
|
To restrict rewriting to only part of the history, specify a revision
|
|
range in addition to the new branch name. The new branch name will
|
|
point to the top-most revision that a 'git rev-list' of this range
|
|
will print.
|
|
|
|
Consider this history:
|
|
|
|
------------------
|
|
D--E--F--G--H
|
|
/ /
|
|
A--B-----C
|
|
------------------
|
|
|
|
To rewrite only commits D,E,F,G,H, but leave A, B and C alone, use:
|
|
|
|
--------------------------------
|
|
git filter-branch ... C..H
|
|
--------------------------------
|
|
|
|
To rewrite commits E,F,G,H, use one of these:
|
|
|
|
----------------------------------------
|
|
git filter-branch ... C..H --not D
|
|
git filter-branch ... D..H --not C
|
|
----------------------------------------
|
|
|
|
To move the whole tree into a subdirectory, or remove it from there:
|
|
|
|
---------------------------------------------------------------
|
|
git filter-branch --index-filter \
|
|
'git ls-files -s | sed "s-\t\"*-&newsubdir/-" |
|
|
GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
|
|
git update-index --index-info &&
|
|
mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"' HEAD
|
|
---------------------------------------------------------------
|
|
|
|
|
|
|
|
CHECKLIST FOR SHRINKING A REPOSITORY
|
|
------------------------------------
|
|
|
|
git-filter-branch can be used to get rid of a subset of files,
|
|
usually with some combination of `--index-filter` and
|
|
`--subdirectory-filter`. People expect the resulting repository to
|
|
be smaller than the original, but you need a few more steps to
|
|
actually make it smaller, because Git tries hard not to lose your
|
|
objects until you tell it to. First make sure that:
|
|
|
|
* You really removed all variants of a filename, if a blob was moved
|
|
over its lifetime. `git log --name-only --follow --all -- filename`
|
|
can help you find renames.
|
|
|
|
* You really filtered all refs: use `--tag-name-filter cat -- --all`
|
|
when calling git-filter-branch.
|
|
|
|
Then there are two ways to get a smaller repository. A safer way is
|
|
to clone, that keeps your original intact.
|
|
|
|
* Clone it with `git clone file:///path/to/repo`. The clone
|
|
will not have the removed objects. See linkgit:git-clone[1]. (Note
|
|
that cloning with a plain path just hardlinks everything!)
|
|
|
|
If you really don't want to clone it, for whatever reasons, check the
|
|
following points instead (in this order). This is a very destructive
|
|
approach, so *make a backup* or go back to cloning it. You have been
|
|
warned.
|
|
|
|
* Remove the original refs backed up by git-filter-branch: say `git
|
|
for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git
|
|
update-ref -d`.
|
|
|
|
* Expire all reflogs with `git reflog expire --expire=now --all`.
|
|
|
|
* Garbage collect all unreferenced objects with `git gc --prune=now`
|
|
(or if your git-gc is not new enough to support arguments to
|
|
`--prune`, use `git repack -ad; git prune` instead).
|
|
|
|
[[PERFORMANCE]]
|
|
PERFORMANCE
|
|
-----------
|
|
|
|
The performance of git-filter-branch is glacially slow; its design makes it
|
|
impossible for a backward-compatible implementation to ever be fast:
|
|
|
|
* In editing files, git-filter-branch by design checks out each and
|
|
every commit as it existed in the original repo. If your repo has
|
|
`10^5` files and `10^5` commits, but each commit only modifies five
|
|
files, then git-filter-branch will make you do `10^10` modifications,
|
|
despite only having (at most) `5*10^5` unique blobs.
|
|
|
|
* If you try and cheat and try to make git-filter-branch only work on
|
|
files modified in a commit, then two things happen
|
|
|
|
** you run into problems with deletions whenever the user is simply
|
|
trying to rename files (because attempting to delete files that
|
|
don't exist looks like a no-op; it takes some chicanery to remap
|
|
deletes across file renames when the renames happen via arbitrary
|
|
user-provided shell)
|
|
|
|
** even if you succeed at the map-deletes-for-renames chicanery, you
|
|
still technically violate backward compatibility because users
|
|
are allowed to filter files in ways that depend upon topology of
|
|
commits instead of filtering solely based on file contents or
|
|
names (though this has not been observed in the wild).
|
|
|
|
* Even if you don't need to edit files but only want to e.g. rename or
|
|
remove some and thus can avoid checking out each file (i.e. you can
|
|
use --index-filter), you still are passing shell snippets for your
|
|
filters. This means that for every commit, you have to have a
|
|
prepared git repo where those filters can be run. That's a
|
|
significant setup.
|
|
|
|
* Further, several additional files are created or updated per commit
|
|
by git-filter-branch. Some of these are for supporting the
|
|
convenience functions provided by git-filter-branch (such as map()),
|
|
while others are for keeping track of internal state (but could have
|
|
also been accessed by user filters; one of git-filter-branch's
|
|
regression tests does so). This essentially amounts to using the
|
|
filesystem as an IPC mechanism between git-filter-branch and the
|
|
user-provided filters. Disks tend to be a slow IPC mechanism, and
|
|
writing these files also effectively represents a forced
|
|
synchronization point between separate processes that we hit with
|
|
every commit.
|
|
|
|
* The user-provided shell commands will likely involve a pipeline of
|
|
commands, resulting in the creation of many processes per commit.
|
|
Creating and running another process takes a widely varying amount
|
|
of time between operating systems, but on any platform it is very
|
|
slow relative to invoking a function.
|
|
|
|
* git-filter-branch itself is written in shell, which is kind of slow.
|
|
This is the one performance issue that could be backward-compatibly
|
|
fixed, but compared to the above problems that are intrinsic to the
|
|
design of git-filter-branch, the language of the tool itself is a
|
|
relatively minor issue.
|
|
|
|
** Side note: Unfortunately, people tend to fixate on the
|
|
written-in-shell aspect and periodically ask if git-filter-branch
|
|
could be rewritten in another language to fix the performance
|
|
issues. Not only does that ignore the bigger intrinsic problems
|
|
with the design, it'd help less than you'd expect: if
|
|
git-filter-branch itself were not shell, then the convenience
|
|
functions (map(), skip_commit(), etc) and the `--setup` argument
|
|
could no longer be executed once at the beginning of the program
|
|
but would instead need to be prepended to every user filter (and
|
|
thus re-executed with every commit).
|
|
|
|
The https://github.com/newren/git-filter-repo/[git filter-repo] tool is
|
|
an alternative to git-filter-branch which does not suffer from these
|
|
performance problems or the safety problems (mentioned below). For those
|
|
with existing tooling which relies upon git-filter-branch, 'git
|
|
filter-repo' also provides
|
|
https://github.com/newren/git-filter-repo/blob/master/contrib/filter-repo-demos/filter-lamely[filter-lamely],
|
|
a drop-in git-filter-branch replacement (with a few caveats). While
|
|
filter-lamely suffers from all the same safety issues as
|
|
git-filter-branch, it at least ameliorates the performance issues a
|
|
little.
|
|
|
|
[[SAFETY]]
|
|
SAFETY
|
|
------
|
|
|
|
git-filter-branch is riddled with gotchas resulting in various ways to
|
|
easily corrupt repos or end up with a mess worse than what you started
|
|
with:
|
|
|
|
* Someone can have a set of "working and tested filters" which they
|
|
document or provide to a coworker, who then runs them on a different
|
|
OS where the same commands are not working/tested (some examples in
|
|
the git-filter-branch manpage are also affected by this).
|
|
BSD vs. GNU userland differences can really bite. If lucky, error
|
|
messages are spewed. But just as likely, the commands either don't
|
|
do the filtering requested, or silently corrupt by making some
|
|
unwanted change. The unwanted change may only affect a few commits,
|
|
so it's not necessarily obvious either. (The fact that problems
|
|
won't necessarily be obvious means they are likely to go unnoticed
|
|
until the rewritten history is in use for quite a while, at which
|
|
point it's really hard to justify another flag-day for another
|
|
rewrite.)
|
|
|
|
* Filenames with spaces are often mishandled by shell snippets since
|
|
they cause problems for shell pipelines. Not everyone is familiar
|
|
with find -print0, xargs -0, git-ls-files -z, etc. Even people who
|
|
are familiar with these may assume such flags are not relevant
|
|
because someone else renamed any such files in their repo back
|
|
before the person doing the filtering joined the project. And
|
|
often, even those familiar with handling arguments with spaces may
|
|
not do so just because they aren't in the mindset of thinking about
|
|
everything that could possibly go wrong.
|
|
|
|
* Non-ascii filenames can be silently removed despite being in a
|
|
desired directory. Keeping only wanted paths is often done using
|
|
pipelines like `git ls-files | grep -v ^WANTED_DIR/ | xargs git rm`.
|
|
ls-files will only quote filenames if needed, so folks may not
|
|
notice that one of the files didn't match the regex (at least not
|
|
until it's much too late). Yes, someone who knows about
|
|
core.quotePath can avoid this (unless they have other special
|
|
characters like \t, \n, or "), and people who use ls-files -z with
|
|
something other than grep can avoid this, but that doesn't mean they
|
|
will.
|
|
|
|
* Similarly, when moving files around, one can find that filenames
|
|
with non-ascii or special characters end up in a different
|
|
directory, one that includes a double quote character. (This is
|
|
technically the same issue as above with quoting, but perhaps an
|
|
interesting different way that it can and has manifested as a
|
|
problem.)
|
|
|
|
* It's far too easy to accidentally mix up old and new history. It's
|
|
still possible with any tool, but git-filter-branch almost
|
|
invites it. If lucky, the only downside is users getting frustrated
|
|
that they don't know how to shrink their repo and remove the old
|
|
stuff. If unlucky, they merge old and new history and end up with
|
|
multiple "copies" of each commit, some of which have unwanted or
|
|
sensitive files and others which don't. This comes about in
|
|
multiple different ways:
|
|
|
|
** the default to only doing a partial history rewrite ('--all' is not
|
|
the default and few examples show it)
|
|
|
|
** the fact that there's no automatic post-run cleanup
|
|
|
|
** the fact that --tag-name-filter (when used to rename tags) doesn't
|
|
remove the old tags but just adds new ones with the new name
|
|
|
|
** the fact that little educational information is provided to inform
|
|
users of the ramifications of a rewrite and how to avoid mixing old
|
|
and new history. For example, this man page discusses how users
|
|
need to understand that they need to rebase their changes for all
|
|
their branches on top of new history (or delete and reclone), but
|
|
that's only one of multiple concerns to consider. See the
|
|
"DISCUSSION" section of the git filter-repo manual page for more
|
|
details.
|
|
|
|
* Annotated tags can be accidentally converted to lightweight tags,
|
|
due to either of two issues:
|
|
|
|
** Someone can do a history rewrite, realize they messed up, restore
|
|
from the backups in refs/original/, and then redo their
|
|
git-filter-branch command. (The backup in refs/original/ is not a
|
|
real backup; it dereferences tags first.)
|
|
|
|
** Running git-filter-branch with either --tags or --all in your
|
|
<rev-list options>. In order to retain annotated tags as
|
|
annotated, you must use --tag-name-filter (and must not have
|
|
restored from refs/original/ in a previously botched rewrite).
|
|
|
|
* Any commit messages that specify an encoding will become corrupted
|
|
by the rewrite; git-filter-branch ignores the encoding, takes the
|
|
original bytes, and feeds it to commit-tree without telling it the
|
|
proper encoding. (This happens whether or not --msg-filter is
|
|
used.)
|
|
|
|
* Commit messages (even if they are all UTF-8) by default become
|
|
corrupted due to not being updated -- any references to other commit
|
|
hashes in commit messages will now refer to no-longer-extant
|
|
commits.
|
|
|
|
* There are no facilities for helping users find what unwanted crud
|
|
they should delete, which means they are much more likely to have
|
|
incomplete or partial cleanups that sometimes result in confusion
|
|
and people wasting time trying to understand. (For example, folks
|
|
tend to just look for big files to delete instead of big directories
|
|
or extensions, and once they do so, then sometime later folks using
|
|
the new repository who are going through history will notice a build
|
|
artifact directory that has some files but not others, or a cache of
|
|
dependencies (node_modules or similar) which couldn't have ever been
|
|
functional since it's missing some files.)
|
|
|
|
* If --prune-empty isn't specified, then the filtering process can
|
|
create hoards of confusing empty commits
|
|
|
|
* If --prune-empty is specified, then intentionally placed empty
|
|
commits from before the filtering operation are also pruned instead
|
|
of just pruning commits that became empty due to filtering rules.
|
|
|
|
* If --prune-empty is specified, sometimes empty commits are missed
|
|
and left around anyway (a somewhat rare bug, but it happens...)
|
|
|
|
* A minor issue, but users who have a goal to update all names and
|
|
emails in a repository may be led to --env-filter which will only
|
|
update authors and committers, missing taggers.
|
|
|
|
* If the user provides a --tag-name-filter that maps multiple tags to
|
|
the same name, no warning or error is provided; git-filter-branch
|
|
simply overwrites each tag in some undocumented pre-defined order
|
|
resulting in only one tag at the end. (A git-filter-branch
|
|
regression test requires this surprising behavior.)
|
|
|
|
Also, the poor performance of git-filter-branch often leads to safety
|
|
issues:
|
|
|
|
* Coming up with the correct shell snippet to do the filtering you
|
|
want is sometimes difficult unless you're just doing a trivial
|
|
modification such as deleting a couple files. Unfortunately, people
|
|
often learn if the snippet is right or wrong by trying it out, but
|
|
the rightness or wrongness can vary depending on special
|
|
circumstances (spaces in filenames, non-ascii filenames, funny
|
|
author names or emails, invalid timezones, presence of grafts or
|
|
replace objects, etc.), meaning they may have to wait a long time,
|
|
hit an error, then restart. The performance of git-filter-branch is
|
|
so bad that this cycle is painful, reducing the time available to
|
|
carefully re-check (to say nothing about what it does to the
|
|
patience of the person doing the rewrite even if they do technically
|
|
have more time available). This problem is extra compounded because
|
|
errors from broken filters may not be shown for a long time and/or
|
|
get lost in a sea of output. Even worse, broken filters often just
|
|
result in silent incorrect rewrites.
|
|
|
|
* To top it all off, even when users finally find working commands,
|
|
they naturally want to share them. But they may be unaware that
|
|
their repo didn't have some special cases that someone else's does.
|
|
So, when someone else with a different repository runs the same
|
|
commands, they get hit by the problems above. Or, the user just
|
|
runs commands that really were vetted for special cases, but they
|
|
run it on a different OS where it doesn't work, as noted above.
|
|
|
|
GIT
|
|
---
|
|
Part of the linkgit:git[1] suite
|