Even after a handfle attempts, match_beginning logic still has corner
cases:
1bf1a85 (apply: treat EOF as proper context., 2006-05-23)
65aadb9 (apply: force matching at the beginning., 2006-05-24)
4be6096 (apply --unidiff-zero: loosen sanity checks ..., 2006-09-17)
ee5a317 (Fix "git apply" to correctly enforce "match ..., 2008-04-06)
This is a tricky piece of code.
We still incorrectly enforce "match_beginning" for -U0 matches.
I noticed this while trying out an example sequence from Clemens Buchacher:
$ echo a >victim
$ git add victim
$ echo b >>victim
$ git diff -U0 >patch
$ cat patch
diff --git i/victim w/victim
index 7898192..422c2b7 100644
--- i/victim
+++ w/victim
@@ -1,0 +2 @@ a
+b
$ git apply --cached --unidiff-zero <patch
$ git show :victim
b
a
The change inserts a new line before the second line, but we insist it to
be applied at the beginning. As the result, the code refuses to apply it
at the original offset, and we end up adding the line at the beginning.
Updates to the test script are by Clemens Buchacher.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The name path_list was correct for the first usage of that data structure,
but it really is a general-purpose string list.
$ perl -i -pe 's/path-list/string-list/g' $(git grep -l path-list)
$ perl -i -pe 's/path_list/string_list/g' $(git grep -l path_list)
$ git mv path-list.h string-list.h
$ git mv path-list.c string-list.c
$ perl -i -pe 's/has_path/has_string/g' $(git grep -l has_path)
$ perl -i -pe 's/path/string/g' string-list.[ch]
$ git mv Documentation/technical/api-path-list.txt \
Documentation/technical/api-string-list.txt
$ perl -i -pe 's/strdup_paths/strdup_strings/g' $(git grep -l strdup_paths)
... and then fix all users of string-list to access the member "string"
instead of "path".
Documentation/technical/api-string-list.txt needed some rewrapping, too.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When you misuse a git command, you are shown the usage string.
But this is currently shown in the dashed form. So if you just
copy what you see, it will not work, when the dashed form
is no longer supported.
This patch makes git commands show the dash-less version.
For shell scripts that do not specify OPTIONS_SPEC, git-sh-setup.sh
generates a dash-less usage string now.
Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
7ebd52a (Merge branch 'dz/apply-again', 2008-07-01) taught "git-apply" to
grok a (non-git) patch that is a concatenation of separate patches that
touch the same file number of times, by recording the postimage of patch
application of previous round and using it as the preimage for later
rounds.
This "incremental" mode of patch application fundamentally contradicts
with the way git rename/copy patches are designed. When a git patch talks
about a file A getting modified, and a new file B created out of A, like
this:
diff --git a/A b/A
--- a/A
+++ b/A
... change text here ...
diff --git a/A b/B
copy from A
copy to B
--- a/A
+++ b/B
... change text here ...
the second change to produce B does not depend on what is done to A with
the first change in any way. This is explicitly done so for reviewability
of individual patches.
With this commit, we do not look at 'fn_table' that records the postimage
of previous round when applying a patch to produce a new file out of an
existing file.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* js/apply-root:
git-apply --directory: make --root more similar to GNU diff
apply --root: thinkofix.
Teach "git apply" to prepend a prefix with "--root=<root>"
Applying a patch in the directory that is different from what the patch
records is done with --directory option in GNU diff. The --root option we
introduced previously does the same, and we can call it the same way to
give users more familiar feel.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If a patch modifies the last line of a file that previously had no
terminating '\n', it looks like
-old text
\ No newline at end of file
+new text
Hence, a '\' line does not signal the end of the hunk. This modifies
'git apply --recount' to take this into account.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Acked-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The end of a string is string[length-1], not string[length+1].
I pointed it out during the review, but I forgot about it when applying the
patch. This should fix it.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With "git apply --root=<root>", all file names in the patch are prepended
with <root>. If a "-p" value was given, the paths are stripped _before_
prepending <root>.
Wished for by HPA.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jc/checkdiff:
Fix t4017-diff-retval for white-space from wc
Update sample pre-commit hook to use "diff --check"
diff --check: detect leftover conflict markers
Teach "diff --check" about new blank lines at end
checkdiff: pass diff_options to the callback
check_and_emit_line(): rename and refactor
diff --check: explain why we do not care whether old side is binary
Sometimes, the easiest way to fix up a patch is to edit it directly, even
adding or deleting lines. Now, many people are not as divine as certain
benevolent dictators as to update the hunk headers correctly at the first
try.
So teach the tool to do it for us.
[jc: with tests]
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When working with a lot of people who backport patches all day long, every
once in a while I get a patch that modifies the same file more than once
inside the same patch. git-apply either fails if the second change relies
on the first change or silently drops the first change if the second change
is independent.
The silent part is the scary scenario for us. Also this behaviour is
different from the patch-utils.
I have modified git-apply to create a table of the filenames of files it
modifies such that if a later patch chunk modifies a file in the table it
will buffer the previously changed file instead of reading the original file
from disk.
Logic has been put in to handle creations/deletions/renames/copies. All the
relevant tests of git-apply succeed.
A new test has been added to cover the cases I addressed.
The fix is relatively straight-forward.
Signed-off-by: Don Zickus <dzickus@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function name was too bland and not explicit enough as to what it is
checking. Split it into two, and call the one that checks if there is a
whitespace breakage "ws_check()", and call the other one that checks and
emits the line after color coding "ws_check_emit()".
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we see no context nor deleted line in the patch, we used to declare
that the patch creates a new file. But some people create an empty file
and then apply a patch to it. Similarly, a patch that delete everything
is not a deletion patch either.
This commit corrects these two issues. Together with the previous commit,
it allows a diff between an empty file and a line-ful file to be treated
as both creation patch and "add stuff to an existing empty file",
depending on the context. A new test t4126 demonstrates the fix.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A patch from a foreign SCM (or plain "diff" output) often have both
preimage and postimage filename on ---/+++ lines even for a patch that
creates a new file. However, when there is a filename for preimage, we
used to insist the file to exist (either in the work tree and/or in the
index). When we cannot be sure by parsing the patch that it is not a
creation patch, we shouldn't complain when if there is no such a file.
This commit fixes the logic.
Refactor the code that validates the preimage file into a separate
function while we are at it, as it is getting rather big.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git_config() only had a function parameter, but no callback data
parameter. This assumes that all callback functions only modify
global variables.
With this patch, every callback gets a void * parameter, and it is hoped
that this will help the libification effort.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is the base for making symlink detection in the middle fo a pathname
saner and (much) more efficient.
Under various loads, we want to verify that the full path leading up to a
filename is a real directory tree, and that when we successfully do an
'lstat()' on a filename, we don't get a false positive due to a symlink in
the middle of the path that git should have seen as a symlink, not as a
normal path component.
The 'has_symlink_leading_path()' function already did this, and cached
a single level of symlink information, but didn't cache the _lack_ of a
symlink, so the normal behaviour was actually the wrong way around, and we
ended up doing an 'lstat()' on each path component to check that it was a
real directory.
This caches the last detected full directory and symlink entries, and
speeds up especially deep directory structures a lot by avoiding to
lstat() all the directories leading up to each entry in the index.
[ This can - and should - probably be extended upon so that we eventually
never do a bare 'lstat()' on any path entries at *all* when checking the
index, but always check the full path carefully. Right now we do not
generally check the whole path for all our normal quick index
revalidation.
We should also make sure that we're careful about all the invalidation,
ie when we remove a link and replace it by a directory we should
invalidate the symlink cache if it matches (and vice versa for the
directory cache).
But regardless, the basic function needs to be sane to do that. The old
'has_symlink_leading_path()' was not capable enough - or indeed the code
readable enough - to really do that sanely. So I'm pushing this as not
just an optimization, but as a base for further work. ]
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
git-bisect: make "start", "good" and "skip" succeed or fail atomically
git-am: cope better with an empty Subject: line
Ignore leading empty lines while summarizing merges
bisect: squelch "fatal: ref HEAD not a symref" misleading message
builtin-apply: Show a more descriptive error on failure when opening a patch
Clarify documentation of git-cvsserver, particularly in relation to git-shell
* maint-1.5.4:
git-bisect: make "start", "good" and "skip" succeed or fail atomically
git-am: cope better with an empty Subject: line
Ignore leading empty lines while summarizing merges
bisect: squelch "fatal: ref HEAD not a symref" misleading message
builtin-apply: Show a more descriptive error on failure when opening a patch
Clarify documentation of git-cvsserver, particularly in relation to git-shell
When a patch can't be opened (it doesn't exist, there are permission
problems, etc.) we get the usage text, which is not a proper indication of
failure.
Signed-off-by: Alberto Bertogli <albertito@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
An earlier commit 4be6096 (apply --unidiff-zero: loosen sanity checks for
--unidiff=0 patches, 2006-09-17) made match_beginning and match_end
computed incorrectly. If a hunk inserts at the beginning, old position
recorded at the hunk is line 0, and if a hunk changes at the beginning, it
is line 1. The new test added to t4104 exposes that the old code did not
insist on matching at the beginning for a patch to add a line to an empty
file.
An even older 65aadb9 (apply: force matching at the beginning.,
2006-05-24) was equally wrong in that it tried to take hints from the
number of leading context lines, to decide if the hunk must match at the
beginning, but we can just look at the line number in the hunk to decide.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
setup_git_directory_gently() only modified the value of its *nongit_ok
argument if we were not in a git repository. Now it will always set it
to 0 when we are inside a repository.
Also remove now unnecessary initializations in the callers of this
function.
Signed-off-by: SZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jc/apply-whitespace:
ws_fix_copy(): move the whitespace fixing function to ws.c
apply: do not barf on patch with too large an offset
core.whitespace: cr-at-eol
git-apply --whitespace=fix: fix whitespace fuzz introduced by previous run
builtin-apply.c: pass ws_rule down to match_fragment()
builtin-apply.c: move copy_wsfix() function a bit higher.
builtin-apply.c: do not feed copy_wsfix() leading '+'
builtin-apply.c: simplify calling site to apply_line()
builtin-apply.c: clean-up apply_one_fragment()
builtin-apply.c: mark common context lines in lineinfo structure.
builtin-apply.c: optimize match_beginning/end processing a bit.
builtin-apply.c: make it more line oriented
builtin-apply.c: push match-beginning/end logic down
builtin-apply.c: restructure "offset" matching
builtin-apply.c: refactor small part that matches context
* lt/in-core-index:
lazy index hashing
Create pathname-based hash-table lookup into index
read-cache.c: introduce is_racy_timestamp() helper
read-cache.c: fix a couple more CE_REMOVE conversion
Also use unpack_trees() in do_diff_cache()
Make run_diff_index() use unpack_trees(), not read_tree()
Avoid running lstat(2) on the same cache entry.
index: be careful when handling long names
Make on-disk index representation separate from in-core one
Previously a patch that records too large a line number caused the
offset matching code in git-apply to overstep its internal buffer.
Noticed by Johannes Schindelin.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
CRLF conversion bears a slight chance of corrupting data.
autocrlf=true will convert CRLF to LF during commit and LF to
CRLF during checkout. A file that contains a mixture of LF and
CRLF before the commit cannot be recreated by git. For text
files this is the right thing to do: it corrects line endings
such that we have only LF line endings in the repository.
But for binary files that are accidentally classified as text the
conversion can corrupt data.
If you recognize such corruption early you can easily fix it by
setting the conversion type explicitly in .gitattributes. Right
after committing you still have the original file in your work
tree and this file is not yet corrupted. You can explicitly tell
git that this file is binary and git will handle the file
appropriately.
Unfortunately, the desired effect of cleaning up text files with
mixed line endings and the undesired effect of corrupting binary
files cannot be distinguished. In both cases CRLFs are removed
in an irreversible way. For text files this is the right thing
to do because CRLFs are line endings, while for binary files
converting CRLFs corrupts data.
This patch adds a mechanism that can either warn the user about
an irreversible conversion or can even refuse to convert. The
mechanism is controlled by the variable core.safecrlf, with the
following values:
- false: disable safecrlf mechanism
- warn: warn about irreversible conversions
- true: refuse irreversible conversions
The default is to warn. Users are only affected by this default
if core.autocrlf is set. But the current default of git is to
leave core.autocrlf unset, so users will not see warnings unless
they deliberately chose to activate the autocrlf mechanism.
The safecrlf mechanism's details depend on the git command. The
general principles when safecrlf is active (not false) are:
- we warn/error out if files in the work tree can modified in an
irreversible way without giving the user a chance to backup the
original file.
- for read-only operations that do not modify files in the work tree
we do not not print annoying warnings.
There are exceptions. Even though...
- "git add" itself does not touch the files in the work tree, the
next checkout would, so the safety triggers;
- "git apply" to update a text file with a patch does touch the files
in the work tree, but the operation is about text files and CRLF
conversion is about fixing the line ending inconsistencies, so the
safety does not trigger;
- "git diff" itself does not touch the files in the work tree, it is
often run to inspect the changes you intend to next "git add". To
catch potential problems early, safety triggers.
The concept of a safety check was originally proposed in a similar
way by Linus Torvalds. Thanks to Dimitry Potapov for insisting
on getting the naked LF/autocrlf=true case right.
Signed-off-by: Steffen Prohaska <prohaska@zib.de>
This new error mode allows a line to have a carriage return at the
end of the line when checking and fixing trailing whitespace errors.
Some people like to keep CRLF line ending recorded in the repository,
and still want to take advantage of the automated trailing whitespace
stripping. We still show ^M in the diff output piped to "less" to
remind them that they do have the CR at the end, but these carriage
return characters at the end are no longer flagged as errors.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When you have more than one patch series, an earlier one of which
tries to introduce whitespace breakages and a later one of which
has such a new line in its context, "git-apply --whitespace=fix"
will apply and fix the whitespace breakages in the earlier one,
making the resulting file not to match the context of the later
patch.
A short demonstration is in the new test, t4125.
For example, suppose the first patch is:
diff a/hello.txt b/hello.txt
--- a/hello.txt
+++ b/hello.txt
@@ -20,3 +20,3 @@
Hello world.$
-How Are you$
-Today?$
+How are you $
+today? $
to fix broken case in the string, but it introduces unwanted
trailing whitespaces to the result (pretend you are looking at
"cat -e" output of the patch --- '$' signs are not in the patch
but are shown to make the EOL stand out). And the second patch
is to change the wording of the greeting further:
diff a/hello.txt b/hello.txt
--- a/hello.txt
+++ b/hello.txt
@@ -18,5 +18,5 @@
Greetings $
-Hello world.$
+Hello, everybody. $
How are you $
-today? $
+these days? $
If you apply the first one with --whitespace=fix, you will get
this as the result:
Hello world.$
How are you$
today?$
and this does not match the preimage of the second patch, which
demands extra whitespace after "How are you" and "today?".
This series is about teaching "git apply --whitespace=fix" to
cope with this situation better. If the patch does not apply,
it rewrites the second patch like this and retries:
diff a/hello.txt b/hello.txt
--- a/hello.txt
+++ b/hello.txt
@@ -18,5 +18,5 @@
Greetings$
-Hello world.$
+Hello, everybody.$
How are you$
-today?$
+these days?$
This is done by rewriting the preimage lines in the hunk
(i.e. the lines that begin with ' ' or '-'), using the same
whitespace fixing rules as it is using to apply the patches, so
that it can notice what it did to the previous ones in the
series.
A careful reader may notice that the first patch in the example
did not touch the "Greetings" line, so the trailing whitespace
that is in the original preimage of the second patch is not from
the series. Is rewriting this context line a problem?
If you think about it, you will realize that the reason for the
difference is because the submitter's tree was based on an
earlier version of the file that had whitespaces wrong on that
"Greetings" line, and the change that introduced the "Greetings"
line was added independently of this two-patch series to our
tree already with an earlier "git apply --whitespace=fix".
So it may appear this logic is rewriting too much, it is not
so. It is just rewriting what we would have rewritten in the
past.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is necessary to allow match_fragment() to attempt a match
with a preimage that is based on a version before whitespace
errors were fixed.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "patch" parameter used to include leading '+' of an added
line in the patch, and the array was treated as 1-based. Make
it accept the contents of the line alone and simplify the code.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function apply_line() changed its behaviour depending on the
ws_error_action, whitespace_error and if the input was a context.
Make its caller responsible for such checking so that we can convert
the function to copy the contents of line while fixing whitespace
breakage more easily.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We had two pointer variables pointing to the same buffer and an
integer variable used to index into its tail part that was
active (old, oldlines and oldsize for the preimage, and their
'new' counterparts for the postimage).
To help readability, use 'oldlines' as the allocated pointer,
and use 'old' as the pointer to the tail that advances while the
code builds up the contents in the buffer. The size 'oldsize'
can be computed as (old-oldines).
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This updates the way preimage and postimage in a patch hunk is
parsed and prepared for applying. By looking at image->line[n].flag,
the code can tell if it is a common context line that is the
same between the preimage and the postimage.
This matters when we actually start applying a patch with
contexts that have whitespace breakages that have already been
fixed in the target file.
Wnen the caller knows the hunk needs to match at the beginning
or at the end, there is no point starting from the line number
that is found in the patch and trying match with increasing
offset. The logic to find matching lines was made more line
oriented with the previous patch and this optimization is now
trivial.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This changes the way git-apply internally works to be more line
oriented. The logic to find where the patch applies with offset
used to count line numbers by always counting LF from the
beginning of the buffer, but it is simplified because we count
the line length of the target file and the preimage snippet
upfront now.
The ultimate motivation is to allow applying patches
whose preimage context has whitespace corruption that has
already been corrected in the local copy. For that purpose, we
introduce a table of line-hash that allows us to match lines
that differ only in whitespaces.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This moves the logic to force match at the beginning and/or at
the end of the buffer to the actual function that finds the
match from its caller. This is a necessary preparation for the
next step to allow matching disregarding certain differences,
such as whitespace changes.
We probably could optimize this even more by taking advantage of
the fact that match_beginning and match_end forces the match to
be at an exact location (anchored at the beginning and/or the
end), but that's for another commit.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This restructures code to find matching location with offset
in find_offset() function, so that there is need for only one
call site of match_fragment() function. There still isn't a
change in the logic of the program.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This moves three "if" conditions out of line from find_offset()
function, which is responsible for finding the matching place in
the preimage to apply the patch. There is no change in the
logic of the program.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This converts the index explicitly on read and write to its on-disk
format, allowing the in-core format to contain more flags, and be
simpler.
In particular, the in-core format is now host-endian (as opposed to the
on-disk one that is network endian in order to be able to be shared
across machines) and as a result we can dispense with all the
htonl/ntohl on accesses to the cache_entry fields.
This will make it easier to make use of various temporary flags that do
not exist in the on-disk format.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
When running "git apply --check" while --whitespace=fix is
enabled (either from the command line or via the configuration),
we reported that "N line(s) applied after _fixing_", but --check
by itself does not apply and this message was alarming.
We could even reword the message to say "N line(s) would have
been applied after fixing...", but this patch does not go that
far. Instead, we just make it use the "N lines add whitespace
errors" warning, which happens to be a good diagnostic message a
user would expect from the --check option.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano
Johannes Schindelin
Stephan Beyer
Thomas Rast
Don Zickus
Linus Torvalds
Alberto Bertogli
SZEDER Gábor
Steffen Prohaska
Brandon Casey