We store and manipulate the cumulative traverse_info.pathlen as an
"int", which can overflow when we are fed ridiculously long pathnames
(e.g., ones at the edge of 2GB or 4GB, even if the individual tree entry
names are smaller than that). The results can be confusing, though
after some prodding I was not able to use this integer overflow to cause
an under-allocated buffer.
Let's consistently use size_t to generate and store these, and make
sure our addition doesn't overflow.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As the previous commit shows, the presence of an oid in each level of
the traverse_info is confusing and ultimately not necessary. Let's drop
it to make it clear that it will not always be set (as well as convince
us that it's unused, and let the compiler catch any merges with other
branches that do add new uses).
Since the oid is part of name_entry, we'll actually stop embedding a
name_entry entirely, and instead just separately hold the pathname, its
length, and the mode.
This makes the resulting code slightly more verbose as we have to pass
those elements around individually. But it also makes it more clear what
each code path is going to use (and in most of the paths, we really only
care about the pathname itself).
A few of these conversions are noisier than they need to be, as they
also take the opportunity to rename "len" to "namelen" for clarity
(especially where we also have "pathlen" or "ce_len" alongside).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We assume that if setup_traverse_info() is passed a non-empty "base"
string, that string is pointing into a tree object and we can read the
object oid by skipping past the trailing NUL.
As it turns out, this is not true for either of the two calls, and we
may end up reading garbage bytes:
1. In git-merge-tree, our base string is either empty (in which case
we'd never run this code), or it comes from our traverse_path()
helper. The latter overallocates a buffer by the_hash_algo->rawsz
bytes, but then fills it with only make_traverse_path(), leaving
those extra bytes uninitialized (but part of a legitimate heap
buffer).
2. In unpack_trees(), we pass o->prefix, which is some arbitrary
string from the caller. In "git read-tree --prefix=foo", for
instance, it will point to the command-line parameter, and we'll
read 20 bytes past the end of the string.
Interestingly, tools like ASan do not detect (2) because the process
argv is part of a big pre-allocated buffer. So we're reading trash, but
it's trash that's probably part of the next argument, or the
environment.
You can convince it to fail by putting something like this at the
beginning of common-main.c's main() function:
{
int i;
for (i = 0; i < argc; i++)
argv[i] = xstrdup_or_null(argv[i]);
}
That puts the arguments into their own heap buffers, so running:
make SANITIZE=address test
will find problems when "read-tree --prefix" is used (e.g., in t3030).
Doubly interesting, even with the hackery above, this does not fail
prior to ea82b2a085 (tree-walk: store object_id in a separate member,
2019-01-15). That commit switched setup_traverse_info() to actually
copying the hash, rather than simply pointing to it. That pointer was
always pointing to garbage memory, but that commit started actually
dereferencing the bytes, which is what triggers ASan.
That also implies that nobody actually cares about reading these oid
bytes anyway (or at least no path covered by our tests). And manual
inspection of the code backs that up (I'll follow this patch with some
cleanups that show definitively this is the case, but they're quite
invasive, so it's worth doing this fix on its own).
So let's drop the bogus hashcpy(), along with the confusing oversizing
in merge-tree.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We have been trying out a few language features outside c89; the
coding guidelines document did not talk about them and instead had
a blanket ban against them.
* jc/post-c89-rules-doc:
CodingGuidelines: spell out post-C89 rules
Code restructuring during 2.20 period broke fetching tags via
"import" based transports.
* fc/fetch-with-import-fix:
fetch: fix regression with transport helpers
fetch: make the code more understandable
fetch: trivial cleanup
t5801 (remote-helpers): add test to fetch tags
t5801 (remote-helpers): cleanup refspec stuff
The commit-graph file is now part of the "files that the runtime
may keep open file descriptors on, all of which would need to be
closed when done with the object store", and the file descriptor to
an existing commit-graph file now is closed before "gc" finalizes a
new instance to replace it.
* ds/close-object-store:
packfile: rename close_all_packs to close_object_store
packfile: close commit-graph in close_all_packs
commit-graph: use raw_object_store when closing
commit-graph: extract write_commit_graph_file()
commit-graph: extract copy_oids_to_commits()
commit-graph: extract count_distinct_commits()
commit-graph: extract fill_oids_from_all_packs()
commit-graph: extract fill_oids_from_commit_hex()
commit-graph: extract fill_oids_from_packs()
commit-graph: create write_commit_graph_context
commit-graph: remove Future Work section
commit-graph: collapse parameters into flags
commit-graph: return with errors during write
commit-graph: fix the_repository reference
"git checkout -p" needs to selectively apply a patch in reverse,
which did not work well.
* pw/add-p-recount:
add -p: fix checkout -p with pathological context
Code clean-up to avoid signed integer overlaps during binary search.
* rs/avoid-overflow-in-midpoint-computation:
cleanup: fix possible overflow errors in binary search, part 2
"git interpret-trailers" always treated '#' as the comment
character, regardless of core.commentChar setting, which has been
corrected.
* jk/trailers-use-config:
interpret-trailers: load default config
"git stash show 23" used to work, but no more after getting
rewritten in C; this regression has been corrected.
* tg/stash-ref-by-index-fix:
stash: fix show referencing stash index
"git rebase --abort" used to leave refs/rewritten/ when concluding
"git rebase -r", which has been corrected.
* pw/rebase-abort-clean-rewritten:
rebase --abort/--quit: cleanup refs/rewritten
sequencer: return errors from sequencer_remove_state()
rebase: warn if state directory cannot be removed
rebase: fix a memory leak
An incorrect list of options was cached after command line
completion failed (e.g. trying to complete a command that requires
a repository outside one), which has been corrected.
* nd/completion-no-cache-failure:
completion: do not cache if --git-completion-helper fails
The code to parse scaled numbers out of configuration files has
been made more robust and also easier to follow.
* rs/config-unit-parsing:
config: simplify parsing of unit factors
config: don't multiply in parse_unit_factor()
config: use unsigned_mult_overflows to check for overflows
The codepath to compute delta islands used to spew progress output
without giving the callers any way to squelch it, which has been
fixed.
* jk/delta-islands-progress-fix:
delta-islands: respect progress flag
Use "Erase in Line" CSI sequence that is already used in the editor
support to clear cruft in the progress output.
* sg/rebase-progress:
progress: use term_clear_line()
rebase: fix garbled progress display with '-x'
pager: add a helper function to clear the last line in the terminal
t3404: make the 'rebase.missingCommitsCheck=ignore' test more focused
t3404: modernize here doc style
"git submodule foreach" did not protect command line options passed
to the command to be run in each submodule correctly, when the
"--recursive" option was in use.
* ms/submodule-foreach-fix:
submodule foreach: fix recursion of options
The configuration variable rebase.rescheduleFailedExec should be
effective only while running an interactive rebase and should not
affect anything when running an non-interactive one, which was not
the case. This has been corrected.
* js/rebase-reschedule-applies-only-to-interactive:
rebase --am: ignore rebase.rescheduleFailedExec
The "git clone" documentation refers to command line options in its
description in the short form; they have been replaced with long
forms to make them more recognisable.
* qn/clone-doc-use-long-form:
docs: git-clone: list short form of options first
docs: git-clone: refer to long form of options
"git rm" to resolve a conflicted path leaked an internal message
"needs merge" before actually removing the path, which was
confusing. This has been corrected.
* jc/denoise-rm-to-resolve:
rm: resolving by removal is not a warning-worthy event
A codepath that reads from GPG for signed object verification read
past the end of allocated buffer, which has been fixed.
* sr/gpg-interface-stop-at-the-end:
gpg-interface: do not scan past the end of buffer
"git clean" silently skipped a path when it cannot lstat() it; now
it gives a warning.
* js/clean-report-too-long-a-path:
clean: show an error message when the path is too long
"git push --atomic" that goes over the transport-helper (namely,
the smart http transport) failed to prevent refs to be pushed when
it can locally tell that one of the ref update will fail without
having to consult the other end, which has been corrected.
* es/local-atomic-push-failure-with-http:
transport-helper: avoid var decl in for () loop control
transport-helper: enforce atomic in push_refs_with_push
Dev support update.
* sg/ci-brew-gcc-workaround:
ci/lib.sh: update a comment about installed P4 and Git-LFS versions
ci: disable Homebrew's auto cleanup
ci: don't update Homebrew
Code clean-up for new compilers.
The 'kwset' one may get a wholesale replacement, either with new
version of kwset from upstream or removal of its users, but in the
meantime, it is probably OK to merge it down.
* js/gcc-8-and-9:
config: avoid calling `labs()` on too-large data type
winansi: simplify loading the GetCurrentConsoleFontEx() function
kwset: allow building with GCC 8
poll (mingw): allow compiling with GCC 8 and DEVELOPER=1
The list of for-each like macros used by clang-format has been
updated.
* mo/clang-format-for-each-update:
clang-format: use git grep to generate the ForEachMacros list
The URL decoding code has been updated to avoid going past the end
of the string while parsing %-<hex>-<hex> sequence.
* md/url-parse-harden:
url: do not allow %00 to represent NUL in URLs
url: do not read past end of buffer
Jeff King
Junio C Hamano