From b7916422c741b50925d454819b1977449fccc111 Mon Sep 17 00:00:00 2001 From: Jeff King Date: Thu, 19 Mar 2015 16:34:51 -0400 Subject: [PATCH 1/4] filter_ref: avoid overwriting ref->old_sha1 with garbage If the server supports allow_tip_sha1_in_want, then fetch-pack's filter_refs function tries to check whether a ref is a request for a straight sha1 by running: if (get_sha1_hex(ref->name, ref->old_sha1)) ... I.e., we are using get_sha1_hex to ask "is this ref name a sha1?". If it is true, then the contents of ref->old_sha1 will end up unchanged. But if it is false, then get_sha1_hex makes no guarantees about what it has written. With a ref name like "abcdefoo", we would overwrite 3 bytes of ref->old_sha1 before realizing that it was not a sha1. This is likely not a problem in practice, as anything in refs->name (besides a sha1) will start with "refs/", meaning that we would notice on the first character that there is a problem. Still, we are making assumptions about the state left in the output when get_sha1_hex returns an error (e.g., it could start from the end of the string, or error check the values only once they were placed in the output). It's better to be defensive. We could just check that we have exactly 40 characters of sha1. But let's be even more careful and make sure that we have a 40-char hex refname that matches what is in old_sha1. This is perhaps overly defensive, but spells out our assumptions clearly. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano --- fetch-pack.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/fetch-pack.c b/fetch-pack.c index 655ee64256..058c25837d 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -544,10 +544,14 @@ static void filter_refs(struct fetch_pack_args *args, /* Append unmatched requests to the list */ if (allow_tip_sha1_in_want) { for (i = 0; i < nr_sought; i++) { + unsigned char sha1[20]; + ref = sought[i]; if (ref->matched) continue; - if (get_sha1_hex(ref->name, ref->old_sha1)) + if (get_sha1_hex(ref->name, sha1) || + ref->name[40] != '\0' || + hashcmp(sha1, ref->old_sha1)) continue; ref->matched = 1; From c3c17bf10725149e2f806c73aceb522509afcc83 Mon Sep 17 00:00:00 2001 From: Jeff King Date: Thu, 19 Mar 2015 16:37:09 -0400 Subject: [PATCH 2/4] filter_ref: make a copy of extra "sought" entries If the server supports allow_tip_sha1_in_want, we add any unmatched raw-sha1 entries in our "sought" list of refs to the list of refs we will ask the other side for. We do so by inserting the original "struct ref" directly into our list, rather than making a copy. This has several problems. The most minor problem is that one cannot ever free the resulting list; it contains structs that are copies of the remote refs (made earlier by fetch_pack) along with sought refs that are referenced elsewhere. But more importantly that we set the ref->next pointer to NULL, chopping off the remainder of any existing list that the ref was a part of. We get the set of "sought" refs in an array rather than a linked list, but that array is often in turn generated from a list. The test modification in t5516 demonstrates this. Rather than fetching just an exact sha1, we fetch that sha1 plus another ref: - we build a linked list of refs to fetch when do_fetch calls get_ref_map; the exact sha1 is first, followed by the named ref ("refs/heads/extra" in this case). - we pass that linked list to transport_fetch_ref, which squashes it into an array of pointers - that array goes to fetch_pack, which calls filter_ref. There we generate the want list from a mix of what the remote side has advertised, and the "sought" entry for the exact sha1. We set the sought entry's "next" pointer to NULL. - after we return from transport_fetch_refs, we then try to update the refs by following the linked list. But our list is now truncated, and we do not update refs/heads/extra at all. We can fix this by making a copy of the ref. There's nothing that fetch_pack does to it that must be reflected in the original "sought" list (and indeed, if that were the case we would have a serious bug, because it is only exact-sha1 entries which are treated this way). Signed-off-by: Jeff King Signed-off-by: Junio C Hamano --- fetch-pack.c | 5 ++--- t/t5516-fetch-push.sh | 13 ++++++++++--- 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/fetch-pack.c b/fetch-pack.c index 058c25837d..84d5a66967 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -555,9 +555,8 @@ static void filter_refs(struct fetch_pack_args *args, continue; ref->matched = 1; - *newtail = ref; - ref->next = NULL; - newtail = &ref->next; + *newtail = copy_ref(ref); + newtail = &(*newtail)->next; } } *refs = newlist; diff --git a/t/t5516-fetch-push.sh b/t/t5516-fetch-push.sh index f4da20aa9b..f7947315ba 100755 --- a/t/t5516-fetch-push.sh +++ b/t/t5516-fetch-push.sh @@ -1107,9 +1107,16 @@ test_expect_success 'fetch exact SHA1' ' git config uploadpack.allowtipsha1inwant true ) && - git fetch -v ../testrepo $the_commit:refs/heads/copy && - result=$(git rev-parse --verify refs/heads/copy) && - test "$the_commit" = "$result" + git fetch -v ../testrepo $the_commit:refs/heads/copy master:refs/heads/extra && + cat >expect <<-EOF && + $the_commit + $the_first_commit + EOF + { + git rev-parse --verify refs/heads/copy && + git rev-parse --verify refs/heads/extra + } >actual && + test_cmp expect actual ) ' From 626df76e3df3a0d8ecee07098706547648e6c96f Mon Sep 17 00:00:00 2001 From: Jeff King Date: Thu, 19 Mar 2015 16:38:35 -0400 Subject: [PATCH 3/4] fetch_refs_via_pack: free extra copy of refs When fetch_refs_via_pack calls fetch_pack(), we pass a list of refs to fetch, and the function returns either a copy of that list, with the fetched items filled in, or NULL. We check the return value to see whether the fetch was successful, but do not otherwise look at the copy, and simply leak it at the end of the function. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano --- transport.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/transport.c b/transport.c index 70d38e4c4b..2e11f8ea82 100644 --- a/transport.c +++ b/transport.c @@ -519,7 +519,7 @@ static int fetch_refs_via_pack(struct transport *transport, int nr_heads, struct ref **to_fetch) { struct git_transport_data *data = transport->data; - const struct ref *refs; + struct ref *refs; char *dest = xstrdup(transport->url); struct fetch_pack_args args; struct ref *refs_tmp = NULL; @@ -552,15 +552,17 @@ static int fetch_refs_via_pack(struct transport *transport, &transport->pack_lockfile); close(data->fd[0]); close(data->fd[1]); - if (finish_connect(data->conn)) + if (finish_connect(data->conn)) { + free_refs(refs); refs = NULL; + } data->conn = NULL; data->got_remote_heads = 0; data->options.self_contained_and_connected = args.self_contained_and_connected; free_refs(refs_tmp); - + free_refs(refs); free(dest); return (refs ? 0 : -1); } From 32d0462f8da9cc4e26ca1e785098e2ae1bee4d02 Mon Sep 17 00:00:00 2001 From: Jeff King Date: Thu, 19 Mar 2015 16:39:36 -0400 Subject: [PATCH 4/4] fetch-pack: remove dead assignment to ref->new_sha1 In everything_local(), we used to assign the current ref's value found in ref->old_sha1 to ref->new_sha1 when we already have all the necessary objects to complete the history leading to that commit. This copying was broken at 49bb805e (Do not ask for objects known to be complete., 2005-10-19) and ever since we instead stuffed a random bytes in ref->new_sha1 here. No code complained or failed due to this breakage. It turns out that no code path that comes after this assignment even looks at ref->new_sha1 at all. - The only caller of everything_local(), do_fetch_pack(), returns this list of refs, whose element has bogus new_sha1 values, to its caller. It does not look at the elements itself, but does pass them to find_common, which looks only at the name and old_sha1 fields. - The only caller of do_fetch_pack(), fetch_pack(), returns this list to its caller. It does not look at the elements nor act on them. - One of the two callers of fetch_pack() is cmd_fetch_pack(), the top-level that implements "git fetch-pack". The only thing it looks at in the elements of the returned ref list is the old_sha1 and name fields. - The other caller of fetch_pack() is fetch_refs_via_pack() in the transport layer, which is a helper that implements "git fetch". It only cares about whether the returned list is empty (i.e. failed to fetch anything). Just drop the bogus assignment, that is not even necessary. The remote-tracking refs are updated based on a different list and not using the ref list being manipulated by this code path; the caller do_fetch_pack() created a copy of that real ref list and passed the copy down to this function, and modifying the elements here does not affect anything. Noticed-by: Kyle J. McKay Signed-off-by: Jeff King Signed-off-by: Junio C Hamano --- fetch-pack.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/fetch-pack.c b/fetch-pack.c index 84d5a66967..48526aa54b 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -628,7 +628,6 @@ static int everything_local(struct fetch_pack_args *args, for (retval = 1, ref = *refs; ref ; ref = ref->next) { const unsigned char *remote = ref->old_sha1; - unsigned char local[20]; struct object *o; o = lookup_object(remote); @@ -641,8 +640,6 @@ static int everything_local(struct fetch_pack_args *args, ref->name); continue; } - - hashcpy(ref->new_sha1, local); if (!args->verbose) continue; fprintf(stderr,