Граф коммитов

39 Коммитов

Автор SHA1 Сообщение Дата
Jeff King 90108a2441 upload-pack: avoid parsing tag destinations
When upload-pack advertises refs, it dereferences any tags
it sees, and shows the resulting sha1 to the client. It does
this by calling deref_tag. That function must load and parse
each tag object to find the sha1 of the tagged object.
However, it also ends up parsing the tagged object itself,
which is not strictly necessary for upload-pack's use.

Each tag produces two object loads (assuming it is not a
recursive tag), when it could get away with only a single
one. Dropping the second load halves the effort we spend.

The downside is that we are no longer verifying the
resulting object by loading it. In particular:

  1. We never cross-check the "type" field given in the tag
     object with the type of the pointed-to object.  If the
     tag says it points to a tag but doesn't, then we will
     keep peeling and realize the error.  If the tag says it
     points to a non-tag but actually points to a tag, we
     will stop peeling and just advertise the pointed-to
     tag.

  2. If we are missing the pointed-to object, we will not
     realize (because we never even look it up in the object
     db).

However, both of these are errors in the object database,
and both will be detected if a client actually requests the
broken objects in question. So we are simply pushing the
verification away from the advertising stage, and down to
the actual fetching stage.

On my test repo with 120K refs, this drops the time to
advertise the refs from ~3.2s to ~2.0s.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-01-06 13:28:57 -08:00
Junio C Hamano 2f47eae2a1 Split GPG interface into its own helper library
This mostly moves existing code from builtin/tag.c (for signing)
and builtin/verify-tag.c (for verifying) to a new gpg-interface.c
file to provide a more generic library interface.

 - sign_buffer() takes a payload strbuf, a signature strbuf, and a signing
   key, runs "gpg" to produce a detached signature for the payload, and
   appends it to the signature strbuf. The contents of a signed tag that
   concatenates the payload and the detached signature can be produced by
   giving the same strbuf as payload and signature strbuf.

 - verify_signed_buffer() takes a payload and a detached signature as
   <ptr, len> pairs, and runs "gpg --verify" to see if the payload matches
   the signature. It can optionally capture the output from GPG to allow
   the callers to pretty-print it in a way more suitable for their
   contexts.

"verify-tag" (aka "tag -v") used to save the whole tag contents as if it
is a detached signature, and fed gpg the payload part of the tag. It
relied on gpg to fail when the given tag is not signed but just is
annotated.  The updated run_gpg_verify() function detects the lack of
detached signature in the input, and errors out without bothering "gpg".

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-11-04 21:40:25 -07:00
Nguyễn Thái Ngọc Duy 855942528e parse_tag_buffer(): do not prefixcmp() out of range
There is a check (size < 64) at the beginning of the function, but
that only covers object+type lines.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-02-16 10:05:14 -08:00
Nguyễn Thái Ngọc Duy cf7b1cad0e Add const to parse_{commit,tag}_buffer()
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-02-07 15:04:42 -08:00
Michael J Gruber 3d5854e765 tag: recognize rfc1991 signatures
We have always been creating rfc1991 signatures for users with "rfc1991"
in their gpg config but failed to recognize them (tag -l -n largenumber)
and verify them (tag -v, verify-tag).

Make good use of the refactored signature detection and let us recognize
and verify those signatures also.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-11-10 09:40:35 -08:00
Michael J Gruber ac58c4c795 verify-tag: factor out signature detection
into tag.h/c for later reuse and modification.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-11-10 09:39:56 -08:00
Junio C Hamano af24059fa2 Merge branch 'xx/trivial' into maint
* xx/trivial:
  tag.c: whitespace breakages fix
  Fix whitespace issue in object.c
  t5505: add missing &&
2010-09-06 00:11:59 -07:00
Junio C Hamano d2c030d477 tag.c: whitespace breakages fix
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-09-05 22:32:05 -07:00
Shawn O. Pearce e451d06bf3 tag.c: Parse tagger date (if present)
Just like with committer dates, we parse the tagger date into the
struct tag so its available for further downstream processing.
However since the tagger header was not introduced until Git 0.99.1
we must consider it optional.  For tags missing this header we use
the default date of 0.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-12 21:45:17 -07:00
Shawn O. Pearce 28de5b6b40 tag.c: Refactor parse_tag_buffer to be saner to program
This code was horribly ugly to follow.  The structure of the headers
in an annotated tag object must follow a prescribed order, and most
of these are required.  Simplify the entire parsing logic by going
through the headers in the order they are supposed to appear in,
acting on each header as its identified in the buffer.

This change has the same behavior as the older version, its just
easier to read and maintain.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-12 21:45:13 -07:00
Shawn O. Pearce 2e0052a5eb tag.c: Correct indentation
These lines were incorrectly indented with spaces, violating our
coding style.  Its annoying to read with 4 position tab stops, so
fix the indentation to be correct.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-12 21:45:03 -07:00
Junio C Hamano eadbcd498a Merge branch 'mk/maint-parse-careful'
* mk/maint-parse-careful:
  receive-pack: use strict mode for unpacking objects
  index-pack: introduce checking mode
  unpack-objects: prevent writing of inconsistent objects
  unpack-object: cache for non written objects
  add common fsck error printing function
  builtin-fsck: move common object checking code to fsck.c
  builtin-fsck: reports missing parent commits
  Remove unused object-ref code
  builtin-fsck: move away from object-refs to fsck_walk
  add generic, type aware object chain walker

Conflicts:

	Makefile
	builtin-fsck.c
2008-03-02 15:11:07 -08:00
Martin Koegler 7914053ba9 Remove unused object-ref code
Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-25 23:57:35 -08:00
Martin Koegler 24e8a3c946 deref_tag: handle tag->tagged = NULL
Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-02-17 23:49:33 -08:00
Martin Koegler a0393ef676 parse_tag_buffer: don't parse invalid tags
The current tag parsing code can access memory outside the tag buffer,
if \n are missing. This patch prevent this behaviour.

Signed-off-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-01-06 18:41:44 -08:00
Pierre Habouzit 182af8343c Use xmemdupz() in many places.
Signed-off-by: Pierre Habouzit <madcoder@debian.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-09-18 17:42:17 -07:00
Junio C Hamano cdd5b82ee8 Merge branch 'maint-1.5.1' into maint
* maint-1.5.1:
  git-svn: avoid md5 calculation entirely if SVN doesn't provide one
  Fix stupid typo in lookup_tag()
2007-05-28 23:54:26 -07:00
Johan Herland eb09626b94 Fix stupid typo in lookup_tag()
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-05-28 16:51:55 -07:00
Linus Torvalds 100c5f3b0b Clean up object creation to use more common code
This replaces the fairly odd "created_object()" function that did _most_
of the object setup with a more complete "create_object()" function that
also has a more natural calling convention.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-04-16 23:36:16 -07:00
Nicolas Pitre 0ab179504a get rid of lookup_object_type()
This function is called only once in the whole source tree.  Let's move
its code inline instead, which is also in the spirit of removing as much
object type char arrays as possible (not that this patch does anything for
that but at least it is now a local matter).

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-02-27 01:34:21 -08:00
Nicolas Pitre 21666f1aae convert object type handling from a string to a number
We currently have two parallel notation for dealing with object types
in the code: a string and a numerical value.  One of them is obviously
redundent, and the most used one requires more stack space and a bunch
of strcmp() all over the place.

This is an initial step for the removal of the version using a char array
found in object reading code paths.  The patch is unfortunately large but
there is no sane way to split it in smaller parts without breaking the
system.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-02-27 01:34:21 -08:00
Linus Torvalds 1974632c66 Remove TYPE_* constant macros and use object_type enums consistently.
This updates the type-enumeration constants introduced to reduce
the memory footprint of "struct object" to match the type bits
already used in the packfile format, by removing the former
(i.e. TYPE_* constant macros) and using the latter (i.e. enum
object_type) throughout the code for consistency.

Eventually we can stop passing around the "type strings"
entirely, and this will help - no confusion about two different
integer enumeration.

Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-07-12 23:18:03 -07:00
Junio C Hamano 9d24ed4f01 Merge branch 'ff/c99' into next
* ff/c99:
  Remove all void-pointer arithmetic.
2006-06-21 03:51:59 -07:00
Florian Forster 1d7f171c3a Remove all void-pointer arithmetic.
ANSI C99 doesn't allow void-pointer arithmetic. This patch fixes this in
various ways. Usually the strategy that required the least changes was used.

Signed-off-by: Florian Forster <octo@verplant.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-06-20 01:59:46 -07:00
Linus Torvalds 855419f764 Add specialized object allocator
This creates a simple specialized object allocator for basic
objects.

This avoids wasting space with malloc overhead (metadata and
extra alignment), since the specialized allocator knows the
alignment, and that objects, once allocated, are never freed.

It also allows us to track some basic statistics about object
allocations. For example, for the mozilla import, it shows
object usage as follows:

     blobs:   627629 (14710 kB)
     trees:  1119035 (34969 kB)
   commits:   196423  (8440 kB)
      tags:     1336    (46 kB)

and the simpler allocator shaves off about 2.5% off the memory
footprint off a "git-rev-list --all --objects", and is a bit
faster too.

[ Side note: this concludes the series of "save memory in object storage".
  The thing is, there simply isn't much more to be saved on the objects.

  Doing "git-rev-list --all --objects" on the mozilla archive has a final
  total RSS of 131498 pages for me: that's about 513MB. Of that, the
  object overhead is now just 56MB, the rest is going somewhere else (put
  another way: the fact that this patch shaves off 2.5% of the total
  memory overhead, considering that objects are now not much more than 10%
  of the total shows how big the wasted space really was: this makes
  object allocations much more memory- and time-efficient).

  I haven't looked at where the rest is, but I suspect the bulk of it is
  just the pack-file loading. It may be that we should pack the tree
  objects separately from the blob objects: for git-rev-list --objects, we
  don't actually ever need to even look at the blobs, but since trees and
  blobs are interspersed in the pack-file, we end up not being dense in
  the tree accesses, so we end up looking at more pages than we strictly
  need to.

  So with a 535MB pack-file, it's entirely possible - even likely - that
  most of the remaining RSS is just the mmap of the pack-file itself. We
  don't need to map in _all_ of it, but we do end up mapping a fair
  amount. ]

Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-06-19 18:42:21 -07:00
Linus Torvalds 885a86abe2 Shrink "struct object" a bit
This shrinks "struct object" by a small amount, by getting rid of the
"struct type *" pointer and replacing it with a 3-bit bitfield instead.

In addition, we merge the bitfields and the "flags" field, which
incidentally should also remove a useless 4-byte padding from the object
when in 64-bit mode.

Now, our "struct object" is still too damn large, but it's now less
obviously bloated, and of the remaining fields, only the "util" (which is
not used by most things) is clearly something that should be eventually
discarded.

This shrinks the "git-rev-list --all" memory use by about 2.5% on the
kernel archive (and, perhaps more importantly, on the larger mozilla
archive). That may not sound like much, but I suspect it's more on a
64-bit platform.

There are other remaining inefficiencies (the parent lists, for example,
probably have horrible malloc overhead), but this was pretty obvious.

Most of the patch is just changing the comparison of the "type" pointer
from one of the constant string pointers to the appropriate new TYPE_xxx
small integer constant.

Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-06-17 18:49:18 -07:00
Peter Eriksen 90321c106c Replace xmalloc+memset(0) with xcalloc.
Signed-off-by: Peter Eriksen <s022018@student.dtu.dk>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-04-04 00:11:19 -07:00
Junio C Hamano 8f1d2e6f49 [PATCH] Compilation: zero-length array declaration.
ISO C99 (and GCC 3.x or later) lets you write a flexible array
at the end of a structure, like this:

	struct frotz {
		int xyzzy;
		char nitfol[]; /* more */
	};

GCC 2.95 and 2.96 let you to do this with "char nitfol[0]";
unfortunately this is not allowed by ISO C90.

This declares such construct like this:

	struct frotz {
		int xyzzy;
		char nitfol[FLEX_ARRAY]; /* more */
	};

and git-compat-util.h defines FLEX_ARRAY to 0 for gcc 2.95 and
empty for others.

If you are using a C90 C compiler, you should be able
to override this with CFLAGS=-DFLEX_ARRAY=1 from the
command line of "make".

Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-01-07 10:51:06 -08:00
Junio C Hamano 27dedf0c3b GIT 0.99.9j aka 1.0rc3
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-11-16 21:32:44 -08:00
Junio C Hamano 9534f40bc4 Be careful when dereferencing tags.
One caller of deref_tag() was not careful enough to make sure
what deref_tag() returned was not NULL (i.e. we found a tag
object that points at an object we do not have).  Fix it, and
warn about refs that point at such an incomplete tag where
needed.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-11-02 16:50:58 -08:00
Junio C Hamano 37fde874c2 Fix send-pack for non-commitish tags.
Again I left the v2.6.11-tree tag behind.  My bad.

This commit makes sure that we do not barf when pushing a ref
that is a non-commitish tag.  You can update a remote ref under
the following conditions:

 * You can always use --force.
 * Creating a brand new ref is OK.
 * If the remote ref is exactly the same as what you are
   pushing, it is OK (nothing is pushed).
 * You can replace a commitish with another commitish which is a
   descendant of it, if you can verify the ancestry between them;
   this and the above means you have to have what you are replacing.
 * Otherwise you cannot update; you need to use --force.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-08-05 00:47:56 -07:00
Daniel Barkalow 89e4202f98 [PATCH] Parse tags for absent objects
Handle parsing a tag for a non-present object. This adds a function to lookup
an object with lookup_* for * in a string, so that it can get the right storage
based on the "type" line in the tag.

Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-06-21 18:29:12 -07:00
Jason McMullan 5d6ccf5ce7 [PATCH] Anal retentive 'const unsigned char *sha1'
Make 'sha1' parameters const where possible

Signed-off-by: Jason McMullan <jason.mcmullan@timesys.com>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-06-08 13:04:53 -07:00
Nicolas Pitre d1af002dc6 [PATCH] delta check
This adds knowledge of delta objects to fsck-cache and various object
parsing code.  A new switch to git-fsck-cache is provided to display the
maximum delta depth found in a repository.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-05-20 15:41:45 -07:00
Nicolas Pitre bd2c39f58f [PATCH] don't load and decompress objects twice with parse_object()
It turns out that parse_object() is loading and decompressing given
object to free it just before calling the specific object parsing
function which does mmap and decompress the same object again. This
patch introduces the ability to parse specific objects directly from a
memory buffer.

Without this patch, running git-fsck-cache on the kernel repositorytake:

	real    0m13.006s
	user    0m11.421s
	sys     0m1.218s

With this patch applied:

	real    0m8.060s
	user    0m7.071s
	sys     0m0.710s

The performance increase is significant, and this is kind of a
prerequisite for sane delta object support with fsck.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-05-06 11:02:01 -07:00
Linus Torvalds 770896e548 Teach fsck-cache to accept non-commits for reachability analysis.
In particular, teach it about tags. Also, to make reachability actually
work for tags, we need to add the ref to the tagged object.
2005-05-04 17:03:09 -07:00
Sergey Vlasov 13019d4136 [PATCH] Fix memory leaks in git-fsck-cache
This patch fixes memory leaks in parse_object() and related functions;
these leaks were very noticeable when running git-fsck-cache.

Signed-off-by: Sergey Vlasov <vsu@altlinux.ru>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-05-04 10:58:15 -07:00
Edgar Toernig ae200ee594 [PATCH] compat: replace AF_LOCAL with AF_UNIX
There's no AF_LOCAL in POSIX
2005-04-30 09:51:03 -07:00
Daniel Barkalow 2636f61437 [PATCH] Add tag header/parser to library
This adds preliminary support for tags in the library. It doesn't even
store the signature, however, let alone provide any way of checking it.

Signed-Off-By: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-04-28 07:46:33 -07:00