Граф коммитов

233 Коммитов

Автор SHA1 Сообщение Дата
Michael Lukashov 1b22b6c897 refactor duplicated encode_header in pack-objects and fast-import
The following function is duplicated:

  encode_header

Move this function to sha1_file.c and rename it 'encode_in_pack_object_header',
as suggested by Junio C Hamano

Signed-off-by: Michael Lukashov <michael.lukashov@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-02-17 15:30:20 -08:00
Nicolas Pitre b500d5e11e fast-import: use the diff_delta() max_delta_size argument
This let diff_delta() abort early if it is going to bust the given
size limit.  Also, only objects larger than 20 bytes are considered
as objects smaller than that are most certainly going to produce
larger deltas than the original object due to the additional headers.

Signed-off-by: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-02-17 11:08:44 -08:00
Nicolas Pitre 8c2ca8dd8a fast-import: honor pack.indexversion and pack.packsizelimit config vars
Signed-off-by: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-02-17 11:08:44 -08:00
Nicolas Pitre 89e0a3a131 fast-import: make default pack size unlimited
Now that fast-import is creating packs with index version 2, there is
no point limiting the pack size by default.  A pack split will still
happen if off_t is not sufficiently large to hold large offsets.

While updating the doc, let's remove the "packfiles fit on CDs"
suggestion.  Pack files created by fast-import are still suboptimal and
a 'git repack -a -f -d' or even 'git gc --aggressive' would be a pretty
good idea before considering storage on CDs.

Signed-off-by: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-02-17 11:08:43 -08:00
Nicolas Pitre 427cb22c40 fast-import: use write_idx_file() instead of custom code
This allows for the creation of pack index version 2 with its object
CRC and the possibility for a pack to be larger than 4 GB.

Signed-off-by: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-02-17 11:08:43 -08:00
Nicolas Pitre 212818160d fast-import: use sha1write() for pack data
This is in preparation for using write_idx_file().  Also, by using
sha1write() we get some buffering to reduces the number of write
syscalls, and the written data is SHA1 summed which allows for the extra
data integrity validation check performed in fixup_pack_header_footer()
(details on this in commit abeb40e5aa).

Signed-off-by: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-02-17 11:08:42 -08:00
Nicolas Pitre 3fc366bdbb fast-import: start using struct pack_idx_entry
This is in preparation for using write_idx_file().

Signed-off-by: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-02-17 11:08:42 -08:00
Junio C Hamano 4d0cc22437 fast-import: count --max-pack-size in bytes
Similar in spirit to 07cf0f2 (make --max-pack-size argument to 'git
pack-object' count in bytes, 2010-02-03) which made the option by the same
name to pack-objects, this counts the pack size limit in bytes.

In order not to cause havoc with people used to the previous megabyte
scale an integer smaller than 8192 is interpreted in megabytes but the
user gets a warning.  Also a minimum size of 1 MiB is enforced to avoid an
explosion of pack files.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Shawn O. Pearce <spearce@spearce.org>
Acked-by: Nicolas Pitre <nico@fluxnic.net>
2010-02-04 15:12:17 -08:00
Junio C Hamano 76ea93ccb5 fast-import.c: Fix big-file-threshold parsing bug
Manual merge made at 844ad3d (Merge branch 'sp/maint-fast-import-large-blob'
into sp/fast-import-large-blob, 2010-02-01) did not correctly reflect the change
of unit in which this variable's value is counted from its previous version.

Now it counts in bytes, not in megabytes.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Shawn O. Pearce <spearce@spearce.org>
2010-02-04 09:09:50 -08:00
Junio C Hamano 844ad3d9a0 Merge branch 'sp/maint-fast-import-large-blob' into sp/fast-import-large-blob
* sp/maint-fast-import-large-blob:
  fast-import: Stream very large blobs directly to pack
  bash: don't offer remote transport helpers as subcommands

Conflicts:
	fast-import.c
2010-02-01 12:42:00 -08:00
Shawn O. Pearce 5eef828bc0 fast-import: Stream very large blobs directly to pack
If a blob is larger than the configured big-file-threshold, instead
of reading it into a single buffer obtained from malloc, stream it
onto the end of the current pack file.  Streaming the larger objects
into the pack avoids the 4+ GiB memory footprint that occurs when
fast-import is processing 2+ GiB blobs.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-02-01 12:09:47 -08:00
Junio C Hamano 5fc9df08b5 Merge branch 'jh/notes' (early part)
* 'jh/notes' (early part):
  Add more testcases to test fast-import of notes
  Rename t9301 to t9350, to make room for more fast-import tests
  fast-import: Proper notes tree manipulation
2010-01-20 20:28:49 -08:00
Junio C Hamano e33fd3c326 Merge branch 'maint'
* maint:
  Update draft release notes to 1.6.6.1
  grep: NUL terminate input from a file
  fast-import: tag may point to any object type
2010-01-18 18:16:19 -08:00
Junio C Hamano 6304c4068e Merge branch 'dp/maint-1.6.5-fast-import-non-commit-tag' into maint
* dp/maint-1.6.5-fast-import-non-commit-tag:
  fast-import: tag may point to any object type
2010-01-18 18:15:12 -08:00
Junio C Hamano fa232d457e Merge branch 'sr/gfi-options'
* sr/gfi-options:
  fast-import: add (non-)relative-marks feature
  fast-import: allow for multiple --import-marks= arguments
  fast-import: test the new option command
  fast-import: add option command
  fast-import: add feature command
  fast-import: put marks reading in its own function
  fast-import: put option parsing code in separate functions
2010-01-17 15:58:11 -08:00
Dmitry Potapov 8db751a8f9 fast-import: tag may point to any object type
If you tried to export the official git repository, and then to import it
back then git-fast-import would die complaining that "Mark :1 not a commit".

Accordingly to a generated crash file, Mark 1 is not a commit but a blob,
which is pointed by junio-gpg-pub tag. Because git-tag allows to create such
tags, git-fast-import should import them.

Signed-off-by: Dmitry Potapov <dpotapov@gmail.com>
Acked-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-01-14 21:51:06 -08:00
Shawn O. Pearce 74fbd1182a fast-import: Document author/committer/tagger name is optional
The fast-import parser does not validate that the author, committer
or tagger name component contains both a name and an email address.
Therefore the name component has always been optional.  Correct the
documentation to match the implementation.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-12-31 14:46:08 -08:00
Johan Herland 2a113aee9b fast-import: Proper notes tree manipulation
This patch teaches 'git fast-import' to automatically organize note objects
in a fast-import stream into an appropriate fanout structure. The notes API
in notes.h is NOT used to accomplish this, because trying to keep the
fast-import and notes data structures in sync would yield a significantly
larger patch with higher complexity.

Note objects are added with the 'N' command, and accounted for with a
per-branch counter, which is used to trigger fanout restructuring when
needed. Note that when restructuring the branch tree, _any_ entry whose
path consists of 40 hex chars (not including directory separators) will
be recognized as a note object. It is therefore not advisable to
manipulate note entries with M/D/R/C commands.

Since note objects are stored in the same tree structure as other objects,
the unloading and reloading of a fast-import branches handle note objects
transparently.

This patch has been improved by the following contributions:
- Shawn O. Pearce: Several style- and logic-related improvements

Cc: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-12-07 13:52:52 -08:00
Sverre Rabbelier bc3c79aefc fast-import: add (non-)relative-marks feature
After specifying 'feature relative-marks' the paths specified with
'feature import-marks' and 'feature export-marks' are relative to an
internal directory in the current repository.

In git-fast-import this means that the paths are relative to the
'.git/info/fast-import' directory. However, other importers may use a
different location.

Add 'feature non-relative-marks' to disable this behavior, this way
it is possible to, for example, specify the import-marks location as
relative, and the export-marks location as non-relative.

Also add tests to verify this behavior.

Cc: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-12-05 12:43:24 -08:00
Sverre Rabbelier 081751c882 fast-import: allow for multiple --import-marks= arguments
The --import-marks= option may be specified multiple times on the
commandline and should result in all marks being read in. Only one
import-marks feature may be specified in the stream, which is
overriden by any --import-marks= commandline options.

If one wishes to specify import-marks files in addition to the one
specified in the stream, it is easy to repeat the stream option as a
--import-marks= commandline option.

Also verify this behavior with tests.

Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-12-04 16:10:59 -08:00
Sverre Rabbelier 9c8398f0c9 fast-import: add option command
This allows the frontend to specify any of the supported options as
long as no non-option command has been given. This way the
user does not have to include any frontend-specific options, but
instead she can rely on the frontend to tell fast-import what it
needs.

Also factor out parsing of argv and have it execute when we reach the
first non-option command, or after all commands have been read and
no non-option command has been encountered.

Non-git options are ignored, unrecognised options result in an error.

Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-12-04 16:10:22 -08:00
Sverre Rabbelier f963bd5d71 fast-import: add feature command
This allows the fronted to require a specific feature to be supported
by the backend, or abort.

Also add support for four initial feature, date-format=, force=,
import-marks=, export-marks=.

Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-12-04 16:08:55 -08:00
Sverre Rabbelier 07cd9328b6 fast-import: put marks reading in its own function
All options do nothing but set settings, with the exception of the
--input-marks option. Delay the reading of the marks file till after
all options have been parsed.

Also, rename mark_file to export_marks_file as it is now ambiguous.

Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-12-04 16:08:54 -08:00
Sverre Rabbelier 0f6927c229 fast-import: put option parsing code in separate functions
Putting the options in their own functions increases readability of
the option parsing block and makes it easier to reuse the option
parsing code later on.

Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-12-04 16:08:53 -08:00
Junio C Hamano 885d492f69 Merge branch 'jh/notes' (early part)
* 'jh/notes' (early part):
  Add selftests verifying concatenation of multiple notes for the same commit
  Refactor notes code to concatenate multiple notes annotating the same object
  Add selftests verifying that we can parse notes trees with various fanouts
  Teach the notes lookup code to parse notes trees with various fanout schemes
  Teach notes code to free its internal data structures on request
  Add '%N'-format for pretty-printing commit notes
  Add flags to get_commit_notes() to control the format of the note string
  t3302-notes-index-expensive: Speed up create_repo()
  fast-import: Add support for importing commit notes
  Teach "-m <msg>" and "-F <file>" to "git notes edit"
  Add an expensive test for git-notes
  Speed up git notes lookup
  Add a script to edit/inspect notes
  Introduce commit notes

Conflicts:
	.gitignore
	Documentation/pretty-formats.txt
	pretty.c
2009-11-20 23:53:55 -08:00
Jonathan Nieder 71a04a8b52 Show usage string for 'git fast-import -h'
Let "git fast-import -h" (with no other arguments) print usage
before exiting, even when run outside any repository.

Cc: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-11-10 11:06:56 -08:00
Johan Herland a8dd2e7d2b fast-import: Add support for importing commit notes
Introduce a 'notemodify' subcommand of the 'commit' command. This subcommand
is similar to 'filemodify', except that no mode is supplied (all notes have
mode 0644), and the path is set to the hex SHA1 of the given "comittish".

This enables fast import of note objects along with their associated commits,
since the notes can now be named using the mark references of their
corresponding commits.

The patch also includes a test case of the added functionality.

Signed-off-by: Johan Herland <johan@herland.net>
Acked-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-10-19 19:00:24 -07:00
Junio C Hamano 1cd749cc07 fast-import.c::validate_raw_date(): really validate the value
When reading the "raw format" timestamp from the input stream, make sure
that the timezone offset is a reasonable value by imitating 7122f82
(date.c: improve guess between timezone offset and year., 2006-06-08).

We _might_ want to also check if the timestamp itself is reasonable, but
that is left for a separate commit.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-10-07 13:05:03 -07:00
Thomas Rast 0721c314a5 Use die_errno() instead of die() when checking syscalls
Lots of die() calls did not actually report the kind of error, which
can leave the user confused as to the real problem.  Use die_errno()
where we check a system/library call that sets errno on failure, or
one of the following that wrap such calls:

  Function              Passes on error from
  --------              --------------------
  odb_pack_keep         open
  read_ancestry         fopen
  read_in_full          xread
  strbuf_read           xread
  strbuf_read_file      open or strbuf_read_file
  strbuf_readlink       readlink
  write_in_full         xwrite

Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-06-27 11:14:53 -07:00
Thomas Rast d824cbba02 Convert existing die(..., strerror(errno)) to die_errno()
Change calls to die(..., strerror(errno)) to use the new die_errno().

In the process, also make slight style adjustments: at least state
_something_ about the function that failed (instead of just printing
the pathname), and put paths in single quotes.

Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-06-27 11:14:53 -07:00
Junio C Hamano 36587681b4 Merge branch 'ar/unlink-err'
* ar/unlink-err:
  print unlink(2) errno in copy_or_link_directory
  replace direct calls to unlink(2) with unlink_or_warn
  Introduce an unlink(2) wrapper which gives warning if unlink failed
2009-05-18 09:01:06 -07:00
Felipe Contreras 4b25d091ba Fix a bunch of pointer declarations (codestyle)
Essentially; s/type* /type */ as per the coding guidelines.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-05-01 15:17:31 -07:00
Alex Riesen 691f1a28bf replace direct calls to unlink(2) with unlink_or_warn
This helps to notice when something's going wrong, especially on
systems which lock open files.

I used the following criteria when selecting the code for replacement:
- it was already printing a warning for the unlink failures
- it is in a function which already printing something or is
  called from such a function
- it is in a static function, returning void and the function is only
  called from a builtin main function (cmd_)
- it is in a function which handles emergency exit (signal handlers)
- it is in a function which is obvously cleaning up the lockfiles

Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-04-29 18:37:41 -07:00
Michael J Gruber b18cc5a3b2 Fix more typos/spelling in comments
A few more fixes on top of the automatic spell checker generated ones.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-04-22 19:03:39 -07:00
Mike Ralphson 3ea3c215c0 Fix typos / spelling in comments
Signed-off-by: Mike Ralphson <mike@abacus.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-04-22 19:02:12 -07:00
Junio C Hamano 03a39a9184 Merge branch 'jc/shared-literally'
* jc/shared-literally:
  t1301: loosen test for forced modes
  set_shared_perm(): sometimes we know what the final mode bits should look like
  move_temp_to_file(): do not forget to chmod() in "Coda hack" codepath
  Move chmod(foo, 0444) into move_temp_to_file()
  "core.sharedrepository = 0mode" should set, not loosen
2009-04-06 00:42:52 -07:00
Johan Herland fb8b193670 Move chmod(foo, 0444) into move_temp_to_file()
When writing out a loose object or a pack (index), move_temp_to_file() is
called to finalize the resulting file. These files (loose files and packs)
should all have permission mode 0444 (modulo adjust_shared_perm()).
Therefore, instead of doing chmod(foo, 0444) explicitly from each callsite
(or even forgetting to chmod() at all), do the chmod() call from within
move_temp_to_file().

Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-03-27 22:10:58 -07:00
Elijah Newren 98e1a4186a Correct missing SP characters in grammar comment at top of fast-import.c
Signed-off-by: Elijah Newren <newren@gmail.com>
Acked-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-03-25 16:36:16 -07:00
Benjamin Kramer eb3a9dd327 Remove unused function scope local variables
These variables were unused and can be removed safely:

  builtin-clone.c::cmd_clone(): use_local_hardlinks, use_separate_remote
  builtin-fetch-pack.c::find_common(): len
  builtin-remote.c::mv(): symref
  diff.c::show_stats():show_stats(): total
  diffcore-break.c::should_break(): base_size
  fast-import.c::validate_raw_date(): date, sign
  fsck.c::fsck_tree(): o_sha1, sha1
  xdiff-interface.c::parse_num(): read_some

Signed-off-by: Benjamin Kramer <benny.kra@googlemail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-03-07 20:52:17 -08:00
Junio C Hamano bb0cebd7d0 Merge branch 'jc/maint-1.6.0-pack-directory'
* jc/maint-1.6.0-pack-directory:
  Make sure objects/pack exists before creating a new pack
2009-02-25 14:50:05 -08:00
Junio C Hamano 6e180cdcec Make sure objects/pack exists before creating a new pack
In a repository created with git older than f49fb35 (git-init-db: create
"pack" subdirectory under objects, 2005-06-27), objects/pack/ directory is
not created upon initialization.  It was Ok because subdirectories are
created as needed inside directories init-db creates, and back then,
packfiles were recent invention.

After the said commit, new codepaths started relying on the presense of
objects/pack/ directory in the repository.  This was exacerbated with
8b4eb6b (Do not perform cross-directory renames when creating packs,
2008-09-22) that moved the location temporary pack files are created from
objects/ directory to objects/pack/ directory, because moving temporary to
the final location was done carefully with lazy leading directory creation.

Many packfile related operations in such an old repository can fail
mysteriously because of this.

This commit introduces two helper functions to make things work better.

 - odb_mkstemp() is a specialized version of mkstemp() to refactor the
   code and teach it to create leading directories as needed;

 - odb_pack_keep() refactors the code to create a ".keep" file while
   create leading directories as needed.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-02-25 14:39:42 -08:00
Junio C Hamano ba19a808aa Drop double-semicolon in C
The worst offenders are "continue;;" and "break;;" in switch statements.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-02-10 22:26:37 -08:00
Junio C Hamano fd8475d9fb Merge branch 'maint'
* maint:
  Clear the delta base cache during fast-import checkpoint
2009-02-10 21:30:45 -08:00
Junio C Hamano 9b27ea9518 Merge branch 'maint-1.6.0' into maint
* maint-1.6.0:
  Clear the delta base cache during fast-import checkpoint
2009-02-10 15:32:26 -08:00
Shawn O. Pearce 3d20c636af Clear the delta base cache during fast-import checkpoint
Otherwise we may reuse the same memory address for a totally
different "struct packed_git", and a previously cached object from
the prior occupant might be returned when trying to unpack an object
from the new pack.

Found-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-02-10 15:30:59 -08:00
Steffen Prohaska 2fb3f6db96 Add calls to git_extract_argv0_path() in programs that call git_config_*
Programs that use git_config need to find the global configuration.
When runtime prefix computation is enabled, this requires that
git_extract_argv0_path() is called early in the program's main().

This commit adds the necessary calls.

Signed-off-by: Steffen Prohaska <prohaska@zib.de>
Acked-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-01-26 00:26:05 -08:00
Junio C Hamano 4f8b8992ef Merge branch 'maint-1.6.0' into maint
* maint-1.6.0:
  fast-import: Cleanup mode setting.
  Git.pm: call Error::Simple() properly
2009-01-13 23:10:50 -08:00
Felipe Contreras 3d1d81eba2 fast-import: Cleanup mode setting.
"S_IFREG | mode" makes only sense for 0644 and 0755.

Even though doing (S_IFREG | mode) may not hurt when mode is any other
supported value, that is only true because S_IFREG mode bit happens to
be already on for S_IFLNK or S_IFGITLINK.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Acked-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-01-13 22:57:12 -08:00
René Scharfe c55fae43c4 fast-import.c: stricter strtoul check, silence compiler warning
Store the return value of strtoul() in order to avoid compiler
warnings on Ubuntu 8.10.

Also check errno after each call, which is the only way to notice
an overflow without making ULONG_MAX an illegal date.

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-12-21 01:48:26 -08:00
Junio C Hamano efe05b019c Merge branch 'maint' to sync with GIT 1.6.0.6
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-12-19 19:35:55 -08:00