Change commit() to stream data from Perforce and into fast-import
rather than reading into memory first, and then writing out. This
hugely reduces the memory requirements when cloning non-incrementally.
Signed-off-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Without this patch, symbolic links are turned into empty files.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Embarrassingly, the common prefix calculation did not work properly, due
to a mistake in the assignment: instead of assigning the dirname of the
current file name, the dirname of the current common prefix needs to
be assigned to common prefix, when the current prefix does not match the
current file name.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The import-tars script is typically employed to (re)create the past
history of a project from stored tars. Although assigning authorship in
these cases can be a somewhat arbitrary process, it makes sense to set
the author to whoever created the tars in the first place (if it's
known), and (s)he can in general be different from the committer
(whoever is running the script).
Implement this by having separate author and committer data, making them
settable from the usual GIT_* environment variables.
Acked-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some users have tabs in their names, oddly enough. This
causes problems when loading the usercache from disk,
as split separates the fields on the wrong tabs. When
fast-import's parse_ident() tries to parse the committer
field, it is unhappy about the unbalanced <..> angle brackets.
It is easy enough to convert the tabs to single spaces.
Signed-off-by: Pete Wyckoff <pw@padd.com>
Acked-by: Simon Hausmann <simon@lst.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When a particular changeset affects multiple depot paths, it
will appear multiple times in the output of "p4 changes".
Filter out the duplicates to avoid the extra empty commits that
this otherwise would create.
Signed-off-by: Pete Wyckoff <pw@padd.com>
Acked-by: Simon Hausmann <simon@lst.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Unfortunately, I introduced a bug in commit 7f705dc36 (git-p4: Fix bug in
p4Where method). This happens because sometimes the result from
"p4 where <somepath>" doesn't contain a "depotFile" key, but instead a
"data" key that needs further parsing. This commit should ensure that both
of these cases are checked.
Signed-off-by: Tor Arvid Lund <torarvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When running:
p4 where //depot/SomePath/...
The result can in some situations look like:
//depot/SomePath/... //client/SomePath/... /home/user/p4root/SomePath/...
-//depot/SomePath/UndesiredSubdir/... //client/SomePath/UndesiredSubdir/... /home/user/p4root/SomePath/UndesiredSubdir/...
This depends on the users Client view. The current p4Where method will now
return /home/user/p4root/SomePath/UndesiredSubdir/... which is not what we
want. This patch loops through the results from "p4 where", and picks the one
where the depotFile exactly matches the given depotPath (//depot/SomePath/...
in this example).
Signed-off-by: Tor Arvid Lund <torarvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
sha1_file.c: resolve confusion EACCES vs EPERM
sha1_file: avoid bogus "file exists" error message
git checkout: don't warn about unborn branch if -f is already passed
bash: offer refs instead of filenames for 'git revert'
bash: remove dashed command leftovers
git-p4: fix keyword-expansion regex
fast-export: use an unsorted string list for extra_refs
Add new testcase to show fast-export does not always exports all tags
This text:
my $dir = $File::Find::dir;
return if ($dir !~ m,$options->{dirpat}$,);
was improperly converted to:
my $dir = $File$dir !~ m,$options->{dirpat}$,);
by the keyword identifier expansion code. Add a \n
to make sure the regex doesn't go across end-of-line
boundaries.
Signed-off-by: Pete Wyckoff <pw@padd.com>
Acked-by: Simon Hausmann <simon@lst.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This makes git-p4 noticibly faster on Windows.
Signed-off-by: John Chapman <thestar@fussycoder.id.au>
Acked-by: Simon Hausmann <simon@lst.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Purged files are handled as if they are merely deleted, which is not
entirely optimal, but I don't know of any other way to handle them.
File data is deleted from memory as early as they can, and they are more
efficiently handled, at (significant) cost to CPU usage.
Still need to handle p4 branches with spaces in their names.
Still need to make git-p4 clone more reliable.
- Perhaps with a --continue option. (Sometimes the p4 server kills
the connection)
Signed-off-by: John Chapman <thestar@fussycoder.id.au>
Acked-by: Simon Hausmann <simon@lst.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When this option is passed to git p4 clone, the checkout at the end would
previously fail. This patch fixes it by optionally creating the master branch
from refs/heads/p4/master, which is the correct one for this option.
Signed-off-by: Tor Arvid Lund <torarvid@gmail.com>
Acked-By: Simon Hausmann <simon@lst.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently it is not possible to skip the submission of a change to Perforce
when running git-p4 submit. This patch compares the modification time before
and after the submit editor invokation and offers a prompt for skipping if
the submit template file was not saved.
Signed-off-by: Simon Hausmann <simon@lst.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch massages the documentation a bit for improved readability and cleans
it up from outdated options/commands.
Signed-off-by: Simon Hausmann <simon@lst.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function built a p4 command string via the p4_build_cmd function, but
ignored the result.
Signed-off-by: Tor Arvid Lund <torarvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* ak/p4:
Utilise our new p4_read_pipe and p4_write_pipe wrappers
Add p4 read_pipe and write_pipe wrappers
Put in the two other configuration elements found in the source
Put some documentation in about the parameters that have been added
Move git-p4.syncFromOrigin into a configuration parameters section
Consistently use 'git-p4' for the configuration entries
If the user has configured various parameters, use them.
Switch to using 'p4_build_cmd'
If we are in verbose mode, output what we are about to run (or return)
Add a single command that will be used to construct the 'p4' command
Utilise the new 'p4_system' function.
Have a command that specifically invokes 'p4' (via system)
Utilise the new 'p4_read_pipe_lines' command
Create a specific version of the read_pipe_lines command for p4 invocations
Conflicts:
contrib/fast-import/git-p4
P4 on Windows expects the PWD environment variable to be set to the
current working dir, but os.chdir in python doesn't do so.
Signed-off-by: Robert Blum <rob.blum@gmail.com>
Acked-by: Simon Hausmann <simon@lst.de>
Acked-by: Han-Wen Nienhuys <hanwen@xs4all.nl>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Two additional wrappers to cover 3 places where we utilise p4 in piped
form. Found by Tor Arvid Lund.
Signed-off-by: Anand Kumria <wildfire@progsoc.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
I am not entirely clear what these parameters do but felt it
useful to call them out in the documentation.
Signed-off-by: Anand Kumria <wildfire@progsoc.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some repositories require authentication and access to certain
hosts. Allow git-p4 to pull this information from the configuration
Signed-off-by: Anand Kumria <wildfire@progsoc.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Rather than having three locations where the 'p4' command is built up,
refactor this into the one place. This will, eventually, allow us to
have one place where we modify the evironment or pass extra
command-line options to the 'p4' binary.
Signed-off-by: Anand Kumria <wildfire@progsoc.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Similiar to our 'p4_read_pipe_lines' command, we can isolate
specific changes to the invocation method in the one location
with this change.
Signed-off-by: Anand Kumria <wildfire@progsoc.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Now that we have the new command, we can utilise it and then
eventually, isolate any changes required to the one place.
Signed-off-by: Anand Kumria <wildfire@progsoc.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This will make it easier to isolate changes to how 'p4' is invoked
(whether with parameters or not, etc.).
Signed-off-by: Anand Kumria <wildfire@progsoc.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
At least, this is true in 2007.2, according to the documentation.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
I'm working with a perforce repo using git-p4. There are some config
files which I need to change locally according to my environment. I'm
using a 'local' git branch to park these changes. And I want to avoid
accidentally checking them into p4 just by doing "git p4 submit"
mindlessly without realizing which branch I'm actually on.
This patch adds a new git config, 'git-p4.allowSubmit', which is a
whitelist of branch names. "git p4 submit" will only allow submissions
from local branches on the list. Useful for preventing inadvertently
submitting from a strictly local branch.
For backward compatibility, if this config is not set at all,
submissions from all branches are allowed.
Signed-off-by: Jing Xue <jingxue@digizenstudio.com>
Acked-By: Simon Hausmann <simon@lst.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-p4s handling of Windows style EOL was broken after the removal
of the p4 submit template handling in commit f2a6059. Fix that, and
make getP4OpenedType() more robust.
Signed-off-by: Marius Storm-Olsen <marius@trolltech.com>
Signed-off-by: Simon Hausmann <simon@lst.de>
Perforce allows you to set the P4EDITOR environment variable to your
preferred editor for use in perforce. Since we are displaying a
perforce changelog to the user we should use it when it is defined.
Signed-off-by: Shawn Bohrer <shawn.bohrer@gmail.com>
Signed-off-by: Simon Hausmann <simon@lst.de>
A custom diffing utility can be specified for the 'p4 diff' command by
setting the P4DIFF environment variable. However when using a custom
diffing utility such as 'vimdiff' passing options like -du can cause
unexpected behavior.
Since the goal is to generate a unified diff of the changes and attach
them to the bottom of the p4 submit log we should unset P4DIFF if it
has been set in order to generate the diff properly.
Signed-off-by: Shawn Bohrer <shawn.bohrer@gmail.com>
Signed-off-by: Simon Hausmann <simon@lst.de>
Use shallow copies in loop, and join content at the end. Then do the substitution, if needed.
Signed-off-by: Marius Storm-Olsen <marius@trolltech.com>
Signed-off-by: Simon Hausmann <simon@lst.de>
Commit 3a70cdfa42 made readP4Files abort quickly
when the changeset only contains files that are marked for deletion with an empty return
value, which caused the commit to not do anything.
This commit changes readP4Files to distinguish between files that need to be passed to p4
print and files that have no content ("deleted") and merge them in the returned
list.
Signed-off-by: Simon Hausmann <simon@lst.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When syncing, git-p4 will only download files that are included in the active
perforce client spec. This does not change the default behaviour - it requires
that the user either supplies the command line argument --use-client-spec, or
sets the git config option p4.useclientspec to "true".
Signed-off-by: Tor Arvid Lund <torarvid@gmail.com>
Signed-off-by: Simon Hausmann <simon@lst.de>
Removed storing the list of commits in a configuration file. We only need the list
of commits at run-time.
Signed-off-by: Simon Hausmann <simon@lst.de>
This feature was originally meant to allow for quicker direct submits into perforce, but
it turns out that it is not actually quicker than doing a git commit and then running
git-p4 submit.
Signed-off-by: Simon Hausmann <simon@lst.de>
Instead of trying to substitute fields in the p4 submit template we now simply
replace the description of the submit with the log message of the git commit.
Signed-off-by: Simon Hausmann <simon@lst.de>
This turns out to be rarely useful and is already covered by git's commit.template configuration variable.
Signed-off-by: Simon Hausmann <simon@lst.de>
Teach git-p4 about the -/ option which adds depot paths to the exclude
list, used when cloning. The option is chosen such that the natural
Perforce syntax works, eg:
git p4 clone //branch/path/... -//branch/path/{large,old}/...
Trailing ... on exclude paths are optional.
This is a generalization of a change by Dmitry Kakurin (thanks).
Signed-off-by: Tommy Thorn <tommy-git@thorn.ws>
Signed-off-by: Simon Hausmann <simon@lst.de>
The regexp "$," can't match anything. Clearly not intended.
This was introduced in ce6f33c8 which is quite a while ago.
Signed-off-by: Tommy Thorn <tommy-git@thorn.ws>
Acked-by: Simon Hausmann <simon@lst.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch removes the '$Keyword: ...$' '...' data, so that files
don't have spurious megre conflicts between branches.
Handles both +ko and +k styles, and leaves the '$Foo$' in
the original file.
Signed-off-by: Simon Hausmann <simon@lst.de>
When using an existing git repository to cache the perforce import we don't
fetch the branch mapping from perforce as that is a slow operation. However
the origin repository may not be fully up-to-date and therefore it may be
necessary to import more changes directly from Perforce.
Such a direct import needs self.knownBranches to be set up though, so
initialize it from the existing p4/* git branches.
Signed-off-by: Simon Hausmann <simon@lst.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This changeset takes advantage of the new parseDiffTreeEntry(...) function to
detect changes to the execute bit in the git repository. During submit, git-p4
now looks for changes to the executable bit and if it finds them it "reopens"
the file in perforce, which allows it to change the file type.
The logic for adding the executable bit in perforce is straightforward: the +x
modifier can be used. Removing the executable bit in perforce requires that the
entire filetype be redefined (there is no way to join remove the bit with a -x
modifier, for example). This changeset includes logic to remove the executable
bit from the full file type while preserving the base file type and other
modifiers.
Signed-off-by: Chris Pettitt <cpettitt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The current git-p4 implementation does support file renames. However, because
it does not use the "p4 integrate" command, the history for the renamed file is
not linked to the new file.
This changeset adds support for perforce renames with the integrate command.
Currently this feature is only enabled when calling git-p4 submit with the -M
option. This is intended to look and behave similar to the "detect renames"
feature of other git commands.
The following sequence is used for renamed files:
p4 integrate -Dt x x'
p4 edit x'
rm x'
git apply
p4 delete x
By default, perforce will not allow an integration with a target file that has
been deleted. That is, if x' in the example above is the name of a previously
deleted file then perforce will fail the integrate. The -Dt option tells
perforce to allow the target of integrate to be a previously deleted file.
Signed-off-by: Chris Pettitt <cpettitt@gmail.com>
Signed-off-by: Simon Hausmann <simon@lst.de>
Given that git uses 'commit', git-p4's 'sumbit' was a bit confusing at times;
often making me do 'git submit' and 'git-p4 commit' instead.
Signed-off-by: Marius Storm-Olsen <marius@trolltech.com>
Acked-By: Simon Hausmann <simon@lst.de>
Signed-off-by: Lars Hjemli <hjemli@gmail.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* maint:
git-svn: don't attempt to spawn pager if we don't want one
Supplant the "while case ... break ;; esac" idiom
User Manual: add a chapter for submodules
user-manual: don't assume refs are stored under .git/refs
Detect exec bit in more cases.
Conjugate "search" correctly in the git-prune-packed man page.
Move the paragraph specifying where the .idx and .pack files should be
Documentation/git-lost-found.txt: drop unnecessarily duplicated name.
This is based on the git-import.sh script, but is a little
more robust and efficient. More importantly, it should
serve as a quick template for interfacing fast-import with
perl scripts.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This example just puts a directory under git control. It is
significantly slower than using the git tools directly, but
hopefully shows a bit how fast-import works.
[jk: added header comments]
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If a change in a p4 "branch" appears that hasn't seen any previous commit and
that has a known branch mapping we now try to import it properly. First we
find the p4 change of the source branch that the new p4 branch is based on. Then
we using git rev-list --bisect to locate the corresponding git commit to that change.
Finally we import all changes in the new p4 branch up to the current change and resume
with the regular import.
Signed-off-by: Simon Hausmann <simon@lst.de>
Don't create the p4/HEAD symbolic ref if p4/master doesn't exist yet.
Signed-off-by: Simon Hausmann <simon@lst.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After a clone with "git clone" of a repository the p4 branches are only in remotes/origin/p4/* and not in remotes/p4/*.
Separate the code for detection and creation out of the P4Sync command class into standalone methods and use them
from the P4Branches command.
Signed-off-by: Simon Hausmann <simon@lst.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When performing a git-p4 clone operation on a Perforce repository,
where the changelists change in order of magnitude (e.g. 100 to 1000),
the set of changes to import from is not sorted properly. This is
because the data in the list is strings not integers. The other place
where this is done already converts the value to an integer, so it is
not affected.
Acked-by: Simon Hausmann <simon@lst.de>
Parse the files section in the "p4 change -o" output and remove lines with file changes in unrelated depot paths.
Signed-off-by: Simon Hausmann <simon@lst.de>
Signed-off-by: Marius Storm-Olsen <marius@trolltech.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Detect symlinks as file type, set the git file mode accordingly and strip off the trailing newline in the p4 print output.
Make the mode handling a bit more readable at the same time.
Signed-off-by: Simon Hausmann <simon@lst.de>
Acked-by: Brian Swetland <swetland@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
processing
P4 change outputs the changes sorted for each directory separately. We
want the global ordering on the changes, hence we sort.
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Simon Hausmann <simon@lst.de>
Fall back to USERPROFILE if HOME isn't set.
Signed-off-by: Simon Hausmann <shausman@trolltech.com>
Signed-off-by: Marius Storm-Olsen <marius@trolltech.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Don't use git name-rev to locate the upstream git-p4 branch for rebase and submit but instead locate the branch by comparing the depot paths.
name-rev may produce results like wrongbranch~12 as it uses the first match.
Signed-off-by: Simon Hausmann <simon@lst.de>
Signed-off-by: Marius Storm-Olsen <marius@trolltech.com>
This approach, suggested by Alex Riesen, bypasses the need for xargs-style
argument list handling. The handling in question looks broken in a corner
case with SC_ARG_MAX=4096 and final argument over 96 characters.
Signed-off-by: Scott Lamb <slamb@slamb.org>
Signed-off-by: Simon Hausmann <simon@lst.de>
This allows bidirectional piping - useful for "-x -" to avoid commandline
arguments - and is a step toward bypassing the shell.
Signed-off-by: Scott Lamb <slamb@slamb.org>
Signed-off-by: Simon Hausmann <simon@lst.de>
Add condition for Windows, since it doesn't support the os.sysconf module.
We hardcode the commandline limit to 2K, as that should work on most
Windows platforms.
Signed-off-by: Marius Storm-Olsen <marius@trolltech.com>
Acked-by: Simon Hausmann <simon@lst.de>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Simon has asked that the git.git project include the git-p4 project
as at least a contrib/fast-import within git.git. I think it makes
a lot of sense, as git-p4 nicely complements the only other in-tree
fast-import user: import-tars.perl.
git-p4 is offered under the MIT license by its authors.
This isn't used right now in git-p4 but I use it in an external script that loads git-p4 as module.
Signed-off-by: Simon Hausmann <shausman@trolltech.com>
Collect "unknown" source branches separately and register them at the end.
Also added a minor speed up to splitFilesIntoBranches by breaking out of the loop through all branches when it's safe.
Signed-off-by: Simon Hausmann <simon@lst.de>
A perforce command with all the files in the repo is generated to get
all the file content.
Here is a patch to break it into multiple successive perforce command
who uses 4K of parameter max, and collect the output for later.
It works, but not for big depos, because the whole perforce depo
content is stored in memory in P4Sync.run(), and it looks like mine is
bigger than 2 Gigs, so I had to kill the process.
[Simon: I added the bit about using SC_ARG_MAX, as suggested by Han-Wen]
Signed-off-by: Benjamin Sergeant <bsergean@gmail.com>
Signed-off-by: Simon Hausmann <simon@lst.de>
Luke Diamand
Johannes Schindelin
Junio C Hamano
Giuseppe Bilotta
Pete Wyckoff
Tor Arvid Lund
John Chapman
Simon Hausmann
Robert Blum
Anand Kumria
Daniel Barkalow
Jing Xue
Marius Storm-Olsen
Shawn Bohrer
Tommy Thorn
Toby Allsopp
Tommy Thorn
Jason McMullan
Shun Kei Leung
Chris Pettitt
David Brown
Jeff King
Nguyen Thai Ngoc Duy
Reece H. Dunn
Han-Wen Nienhuys
Scott Lamb
Shawn O. Pearce
Simon Hausmann
Benjamin Sergeant
Marius Storm-Olsen