A behavior change, but a worthwhile one: "git submodule foreach"
was treating its arguments as part of a single command to be
concatenated and passed to a shell, making writing buggy
scripts too easy.
This patch preserves the old "just pass it to the shell" behavior
when a single argument is passed to 'git submodule foreach' and
moves to a new "skip the shell and use the arguments passed
unmolested" behavior when more than one argument is passed.
The old behavior (always concatenating and passing to the shell)
was similar to the 'ssh' command, while the new behavior (switching
on the number of arguments) is what 'xterm -e' does.
May need more thought to make sure this change is advertised well
so that scripts that used multiple arguments but added their own
extra layer of quoting are not broken.
* ak/submodule-foreach-quoting:
submodule foreach: skip eval for more than one argument
'eval "$@"' creates an extra layer of shell interpretation, which is
probably not expected by a user who passes multiple arguments to git
submodule foreach:
$ git grep "'"
[searches for single quotes]
$ git submodule foreach git grep "'"
Entering '[submodule]'
/usr/lib/git-core/git-submodule: 1: eval: Syntax error: Unterminated quoted string
Stopping at '[submodule]'; script returned non-zero status.
To fix this, if the user passes more than one argument, execute "$@"
directly instead of passing it to eval.
Examples:
* Typical usage when adding an extra level of quoting is to pass a
single argument representing the entire command to be passed to the
shell. This doesn't change that.
* One can imagine someone feeding untrusted input as an argument:
git submodule foreach git grep "$variable"
That currently results in a nonobvious shell code injection
vulnerability. Executing the command named by the arguments
directly, as in this patch, fixes it.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Acked-by: Johan Herland <johan@herland.net>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
In t/t7407-submodule-foreach.sh there is a typo in one of the
path names given for a test step. The correct path is
nested1/nested2/.git, but nested1/nested1/nested2/.git is
given instead. The typo is hidden because this line also
accidentally omits the && chain operator. The omitted chain
also means the return values of all the previous commands in
this test are also being ignored.
Fix the path and add the chain operator so the entire test
sequence can be properly validated.
Signed-off-by: Phil Hord <hordp@cisco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use the new rev-parse --prefix option to process all paths given to the
submodule command, dropping the requirement that it be run from the
top-level of the repository.
Since the interpretation of a relative submodule URL depends on whether
or not "remote.origin.url" is configured, explicitly block relative URLs
in "git submodule add" when not at the top level of the working tree.
Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A test in t7404-submodule-foreach purports to test that
the --cached flag is properly noticed by --recursive calls
to the foreach command as it descends into nested
submodules. However, the test really does not perform this
test since the change it looks for is in a top-level
submodule handled by the first invocation of the command.
To properly test for the flag being passed to recursive
invocations, the change must be buried deeper in the
hierarchy.
Move the change one level deeper so it properly verifies
the recursive machinery of the 'git submodule status'
command.
Signed-off-by: Phil Hord <hordp@cisco.com>
Signed-off-by: Jeff King <peff@peff.net>
Check if <path> is a valid git-dir or a valid git-file that points
to a valid git-dir.
We want tests to be independent from the fact that a git-dir may
be a git-file. Thus we changed tests to use this feature.
Signed-off-by: Fredrik Gustafsson <iveqy@iveqy.com>
Mentored-by: Jens Lehmann <Jens.Lehmann@web.de>
Mentored-by: Heiko Voigt <hvoigt@hvoigt.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* bc/submodule-foreach-stdin-fix-1.7.4:
git-submodule.sh: preserve stdin for the command spawned by foreach
t/t7407: demonstrate that the command called by 'submodule foreach' loses stdin
Conflicts:
git-submodule.sh
The user-supplied command spawned by 'submodule foreach' loses its
connection to the original standard input. Instead, it is connected to the
output of a pipe within the git-submodule script. The user-supplied
command supplied to 'submodule foreach' is spawned within a while loop
which is being piped into. Due to the way shells implement piping output
to a while loop, a subshell is created with its standard input attached to
the output of the pipe. This results in all of the commands executed
within the while loop to have their stdins modified in the same way,
including the user-supplied command.
This can cause a problem if the command requires reading from stdin or if
it changes its behavior based on whether stdin is a tty or not. For
example, this problem was noticed when trying to execute the following:
git submodule foreach git shortlog --since=two.weeks.ago
which printed a message about entering the first submodule and produced no
further output and exited with a status of zero. In this case, shortlog
detected that it was not connected to a tty, and since no revision was
supplied as an argument, it attempted to read the list of revisions from
standard input. Instead, it slurped up the list of submodules that was
being piped to the enclosing while loop and caused that loop to end early
without processing the remaining submodules.
Work around this behavior by saving the original standard input file
descriptor before the while loop, and restoring it when spawning the
user-supplied command.
This fixes the tests in t7407.
Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The user-supplied command spawned by 'submodule foreach' loses its
connection to the original standard input. Instead, it is connected to the
output of a pipe within the git-submodule script. This can cause a problem
if the command requires reading from stdin or if it changes its behavior
based on whether stdin is a tty or not (e.g. git shortlog). Demonstrate
this flaw.
Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Gettextize the "Entering [...]" message. This is explicitly tested for
so we need to skip a portion of a test with test_i18ncmp.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Recursive invocations of submodule update/status preserve all arguments,
so executing
git submodule update --recursive -- foo
attempts to recursively update a submodule named "foo".
Naturally, this fails as one cannot have an infinitely-deep stack of
submodules each containing a submodule named "foo". The desired behavior
is instead to update foo and then recursively update all submodules
inside of foo.
This commit accomplishes that by only saving the flags for use in the
recursive invocation.
Signed-off-by: Kevin Ballard <kevin@sb.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Shell variables only hold strings, not lists of parameters,
so $orig_args after
orig_args="$@"
fails to remember where each parameter starts and ends, if
some include whitespace. So
git submodule update \
--reference='/var/lib/common objects.git' \
--recursive --init
becomes
git submodule update --reference=/var/lib/common \
objects.git --recursive --init
in the inner repositories. Use "git rev-parse --sq-quote" to
save parameters in quoted form ready for evaluation by the
shell, avoiding this problem.
Helped-By: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Kevin Ballard <kevin@sb.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a $toplevel variable accessible to `git submodule foreach`, it
contains the absolute path of the top level directory (where
.gitmodules is).
This makes it possible to e.g. read data in .gitmodules from within
foreach commands. I'm using this to configure the branch names I want
to track for each submodule:
git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
For a little history: This patch is borne out of my continuing fight
of trying to have Git track the branches of submodules, not just their
commits.
Obviously that's not how they work (they only track commits), but I'm
just interested in being able to do:
git submodule foreach 'git pull'
Of course that won't work because the submodule is in a disconnected
head, so I first have to connect it, but connect it *to what*.
For a while I was happy with this because as fate had it, it just so
happened to do what I meant:
git submodule foreach 'git checkout $(git describe --all --always) && git pull'
But then that broke down, if there's a tag and a branch the tag will
win out, and I can't git pull a branch:
$ git branch -a
* master
remotes/origin/HEAD -> origin/master
remotes/origin/master
$ git tag -l
release-0.0.6
$ git describe --always --all
release-0.0.6
So I figured that I might as well start tracking the branches I want
in .gitmodules itself:
[submodule "yaml-mode"]
path = yaml-mode
url = git://github.com/yoshiki/yaml-mode.git
branch = master
So now I can just do (as stated above):
git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'
Maybe there's a less painful way to do *that* (I'd love to hear about
it). But regardless of that I think it's a good idea to be able to
know what the top-level is from git submodule foreach.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Many projects using submodules expect all submodules to be checked out
in order to build/work correctly. A common command sequence for
developers on such projects is:
git clone url/to/project
cd project
git submodule update --init (--recursive)
This patch introduces the --recursive option to git-clone. The new
option causes git-clone to recursively clone and checkout all
submodules of the cloned project. Hence, the above command sequence
can be reduced to:
git clone --recursive url/to/project
--recursive is ignored if no checkout is done by the git-clone.
The patch also includes documentation and a selftest.
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The substring expansion notation is a bashism that we have not so far
adopted. Use 'git rev-parse --short' instead, as this also handles
the case where the unique abbreviation is longer than 7 characters.
Also fix the typo; the object name for submodule #2 was copied from
submodule #1's by mistake.
Suggested-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In very large and hierarchically structured projects, one may encounter
nested submodules. In these situations, it is valuable to not only show
status for all the submodules in the current repo (which is what is
currently done by 'git submodule status'), but also to show status for
all submodules at all levels (i.e. recursing into nested submodules as
well).
This patch teaches the new --recursive option to the 'git submodule status'
command. The patch also includes documentation and selftests.
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In very large and hierarchically structured projects, one may encounter
nested submodules. In these situations, it is valuable to not only update
the submodules in the current repo (which is what is currently done by
'git submodule update'), but also to operate on all submodules at all
levels (i.e. recursing into nested submodules as well).
This patch teaches the new --recursive option to the 'git submodule update'
command. The patch also includes documentation and selftests.
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In very large and hierarchically structured projects, one may encounter
nested submodules. In these situations, it is valuable to not only operate
on all the submodules in the current repo (which is what is currently done
by 'git submodule foreach'), but also to operate on all submodules at all
levels (i.e. recursing into nested submodules as well).
This patch teaches the new --recursive option to the 'git submodule foreach'
command. The patch also includes documentation and selftests.
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add verification of the behaviour of '$name' to the git submodule
foreach selftest.
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The selftest verifies that:
- only checked out submodules are visited by 'git submodule foreach'
- the $path, and $sha1 variables are set correctly for each submodule
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano
Johannes Sixt
Anders Kaseorg
Phil Hord
John Keeping
Fredrik Gustafsson
Brandon Casey
Ævar Arnfjörð Bjarmason
Pat Thoyts
Kevin Ballard
Jens Lehmann
Johan Herland