A fork of Git containing Microsoft-specific patches.
Перейти к файлу
Ævar Arnfjörð Bjarmason f1c903debd cocci: make "coccicheck" rule incremental
Optimize the very slow "coccicheck" target to take advantage of
incremental rebuilding, and fix outstanding dependency problems with
the existing rule.

The rule is now faster both on the initial run as we can make better
use of GNU make's parallelism than the old ad-hoc combination of
make's parallelism combined with $(SPATCH_BATCH_SIZE) and/or the
"--jobs" argument to "spatch(1)".

It also makes us *much* faster when incrementally building, it's now
viable to "make coccicheck" as topic branches are merged down.

The rule didn't use FORCE (or its equivalents) before, so a:

	make coccicheck
	make coccicheck

Would report nothing to do on the second iteration. But all of our
patch output depended on all $(COCCI_SOURCES) files, therefore e.g.:

    make -W grep.c coccicheck

Would do a full re-run, i.e. a a change in a single file would force
us to do a full re-run.

The reason for this (not the initial rationale, but my analysis) is:

* Since we create a single "*.cocci.patch+" we don't know where to
  pick up where we left off, or how to incrementally merge e.g. a
  "grep.c" change with an existing *.cocci.patch.

* We've been carrying forward the dependency on the *.c files since
  63f0a758a0 (add coccicheck make target, 2016-09-15) the rule was
  initially added as a sort of poor man's dependency discovery.

  As we don't include other *.c files depending on other *.c files
  has always been broken, as could be trivially demonstrated
  e.g. with:

       make coccicheck
       make -W strbuf.h coccicheck

  However, depending on the corresponding *.c files has been doing
  something, namely that *if* an API change modified both *.c and *.h
  files we'd catch the change to the *.h we care about via the *.c
  being changed.

  For API changes that happened only via *.h files we'd do the wrong
  thing before this change, but e.g. for function additions (not
  "static inline" ones) catch the *.h change by proxy.

Now we'll instead:

 * Create a <RULE>/<FILE> pair in the .build directory, E.g. for
   swap.cocci and grep.c we'll create
   .build/contrib/coccinelle/swap.cocci.patch/grep.c.

   That file is the diff we'll apply for that <RULE>-<FILE>
   combination, if there's no changes to me made (the common case)
   it'll be an empty file.

 * Our generated *.patch
   file (e.g. contrib/coccinelle/swap.cocci.patch) is now a simple "cat
   $^" of all of all of the <RULE>/<FILE> files for a given <RULE>.

   In the case discussed above of "grep.c" being changed we'll do the
   full "cat" every time, so they resulting *.cocci.patch will always
   be correct and up-to-date, even if it's "incrementally updated".

   See 1cc0425a27 (Makefile: have "make pot" not "reset --hard",
   2022-05-26) for another recent rule that used that technique.

As before we'll:

 * End up generating a contrib/coccinelle/swap.cocci.patch, if we
   "fail" by creating a non-empty patch we'll still exit with a zero
   exit code.

   Arguably we should move to a more Makefile-native way of doing
   this, i.e. fail early, and if we want all of the "failed" changes
   we can use "make -k", but as the current
   "ci/run-static-analysis.sh" expects us to behave this way let's
   keep the existing behavior of exhaustively discovering all cocci
   changes, and only failing if spatch itself errors out.

Further implementation details & notes:

 * Before this change running "make coccicheck" would by default end
   up pegging just one CPU at the very end for a while, usually as
   we'd finish whichever *.cocci rule was the most expensive.

   This could be mitigated by combining "make -jN" with
   SPATCH_BATCH_SIZE, see 960154b9c1 (coccicheck: optionally batch
   spatch invocations, 2019-05-06).

   There will be cases where getting rid of "SPATCH_BATCH_SIZE" makes
   things worse, but a from-scratch "make coccicheck" with the default
   of SPATCH_BATCH_SIZE=1 (and tweaking it doesn't make a difference)
   is faster (~3m36s v.s. ~3m56s) with this approach, as we can feed
   the CPU more work in a less staggered way.

 * Getting rid of "SPATCH_BATCH_SIZE" particularly helps in cases
   where the default of 1 yields parallelism under "make coccicheck",
   but then running e.g.:

       make -W contrib/coccinelle/swap.cocci coccicheck

   I.e. before that would use only one CPU core, until the user
   remembered to adjust "SPATCH_BATCH_SIZE" differently than the
   setting that makes sense when doing a non-incremental run of "make
   coccicheck".

 * Before the "make coccicheck" rule would have to clean
   "contrib/coccinelle/*.cocci.patch*", since we'd create "*+" and
   "*.log" files there. Now those are created in
   .build/contrib/coccinelle/, which is covered by the "cocciclean" rule
   already.

Outstanding issues & future work:

 * We could get rid of "--all-includes" in favor of manually
   specifying a list of includes to give to "spatch(1)".

   As noted upthread of [1] a naïve removal of "--all-includes" will
   result in broken *.cocci patches, but if we know the exhaustive
   list of includes via COMPUTE_HEADER_DEPENDENCIES we don't need to
   re-scan for them, we could grab the headers to include from the
   .depend.d/<file>.o.d and supply them with the "--include" option to
   spatch(1).q

1. https://lore.kernel.org/git/87ft18tcog.fsf@evledraar.gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-11-02 21:22:16 -04:00
.github ci: update 'static-analysis' to Ubuntu 22.04 2022-08-24 13:02:12 -07:00
Documentation Git 2.37.3 2022-08-30 10:27:16 -07:00
block-sha1
builtin Merge branch 'jk/unused-fixes' 2022-08-29 14:55:12 -07:00
ci CI: use "GIT_TEST_SANITIZE_LEAK_LOG=true" in linux-leaks 2022-07-27 16:35:40 -07:00
compat Merge branch 'js/safe-directory-plus' into maint 2022-08-26 11:13:12 -07:00
contrib cocci: make "coccicheck" rule incremental 2022-11-02 21:22:16 -04:00
ewah
git-gui
gitk-git
gitweb Merge branch 'jr/gitweb-title-shortening' 2022-08-05 15:52:14 -07:00
mergetools mergetools: vimdiff: simplify tabfirst 2022-08-10 12:39:39 -07:00
negotiator
perl
po
ppc
refs
reftable reftable: drop unused parameter from reader_seek_linear() 2022-08-20 14:14:55 -07:00
sha1collisiondetection@855827c583
sha1dc
sha256
t Merge branch 'es/fix-chained-tests' 2022-08-29 14:55:15 -07:00
templates
trace2 tr2: shows scope unconditionally in addition to key-value pair 2022-08-11 21:05:00 -07:00
xdiff xdiff: drop unused mmfile parameters from xdl_do_patience_diff() 2022-08-20 14:14:55 -07:00
.cirrus.yml
.clang-format
.editorconfig
.gitattributes
.gitignore Makefile: have "coccicheck" re-run if flags change 2022-11-02 21:22:16 -04:00
.gitmodules
.mailmap
.tsan-suppressions
CODE_OF_CONDUCT.md
COPYING
GIT-VERSION-GEN Git 2.37.3 2022-08-30 10:22:10 -07:00
INSTALL
LGPL-2.1
Makefile cocci: make "coccicheck" rule incremental 2022-11-02 21:22:16 -04:00
README.md
RelNotes A handful more topics from the 'master' front for 2.37.3 2022-08-26 11:13:13 -07:00
SECURITY.md
abspath.c
aclocal.m4
add-interactive.c
add-interactive.h
add-patch.c
advice.c
advice.h
alias.c
alias.h
alloc.c
alloc.h
apply.c
apply.h
archive-tar.c
archive-zip.c
archive.c
archive.h
attr.c match_pathname(): drop unused "flags" parameter 2022-08-19 12:20:56 -07:00
attr.h
banned.h
base85.c
bisect.c revisions API: don't leak memory on argv elements that need free()-ing 2022-08-03 11:12:36 -07:00
bisect.h
blame.c Merge branch 'rs/mergesort' 2022-08-03 13:36:09 -07:00
blame.h
blob.c
blob.h
bloom.c
bloom.h
branch.c branch: consider refs under 'update-refs' 2022-07-19 12:49:03 -07:00
branch.h
builtin.h builtin/diagnose.c: create 'git diagnose' builtin 2022-08-12 13:20:02 -07:00
bulk-checkin.c t5351: avoid relying on `core.fsyncMethod = batch` to be supported 2022-07-29 09:08:57 -07:00
bulk-checkin.h
bundle.c
bundle.h
cache-tree.c verify_one_sparse(): drop unused parameters 2022-08-20 14:14:17 -07:00
cache-tree.h
cache.h Merge branch 'ds/decorate-filter-tweak' 2022-08-29 14:55:11 -07:00
cbtree.c
cbtree.h
chdir-notify.c
chdir-notify.h
check-builtins.sh
checkout.c
checkout.h
chunk-format.c
chunk-format.h
color.c
color.h
column.c
column.h
combine-diff.c Merge branch 'rs/combine-diff-with-incompatible-options' into maint 2022-07-27 13:00:32 -07:00
command-list.txt docs: move http-protocol docs to man section 5 2022-08-04 14:12:24 -07:00
commit-graph.c Merge branch 'tb/commit-graph-genv2-upgrade-fix' into maint 2022-08-10 21:52:35 -07:00
commit-graph.h Merge branch 'tb/commit-graph-genv2-upgrade-fix' 2022-08-03 13:36:08 -07:00
commit-reach.c
commit-reach.h
commit-slab-decl.h
commit-slab-impl.h
commit-slab.h
commit.c Merge branch 'rs/mergesort' 2022-08-03 13:36:09 -07:00
commit.h
common-main.c
config.c config.c: NULL check when reading protected config 2022-07-26 23:46:01 -07:00
config.h
config.mak.dev config.mak.dev: squelch -Wno-missing-braces for older gcc 2022-07-31 11:50:07 -07:00
config.mak.in
config.mak.uname mingw: remove unneeded `NO_CURL` directive 2022-07-29 09:48:17 -07:00
configure.ac
connect.c
connect.h
connected.c
connected.h
convert.c
convert.h
copy.c
credential.c
credential.h
csum-file.c
csum-file.h
ctype.c
daemon.c daemon: clarify directory arguments 2022-07-19 12:45:31 -07:00
date.c
date.h
decorate.c
decorate.h
delta-islands.c
delta-islands.h
delta.h
detect-compiler
diagnose.c builtin/diagnose.c: add '--mode' option 2022-08-12 13:20:02 -07:00
diagnose.h builtin/diagnose.c: add '--mode' option 2022-08-12 13:20:02 -07:00
diff-delta.c
diff-lib.c oneway_diff: handle removed sparse directories 2022-08-08 13:21:49 -07:00
diff-merges.c
diff-merges.h
diff-no-index.c
diff.c
diff.h
diffcore-break.c
diffcore-delta.c
diffcore-order.c
diffcore-pickaxe.c
diffcore-rename.c
diffcore-rotate.c
diffcore.h
dir-iterator.c
dir-iterator.h
dir.c match_pathname(): drop unused "flags" parameter 2022-08-19 12:20:56 -07:00
dir.h match_pathname(): drop unused "flags" parameter 2022-08-19 12:20:56 -07:00
editor.c
entry.c
entry.h
environment.c refs: use ref_namespaces for replace refs base 2022-08-05 14:13:12 -07:00
environment.h
exec-cmd.c
exec-cmd.h
fetch-negotiator.c
fetch-negotiator.h
fetch-pack.c fetch-pack: add tracing for negotiation rounds 2022-08-15 09:17:03 -07:00
fetch-pack.h
fmt-merge-msg.c
fmt-merge-msg.h
fsck.c fsck: actually detect bad file modes in trees 2022-08-10 14:26:27 -07:00
fsck.h fsck: downgrade tree badFilemode to "info" 2022-08-10 14:26:29 -07:00
fsmonitor--daemon.h
fsmonitor-ipc.c
fsmonitor-ipc.h
fsmonitor-settings.c
fsmonitor-settings.h
fsmonitor.c
fsmonitor.h
fuzz-commit-graph.c
fuzz-pack-headers.c
fuzz-pack-idx.c
generate-cmdlist.sh
generate-configlist.sh
generate-hooklist.sh
gettext.c
gettext.h
git-add--interactive.perl
git-archimport.perl
git-bisect.sh
git-compat-util.h Merge branch 'vd/scalar-generalize-diagnose' 2022-08-25 14:42:32 -07:00
git-curl-compat.h
git-cvsexportcommit.perl
git-cvsimport.perl
git-cvsserver.perl git-cvsserver: clarify directory list 2022-07-19 12:45:31 -07:00
git-difftool--helper.sh
git-filter-branch.sh
git-instaweb.sh
git-merge-octopus.sh
git-merge-one-file.sh
git-merge-resolve.sh merge-resolve: abort if index does not match HEAD 2022-07-22 21:45:22 -07:00
git-mergetool--lib.sh
git-mergetool.sh
git-p4.py Merge branch 'kk/p4-client-name-encoding-fix' into maint 2022-08-10 21:52:33 -07:00
git-quiltimport.sh
git-request-pull.sh
git-send-email.perl
git-sh-i18n.sh
git-sh-setup.sh
git-submodule.sh
git-svn.perl
git-web--browse.sh
git.c builtin/diagnose.c: create 'git diagnose' builtin 2022-08-12 13:20:02 -07:00
git.rc
gpg-interface.c
gpg-interface.h
graph.c
graph.h
grep.c
grep.h
hash-lookup.c
hash-lookup.h
hash.h
hashmap.c
hashmap.h
help.c git docs: add a category for file formats, protocols and interfaces 2022-08-04 14:12:23 -07:00
help.h git docs: add a category for file formats, protocols and interfaces 2022-08-04 14:12:23 -07:00
hex.c
hook.c Merge branch 'ab/hooks-regression-fix' into maint 2022-08-26 11:13:12 -07:00
hook.h
http-backend.c
http-fetch.c
http-push.c
http-walker.c
http.c Merge branch 'll/curl-accept-language' 2022-07-19 16:40:19 -07:00
http.h
ident.c
imap-send.c
iterator.h
json-writer.c
json-writer.h
khash.h
kwset.c
kwset.h
levenshtein.c
levenshtein.h
line-log.c
line-log.h
line-range.c
line-range.h
linear-assignment.c
linear-assignment.h
list-objects-filter-options.c
list-objects-filter-options.h
list-objects-filter.c
list-objects-filter.h
list-objects.c
list-objects.h
list.h
ll-merge.c
ll-merge.h
lockfile.c
lockfile.h
log-tree.c Merge branch 'jk/unused-fixes' 2022-08-29 14:55:12 -07:00
log-tree.h
ls-refs.c
ls-refs.h
mailinfo.c
mailinfo.h
mailmap.c
mailmap.h
match-trees.c
mem-pool.c
mem-pool.h
merge-blobs.c
merge-blobs.h
merge-ort-wrappers.c merge-ort-wrappers: make printed message match the one from recursive 2022-07-22 21:45:22 -07:00
merge-ort-wrappers.h
merge-ort.c Merge branch 'en/ort-unused-code-removal' 2022-08-29 14:55:14 -07:00
merge-ort.h
merge-recursive.c
merge-recursive.h
merge.c
mergesort.h
midx.c write_midx_bitmap(): drop unused refs_snapshot parameter 2022-07-27 00:02:45 -07:00
midx.h
name-hash.c
notes-cache.c
notes-cache.h
notes-merge.c
notes-merge.h
notes-utils.c
notes-utils.h
notes.c refs: add array of ref namespaces 2022-08-05 14:13:12 -07:00
notes.h
object-file.c
object-name.c
object-store.h
object.c
object.h revision: allow --ancestry-path to take an argument 2022-08-19 10:45:08 -07:00
oid-array.c
oid-array.h
oidmap.c
oidmap.h
oidset.c
oidset.h
oidtree.c
oidtree.h
pack-bitmap-write.c pack-bitmap-write: use const for hashes 2022-07-19 08:38:17 -07:00
pack-bitmap.c Merge branch 'tl/pack-bitmap-error-messages' 2022-07-27 09:16:52 -07:00
pack-bitmap.h pack-bitmap-write: use const for hashes 2022-07-19 08:38:17 -07:00
pack-check.c
pack-mtimes.c
pack-mtimes.h
pack-objects.c
pack-objects.h
pack-revindex.c
pack-revindex.h docs: move pack format docs to man section 5 2022-08-04 14:12:24 -07:00
pack-write.c
pack.h
packfile.c Merge branch 'jk/is-promisor-object-keep-tree-in-use' 2022-08-25 14:42:31 -07:00
packfile.h
pager.c
parallel-checkout.c
parallel-checkout.h
parse-options-cb.c
parse-options.c
parse-options.h
patch-delta.c
patch-ids.c
patch-ids.h
path.c
path.h
pathspec.c pathspec.h: move pathspec_needs_expanded_index() from reset.c to here 2022-08-08 13:23:26 -07:00
pathspec.h pathspec.h: move pathspec_needs_expanded_index() from reset.c to here 2022-08-08 13:23:26 -07:00
pkt-line.c t0021: implementation the rot13-filter.pl script in C 2022-08-14 22:57:12 -07:00
pkt-line.h Merge branch 'mt/rot13-in-c' 2022-08-29 14:55:11 -07:00
preload-index.c preload-index: fix memleak 2022-08-22 15:08:30 -07:00
pretty.c
pretty.h
prio-queue.c
prio-queue.h
progress.c
progress.h
promisor-remote.c Merge branch 'sg/xcalloc-cocci-fix' 2022-08-29 14:55:14 -07:00
promisor-remote.h
prompt.c
prompt.h
protocol-caps.c
protocol-caps.h
protocol.c
protocol.h
prune-packed.c
prune-packed.h
quote.c
quote.h
range-diff.c
range-diff.h
reachable.c
reachable.h
read-cache.c Merge branch 'vd/sparse-reset-checkout-fixes' into maint 2022-08-26 11:13:13 -07:00
rebase-interactive.c sequencer: rewrite update-refs as user edits todo list 2022-07-19 12:49:04 -07:00
rebase-interactive.h
rebase.c
rebase.h
ref-filter.c
ref-filter.h
reflog-walk.c
reflog-walk.h
reflog.c
reflog.h
refs.c refs: use ref_namespaces for replace refs base 2022-08-05 14:13:12 -07:00
refs.h refs: add array of ref namespaces 2022-08-05 14:13:12 -07:00
refspec.c
refspec.h docs: move protocol-related docs to man section 5 2022-08-04 14:12:23 -07:00
remote-curl.c
remote.c Merge branch 'ab/plug-revisions-leak' 2022-08-12 13:19:08 -07:00
remote.h Merge branch 'rs/mergesort' 2022-08-03 13:36:09 -07:00
replace-object.c
replace-object.h
repo-settings.c
repository.c
repository.h
rerere.c
rerere.h
reset.c
reset.h
resolve-undo.c
resolve-undo.h
revision.c revision: allow --ancestry-path to take an argument 2022-08-19 10:45:08 -07:00
revision.h revision: allow --ancestry-path to take an argument 2022-08-19 10:45:08 -07:00
run-command.c pipe_command(): mark stdin descriptor as non-blocking 2022-08-17 09:21:41 -07:00
run-command.h
send-pack.c
send-pack.h
sequencer.c sequencer: do not translate command names 2022-08-19 13:46:37 -07:00
sequencer.h sequencer: rewrite update-refs as user edits todo list 2022-07-19 12:49:04 -07:00
serve.c
serve.h
server-info.c
setup.c Merge branch 'js/safe-directory-plus' into maint 2022-08-26 11:13:12 -07:00
sh-i18n--envsubst.c
sha1dc_git.c
sha1dc_git.h
shallow.c
shallow.h
shared.mak cocci: make "coccicheck" rule incremental 2022-11-02 21:22:16 -04:00
shell.c
shortlog.h
sideband.c
sideband.h
sigchain.c
sigchain.h
simple-ipc.h
sparse-index.c
sparse-index.h
split-index.c
split-index.h
stable-qsort.c
strbuf.c
strbuf.h
streaming.c
streaming.h
string-list.c
string-list.h
strmap.c
strmap.h
strvec.c
strvec.h
sub-process.c
sub-process.h
submodule-config.c
submodule-config.h
submodule.c
submodule.h
symlinks.c
tag.c
tag.h
tar.h
tempfile.c
tempfile.h
thread-utils.c
thread-utils.h
tmp-objdir.c
tmp-objdir.h
trace.c
trace.h
trace2.c
trace2.h
trailer.c
trailer.h
transport-helper.c
transport-internal.h
transport.c transport.c: avoid "whitelist" 2022-07-19 12:45:31 -07:00
transport.h
tree-diff.c
tree-walk.c tree-walk: add a mechanism for getting non-canonicalized modes 2022-08-10 14:26:25 -07:00
tree-walk.h tree-walk: add a mechanism for getting non-canonicalized modes 2022-08-10 14:26:25 -07:00
tree.c
tree.h
unicode-width.h
unimplemented.sh
unix-socket.c
unix-socket.h
unix-stream-server.c
unix-stream-server.h
unpack-trees.c Merge branch 'vd/sparse-reset-checkout-fixes' into maint 2022-08-26 11:13:13 -07:00
unpack-trees.h
upload-pack.c upload-pack: fix a memory leak in create_pack_file() 2022-07-27 16:35:40 -07:00
upload-pack.h
url.c
url.h
urlmatch.c
urlmatch.h
usage.c
userdiff.c
userdiff.h
utf8.c
utf8.h
varint.c
varint.h
version.c
version.h
versioncmp.c
walker.c
walker.h
wildmatch.c
wildmatch.h
worktree.c
worktree.h
wrap-for-bin.sh
wrapper.c Merge branch 'jk/pipe-command-nonblock' 2022-08-25 14:42:32 -07:00
write-or-die.c
ws.c
wt-status.c
wt-status.h
xdiff-interface.c
xdiff-interface.h
zlib.c

README.md

Build status

Git - fast, scalable, distributed revision control system

Git is a fast, scalable, distributed revision control system with an unusually rich command set that provides both high-level operations and full access to internals.

Git is an Open Source project covered by the GNU General Public License version 2 (some parts of it are under different licenses, compatible with the GPLv2). It was originally written by Linus Torvalds with help of a group of hackers around the net.

Please read the file INSTALL for installation instructions.

Many Git online resources are accessible from https://git-scm.com/ including full documentation and Git related tools.

See Documentation/gittutorial.txt to get started, then see Documentation/giteveryday.txt for a useful minimum set of commands, and Documentation/git-<commandname>.txt for documentation of each command. If git has been correctly installed, then the tutorial can also be read with man gittutorial or git help tutorial, and the documentation of each command with man git-<commandname> or git help <commandname>.

CVS users may also want to read Documentation/gitcvs-migration.txt (man gitcvs-migration or git help cvs-migration if git is installed).

The user discussion and development of Git take place on the Git mailing list -- everyone is welcome to post bug reports, feature requests, comments and patches to git@vger.kernel.org (read Documentation/SubmittingPatches for instructions on patch submission and Documentation/CodingGuidelines).

Those wishing to help with error message, usage and informational message string translations (localization l10) should see po/README.md (a po file is a Portable Object file that holds the translations).

To subscribe to the list, send an email with just "subscribe git" in the body to majordomo@vger.kernel.org (not the Git list). The mailing list archives are available at https://lore.kernel.org/git/, http://marc.info/?l=git and other archival sites.

Issues which are security relevant should be disclosed privately to the Git Security mailing list git-security@googlegroups.com.

The maintainer frequently sends the "What's cooking" reports that list the current status of various development topics to the mailing list. The discussion following them give a good reference for project status, development direction and remaining tasks.

The name "git" was given by Linus Torvalds when he wrote the very first version. He described the tool as "the stupid content tracker" and the name as (depending on your mood):

  • random three-letter combination that is pronounceable, and not actually used by any common UNIX command. The fact that it is a mispronunciation of "get" may or may not be relevant.
  • stupid. contemptible and despicable. simple. Take your pick from the dictionary of slang.
  • "global information tracker": you're in a good mood, and it actually works for you. Angels sing, and a light suddenly fills the room.
  • "goddamn idiotic truckload of sh*t": when it breaks