In short, the motivation of this commit is to build a source package
without cleaning the source tree.
The deb-pkg and (src)rpm-pkg targets first run 'make clean' before
creating a source tarball. Otherwise build artifacts such as *.o,
*.a, etc. would be included in the tarball. Yet, the tarball ends up
containing several garbage files since 'make clean' does not clean
everything.
Cleaning the tree every time is annoying since it makes the incremental
build impossible. It is desirable to create a source tarball without
cleaning the tree.
In fact, there are some ways to achieve this.
The easiest solution is 'git archive'. 'make perf-tar*-src-pkg' uses
it, but I do not like it because it works only when the source tree is
managed by git, and all files you want in the tarball must be committed
in advance.
I want to make it work without relying on git. We can do this.
Files that are ignored by git are generated files, so should be excluded
from the source tarball. We can list them out by parsing the .gitignore
files. Of course, .gitignore does not cover all the cases, but it works
well enough.
tar(1) claims to support it:
--exclude-vcs-ignores
Exclude files that match patterns read from VCS-specific ignore files.
Supported files are: .cvsignore, .gitignore, .bzrignore, and .hgignore.
The best scenario would be to use 'tar --exclude-vcs-ignores', but this
option does not work. --exclude-vcs-ignore does not understand any of
the negation (!), preceding slash, following slash, etc.. So, this option
is just useless.
Hence, I wrote this gitignore parser. The previous version [1], written
in Python, was so slow. This version is implemented in C, so it works
much faster.
I imported the code from git (commit: 23c56f7bd5f1), so we get the same
result.
This tool traverses the source tree, parsing all .gitignore files, and
prints file paths that are ignored by git.
The output is similar to 'git ls-files --ignored --directory --others
--exclude-per-directory=.gitignore', except
[1] Not sorted
[2] No trailing slash for directories
[2] is intentional because tar's --exclude-from option cannot handle
trailing slashes.
[How to test this tool]
$ git clean -dfx
$ make -s -j$(nproc) defconfig all # or allmodconifg or whatever
$ git archive -o ../linux1.tar --prefix=./ HEAD
$ tar tf ../linux1.tar | LANG=C sort > ../file-list1 # files emitted by 'git archive'
$ make scripts_package
HOSTCC scripts/list-gitignored
$ scripts/list-gitignored --prefix=./ -o ../exclude-list
$ tar cf ../linux2.tar --exclude-from=../exclude-list .
$ tar tf ../linux2.tar | LANG=C sort > ../file-list2 # files emitted by 'tar'
$ diff ../file-list1 ../file-list2 | grep -E '^(<|>)'
< ./Documentation/devicetree/bindings/.yamllint
< ./drivers/clk/.kunitconfig
< ./drivers/gpu/drm/tests/.kunitconfig
< ./drivers/hid/.kunitconfig
< ./fs/ext4/.kunitconfig
< ./fs/fat/.kunitconfig
< ./kernel/kcsan/.kunitconfig
< ./lib/kunit/.kunitconfig
< ./mm/kfence/.kunitconfig
< ./tools/testing/selftests/arm64/tags/
< ./tools/testing/selftests/arm64/tags/.gitignore
< ./tools/testing/selftests/arm64/tags/Makefile
< ./tools/testing/selftests/arm64/tags/run_tags_test.sh
< ./tools/testing/selftests/arm64/tags/tags_test.c
< ./tools/testing/selftests/kvm/.gitignore
< ./tools/testing/selftests/kvm/Makefile
< ./tools/testing/selftests/kvm/config
< ./tools/testing/selftests/kvm/settings
The source tarball contains most of files that are tracked by git. You
see some diffs, but it is just because some .gitignore files are wrong.
$ git ls-files -i -c --exclude-per-directory=.gitignore
Documentation/devicetree/bindings/.yamllint
drivers/clk/.kunitconfig
drivers/gpu/drm/tests/.kunitconfig
drivers/hid/.kunitconfig
fs/ext4/.kunitconfig
fs/fat/.kunitconfig
kernel/kcsan/.kunitconfig
lib/kunit/.kunitconfig
mm/kfence/.kunitconfig
tools/testing/selftests/arm64/tags/.gitignore
tools/testing/selftests/arm64/tags/Makefile
tools/testing/selftests/arm64/tags/run_tags_test.sh
tools/testing/selftests/arm64/tags/tags_test.c
tools/testing/selftests/kvm/.gitignore
tools/testing/selftests/kvm/Makefile
tools/testing/selftests/kvm/config
tools/testing/selftests/kvm/settings
[1]: https://lore.kernel.org/all/20230128173843.765212-1-masahiroy@kernel.org/
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Commit 80f8be7af0 ("tomoyo: Omit use of bin2c") removed the last
use of bin2c.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Reviewed-by: Sedat Dilek <sedat.dilek@gmail.com>
scripts/ is a better place to generate files used treewide.
With target.json moved to scripts/, you do not need to add target.json
to no-clean-files or MRPROPER_FILES.
'make clean' does not visit scripts/, but 'make mrproper' does.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Miguel Ojeda <ojeda@kernel.org>
Tested-by: Miguel Ojeda <ojeda@kernel.org>
This script takes care of generating the custom target specification
file for `rustc`, based on the kernel configuration.
It also serves as an example of a Rust host program.
A dummy architecture is kept in this patch so that a later patch
adds x86 support on top with as few changes as possible.
Reviewed-by: Kees Cook <keescook@chromium.org>
Co-developed-by: Alex Gaynor <alex.gaynor@gmail.com>
Signed-off-by: Alex Gaynor <alex.gaynor@gmail.com>
Co-developed-by: Wedson Almeida Filho <wedsonaf@google.com>
Signed-off-by: Wedson Almeida Filho <wedsonaf@google.com>
Co-developed-by: David Gow <davidgow@google.com>
Signed-off-by: David Gow <davidgow@google.com>
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
The pattern prefixed with '/' matches files in the same directory,
but not ones in sub-directories.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Acked-by: Rob Herring <robh@kernel.org>
Acked-by: Andra Paraschiv <andraprs@amazon.com>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Gabriel Krisman Bertazi <krisman@collabora.com>
There was a request to preprocess the module linker script like we
do for the vmlinux one. (https://lkml.org/lkml/2020/8/21/512)
The difference between vmlinux.lds and module.lds is that the latter
is needed for external module builds, thus must be cleaned up by
'make mrproper' instead of 'make clean'. Also, it must be created
by 'make modules_prepare'.
You cannot put it in arch/$(SRCARCH)/kernel/, which is cleaned up by
'make clean'. I moved arch/$(SRCARCH)/kernel/module.lds to
arch/$(SRCARCH)/include/asm/module.lds.h, which is included from
scripts/module.lds.S.
scripts/module.lds is fine because 'make clean' keeps all the
build artifacts under scripts/.
You can add arch-specific sections in <asm/module.lds.h>.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Jessica Yu <jeyu@kernel.org>
Acked-by: Will Deacon <will@kernel.org>
Acked-by: Geert Uytterhoeven <geert@linux-m68k.org>
Acked-by: Palmer Dabbelt <palmerdabbelt@google.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Acked-by: Jessica Yu <jeyu@kernel.org>
Some .gitignore files have comments like "Generated files",
"Ignore generated files" at the header part, but they are
too obvious.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Here are the big set of tty and serial driver updates for 5.6-rc1
Included in here are:
- dummy_con cleanups (touches lots of arch code)
- sysrq logic cleanups (touches lots of serial drivers)
- samsung driver fixes (wasn't really being built)
- conmakeshash move to tty subdir out of scripts
- lots of small tty/serial driver updates
All of these have been in linux-next for a while with no reported
issues.
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
-----BEGIN PGP SIGNATURE-----
iG0EABECAC0WIQT0tgzFv3jCIUoxPcsxR9QN2y37KQUCXjFRBg8cZ3JlZ0Brcm9h
aC5jb20ACgkQMUfUDdst+yn2VACgkge7vTeUNeZFc+6F4NWphAQ5tCQAoK/MMbU6
0O8ef7PjFwCU4s227UTv
=6m40
-----END PGP SIGNATURE-----
Merge tag 'tty-5.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty
Pull tty/serial driver updates from Greg KH:
"Here are the big set of tty and serial driver updates for 5.6-rc1
Included in here are:
- dummy_con cleanups (touches lots of arch code)
- sysrq logic cleanups (touches lots of serial drivers)
- samsung driver fixes (wasn't really being built)
- conmakeshash move to tty subdir out of scripts
- lots of small tty/serial driver updates
All of these have been in linux-next for a while with no reported
issues"
* tag 'tty-5.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty: (140 commits)
tty: n_hdlc: Use flexible-array member and struct_size() helper
tty: baudrate: SPARC supports few more baud rates
tty: baudrate: Synchronise baud_table[] and baud_bits[]
tty: serial: meson_uart: Add support for kernel debugger
serial: imx: fix a race condition in receive path
serial: 8250_bcm2835aux: Document struct bcm2835aux_data
serial: 8250_bcm2835aux: Use generic remapping code
serial: 8250_bcm2835aux: Allocate uart_8250_port on stack
serial: 8250_bcm2835aux: Suppress register_port error on -EPROBE_DEFER
serial: 8250_bcm2835aux: Suppress clk_get error on -EPROBE_DEFER
serial: 8250_bcm2835aux: Fix line mismatch on driver unbind
serial_core: Remove unused member in uart_port
vt: Correct comment documenting do_take_over_console()
vt: Delete comment referencing non-existent unbind_con_driver()
arch/xtensa/setup: Drop dummy_con initialization
arch/x86/setup: Drop dummy_con initialization
arch/unicore32/setup: Drop dummy_con initialization
arch/sparc/setup: Drop dummy_con initialization
arch/sh/setup: Drop dummy_con initialization
arch/s390/setup: Drop dummy_con initialization
...
scripts/conmakehash is only used for generating
drivers/tty/vt/consolemap_deftbl.c
Move it to the related directory.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Link: https://lore.kernel.org/r/20191217110633.8796-1-masahiroy@kernel.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Use a more generic name for additional table sorting usecases,
such as the upcoming ORC table sorting feature. This tool is
not tied to exception table sorting anymore.
No functional changes intended.
[ mingo: Rewrote the changelog. ]
Signed-off-by: Shile Zhang <shile.zhang@linux.alibaba.com>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: linux-kbuild@vger.kernel.org
Link: https://lkml.kernel.org/r/20191204004633.88660-6-shile.zhang@linux.alibaba.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
This tool is only used by drivers/video/logo/Makefile. No reason to
keep it in scripts/.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Commit c512d2544c ("gitignore: ignore scripts/ihex2fw") was unneeded.
ihex2fw was generated in firmware/ instead of scripts/ at that time
although ihex2fw.c was pushed back and forth between those directories
in the past.
check-lc_ctype was removed by commit cb43fb5775 ("docs: remove
DocBook from the building system").
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Commit 8370edea81 ("bin2c: move bin2c in scripts/basic") moved bin2c
to the scripts/basic/ directory, incorrectly stating "Kexec wants to
use bin2c and it wants to use it really early in the build process.
See arch/x86/purgatory/ code in later patches."
Commit bdab125c93 ("Revert "kexec/purgatory: Add clean-up for
purgatory directory"") and commit d6605b6bbe ("x86/build: Remove
unnecessary preparation for purgatory") removed the redundant
purgatory build magic entirely.
That means that the move of bin2c was unnecessary in the first place.
fixdep is the only host program that deserves to sit in the
scripts/basic/ directory.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
There were a few bits and pieces left over from the now-disused DocBook
toolchain; git rid of them.
Reported-by: Markus Heiser <markus.heiser@darmarit.de>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Place a system_extra_cert buffer of configurable size, right after the
system_certificate_list, so that inserted keys can be readily processed by
the existing mechanism. Added script takes a key file and a kernel image
and inserts its contents to the reserved area. The
system_certificate_list_size is also adjusted accordingly.
Call the script as:
scripts/insert-sys-cert -b <vmlinux> -c <certfile>
If vmlinux has no symbol table, supply System.map file with -s flag.
Subsequent runs replace the previously inserted key, instead of appending
the new one.
Signed-off-by: Mehmet Kayaalp <mkayaalp@linux.vnet.ibm.com>
Signed-off-by: David Howells <dhowells@redhat.com>
Acked-by: Mimi Zohar <zohar@linux.vnet.ibm.com>
Currently the encoding of documents generated by DocBook depends on
the current locale. Make the output reproducible independently of
the locale, by setting the encoding to UTF-8 (LC_CTYPE=C.UTF-8) by
preference, or ASCII (LC_CTYPE=C) as a fallback.
LC_CTYPE can normally be overridden by LC_ALL, but the top-level
Makefile unsets that.
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
[jc: added check-lc_ctype to .gitignore]
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
...so "git status" doesn't nag us about them.
Cc: David Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: David Howells <dhowells@redhat.com>
Signed-off-by: James Morris <james.l.morris@oracle.com>
This patch series does not do kernel signature verification yet. I plan
to post another patch series for that. Now distributions are already
signing PE/COFF bzImage with PKCS7 signature I plan to parse and verify
those signatures.
Primary goal of this patchset is to prepare groundwork so that kernel
image can be signed and signatures be verified during kexec load. This
should help with two things.
- It should allow kexec/kdump on secureboot enabled machines.
- In general it can help even without secureboot. By being able to verify
kernel image signature in kexec, it should help with avoiding module
signing restrictions. Matthew Garret showed how to boot into a custom
kernel, modify first kernel's memory and then jump back to old kernel and
bypass any policy one wants to.
This patch (of 15):
Kexec wants to use bin2c and it wants to use it really early in the build
process. See arch/x86/purgatory/ code in later patches.
So move bin2c in scripts/basic so that it can be built very early and
be usable by arch/x86/purgatory/
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
Cc: Borislav Petkov <bp@suse.de>
Cc: Michael Kerrisk <mtk.manpages@gmail.com>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Eric Biederman <ebiederm@xmission.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Matthew Garrett <mjg59@srcf.ucam.org>
Cc: Greg Kroah-Hartman <greg@kroah.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: WANG Chao <chaowang@redhat.com>
Cc: Baoquan He <bhe@redhat.com>
Cc: Andy Lutomirski <luto@amacapital.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Add a simple ASN.1 grammar compiler. This produces a bytecode output that can
be fed to a decoder to inform the decoder how to interpret the ASN.1 stream it
is trying to parse.
Action functions can be specified in the grammar by interpolating:
({ foo })
after a type, for example:
SubjectPublicKeyInfo ::= SEQUENCE {
algorithm AlgorithmIdentifier,
subjectPublicKey BIT STRING ({ do_key_data })
}
The decoder is expected to call these after matching this type and parsing the
contents if it is a constructed type.
The grammar compiler does not currently support the SET type (though it does
support SET OF) as I can't see a good way of tracking which members have been
encountered yet without using up extra stack space.
Currently, the grammar compiler will fail if more than 256 bytes of bytecode
would be produced or more than 256 actions have been specified as it uses
8-bit jump values and action indices to keep space usage down.
Signed-off-by: David Howells <dhowells@redhat.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I've rewritten the extract-ikconfig script to extract the kernel
configuration from a kernel compiled with CONFIG_IKCONFIG. The main
motivation for the rewrite was to remove the dependency on the
external C program binoffset.c, which is compiled on the initial run.
The binoffset executable is invoked with a relative path, which means
that the old script can only be run from the top of the kernel tree,
and only when you have write permission in the scripts directory.
The new script uses tr/grep/tail/zcat only, and can be invoked from
anywhere. The binoffset.c program has been removed. This script
requires GNU grep 2.5 (released 2002-03-13) or higher, because the -o
option was introduced in that version.
Signed-off-by: Dick Streefland <dick@streefland.net>
LKML-Reference: <20091006203540.GA14634@streefland.net>
Tested-by: Steven Rostedt <rostedt@goodmis.org>
Cc: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
scripts/ihex2fw is a generated binary and should be ignored
Signed-off-by: Jaswinder Singh Rajput <jaswinderrajput@gmail.com>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
This reverts commit 8b249b6856.
This 'fix' is not necessary; we just need to undo the damage caused
accidentally by Igor/Mauro in 4b29631db3
("V4L/DVB (9533): cx88: Add support for TurboSight TBS8910 DVB-S PCI card")
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
Rafael reported:
I get the following error from 'make modules_install' on my test boxes:
HOSTCC firmware/ihex2fw
/home/rafael/src/linux-2.6/firmware/ihex2fw.c:268: fatal error: opening dependency file firmware/.ihex2fw.d: Read-only file system
compilation terminated.
make[3]: *** [firmware/ihex2fw] Error 1
make[2]: *** [_modinst_post] Error 2
make[1]: *** [sub-make] Error 2
make: *** [all] Error 2
where the configuration is that the kernel is compiled on a build box
with 'make O=<destdir> -j5' and then <destdir> is mounted over NFS read-only by
each test box (full path to this directory is the same on the build box and on
the test boxes). Then, I cd into <destdir>, run 'make modules_install' and get
the error above.
The issue turns out to be that we when we install firmware pick
up the list of firmware blobs from firmware/Makefile.
And this triggers the Makefile rules to update ihex2fw.
There were two solutions for this issue:
1) Move the list of firmware blobs to a separate file
2) Avoid ihex2fw rebuild by moving it to scripts
As I seriously beleive that the list of firmware blobs should be
done in a fundamental different way solution 2) was selected.
Reported-and-tested-by: "Rafael J. Wysocki" <rjw@sisk.pl>
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Cc: David Woodhouse <dwmw2@infradead.org>
This seems to have been missed when unifdef went in
via Sam's tree..
Signed-off-by: Paul Mundt <lethal@linux-sh.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This still leaves driver and architecture-specific subdirectories alone,
but gets rid of the bulk of the "generic" generated files that we should
ignore.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Masahiro Yamada
Miguel Ojeda
Linus Torvalds
Shile Zhang
Masahiro Yamada
Jonathan Corbet
Mehmet Kayaalp
Ben Hutchings
Paul Gortmaker
Vivek Goyal
David Howells
David Daney
Peter Foley
Américo Wang
Dick Streefland
Jaswinder Singh Rajput
David Woodhouse
Sam Ravnborg
Uwe Kleine-König
Paul Mundt
Linus Torvalds