checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
#!/usr/bin/env python
|
|
|
|
|
2015-03-16 14:16:14 +03:00
|
|
|
"""Find Kconfig symbols that are referenced but not defined."""
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
|
2015-02-25 17:15:23 +03:00
|
|
|
# (c) 2014-2015 Valentin Rothberg <Valentin.Rothberg@lip6.fr>
|
2014-11-08 22:56:35 +03:00
|
|
|
# (c) 2014 Stefan Hengelein <stefan.hengelein@fau.de>
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
#
|
2014-11-08 22:56:35 +03:00
|
|
|
# Licensed under the terms of the GNU GPL License version 2
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
|
|
|
|
|
|
|
|
import os
|
|
|
|
import re
|
2015-03-16 14:16:14 +03:00
|
|
|
import sys
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
from subprocess import Popen, PIPE, STDOUT
|
2015-03-16 14:16:14 +03:00
|
|
|
from optparse import OptionParser
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
|
2014-11-08 22:56:35 +03:00
|
|
|
|
|
|
|
# regex expressions
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
OPERATORS = r"&|\(|\)|\||\!"
|
2014-11-08 22:56:35 +03:00
|
|
|
FEATURE = r"(?:\w*[A-Z0-9]\w*){2,}"
|
|
|
|
DEF = r"^\s*(?:menu){,1}config\s+(" + FEATURE + r")\s*"
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
EXPR = r"(?:" + OPERATORS + r"|\s|" + FEATURE + r")+"
|
|
|
|
STMT = r"^\s*(?:if|select|depends\s+on)\s+" + EXPR
|
2014-11-08 22:56:35 +03:00
|
|
|
SOURCE_FEATURE = r"(?:\W|\b)+[D]{,1}CONFIG_(" + FEATURE + r")"
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
|
2014-11-08 22:56:35 +03:00
|
|
|
# regex objects
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
REGEX_FILE_KCONFIG = re.compile(r".*Kconfig[\.\w+\-]*$")
|
|
|
|
REGEX_FEATURE = re.compile(r"(" + FEATURE + r")")
|
2014-11-08 22:56:35 +03:00
|
|
|
REGEX_SOURCE_FEATURE = re.compile(SOURCE_FEATURE)
|
|
|
|
REGEX_KCONFIG_DEF = re.compile(DEF)
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
REGEX_KCONFIG_EXPR = re.compile(EXPR)
|
|
|
|
REGEX_KCONFIG_STMT = re.compile(STMT)
|
|
|
|
REGEX_KCONFIG_HELP = re.compile(r"^\s+(help|---help---)\s*$")
|
|
|
|
REGEX_FILTER_FEATURES = re.compile(r"[A-Za-z0-9]$")
|
|
|
|
|
|
|
|
|
2015-03-16 14:16:14 +03:00
|
|
|
def parse_options():
|
|
|
|
"""The user interface of this module."""
|
|
|
|
usage = "%prog [options]\n\n" \
|
|
|
|
"Run this tool to detect Kconfig symbols that are referenced but " \
|
|
|
|
"not defined in\nKconfig. The output of this tool has the " \
|
|
|
|
"format \'Undefined symbol\\tFile list\'\n\n" \
|
|
|
|
"If no option is specified, %prog will default to check your\n" \
|
|
|
|
"current tree. Please note that specifying commits will " \
|
|
|
|
"\'git reset --hard\'\nyour current tree! You may save " \
|
|
|
|
"uncommitted changes to avoid losing data."
|
|
|
|
|
|
|
|
parser = OptionParser(usage=usage)
|
|
|
|
|
|
|
|
parser.add_option('-c', '--commit', dest='commit', action='store',
|
|
|
|
default="",
|
|
|
|
help="Check if the specified commit (hash) introduces "
|
|
|
|
"undefined Kconfig symbols.")
|
|
|
|
|
|
|
|
parser.add_option('-d', '--diff', dest='diff', action='store',
|
|
|
|
default="",
|
|
|
|
help="Diff undefined symbols between two commits. The "
|
|
|
|
"input format bases on Git log's "
|
|
|
|
"\'commmit1..commit2\'.")
|
|
|
|
|
|
|
|
parser.add_option('', '--force', dest='force', action='store_true',
|
|
|
|
default=False,
|
|
|
|
help="Reset current Git tree even when it's dirty.")
|
|
|
|
|
|
|
|
(opts, _) = parser.parse_args()
|
|
|
|
|
|
|
|
if opts.commit and opts.diff:
|
|
|
|
sys.exit("Please specify only one option at once.")
|
|
|
|
|
|
|
|
if opts.diff and not re.match(r"^[\w\-\.]+\.\.[\w\-\.]+$", opts.diff):
|
|
|
|
sys.exit("Please specify valid input in the following format: "
|
|
|
|
"\'commmit1..commit2\'")
|
|
|
|
|
|
|
|
if opts.commit or opts.diff:
|
|
|
|
if not opts.force and tree_is_dirty():
|
|
|
|
sys.exit("The current Git tree is dirty (see 'git status'). "
|
|
|
|
"Running this script may\ndelete important data since it "
|
|
|
|
"calls 'git reset --hard' for some performance\nreasons. "
|
|
|
|
" Please run this script in a clean Git tree or pass "
|
|
|
|
"'--force' if you\nwant to ignore this warning and "
|
|
|
|
"continue.")
|
|
|
|
|
|
|
|
return opts
|
|
|
|
|
|
|
|
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
def main():
|
|
|
|
"""Main function of this module."""
|
2015-03-16 14:16:14 +03:00
|
|
|
opts = parse_options()
|
|
|
|
|
|
|
|
if opts.commit or opts.diff:
|
|
|
|
head = get_head()
|
|
|
|
|
|
|
|
# get commit range
|
|
|
|
commit_a = None
|
|
|
|
commit_b = None
|
|
|
|
if opts.commit:
|
|
|
|
commit_a = opts.commit + "~"
|
|
|
|
commit_b = opts.commit
|
|
|
|
elif opts.diff:
|
|
|
|
split = opts.diff.split("..")
|
|
|
|
commit_a = split[0]
|
|
|
|
commit_b = split[1]
|
|
|
|
undefined_a = {}
|
|
|
|
undefined_b = {}
|
|
|
|
|
|
|
|
# get undefined items before the commit
|
|
|
|
execute("git reset --hard %s" % commit_a)
|
|
|
|
undefined_a = check_symbols()
|
|
|
|
|
|
|
|
# get undefined items for the commit
|
|
|
|
execute("git reset --hard %s" % commit_b)
|
|
|
|
undefined_b = check_symbols()
|
|
|
|
|
|
|
|
# report cases that are present for the commit but not before
|
2015-03-23 20:40:49 +03:00
|
|
|
for feature in sorted(undefined_b):
|
2015-03-16 14:16:14 +03:00
|
|
|
# feature has not been undefined before
|
|
|
|
if not feature in undefined_a:
|
2015-03-23 20:40:49 +03:00
|
|
|
files = sorted(undefined_b.get(feature))
|
2015-03-16 14:16:14 +03:00
|
|
|
print "%s\t%s" % (feature, ", ".join(files))
|
|
|
|
# check if there are new files that reference the undefined feature
|
|
|
|
else:
|
2015-03-23 20:40:49 +03:00
|
|
|
files = sorted(undefined_b.get(feature) -
|
|
|
|
undefined_a.get(feature))
|
2015-03-16 14:16:14 +03:00
|
|
|
if files:
|
|
|
|
print "%s\t%s" % (feature, ", ".join(files))
|
|
|
|
|
|
|
|
# reset to head
|
|
|
|
execute("git reset --hard %s" % head)
|
|
|
|
|
|
|
|
# default to check the entire tree
|
|
|
|
else:
|
|
|
|
undefined = check_symbols()
|
2015-03-23 20:40:49 +03:00
|
|
|
for feature in sorted(undefined):
|
|
|
|
files = sorted(undefined.get(feature))
|
|
|
|
print "%s\t%s" % (feature, ", ".join(files))
|
2015-03-16 14:16:14 +03:00
|
|
|
|
|
|
|
|
|
|
|
def execute(cmd):
|
|
|
|
"""Execute %cmd and return stdout. Exit in case of error."""
|
|
|
|
pop = Popen(cmd, stdout=PIPE, stderr=STDOUT, shell=True)
|
|
|
|
(stdout, _) = pop.communicate() # wait until finished
|
|
|
|
if pop.returncode != 0:
|
|
|
|
sys.exit(stdout)
|
|
|
|
return stdout
|
|
|
|
|
|
|
|
|
|
|
|
def tree_is_dirty():
|
|
|
|
"""Return true if the current working tree is dirty (i.e., if any file has
|
|
|
|
been added, deleted, modified, renamed or copied but not committed)."""
|
|
|
|
stdout = execute("git status --porcelain")
|
|
|
|
for line in stdout:
|
|
|
|
if re.findall(r"[URMADC]{1}", line[:2]):
|
|
|
|
return True
|
|
|
|
return False
|
|
|
|
|
|
|
|
|
|
|
|
def get_head():
|
|
|
|
"""Return commit hash of current HEAD."""
|
|
|
|
stdout = execute("git rev-parse HEAD")
|
|
|
|
return stdout.strip('\n')
|
|
|
|
|
|
|
|
|
|
|
|
def check_symbols():
|
|
|
|
"""Find undefined Kconfig symbols and return a dict with the symbol as key
|
|
|
|
and a list of referencing files as value."""
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
source_files = []
|
|
|
|
kconfig_files = []
|
|
|
|
defined_features = set()
|
2014-11-08 22:56:35 +03:00
|
|
|
referenced_features = dict() # {feature: [files]}
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
|
|
|
|
# use 'git ls-files' to get the worklist
|
2015-03-16 14:16:14 +03:00
|
|
|
stdout = execute("git ls-files")
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
if len(stdout) > 0 and stdout[-1] == "\n":
|
|
|
|
stdout = stdout[:-1]
|
|
|
|
|
|
|
|
for gitfile in stdout.rsplit("\n"):
|
2015-02-25 17:15:23 +03:00
|
|
|
if ".git" in gitfile or "ChangeLog" in gitfile or \
|
|
|
|
".log" in gitfile or os.path.isdir(gitfile) or \
|
|
|
|
gitfile.startswith("tools/"):
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
continue
|
|
|
|
if REGEX_FILE_KCONFIG.match(gitfile):
|
|
|
|
kconfig_files.append(gitfile)
|
|
|
|
else:
|
2014-11-08 22:56:35 +03:00
|
|
|
# all non-Kconfig files are checked for consistency
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
source_files.append(gitfile)
|
|
|
|
|
|
|
|
for sfile in source_files:
|
|
|
|
parse_source_file(sfile, referenced_features)
|
|
|
|
|
|
|
|
for kfile in kconfig_files:
|
|
|
|
parse_kconfig_file(kfile, defined_features, referenced_features)
|
|
|
|
|
2015-03-16 14:16:14 +03:00
|
|
|
undefined = {} # {feature: [files]}
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
for feature in sorted(referenced_features):
|
2014-11-08 22:56:35 +03:00
|
|
|
# filter some false positives
|
|
|
|
if feature == "FOO" or feature == "BAR" or \
|
|
|
|
feature == "FOO_BAR" or feature == "XXX":
|
|
|
|
continue
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
if feature not in defined_features:
|
|
|
|
if feature.endswith("_MODULE"):
|
2014-11-08 22:56:35 +03:00
|
|
|
# avoid false positives for kernel modules
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
if feature[:-len("_MODULE")] in defined_features:
|
|
|
|
continue
|
2015-03-16 14:16:14 +03:00
|
|
|
undefined[feature] = referenced_features.get(feature)
|
|
|
|
return undefined
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
|
|
|
|
|
|
|
|
def parse_source_file(sfile, referenced_features):
|
|
|
|
"""Parse @sfile for referenced Kconfig features."""
|
|
|
|
lines = []
|
|
|
|
with open(sfile, "r") as stream:
|
|
|
|
lines = stream.readlines()
|
|
|
|
|
|
|
|
for line in lines:
|
|
|
|
if not "CONFIG_" in line:
|
|
|
|
continue
|
|
|
|
features = REGEX_SOURCE_FEATURE.findall(line)
|
|
|
|
for feature in features:
|
|
|
|
if not REGEX_FILTER_FEATURES.search(feature):
|
|
|
|
continue
|
2014-11-08 22:56:35 +03:00
|
|
|
sfiles = referenced_features.get(feature, set())
|
|
|
|
sfiles.add(sfile)
|
|
|
|
referenced_features[feature] = sfiles
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
|
|
|
|
|
|
|
|
def get_features_in_line(line):
|
|
|
|
"""Return mentioned Kconfig features in @line."""
|
|
|
|
return REGEX_FEATURE.findall(line)
|
|
|
|
|
|
|
|
|
|
|
|
def parse_kconfig_file(kfile, defined_features, referenced_features):
|
|
|
|
"""Parse @kfile and update feature definitions and references."""
|
|
|
|
lines = []
|
|
|
|
skip = False
|
|
|
|
|
|
|
|
with open(kfile, "r") as stream:
|
|
|
|
lines = stream.readlines()
|
|
|
|
|
|
|
|
for i in range(len(lines)):
|
|
|
|
line = lines[i]
|
|
|
|
line = line.strip('\n')
|
2014-11-08 22:56:35 +03:00
|
|
|
line = line.split("#")[0] # ignore comments
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
|
|
|
|
if REGEX_KCONFIG_DEF.match(line):
|
|
|
|
feature_def = REGEX_KCONFIG_DEF.findall(line)
|
|
|
|
defined_features.add(feature_def[0])
|
|
|
|
skip = False
|
|
|
|
elif REGEX_KCONFIG_HELP.match(line):
|
|
|
|
skip = True
|
|
|
|
elif skip:
|
2014-11-08 22:56:35 +03:00
|
|
|
# ignore content of help messages
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
pass
|
|
|
|
elif REGEX_KCONFIG_STMT.match(line):
|
|
|
|
features = get_features_in_line(line)
|
2014-11-08 22:56:35 +03:00
|
|
|
# multi-line statements
|
checkkconfigsymbols.sh: reimplementation in python
The scripts/checkkconfigsymbols.sh script searches Kconfig features
in the source code that are not defined in Kconfig. Such identifiers
always evaluate to false and are the source of various kinds of bugs.
However, the shell script is slow and it does not detect such broken
references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´).
Furthermore, it generates false positives. The script is also hard to
read and understand, and is thereby difficult to maintain.
This patch replaces the shell script with an implementation in Python,
which:
(a) detects the same bugs, but does not report previous false positives
(b) additionally detects broken references in Kconfig and all
non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc.
(c) is up to 75 times faster than the shell script
(d) only checks files under version control
The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD)
from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1;
419 additional reports of which 16 are located in Kconfig files,
287 in defconfigs, 63 in ./Documentation, 1 in Kbuild.
Moreover, we intentionally include references in comments, which have been
ignored until now. Such comments may be leftovers of features that have
been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´).
These references can be misleading and should be removed or replaced.
Note that the output format changed from (file list <tab> feature) to
(feature <tab> file list) as it simplifies the detection of the Kconfig
feature for long file lists.
Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com>
Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 18:30:45 +04:00
|
|
|
while line.endswith("\\"):
|
|
|
|
i += 1
|
|
|
|
line = lines[i]
|
|
|
|
line = line.strip('\n')
|
|
|
|
features.extend(get_features_in_line(line))
|
|
|
|
for feature in set(features):
|
|
|
|
paths = referenced_features.get(feature, set())
|
|
|
|
paths.add(kfile)
|
|
|
|
referenced_features[feature] = paths
|
|
|
|
|
|
|
|
|
|
|
|
if __name__ == "__main__":
|
|
|
|
main()
|