Clone of https://chromium.googlesource.com/chromium/src/build with Mozilla's local modifications
Перейти к файлу
David 'Digit' Turner f24ca38e53 android: build: Use .pydeps file to store Python dependencies.
NOTE: This is a reland of the following CL which was reverted:

  https://chromium-review.googlesource.com/c/chromium/src/+/1131190

The revert was due to a problem with an internal clank/DEPS runhooks
step that calls jinja_template.py (see BUG entry below for details).
The new CL was checking inputs by defaults, which would raise an
error during gclient sync.

To ease review, the first CL on gerrit is the original one,
unmodified, and the second fixes the issue.

---- end of NOTE ------

ProTip: start by looking at the action_with_pydefps() definition
in internal_rules.gni to review changes in this CL.

This CL tries to make "gn analyze" smarter by ensuring that
it knows about Python module dependencies, for any python
script under //build.

At the moment, these dependencies are computed by
build_utils.WriteDepFile() automatically (unless the
add_pydeps=False argument is passed). This adds all imported
module paths to the generated .d file written by the script.

Unfortunately, because this .d file is generated by the
actions, the corresponding Python sources are not part of
the target's inputs, known by GN at parse time, and thus do
not appear in "gn analyze" properly.

This CL tries to solve the problem by adding a new template
named "action_with_pydeps", which acts as "action", but also
expects a foo.pydeps file for every foo.py invoked through
the 'script' scope variable.

This '.pydeps' file is read directly at GN parse time and
added to the action's target inputs.

These .pydeps contain the path of each imported module, one
per line, and are generated with build/print_python_deps.py

Benchmarking shows no significant difference in the time
taken to perform a "gn gen out/Release" on a full Chrome
for Android checkout.

This also needs a PRESUBMIT.py step that ensures that all .pydeps
files are up-to-date with regards to their corresponding .py source
files (this can be done with 'gen_pydeps.py --check .../foo.py')

+ Remove some --depfile options on a few Python scripts that
  don't need it anymore (e.g. when their inputs and outputs
  are now fully known by GN at parse time).

+ Remove uses of compute_inputs_for_analyze in rules.gni and
  internal_rules.gni (as well as exec_script() calls to
  print_python_deps.py). Note that the variable itself is still
  needed for grit inputs, see tools/grit/grit_rule.gni for
  details).

BUG=870845,843562
R=agrieve@chromium.org, estevenson@chromium.org

Change-Id: Id1f606a0c9df9e4e8971fd885ac0394103ca7b03
Reviewed-on: https://chromium-review.googlesource.com/1163512
Reviewed-by: agrieve <agrieve@chromium.org>
Reviewed-by: Eric Stevenson <estevenson@chromium.org>
Commit-Queue: David Turner <digit@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#581158}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 0006f4737647ad0790431737586fd700085f8b0e
2018-08-07 07:12:36 +00:00
android android: build: Use .pydeps file to store Python dependencies. 2018-08-07 07:12:36 +00:00
args Enable WebRTC on Fuchsia by default. 2017-12-15 02:27:25 +00:00
chromeos Deploy a locally built browser when running telemetry tests in cros VMs. 2018-08-06 20:18:34 +00:00
cipd [android] Move CIPD dependencies into DEPS. (RELAND) 2018-03-24 00:23:27 +00:00
config android: build: Use .pydeps file to store Python dependencies. 2018-08-07 07:12:36 +00:00
docs build: Fix common misspellings 2017-08-01 09:03:13 +00:00
experimental Replace kochi-*.ttf with IPA Fonts 2017-08-28 23:01:11 +00:00
fuchsia [Fuchsia] Save system logs to a file and remove LogReader 2018-08-07 00:50:08 +00:00
git-hooks Silence git-hooks/pre-commit in the absence of .gitmodules (take 2). 2014-04-05 00:29:14 +00:00
internal Remove GYP files in breakpad, build, and tools. 2016-09-12 19:55:25 +00:00
ios Set Python shebang in build to /usr/bin/env python 2017-07-05 14:42:43 +00:00
linux unbundle/ffmpeg: revert rename to ffmpeg_buildflags 2018-07-10 22:28:03 +00:00
mac Support dummy commit for LASTCHANGE 2018-08-01 02:09:56 +00:00
sanitizers Update gin_unittests bot suppressions. 2018-06-06 15:58:47 +00:00
secondary Android: Use android_deps for play services APIs 2018-07-26 20:55:39 +00:00
toolchain Clang: the tip-of-tree version is now 8.0.0svn 2018-08-01 14:47:00 +00:00
util Enable use_dummy_lastchange=true for linux_chromium_rel_ng builder 2018-08-01 13:12:50 +00:00
win Make copy_cdb_to_output depend on vs_toolchain.py 2018-05-25 15:52:19 +00:00
.gitignore Support dummy commit for LASTCHANGE 2018-08-01 02:09:56 +00:00
BUILD.gn Add support for include_dirs to gn check. 2018-01-23 20:30:03 +00:00
OWNERS Add tikuta to OWNERS under build directory 2018-07-30 11:38:07 +00:00
OWNERS.status Add global status comments for owners 2017-04-07 07:03:46 +00:00
PRESUBMIT.py Fix buildbot linux_chromium_dbg_ng reference in PRESUBMIT. 2018-05-04 21:12:43 +00:00
apply_locales.py Move apply_locales.py from src/chrome/tools/build/ to src/build/. 2010-03-09 02:00:21 +00:00
branding_value.sh Adding a general script to fetch values from the BRANDING files. 2009-05-19 14:10:09 +00:00
build-ctags.sh Change build-ctags.sh output file name from ".tags" to "tags" 2014-12-09 20:36:25 +00:00
build_config.h Reland "Fuchsia: Stop setting OS_POSIX" 2018-05-22 23:16:18 +00:00
buildflag.h Tiny fix in buildflag.h comment 2016-03-31 21:02:11 +00:00
buildflag_header.gni Rename buildflag_header targets in base for consistency 2018-02-22 02:08:08 +00:00
check_gn_headers.py Make dirty checking in check_gn_headers.py more robust 2017-06-17 00:12:04 +00:00
check_gn_headers_unittest.py Print detailed info in check_gn_headers.py 2017-06-13 10:21:23 +00:00
check_gn_headers_whitelist.txt Fix missing dependencies with RLZ target. 2018-08-04 02:52:45 +00:00
check_return_value.py check_return_value.py should drop stderr of the command it runs. 2014-04-03 07:36:32 +00:00
ciopfs.sha1 Download ciopfs from a hook in win/cross builds. 2018-02-01 20:36:45 +00:00
clobber.py Fix src/build/clobber.py 2018-04-09 15:32:48 +00:00
common.croc Remove references to sconsbuild. 2013-10-12 02:38:29 +00:00
common.gypi Gut gyp_chromium, common.gypi. 2016-09-21 22:29:06 +00:00
compiled_action.gni compiled_action: Don't add host_executable to inputs. 2018-04-03 01:44:15 +00:00
compute_build_timestamp.py win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time 2018-08-03 17:33:15 +00:00
copy_test_data_ios.py Fix handling of spaces with paths in copy_test_data.py 2012-11-09 18:04:50 +00:00
cp.py Normalize output path to cp.py 2013-04-30 01:57:11 +00:00
detect_host_arch.py Arm64: Fix host architecture detection 2017-11-07 14:26:17 +00:00
dir_exists.py gyp performance: don't invoke python to check dir existance 2014-04-16 21:52:49 +00:00
dotfile_settings.gni win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time 2018-08-03 17:33:15 +00:00
download_nacl_toolchains.py Propagate exit code in download_nacl_toolchains. 2017-11-20 20:18:55 +00:00
download_translation_unit_tool.py Modify download_translation_unit_tool.py to work on Windows. 2018-05-22 22:53:13 +00:00
env_dump.py Set Python shebang in build to /usr/bin/env python 2017-07-05 14:42:43 +00:00
extract_from_cab.py Fix spelling: "supress" -> "suppress" 2013-11-02 02:05:53 +00:00
find_depot_tools.py find_depot_tools: put DEPS'd depot_tools first 2017-10-10 21:36:39 +00:00
find_isolated_tests.py Add a script that can discover *.isolated files in a directory. 2014-02-28 19:46:52 +00:00
fix_gn_headers.py Support checking and fixing non-existing header files in GN 2017-04-29 07:12:17 +00:00
gdb-add-index gdb-add-index now removes its temp directory 2016-06-01 20:21:54 +00:00
get_landmines.py The Great Blink mv for source files, part 2. 2018-04-07 15:32:37 +00:00
gn_helpers.py build: Fix common misspellings 2017-08-01 09:03:13 +00:00
gn_helpers_unittest.py Add FromGNArgs() to gn_helpers. 2016-04-06 00:10:38 +00:00
gn_run_binary.py Print Windows crash codes in hex 2018-02-15 00:08:36 +00:00
gyp_chromium Revert of Remove the top-level GYP files (or stub them out). (patchset #2 id:20001 of https://codereview.chromium.org/2299543008/ ) 2016-09-07 16:43:47 +00:00
gyp_chromium.py Gut gyp_chromium, common.gypi. 2016-09-21 22:29:06 +00:00
gyp_environment.py Remove direct references to hermetic mac toolchain. 2016-10-13 21:38:05 +00:00
gyp_helper.py Expose CC.host_wrapper and CXX.host_wrapper. 2015-04-02 20:42:43 +00:00
gypi_to_gn.py Make gypi_to_gn skip target_defaults sections. 2017-12-15 20:19:18 +00:00
install-build-deps-android.sh Add profile-unpacking build deps to Linux 2018-05-10 22:24:17 +00:00
install-build-deps.sh Require CHROME_DEVEL_SANDBOX to be built with a non-component config 2018-08-01 17:41:34 +00:00
install-chroot.sh build: Fix common misspellings 2017-08-01 09:03:13 +00:00
landmine_utils.py [build] Remove gyp usage from landmines script 2018-02-20 23:27:59 +00:00
landmines.py [build] Remove gyp usage from landmines script 2018-02-20 23:27:59 +00:00
mac_toolchain.py Replace hermetic Xcode installation with CIPD-based flow (take 2) 2018-04-30 21:24:49 +00:00
nocompile.gni Revert "Prevent implicit cast from 0 to StructPtr<T>." 2018-04-27 02:24:00 +00:00
package_mac_toolchain.py Don't strip DNTDocumentationSupport for hermetic bundles. 2017-06-01 14:49:01 +00:00
precompile.cc Use precompiled headers for most large projects where the .gyp file 2011-09-07 14:14:54 +00:00
precompile.h Add base::stack, convert std::stack uses use it. 2017-10-02 18:55:28 +00:00
print_python_deps.py android: build: Use .pydeps file to store Python dependencies. 2018-08-07 07:12:36 +00:00
protoc_java.py android: build: Use .pydeps file to store Python dependencies. 2018-08-07 07:12:36 +00:00
protoc_java.pydeps android: build: Use .pydeps file to store Python dependencies. 2018-08-07 07:12:36 +00:00
redirect_stdout.py [Mac/GN] Add targets to run dump_syms and create the dSYM archive. 2016-06-09 22:59:07 +00:00
rm.py Unbreak mac component build. 2016-09-23 22:48:01 +00:00
run_swarming_xcode_install.py run_swarming_xcode_install: Add --dimension and a name tag 2017-09-28 22:25:40 +00:00
sample_arg_file.gn Add support for customizing GN's args text. 2017-03-04 02:32:43 +00:00
sanitize-mac-build-log.sed Cleanup sanitize build sed and sh scripts. 2013-10-31 04:21:14 +00:00
sanitize-mac-build-log.sh Cleanup sanitize build sed and sh scripts. 2013-10-31 04:21:14 +00:00
sanitize-win-build-log.sed Remove removed (and outdated anyway) Perl IDL compiler line from sed script 2014-02-27 15:00:45 +00:00
sanitize-win-build-log.sh Cleanup sanitize build sed and sh scripts. 2013-10-31 04:21:14 +00:00
shim_headers.gni Unbundle openh264 2017-05-22 12:52:46 +00:00
split_static_library.gni Don't use split_static_library when lld is used 2018-07-18 15:27:42 +00:00
swarming_xcode_install.py Install Xcode via swarming. 2017-09-15 13:17:50 +00:00
symlink.gni [android] Tweak data deps to include two chromium tool binaries. (RELAND) 2016-11-11 19:19:23 +00:00
symlink.py Revert of Add work-around to symlink.gni to avoid "not up-to-date" ninja errors (patchset #1 id:1 of https://codereview.chromium.org/2344243002/ ) 2016-09-20 22:16:30 +00:00
timestamp.gni win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time 2018-08-03 17:33:15 +00:00
tree_truth.sh Print project path in build/tree_truth.sh (fix typo) 2014-11-11 16:53:33 +00:00
update-linux-sandbox.sh Require CHROME_DEVEL_SANDBOX to be built with a non-component config 2018-08-01 17:41:34 +00:00
vs_toolchain.py Delete code to manipulate GYP_DEFINES 2018-06-11 09:07:50 +00:00
whitespace_file.txt First commit to test how it works. 2018-07-05 12:23:42 +00:00
win_is_xtree_patched.py Fix xtree patch check, and in turn C4702 disabling 2015-01-07 23:42:34 +00:00
write_build_date_header.py win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time 2018-08-03 17:33:15 +00:00
write_buildflag_header.py New build flag system, convert Google Now flag 2015-11-26 09:22:27 +00:00