This is a reland of 3a8d37c30ccdc64dc46acb7e92f01adbb8fc9157
Reland fix:
- Only adds the needed test data if `cros_sdk_version != ""`
That will prevent ebuild invocations from trying to add non-existant
items to data (since the ebuild doesn't use the cros chrome-sdk cache).
Original change's description:
> chromeos: Add only the current SDK version's items to a test's data.
>
> Instead of adding every version of a given item, this will add only the
> current version's. This is a bit tricky since individual versions of an
> item are keyed by symlink. And since isolate doesn't follow symlinks,
> we have to follow them ourselves.
>
> Bug: 1027382
> Change-Id: I6b3ea4ec15de3804e99fd6af768daac64f490d5b
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1931155
> Commit-Queue: Ben Pastene <bpastene@chromium.org>
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#771959}
Bug: 1027382
Change-Id: I2ff8c8452f5f6f43e178d547e963b6f8b7d17166
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2218581
Reviewed-by: Nico Weber <thakis@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#774341}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 1d040ab9e07f3f9a5c5777f06958591752ae53c4
This reverts commit 3a8d37c30ccdc64dc46acb7e92f01adbb8fc9157.
Reason for revert: blocking uprevs; crbug.com/1087179
Original change's description:
> chromeos: Add only the current SDK version's items to a test's data.
>
> Instead of adding every version of a given item, this will add only the
> current version's. This is a bit tricky since individual versions of an
> item are keyed by symlink. And since isolate doesn't follow symlinks,
> we have to follow them ourselves.
>
> Bug: 1027382
> Change-Id: I6b3ea4ec15de3804e99fd6af768daac64f490d5b
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1931155
> Commit-Queue: Ben Pastene <bpastene@chromium.org>
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#771959}
TBR=thakis@chromium.org,dpranke@chromium.org,bpastene@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 1027382, 1087179
Change-Id: Ie097ba50d36d7eba3865d5cee1775afd5d605354
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2219315
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#772513}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 719ed7077bd31145239955a4bcaf8fb290f28b3a
Instead of adding every version of a given item, this will add only the
current version's. This is a bit tricky since individual versions of an
item are keyed by symlink. And since isolate doesn't follow symlinks,
we have to follow them ourselves.
Bug: 1027382
Change-Id: I6b3ea4ec15de3804e99fd6af768daac64f490d5b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1931155
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#771959}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 3a8d37c30ccdc64dc46acb7e92f01adbb8fc9157
This teaches Chromium's build system to use the goma client in
depot_tools in $PATH as the default.
If a depot_tools directory containing gomacc doesn't exist in $PATH,
use old default path.
Bug: b/110576906, b/154543259
Change-Id: Iccdeb2c4c2568ab4557f699349a4d1ba325dcc0a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2106326
Commit-Queue: Fumitoshi Ukai <ukai@chromium.org>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Yoshisato Yanagisawa <yyanagisawa@google.com>
Cr-Original-Commit-Position: refs/heads/master@{#761306}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 3e3a399c8bba61fa0328e2bf0f428ee775ff7247
//build/config/linux/BUILD.gn is read by //build/config/compiler/BUILD.gn,
which means it is processed by both host and target toolchains.
Consequently, having the "atspi2" target there means "gn gen" can fail
because the host lacks atspi-2's development files even though they are only
needed by the target toolchain during the build.
Bug: 879147
Change-Id: I8780e6e4d3a2b3a44fdb2cbbb0862a1a669acdaa
Reviewed-on: https://chromium-review.googlesource.com/c/1356979
Reviewed-by: Thomas Anderson <thomasanderson@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com>
Cr-Original-Commit-Position: refs/heads/master@{#612831}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 32c051ee1d76c33d389492b1cee23f0d1d752c55
This reverts commit 2df0c83aee0a00bc0f7c26a22b68b4ea3eefef80.
Reason for revert: official android should be fine with this after #583420
Original change's description:
> Revert "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time"
>
> This reverts commit ef36dc19986a90f49d0d31520c88d310d72bd038.
>
> Reason for revert: This turns out to break the official android build, which apparently was relying on a broken aspect of the build that this fixed :(. See https://crbug.com/871173.
>
> Original change's description:
> > win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time
> >
> > We used to set the timestamp to a hash of the binary, similar to
> > https://blogs.msdn.microsoft.com/oldnewthing/20180103-00/?p=97705
> > However, that caused an appcompat warning on Windows 7 to appear, which
> > interpreted the hash as a timestamp. (It's possible that https://llvm.org/PR38429
> > could help with that, but my guess it won't have an effect on Windows 7,
> > which likely always believes that the the coff timestamp field always stores
> > a timestamp).
> >
> > So currently we write the current time during linking in that field, but that's
> > bad for build determinism and that in turn is bad for swarming test result cachability.
> >
> > build/write_build_date_header.py already creates a deterministic BUILD_DATE
> > with several tradeoffs. Cachability wants this to change infrequently, but
> > things like HSTS need a "real" build date and want this to change frequently.
> > The compromise is: The date changes once per day in official builds, and
> > once a month in regular builds.
> >
> > (We could use /Brepro in ldflags instead of /TIMESTAMP for unofficial builds to get
> > the binary hash in the timestamp, but having the header timestamp match the BUILD_DATE
> > define seems nice.)
> >
> > So let's use that same time as timestamp in the PE/COFF header. lld-link has a
> > /TIMESTAMP: flag we can use to pass in an explicit timestamp.
> >
> > Since tools can't have deps, we need to compute the timestamp at gn time,
> > so split write_build_date_header.py in two pieces: build/compute_build_timestamp.py
> > that just prints the timestamp we want to use, and the old write_build_date_header.py, which
> > now takes that timestamp and writes the header file.
> >
> > Call compute_build_timestamp.py at gn time so that we can pass it in ldflags, and
> > pass the resultl to write_build_date_header.py which keeps running as an action
> > during build time (so that we at least don't need to write a file at gn time).
> >
> > An additional wrinkle here is that the PE/COFF timestamp is used as one of just two
> > keys per binary for uploading PE binaries to the symbol server, the other being file size.
> > https://bugs.llvm.org/show_bug.cgi?id=35914#c0 has a good description of this, and
> > tools/symsrc/img_fingerprint.py's GetImgFingerprint() is our implementation of it.
> > But since we only upload binaries with symbols for official chrome builds to the symbol server,
> > a timestamp that changes once a day should be still enough. (32-bit and 64-bit chromes
> > have the same filename, and we might rarely build canary and beta and stable all on the
> > same day, but them all being the same size seems highly unlikely.)
> >
> > Bug: 843199,804926,330260
> > Change-Id: I1d4193cc537ae0c4b2d6ac9281fad29de754dd6c
> > Reviewed-on: https://chromium-review.googlesource.com/1161104
> > Reviewed-by: Dirk Pranke <dpranke@chromium.org>
> > Reviewed-by: Hans Wennborg <hans@chromium.org>
> > Commit-Queue: Nico Weber <thakis@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#580585}
>
> TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org
> NOTRY=true
>
> # Not skipping CQ checks because original CL landed > 1 day ago.
>
> Bug: 843199, 804926, 330260
> Change-Id: Ib93697a82f8a9d3fb303b763609e82e0612887cd
> Reviewed-on: https://chromium-review.googlesource.com/1166203
> Commit-Queue: Hans Wennborg <hans@chromium.org>
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
> Reviewed-by: Nico Weber <thakis@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#581485}
TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 843199, 804926, 330260
Change-Id: I136e405c84eba3f61a4ac96b2017a34ade0cfba6
Reviewed-on: https://chromium-review.googlesource.com/1179281
Commit-Queue: Nico Weber <thakis@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#583945}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 967d1f1e6adc9b7641a5b0174c48f0883e293880
This reverts commit ef36dc19986a90f49d0d31520c88d310d72bd038.
Reason for revert: This turns out to break the official android build, which apparently was relying on a broken aspect of the build that this fixed :(. See https://crbug.com/871173.
Original change's description:
> win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time
>
> We used to set the timestamp to a hash of the binary, similar to
> https://blogs.msdn.microsoft.com/oldnewthing/20180103-00/?p=97705
> However, that caused an appcompat warning on Windows 7 to appear, which
> interpreted the hash as a timestamp. (It's possible that https://llvm.org/PR38429
> could help with that, but my guess it won't have an effect on Windows 7,
> which likely always believes that the the coff timestamp field always stores
> a timestamp).
>
> So currently we write the current time during linking in that field, but that's
> bad for build determinism and that in turn is bad for swarming test result cachability.
>
> build/write_build_date_header.py already creates a deterministic BUILD_DATE
> with several tradeoffs. Cachability wants this to change infrequently, but
> things like HSTS need a "real" build date and want this to change frequently.
> The compromise is: The date changes once per day in official builds, and
> once a month in regular builds.
>
> (We could use /Brepro in ldflags instead of /TIMESTAMP for unofficial builds to get
> the binary hash in the timestamp, but having the header timestamp match the BUILD_DATE
> define seems nice.)
>
> So let's use that same time as timestamp in the PE/COFF header. lld-link has a
> /TIMESTAMP: flag we can use to pass in an explicit timestamp.
>
> Since tools can't have deps, we need to compute the timestamp at gn time,
> so split write_build_date_header.py in two pieces: build/compute_build_timestamp.py
> that just prints the timestamp we want to use, and the old write_build_date_header.py, which
> now takes that timestamp and writes the header file.
>
> Call compute_build_timestamp.py at gn time so that we can pass it in ldflags, and
> pass the resultl to write_build_date_header.py which keeps running as an action
> during build time (so that we at least don't need to write a file at gn time).
>
> An additional wrinkle here is that the PE/COFF timestamp is used as one of just two
> keys per binary for uploading PE binaries to the symbol server, the other being file size.
> https://bugs.llvm.org/show_bug.cgi?id=35914#c0 has a good description of this, and
> tools/symsrc/img_fingerprint.py's GetImgFingerprint() is our implementation of it.
> But since we only upload binaries with symbols for official chrome builds to the symbol server,
> a timestamp that changes once a day should be still enough. (32-bit and 64-bit chromes
> have the same filename, and we might rarely build canary and beta and stable all on the
> same day, but them all being the same size seems highly unlikely.)
>
> Bug: 843199,804926,330260
> Change-Id: I1d4193cc537ae0c4b2d6ac9281fad29de754dd6c
> Reviewed-on: https://chromium-review.googlesource.com/1161104
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
> Reviewed-by: Hans Wennborg <hans@chromium.org>
> Commit-Queue: Nico Weber <thakis@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#580585}
TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org
NOTRY=true
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 843199, 804926, 330260
Change-Id: Ib93697a82f8a9d3fb303b763609e82e0612887cd
Reviewed-on: https://chromium-review.googlesource.com/1166203
Commit-Queue: Hans Wennborg <hans@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#581485}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 2df0c83aee0a00bc0f7c26a22b68b4ea3eefef80
We used to set the timestamp to a hash of the binary, similar to
https://blogs.msdn.microsoft.com/oldnewthing/20180103-00/?p=97705
However, that caused an appcompat warning on Windows 7 to appear, which
interpreted the hash as a timestamp. (It's possible that https://llvm.org/PR38429
could help with that, but my guess it won't have an effect on Windows 7,
which likely always believes that the the coff timestamp field always stores
a timestamp).
So currently we write the current time during linking in that field, but that's
bad for build determinism and that in turn is bad for swarming test result cachability.
build/write_build_date_header.py already creates a deterministic BUILD_DATE
with several tradeoffs. Cachability wants this to change infrequently, but
things like HSTS need a "real" build date and want this to change frequently.
The compromise is: The date changes once per day in official builds, and
once a month in regular builds.
(We could use /Brepro in ldflags instead of /TIMESTAMP for unofficial builds to get
the binary hash in the timestamp, but having the header timestamp match the BUILD_DATE
define seems nice.)
So let's use that same time as timestamp in the PE/COFF header. lld-link has a
/TIMESTAMP: flag we can use to pass in an explicit timestamp.
Since tools can't have deps, we need to compute the timestamp at gn time,
so split write_build_date_header.py in two pieces: build/compute_build_timestamp.py
that just prints the timestamp we want to use, and the old write_build_date_header.py, which
now takes that timestamp and writes the header file.
Call compute_build_timestamp.py at gn time so that we can pass it in ldflags, and
pass the resultl to write_build_date_header.py which keeps running as an action
during build time (so that we at least don't need to write a file at gn time).
An additional wrinkle here is that the PE/COFF timestamp is used as one of just two
keys per binary for uploading PE binaries to the symbol server, the other being file size.
https://bugs.llvm.org/show_bug.cgi?id=35914#c0 has a good description of this, and
tools/symsrc/img_fingerprint.py's GetImgFingerprint() is our implementation of it.
But since we only upload binaries with symbols for official chrome builds to the symbol server,
a timestamp that changes once a day should be still enough. (32-bit and 64-bit chromes
have the same filename, and we might rarely build canary and beta and stable all on the
same day, but them all being the same size seems highly unlikely.)
Bug: 843199,804926,330260
Change-Id: I1d4193cc537ae0c4b2d6ac9281fad29de754dd6c
Reviewed-on: https://chromium-review.googlesource.com/1161104
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Hans Wennborg <hans@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#580585}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: ef36dc19986a90f49d0d31520c88d310d72bd038
Invoking cpu intensive python processes more than machine cores has some overhead on Mac(crbug.com/695864) and Win.
This change introduces pool mainly for python generator to restrict the number of running process when we specify large parallelism with goma.
I took 3 time build stats using target generate_bindings_modules_v8_interfaces and generate_bindings_core_v8_interfaces which have 1148 python tasks.
With this CL on z840 windows10
TotalSeconds : 18.2953436
TotalSeconds : 18.6283626
TotalSeconds : 19.2731436
Without this CL on Z840 windows10
TotalSeconds : 23.8277797
TotalSeconds : 23.6952018
TotalSeconds : 23.0853999
Linux looks to have good task scheduler.
With this CL on z840 linux
0m9.067s
0m8.771s
0m8.953s
Without this CL on Z840 linux
0m8.998s
0m9.022s
0m8.958s
Also this improves UI's responsiveness when we are building chrome on windows.
Stats of clean chrome build in each major OS is like below.
5 time clean build of chrome on Z840 windows 10 with -j1000 and warm goma backend cache is like below.
With this CL
333.3425057
317.4724857
305.0217898
317.8907203
305.1031952
Avg: 315.76613934
Without this CL
369.9731363
331.296758
329.0041556
329.1472297
333.3883952
Avg: 338.56193496
5 time clean build of chrome on Z840 linux with -j1000 and warm goma backend cache is like below.
With this CL
90.42
87.91
90.45
90.50
89.02
avg: 89.66
Without this CL
89.52
86.34
86.08
85.67
85.89
avg: 86.7
3 time clean build of chrome on 24 thread Mac Pro with -j500 and warm goma backend cache is like below.
With this CL
638.28
627.28
624.69
avg: 630.083
Without this CL
667.52
663.83
655.95
avg: 662.433
Bug: 695864
Change-Id: I6838c0f71b8d8030e6eab58b2990810aaa997dfa
Reviewed-on: https://chromium-review.googlesource.com/882581
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#535589}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 2c098797567226115653c2567dbcfc72ee5af5ae
The dri drivers path is the variable 'dridriverdir' in 'dri.pc'
If found then translate it to the macro DRI_DRIVER_DIR.
It can be useful in various places instead of guessing the path.
For now only use it in gpu_sandbox_hook_linux except for ChromeOS.
Bug: 787787
Change-Id: I7fef8606a4f2cbc04233af820a91cf12a4c7a9a5
Reviewed-on: https://chromium-review.googlesource.com/785371
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Antoine Labour <piman@chromium.org>
Commit-Queue: Julien Isorce <julien.isorce@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#519658}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: b66235077eacfe1691135cd8b1b1a06728cdf810
gn has replaced gyp quite a while ago, so this shouldn't be needed any more,
and it'll make running gn a few milliseconds faster.
BUG=none
Review-Url: https://codereview.chromium.org/2735903005
Cr-Original-Commit-Position: refs/heads/master@{#455277}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: a65e1c607032e3dae2d3652c54ddc8fe96b442fd
This change moves the list of files that are allowed to call
exec_script() into a variable in a file in the //build directory,
so that the list can be shared more easily by other repos that
are pulling in //build via DEPS but have their own custom dotfile.
R=brettw@chromium.org, machenbach@chromium.org
BUG=617873
Review-Url: https://codereview.chromium.org/2512043002
Cr-Original-Commit-Position: refs/heads/master@{#433764}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 1cfa53187039597e5cdaa302cab1921a57060dce
Ben Pastene
Sebastien Marchand
Fumitoshi Ukai
Raphael Kubo da Costa
Nico Weber
Dirk Pranke
Takuto Ikuta
Julien Isorce
rayb