This reverts commit e2a95639125afd08739a36d0e7f367f7b8abb952.
Reason for revert: This breaks tests on account of fooling the Windows loader. See https://bugs.chromium.org/p/chromium/issues/detail?id=900121#c6 for details.
Original change's description:
> win: Make chrome.7z generation more deterministic.
>
> mini_installer.exe embeds chrome.7z, and chrome.7z contains timestamps
> of all files in the archive. To make chrome.7z more deterministic, give
> all files in it deterministic timestamps.
>
> Bug: 330260
> Change-Id: Ia251dd38177acbebdde288b86826986f441b2d4d
> Reviewed-on: https://chromium-review.googlesource.com/c/1302393
> Reviewed-by: Gabriel Charette <gab@chromium.org>
> Commit-Queue: Nico Weber <thakis@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#603171}
TBR=gab@chromium.org,thakis@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 330260
Change-Id: I261401c27226ab9e0cf30bc07a47ecf8399f491b
Reviewed-on: https://chromium-review.googlesource.com/c/1317569
Reviewed-by: Greg Thompson <grt@chromium.org>
Commit-Queue: Greg Thompson <grt@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#605299}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 05eb6e237704dc0e3fcdaac39ac35edb71c0cb58
mini_installer.exe embeds chrome.7z, and chrome.7z contains timestamps
of all files in the archive. To make chrome.7z more deterministic, give
all files in it deterministic timestamps.
Bug: 330260
Change-Id: Ia251dd38177acbebdde288b86826986f441b2d4d
Reviewed-on: https://chromium-review.googlesource.com/c/1302393
Reviewed-by: Gabriel Charette <gab@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#603171}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: e2a95639125afd08739a36d0e7f367f7b8abb952
This reverts commit 2df0c83aee0a00bc0f7c26a22b68b4ea3eefef80.
Reason for revert: official android should be fine with this after #583420
Original change's description:
> Revert "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time"
>
> This reverts commit ef36dc19986a90f49d0d31520c88d310d72bd038.
>
> Reason for revert: This turns out to break the official android build, which apparently was relying on a broken aspect of the build that this fixed :(. See https://crbug.com/871173.
>
> Original change's description:
> > win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time
> >
> > We used to set the timestamp to a hash of the binary, similar to
> > https://blogs.msdn.microsoft.com/oldnewthing/20180103-00/?p=97705
> > However, that caused an appcompat warning on Windows 7 to appear, which
> > interpreted the hash as a timestamp. (It's possible that https://llvm.org/PR38429
> > could help with that, but my guess it won't have an effect on Windows 7,
> > which likely always believes that the the coff timestamp field always stores
> > a timestamp).
> >
> > So currently we write the current time during linking in that field, but that's
> > bad for build determinism and that in turn is bad for swarming test result cachability.
> >
> > build/write_build_date_header.py already creates a deterministic BUILD_DATE
> > with several tradeoffs. Cachability wants this to change infrequently, but
> > things like HSTS need a "real" build date and want this to change frequently.
> > The compromise is: The date changes once per day in official builds, and
> > once a month in regular builds.
> >
> > (We could use /Brepro in ldflags instead of /TIMESTAMP for unofficial builds to get
> > the binary hash in the timestamp, but having the header timestamp match the BUILD_DATE
> > define seems nice.)
> >
> > So let's use that same time as timestamp in the PE/COFF header. lld-link has a
> > /TIMESTAMP: flag we can use to pass in an explicit timestamp.
> >
> > Since tools can't have deps, we need to compute the timestamp at gn time,
> > so split write_build_date_header.py in two pieces: build/compute_build_timestamp.py
> > that just prints the timestamp we want to use, and the old write_build_date_header.py, which
> > now takes that timestamp and writes the header file.
> >
> > Call compute_build_timestamp.py at gn time so that we can pass it in ldflags, and
> > pass the resultl to write_build_date_header.py which keeps running as an action
> > during build time (so that we at least don't need to write a file at gn time).
> >
> > An additional wrinkle here is that the PE/COFF timestamp is used as one of just two
> > keys per binary for uploading PE binaries to the symbol server, the other being file size.
> > https://bugs.llvm.org/show_bug.cgi?id=35914#c0 has a good description of this, and
> > tools/symsrc/img_fingerprint.py's GetImgFingerprint() is our implementation of it.
> > But since we only upload binaries with symbols for official chrome builds to the symbol server,
> > a timestamp that changes once a day should be still enough. (32-bit and 64-bit chromes
> > have the same filename, and we might rarely build canary and beta and stable all on the
> > same day, but them all being the same size seems highly unlikely.)
> >
> > Bug: 843199,804926,330260
> > Change-Id: I1d4193cc537ae0c4b2d6ac9281fad29de754dd6c
> > Reviewed-on: https://chromium-review.googlesource.com/1161104
> > Reviewed-by: Dirk Pranke <dpranke@chromium.org>
> > Reviewed-by: Hans Wennborg <hans@chromium.org>
> > Commit-Queue: Nico Weber <thakis@chromium.org>
> > Cr-Commit-Position: refs/heads/master@{#580585}
>
> TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org
> NOTRY=true
>
> # Not skipping CQ checks because original CL landed > 1 day ago.
>
> Bug: 843199, 804926, 330260
> Change-Id: Ib93697a82f8a9d3fb303b763609e82e0612887cd
> Reviewed-on: https://chromium-review.googlesource.com/1166203
> Commit-Queue: Hans Wennborg <hans@chromium.org>
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
> Reviewed-by: Nico Weber <thakis@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#581485}
TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 843199, 804926, 330260
Change-Id: I136e405c84eba3f61a4ac96b2017a34ade0cfba6
Reviewed-on: https://chromium-review.googlesource.com/1179281
Commit-Queue: Nico Weber <thakis@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#583945}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 967d1f1e6adc9b7641a5b0174c48f0883e293880
BUILD_DATE used to take the current time and then set BUILD_DATE to
the first Sunday of the month (unofficial builds) or to that date at 5am
(official builds), so that the date stays constant for a while, to
keep binary hashes changing less frequently. (BUILD_DATE makes it into
base/, so most binaries contain it.)
This behavior is kept, but it's now based off the timestamp of the most
recent commit.
This has the advantage of being deterministic, so for example if some
Android bot builds the same revision twice, it's now guaranteed that
both builds will end up with the same timestamp. The motivation is that
I want to reland https://chromium-review.googlesource.com/1161104 which
broke Android bots which assume fully deterministic BUILD_DATEs and
currently only work by accident due to write_build_date_header.py not
rerunning in incremental builds.
Since querying git is slow and the timestamp will be queried at
gn gen time in a follow-up change, let lastchange.py write the last
commit's timestamp in a file next to build/util/LASTCHANGE and then read
that in write_build_date_header.py.
For official chrome builds we ship, this shouldn't make much of a difference,
since there's always recent commits when we ship those (if only to
update chrome/VERSION).
When syncing to an old revision (past the revision this change lands in, of
course -- say this lands and then 2 years pass, and then you sync back
1.5 years), that will now have an old date while it had a current date
before. This could happen while bisecting. You'd get more security
interstitials while bisecting.
Embedders might ship old Chromiums without having a recent commit
in src. These would have BUILD_DATEs in the past. I announced this change on
https://groups.google.com/a/chromium.org/forum/#!topic/embedder-dev/Ymk3bHPQ45M
We can add an explicit opt-out if requested.
Build date currently affects:
- HSTS lists (considered valid for 70 days past build date)
-- but only enabled for official chrome builds (GOOGLE_CHROME_BUILD
check in TransportSecurityState())
- certificate transparency (considered valid for 70 days past build date)
- invalid cert date interstitial (warns if system clock is
more than 2 days behind build date, or more than 365 days
in the future)
- browser upgrade detector indicator UI (stops checking after
build is 12 weeks old)
- field trials (kNoExpirationYear is 2 years past build date)
//base:build_date already depends on lastchange_file and only reruns
when that file is touched (and on full builds).
Lots of prior discussion in https://codereview.chromium.org/1641413002/
(this approach but without widening was suggested in comment 26,
comment 46 mentions security concerns from agl, comment 67 onwards are about
security too. Eventually http://crbug.com/584880 got filed with a plan similar
to this CL here, but there was no time to evaluate the proposal at the time
due to internship schedule reasons. I talked to security -- namely,
felt agl ncarter carlosil -- and everyone thinks this has no obvious issues.)
Obsoletes https://chromium-review.googlesource.com/c/564598/, a prior attempt
at getting deterministic builds (opt-in, by setting an env var with the desired
date).
Bug: 314403,584880,871173
Change-Id: I6ccd78bff005572a9fa391e9820f4f5f83583c50
Reviewed-on: https://chromium-review.googlesource.com/1167913
Reviewed-by: Alex Mineer <amineer@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#583420}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 4c984bc7664755d701c75b49b795678e164551e3
This reverts commit ef36dc19986a90f49d0d31520c88d310d72bd038.
Reason for revert: This turns out to break the official android build, which apparently was relying on a broken aspect of the build that this fixed :(. See https://crbug.com/871173.
Original change's description:
> win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time
>
> We used to set the timestamp to a hash of the binary, similar to
> https://blogs.msdn.microsoft.com/oldnewthing/20180103-00/?p=97705
> However, that caused an appcompat warning on Windows 7 to appear, which
> interpreted the hash as a timestamp. (It's possible that https://llvm.org/PR38429
> could help with that, but my guess it won't have an effect on Windows 7,
> which likely always believes that the the coff timestamp field always stores
> a timestamp).
>
> So currently we write the current time during linking in that field, but that's
> bad for build determinism and that in turn is bad for swarming test result cachability.
>
> build/write_build_date_header.py already creates a deterministic BUILD_DATE
> with several tradeoffs. Cachability wants this to change infrequently, but
> things like HSTS need a "real" build date and want this to change frequently.
> The compromise is: The date changes once per day in official builds, and
> once a month in regular builds.
>
> (We could use /Brepro in ldflags instead of /TIMESTAMP for unofficial builds to get
> the binary hash in the timestamp, but having the header timestamp match the BUILD_DATE
> define seems nice.)
>
> So let's use that same time as timestamp in the PE/COFF header. lld-link has a
> /TIMESTAMP: flag we can use to pass in an explicit timestamp.
>
> Since tools can't have deps, we need to compute the timestamp at gn time,
> so split write_build_date_header.py in two pieces: build/compute_build_timestamp.py
> that just prints the timestamp we want to use, and the old write_build_date_header.py, which
> now takes that timestamp and writes the header file.
>
> Call compute_build_timestamp.py at gn time so that we can pass it in ldflags, and
> pass the resultl to write_build_date_header.py which keeps running as an action
> during build time (so that we at least don't need to write a file at gn time).
>
> An additional wrinkle here is that the PE/COFF timestamp is used as one of just two
> keys per binary for uploading PE binaries to the symbol server, the other being file size.
> https://bugs.llvm.org/show_bug.cgi?id=35914#c0 has a good description of this, and
> tools/symsrc/img_fingerprint.py's GetImgFingerprint() is our implementation of it.
> But since we only upload binaries with symbols for official chrome builds to the symbol server,
> a timestamp that changes once a day should be still enough. (32-bit and 64-bit chromes
> have the same filename, and we might rarely build canary and beta and stable all on the
> same day, but them all being the same size seems highly unlikely.)
>
> Bug: 843199,804926,330260
> Change-Id: I1d4193cc537ae0c4b2d6ac9281fad29de754dd6c
> Reviewed-on: https://chromium-review.googlesource.com/1161104
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
> Reviewed-by: Hans Wennborg <hans@chromium.org>
> Commit-Queue: Nico Weber <thakis@chromium.org>
> Cr-Commit-Position: refs/heads/master@{#580585}
TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org
NOTRY=true
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 843199, 804926, 330260
Change-Id: Ib93697a82f8a9d3fb303b763609e82e0612887cd
Reviewed-on: https://chromium-review.googlesource.com/1166203
Commit-Queue: Hans Wennborg <hans@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#581485}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 2df0c83aee0a00bc0f7c26a22b68b4ea3eefef80
We used to set the timestamp to a hash of the binary, similar to
https://blogs.msdn.microsoft.com/oldnewthing/20180103-00/?p=97705
However, that caused an appcompat warning on Windows 7 to appear, which
interpreted the hash as a timestamp. (It's possible that https://llvm.org/PR38429
could help with that, but my guess it won't have an effect on Windows 7,
which likely always believes that the the coff timestamp field always stores
a timestamp).
So currently we write the current time during linking in that field, but that's
bad for build determinism and that in turn is bad for swarming test result cachability.
build/write_build_date_header.py already creates a deterministic BUILD_DATE
with several tradeoffs. Cachability wants this to change infrequently, but
things like HSTS need a "real" build date and want this to change frequently.
The compromise is: The date changes once per day in official builds, and
once a month in regular builds.
(We could use /Brepro in ldflags instead of /TIMESTAMP for unofficial builds to get
the binary hash in the timestamp, but having the header timestamp match the BUILD_DATE
define seems nice.)
So let's use that same time as timestamp in the PE/COFF header. lld-link has a
/TIMESTAMP: flag we can use to pass in an explicit timestamp.
Since tools can't have deps, we need to compute the timestamp at gn time,
so split write_build_date_header.py in two pieces: build/compute_build_timestamp.py
that just prints the timestamp we want to use, and the old write_build_date_header.py, which
now takes that timestamp and writes the header file.
Call compute_build_timestamp.py at gn time so that we can pass it in ldflags, and
pass the resultl to write_build_date_header.py which keeps running as an action
during build time (so that we at least don't need to write a file at gn time).
An additional wrinkle here is that the PE/COFF timestamp is used as one of just two
keys per binary for uploading PE binaries to the symbol server, the other being file size.
https://bugs.llvm.org/show_bug.cgi?id=35914#c0 has a good description of this, and
tools/symsrc/img_fingerprint.py's GetImgFingerprint() is our implementation of it.
But since we only upload binaries with symbols for official chrome builds to the symbol server,
a timestamp that changes once a day should be still enough. (32-bit and 64-bit chromes
have the same filename, and we might rarely build canary and beta and stable all on the
same day, but them all being the same size seems highly unlikely.)
Bug: 843199,804926,330260
Change-Id: I1d4193cc537ae0c4b2d6ac9281fad29de754dd6c
Reviewed-on: https://chromium-review.googlesource.com/1161104
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Hans Wennborg <hans@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#580585}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: ef36dc19986a90f49d0d31520c88d310d72bd038
As far as I can tell, this is unused.
Bug: 489490
Change-Id: I729acb5d1f50ad56848e00b3c2b771e1074dba00
Reviewed-on: https://chromium-review.googlesource.com/1160777
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
Cr-Original-Commit-Position: refs/heads/master@{#580326}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: e335101eedd3ab28d16dcb4b22e5ee0b60044b89
- Change the generated_build_date.h to contain the full timestamp, it makes the
code less brittle.
- For official builds before 5:00am, it would generate a build time in the
future. Fix this.
- Improve the error message in BuildTime.InThePast.
- Add test case to ensure the build is not more than 45 days old.
- Remove dummy test.
- Improve comments in base/BUILD.gn and base/base.gyp.
- Add functional write_build_date_header.py --help.
This change builds upon work in https://codereview.chromium.org/1641413002.
R=thakis@chromium.org,felt@chromium.org,lgarron@chromium.org
BUG=587694
Review URL: https://codereview.chromium.org/1846713002
Cr-Original-Commit-Position: refs/heads/master@{#384991}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 1c9b02233631b0ba9b096d79bf5b8d71a370dbb0
After discussion with maruel and agl, it seems that
(1) for the purposes of build determinism, it's necessary
to be able to arbitrarily set the build time.
(2) for the purposes of continuous integration, longer duration
between cache invalidation is better, but >=1mo is preferable.
(3) for security purposes, timebombs would ideally be as close to
the actual time of the build as possible. It must be in the past.
(4) HSTS certificate pinning is valid for 70 days. To make CI builds enforce
HTST pinning, <=1mo is preferable.
All of these can reasonably be satisfied by using different settings for CI
versus official builds:
- For official build, the build time is set to 5:00am of the day of the build or
the day before.
- For continuous integration build, the build time is set to the current month.
If the current day is within the first week of the month and last Sunday
wasn't part of the current month, the Sunday of the previous month is used.
This results that cache invalidation happens on a Sunday, which is preferable
from an infrastructure standpoint.
- In the case that the build time needs to be set to a specific value (i.e. to
reproduce a build), the GN/GYP variable 'override_build_date' can be used to
set the BUILD_DATE explicitly. Its format is "Mmm DD YYYY".
The way it is done is:
- Generate $target_gen_dir/generated_build_date.h that defines BUILD_DATE. Its
value depends on if an official build is done or not.
- This step depends on build/util/LASTCHANGE so it is run at every sync. The
file is only touched if the content changed to not affect null build.
Most importantly, this change removes the need of both GN/GYP variable
"dont_embed_build_metadata" and C define "DONT_EMBED_BUILD_METADATA"; the build
is always deterministic (up to a month) by default. This removes the risk
oversight of forgetting to set this variable, which already happened.
R=maruel@chromium.org
BUG=489490
Review URL: https://codereview.chromium.org/1641413002
Cr-Original-Commit-Position: refs/heads/master@{#375136}
Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src
Cr-Mirrored-Commit: 08d91b75212b6592f05ff993d5a71c0f5a546563
Greg Thompson
Nico Weber
Dirk Pranke
Marc-Antoine Ruel
zforman