chromium-src-build/compute_build_timestamp.py

127 строки
4.8 KiB
Python
Исходник Обычный вид История

Reland "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time" This reverts commit 2df0c83aee0a00bc0f7c26a22b68b4ea3eefef80. Reason for revert: official android should be fine with this after #583420 Original change's description: > Revert "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time" > > This reverts commit ef36dc19986a90f49d0d31520c88d310d72bd038. > > Reason for revert: This turns out to break the official android build, which apparently was relying on a broken aspect of the build that this fixed :(. See https://crbug.com/871173. > > Original change's description: > > win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time > > > > We used to set the timestamp to a hash of the binary, similar to > > https://blogs.msdn.microsoft.com/oldnewthing/20180103-00/?p=97705 > > However, that caused an appcompat warning on Windows 7 to appear, which > > interpreted the hash as a timestamp. (It's possible that https://llvm.org/PR38429 > > could help with that, but my guess it won't have an effect on Windows 7, > > which likely always believes that the the coff timestamp field always stores > > a timestamp). > > > > So currently we write the current time during linking in that field, but that's > > bad for build determinism and that in turn is bad for swarming test result cachability. > > > > build/write_build_date_header.py already creates a deterministic BUILD_DATE > > with several tradeoffs. Cachability wants this to change infrequently, but > > things like HSTS need a "real" build date and want this to change frequently. > > The compromise is: The date changes once per day in official builds, and > > once a month in regular builds. > > > > (We could use /Brepro in ldflags instead of /TIMESTAMP for unofficial builds to get > > the binary hash in the timestamp, but having the header timestamp match the BUILD_DATE > > define seems nice.) > > > > So let's use that same time as timestamp in the PE/COFF header. lld-link has a > > /TIMESTAMP: flag we can use to pass in an explicit timestamp. > > > > Since tools can't have deps, we need to compute the timestamp at gn time, > > so split write_build_date_header.py in two pieces: build/compute_build_timestamp.py > > that just prints the timestamp we want to use, and the old write_build_date_header.py, which > > now takes that timestamp and writes the header file. > > > > Call compute_build_timestamp.py at gn time so that we can pass it in ldflags, and > > pass the resultl to write_build_date_header.py which keeps running as an action > > during build time (so that we at least don't need to write a file at gn time). > > > > An additional wrinkle here is that the PE/COFF timestamp is used as one of just two > > keys per binary for uploading PE binaries to the symbol server, the other being file size. > > https://bugs.llvm.org/show_bug.cgi?id=35914#c0 has a good description of this, and > > tools/symsrc/img_fingerprint.py's GetImgFingerprint() is our implementation of it. > > But since we only upload binaries with symbols for official chrome builds to the symbol server, > > a timestamp that changes once a day should be still enough. (32-bit and 64-bit chromes > > have the same filename, and we might rarely build canary and beta and stable all on the > > same day, but them all being the same size seems highly unlikely.) > > > > Bug: 843199,804926,330260 > > Change-Id: I1d4193cc537ae0c4b2d6ac9281fad29de754dd6c > > Reviewed-on: https://chromium-review.googlesource.com/1161104 > > Reviewed-by: Dirk Pranke <dpranke@chromium.org> > > Reviewed-by: Hans Wennborg <hans@chromium.org> > > Commit-Queue: Nico Weber <thakis@chromium.org> > > Cr-Commit-Position: refs/heads/master@{#580585} > > TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org > NOTRY=true > > # Not skipping CQ checks because original CL landed > 1 day ago. > > Bug: 843199, 804926, 330260 > Change-Id: Ib93697a82f8a9d3fb303b763609e82e0612887cd > Reviewed-on: https://chromium-review.googlesource.com/1166203 > Commit-Queue: Hans Wennborg <hans@chromium.org> > Reviewed-by: Dirk Pranke <dpranke@chromium.org> > Reviewed-by: Nico Weber <thakis@chromium.org> > Cr-Commit-Position: refs/heads/master@{#581485} TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org # Not skipping CQ checks because original CL landed > 1 day ago. Bug: 843199, 804926, 330260 Change-Id: I136e405c84eba3f61a4ac96b2017a34ade0cfba6 Reviewed-on: https://chromium-review.googlesource.com/1179281 Commit-Queue: Nico Weber <thakis@chromium.org> Reviewed-by: Nico Weber <thakis@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#583945} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 967d1f1e6adc9b7641a5b0174c48f0883e293880
2018-08-17 05:42:02 +03:00
#!/usr/bin/env python
# Copyright 2018 The Chromium Authors. All rights reserved.
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.
"""Returns a timestamp that approximates the build date.
build_type impacts the timestamp generated, both relative to the date of the
last recent commit:
- default: the build date is set to the most recent first Sunday of a month at
5:00am. The reason is that it is a time where invalidating the build cache
shouldn't have major repercussions (due to lower load).
- official: the build date is set to the time of the most recent commit.
Reland "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time" This reverts commit 2df0c83aee0a00bc0f7c26a22b68b4ea3eefef80. Reason for revert: official android should be fine with this after #583420 Original change's description: > Revert "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time" > > This reverts commit ef36dc19986a90f49d0d31520c88d310d72bd038. > > Reason for revert: This turns out to break the official android build, which apparently was relying on a broken aspect of the build that this fixed :(. See https://crbug.com/871173. > > Original change's description: > > win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time > > > > We used to set the timestamp to a hash of the binary, similar to > > https://blogs.msdn.microsoft.com/oldnewthing/20180103-00/?p=97705 > > However, that caused an appcompat warning on Windows 7 to appear, which > > interpreted the hash as a timestamp. (It's possible that https://llvm.org/PR38429 > > could help with that, but my guess it won't have an effect on Windows 7, > > which likely always believes that the the coff timestamp field always stores > > a timestamp). > > > > So currently we write the current time during linking in that field, but that's > > bad for build determinism and that in turn is bad for swarming test result cachability. > > > > build/write_build_date_header.py already creates a deterministic BUILD_DATE > > with several tradeoffs. Cachability wants this to change infrequently, but > > things like HSTS need a "real" build date and want this to change frequently. > > The compromise is: The date changes once per day in official builds, and > > once a month in regular builds. > > > > (We could use /Brepro in ldflags instead of /TIMESTAMP for unofficial builds to get > > the binary hash in the timestamp, but having the header timestamp match the BUILD_DATE > > define seems nice.) > > > > So let's use that same time as timestamp in the PE/COFF header. lld-link has a > > /TIMESTAMP: flag we can use to pass in an explicit timestamp. > > > > Since tools can't have deps, we need to compute the timestamp at gn time, > > so split write_build_date_header.py in two pieces: build/compute_build_timestamp.py > > that just prints the timestamp we want to use, and the old write_build_date_header.py, which > > now takes that timestamp and writes the header file. > > > > Call compute_build_timestamp.py at gn time so that we can pass it in ldflags, and > > pass the resultl to write_build_date_header.py which keeps running as an action > > during build time (so that we at least don't need to write a file at gn time). > > > > An additional wrinkle here is that the PE/COFF timestamp is used as one of just two > > keys per binary for uploading PE binaries to the symbol server, the other being file size. > > https://bugs.llvm.org/show_bug.cgi?id=35914#c0 has a good description of this, and > > tools/symsrc/img_fingerprint.py's GetImgFingerprint() is our implementation of it. > > But since we only upload binaries with symbols for official chrome builds to the symbol server, > > a timestamp that changes once a day should be still enough. (32-bit and 64-bit chromes > > have the same filename, and we might rarely build canary and beta and stable all on the > > same day, but them all being the same size seems highly unlikely.) > > > > Bug: 843199,804926,330260 > > Change-Id: I1d4193cc537ae0c4b2d6ac9281fad29de754dd6c > > Reviewed-on: https://chromium-review.googlesource.com/1161104 > > Reviewed-by: Dirk Pranke <dpranke@chromium.org> > > Reviewed-by: Hans Wennborg <hans@chromium.org> > > Commit-Queue: Nico Weber <thakis@chromium.org> > > Cr-Commit-Position: refs/heads/master@{#580585} > > TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org > NOTRY=true > > # Not skipping CQ checks because original CL landed > 1 day ago. > > Bug: 843199, 804926, 330260 > Change-Id: Ib93697a82f8a9d3fb303b763609e82e0612887cd > Reviewed-on: https://chromium-review.googlesource.com/1166203 > Commit-Queue: Hans Wennborg <hans@chromium.org> > Reviewed-by: Dirk Pranke <dpranke@chromium.org> > Reviewed-by: Nico Weber <thakis@chromium.org> > Cr-Commit-Position: refs/heads/master@{#581485} TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org # Not skipping CQ checks because original CL landed > 1 day ago. Bug: 843199, 804926, 330260 Change-Id: I136e405c84eba3f61a4ac96b2017a34ade0cfba6 Reviewed-on: https://chromium-review.googlesource.com/1179281 Commit-Queue: Nico Weber <thakis@chromium.org> Reviewed-by: Nico Weber <thakis@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#583945} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 967d1f1e6adc9b7641a5b0174c48f0883e293880
2018-08-17 05:42:02 +03:00
Either way, it is guaranteed to be in the past and always in UTC.
"""
# The requirements for the timestamp:
# (1) for the purposes of continuous integration, longer duration
# between cache invalidation is better, but >=1mo is preferable.
# (2) for security purposes, timebombs would ideally be as close to
# the actual time of the build as possible. It must be in the past.
# (3) HSTS certificate pinning is valid for 70 days. To make CI builds enforce
# HTST pinning, <=1mo is preferable.
#
# On Windows, the timestamp is also written in the PE/COFF file header of
# executables of dlls. That timestamp and the executable's file size are
# the only two pieces of information that identify a given executable on
# the symbol server, so rarely changing timestamps can cause conflicts there
# as well. We only upload symbols for official builds to the symbol server.
from __future__ import print_function
Reland "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time" This reverts commit 2df0c83aee0a00bc0f7c26a22b68b4ea3eefef80. Reason for revert: official android should be fine with this after #583420 Original change's description: > Revert "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time" > > This reverts commit ef36dc19986a90f49d0d31520c88d310d72bd038. > > Reason for revert: This turns out to break the official android build, which apparently was relying on a broken aspect of the build that this fixed :(. See https://crbug.com/871173. > > Original change's description: > > win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time > > > > We used to set the timestamp to a hash of the binary, similar to > > https://blogs.msdn.microsoft.com/oldnewthing/20180103-00/?p=97705 > > However, that caused an appcompat warning on Windows 7 to appear, which > > interpreted the hash as a timestamp. (It's possible that https://llvm.org/PR38429 > > could help with that, but my guess it won't have an effect on Windows 7, > > which likely always believes that the the coff timestamp field always stores > > a timestamp). > > > > So currently we write the current time during linking in that field, but that's > > bad for build determinism and that in turn is bad for swarming test result cachability. > > > > build/write_build_date_header.py already creates a deterministic BUILD_DATE > > with several tradeoffs. Cachability wants this to change infrequently, but > > things like HSTS need a "real" build date and want this to change frequently. > > The compromise is: The date changes once per day in official builds, and > > once a month in regular builds. > > > > (We could use /Brepro in ldflags instead of /TIMESTAMP for unofficial builds to get > > the binary hash in the timestamp, but having the header timestamp match the BUILD_DATE > > define seems nice.) > > > > So let's use that same time as timestamp in the PE/COFF header. lld-link has a > > /TIMESTAMP: flag we can use to pass in an explicit timestamp. > > > > Since tools can't have deps, we need to compute the timestamp at gn time, > > so split write_build_date_header.py in two pieces: build/compute_build_timestamp.py > > that just prints the timestamp we want to use, and the old write_build_date_header.py, which > > now takes that timestamp and writes the header file. > > > > Call compute_build_timestamp.py at gn time so that we can pass it in ldflags, and > > pass the resultl to write_build_date_header.py which keeps running as an action > > during build time (so that we at least don't need to write a file at gn time). > > > > An additional wrinkle here is that the PE/COFF timestamp is used as one of just two > > keys per binary for uploading PE binaries to the symbol server, the other being file size. > > https://bugs.llvm.org/show_bug.cgi?id=35914#c0 has a good description of this, and > > tools/symsrc/img_fingerprint.py's GetImgFingerprint() is our implementation of it. > > But since we only upload binaries with symbols for official chrome builds to the symbol server, > > a timestamp that changes once a day should be still enough. (32-bit and 64-bit chromes > > have the same filename, and we might rarely build canary and beta and stable all on the > > same day, but them all being the same size seems highly unlikely.) > > > > Bug: 843199,804926,330260 > > Change-Id: I1d4193cc537ae0c4b2d6ac9281fad29de754dd6c > > Reviewed-on: https://chromium-review.googlesource.com/1161104 > > Reviewed-by: Dirk Pranke <dpranke@chromium.org> > > Reviewed-by: Hans Wennborg <hans@chromium.org> > > Commit-Queue: Nico Weber <thakis@chromium.org> > > Cr-Commit-Position: refs/heads/master@{#580585} > > TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org > NOTRY=true > > # Not skipping CQ checks because original CL landed > 1 day ago. > > Bug: 843199, 804926, 330260 > Change-Id: Ib93697a82f8a9d3fb303b763609e82e0612887cd > Reviewed-on: https://chromium-review.googlesource.com/1166203 > Commit-Queue: Hans Wennborg <hans@chromium.org> > Reviewed-by: Dirk Pranke <dpranke@chromium.org> > Reviewed-by: Nico Weber <thakis@chromium.org> > Cr-Commit-Position: refs/heads/master@{#581485} TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org # Not skipping CQ checks because original CL landed > 1 day ago. Bug: 843199, 804926, 330260 Change-Id: I136e405c84eba3f61a4ac96b2017a34ade0cfba6 Reviewed-on: https://chromium-review.googlesource.com/1179281 Commit-Queue: Nico Weber <thakis@chromium.org> Reviewed-by: Nico Weber <thakis@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#583945} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 967d1f1e6adc9b7641a5b0174c48f0883e293880
2018-08-17 05:42:02 +03:00
import argparse
import calendar
import datetime
import doctest
import os
import sys
THIS_DIR = os.path.abspath(os.path.dirname(__file__))
def GetFirstSundayOfMonth(year, month):
"""Returns the first sunday of the given month of the given year.
>>> GetFirstSundayOfMonth(2016, 2)
7
>>> GetFirstSundayOfMonth(2016, 3)
6
>>> GetFirstSundayOfMonth(2000, 1)
2
"""
weeks = calendar.Calendar().monthdays2calendar(year, month)
# Return the first day in the first week that is a Sunday.
return [date_day[0] for date_day in weeks[0] if date_day[1] == 6][0]
Don't quantize build timestamps on official builds Official builds - the PDBs and the PE files - often get published to a symbol server. For PE files the path name in the symbol server is: "%s\%08X%s\%s" % (peName, timeStamp, imageSize, peName) Since the peName for a particular DLL/EXE never changes and since the size often doesn't change this means that the timeStamp is the only differentiator between nearby builds. With the strategy of setting the build timestamp to 5 am of the last commit time it is easy to get a build that overwrites the previous build on the symbol server. This happened when build 75.0.3770.143 overwrote all of the PE files from build 75.0.3770.142, which complicated the investigation of (restricted view, sorry) crbug.com/964273. This probably happened many other times. The PDB files were never overwritten which is why this was not noticed earlier. When the 5 am quantization was added we were using the current time for the build timestamps. Now that we are using the last commit time it is less important to quantize to 5 am so this change removes that quantization, for official builds only. An increased number of days where we do multiple builds of one channel means that this issue is hit more frequently than when the quantization was initially added. Bug: 993509 Change-Id: Ibfac95569b713ede056d3ff070db0c05b4a38c77 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1754527 Reviewed-by: Nico Weber <thakis@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Commit-Queue: Bruce Dawson <brucedawson@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#687349} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 02f0edc3fb60b55db652fce1c298beda75e7d1d9
2019-08-15 21:23:05 +03:00
def GetUnofficialBuildDate(build_date):
Reland "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time" This reverts commit 2df0c83aee0a00bc0f7c26a22b68b4ea3eefef80. Reason for revert: official android should be fine with this after #583420 Original change's description: > Revert "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time" > > This reverts commit ef36dc19986a90f49d0d31520c88d310d72bd038. > > Reason for revert: This turns out to break the official android build, which apparently was relying on a broken aspect of the build that this fixed :(. See https://crbug.com/871173. > > Original change's description: > > win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time > > > > We used to set the timestamp to a hash of the binary, similar to > > https://blogs.msdn.microsoft.com/oldnewthing/20180103-00/?p=97705 > > However, that caused an appcompat warning on Windows 7 to appear, which > > interpreted the hash as a timestamp. (It's possible that https://llvm.org/PR38429 > > could help with that, but my guess it won't have an effect on Windows 7, > > which likely always believes that the the coff timestamp field always stores > > a timestamp). > > > > So currently we write the current time during linking in that field, but that's > > bad for build determinism and that in turn is bad for swarming test result cachability. > > > > build/write_build_date_header.py already creates a deterministic BUILD_DATE > > with several tradeoffs. Cachability wants this to change infrequently, but > > things like HSTS need a "real" build date and want this to change frequently. > > The compromise is: The date changes once per day in official builds, and > > once a month in regular builds. > > > > (We could use /Brepro in ldflags instead of /TIMESTAMP for unofficial builds to get > > the binary hash in the timestamp, but having the header timestamp match the BUILD_DATE > > define seems nice.) > > > > So let's use that same time as timestamp in the PE/COFF header. lld-link has a > > /TIMESTAMP: flag we can use to pass in an explicit timestamp. > > > > Since tools can't have deps, we need to compute the timestamp at gn time, > > so split write_build_date_header.py in two pieces: build/compute_build_timestamp.py > > that just prints the timestamp we want to use, and the old write_build_date_header.py, which > > now takes that timestamp and writes the header file. > > > > Call compute_build_timestamp.py at gn time so that we can pass it in ldflags, and > > pass the resultl to write_build_date_header.py which keeps running as an action > > during build time (so that we at least don't need to write a file at gn time). > > > > An additional wrinkle here is that the PE/COFF timestamp is used as one of just two > > keys per binary for uploading PE binaries to the symbol server, the other being file size. > > https://bugs.llvm.org/show_bug.cgi?id=35914#c0 has a good description of this, and > > tools/symsrc/img_fingerprint.py's GetImgFingerprint() is our implementation of it. > > But since we only upload binaries with symbols for official chrome builds to the symbol server, > > a timestamp that changes once a day should be still enough. (32-bit and 64-bit chromes > > have the same filename, and we might rarely build canary and beta and stable all on the > > same day, but them all being the same size seems highly unlikely.) > > > > Bug: 843199,804926,330260 > > Change-Id: I1d4193cc537ae0c4b2d6ac9281fad29de754dd6c > > Reviewed-on: https://chromium-review.googlesource.com/1161104 > > Reviewed-by: Dirk Pranke <dpranke@chromium.org> > > Reviewed-by: Hans Wennborg <hans@chromium.org> > > Commit-Queue: Nico Weber <thakis@chromium.org> > > Cr-Commit-Position: refs/heads/master@{#580585} > > TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org > NOTRY=true > > # Not skipping CQ checks because original CL landed > 1 day ago. > > Bug: 843199, 804926, 330260 > Change-Id: Ib93697a82f8a9d3fb303b763609e82e0612887cd > Reviewed-on: https://chromium-review.googlesource.com/1166203 > Commit-Queue: Hans Wennborg <hans@chromium.org> > Reviewed-by: Dirk Pranke <dpranke@chromium.org> > Reviewed-by: Nico Weber <thakis@chromium.org> > Cr-Commit-Position: refs/heads/master@{#581485} TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org # Not skipping CQ checks because original CL landed > 1 day ago. Bug: 843199, 804926, 330260 Change-Id: I136e405c84eba3f61a4ac96b2017a34ade0cfba6 Reviewed-on: https://chromium-review.googlesource.com/1179281 Commit-Queue: Nico Weber <thakis@chromium.org> Reviewed-by: Nico Weber <thakis@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#583945} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 967d1f1e6adc9b7641a5b0174c48f0883e293880
2018-08-17 05:42:02 +03:00
"""Gets the approximate build date given the specific build type.
Don't quantize build timestamps on official builds Official builds - the PDBs and the PE files - often get published to a symbol server. For PE files the path name in the symbol server is: "%s\%08X%s\%s" % (peName, timeStamp, imageSize, peName) Since the peName for a particular DLL/EXE never changes and since the size often doesn't change this means that the timeStamp is the only differentiator between nearby builds. With the strategy of setting the build timestamp to 5 am of the last commit time it is easy to get a build that overwrites the previous build on the symbol server. This happened when build 75.0.3770.143 overwrote all of the PE files from build 75.0.3770.142, which complicated the investigation of (restricted view, sorry) crbug.com/964273. This probably happened many other times. The PDB files were never overwritten which is why this was not noticed earlier. When the 5 am quantization was added we were using the current time for the build timestamps. Now that we are using the last commit time it is less important to quantize to 5 am so this change removes that quantization, for official builds only. An increased number of days where we do multiple builds of one channel means that this issue is hit more frequently than when the quantization was initially added. Bug: 993509 Change-Id: Ibfac95569b713ede056d3ff070db0c05b4a38c77 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1754527 Reviewed-by: Nico Weber <thakis@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Commit-Queue: Bruce Dawson <brucedawson@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#687349} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 02f0edc3fb60b55db652fce1c298beda75e7d1d9
2019-08-15 21:23:05 +03:00
>>> GetUnofficialBuildDate(datetime.datetime(2016, 2, 6, 1, 2, 3))
datetime.datetime(2016, 1, 3, 5, 0)
>>> GetUnofficialBuildDate(datetime.datetime(2016, 2, 7, 5))
Reland "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time" This reverts commit 2df0c83aee0a00bc0f7c26a22b68b4ea3eefef80. Reason for revert: official android should be fine with this after #583420 Original change's description: > Revert "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time" > > This reverts commit ef36dc19986a90f49d0d31520c88d310d72bd038. > > Reason for revert: This turns out to break the official android build, which apparently was relying on a broken aspect of the build that this fixed :(. See https://crbug.com/871173. > > Original change's description: > > win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time > > > > We used to set the timestamp to a hash of the binary, similar to > > https://blogs.msdn.microsoft.com/oldnewthing/20180103-00/?p=97705 > > However, that caused an appcompat warning on Windows 7 to appear, which > > interpreted the hash as a timestamp. (It's possible that https://llvm.org/PR38429 > > could help with that, but my guess it won't have an effect on Windows 7, > > which likely always believes that the the coff timestamp field always stores > > a timestamp). > > > > So currently we write the current time during linking in that field, but that's > > bad for build determinism and that in turn is bad for swarming test result cachability. > > > > build/write_build_date_header.py already creates a deterministic BUILD_DATE > > with several tradeoffs. Cachability wants this to change infrequently, but > > things like HSTS need a "real" build date and want this to change frequently. > > The compromise is: The date changes once per day in official builds, and > > once a month in regular builds. > > > > (We could use /Brepro in ldflags instead of /TIMESTAMP for unofficial builds to get > > the binary hash in the timestamp, but having the header timestamp match the BUILD_DATE > > define seems nice.) > > > > So let's use that same time as timestamp in the PE/COFF header. lld-link has a > > /TIMESTAMP: flag we can use to pass in an explicit timestamp. > > > > Since tools can't have deps, we need to compute the timestamp at gn time, > > so split write_build_date_header.py in two pieces: build/compute_build_timestamp.py > > that just prints the timestamp we want to use, and the old write_build_date_header.py, which > > now takes that timestamp and writes the header file. > > > > Call compute_build_timestamp.py at gn time so that we can pass it in ldflags, and > > pass the resultl to write_build_date_header.py which keeps running as an action > > during build time (so that we at least don't need to write a file at gn time). > > > > An additional wrinkle here is that the PE/COFF timestamp is used as one of just two > > keys per binary for uploading PE binaries to the symbol server, the other being file size. > > https://bugs.llvm.org/show_bug.cgi?id=35914#c0 has a good description of this, and > > tools/symsrc/img_fingerprint.py's GetImgFingerprint() is our implementation of it. > > But since we only upload binaries with symbols for official chrome builds to the symbol server, > > a timestamp that changes once a day should be still enough. (32-bit and 64-bit chromes > > have the same filename, and we might rarely build canary and beta and stable all on the > > same day, but them all being the same size seems highly unlikely.) > > > > Bug: 843199,804926,330260 > > Change-Id: I1d4193cc537ae0c4b2d6ac9281fad29de754dd6c > > Reviewed-on: https://chromium-review.googlesource.com/1161104 > > Reviewed-by: Dirk Pranke <dpranke@chromium.org> > > Reviewed-by: Hans Wennborg <hans@chromium.org> > > Commit-Queue: Nico Weber <thakis@chromium.org> > > Cr-Commit-Position: refs/heads/master@{#580585} > > TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org > NOTRY=true > > # Not skipping CQ checks because original CL landed > 1 day ago. > > Bug: 843199, 804926, 330260 > Change-Id: Ib93697a82f8a9d3fb303b763609e82e0612887cd > Reviewed-on: https://chromium-review.googlesource.com/1166203 > Commit-Queue: Hans Wennborg <hans@chromium.org> > Reviewed-by: Dirk Pranke <dpranke@chromium.org> > Reviewed-by: Nico Weber <thakis@chromium.org> > Cr-Commit-Position: refs/heads/master@{#581485} TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org # Not skipping CQ checks because original CL landed > 1 day ago. Bug: 843199, 804926, 330260 Change-Id: I136e405c84eba3f61a4ac96b2017a34ade0cfba6 Reviewed-on: https://chromium-review.googlesource.com/1179281 Commit-Queue: Nico Weber <thakis@chromium.org> Reviewed-by: Nico Weber <thakis@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#583945} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 967d1f1e6adc9b7641a5b0174c48f0883e293880
2018-08-17 05:42:02 +03:00
datetime.datetime(2016, 2, 7, 5, 0)
Don't quantize build timestamps on official builds Official builds - the PDBs and the PE files - often get published to a symbol server. For PE files the path name in the symbol server is: "%s\%08X%s\%s" % (peName, timeStamp, imageSize, peName) Since the peName for a particular DLL/EXE never changes and since the size often doesn't change this means that the timeStamp is the only differentiator between nearby builds. With the strategy of setting the build timestamp to 5 am of the last commit time it is easy to get a build that overwrites the previous build on the symbol server. This happened when build 75.0.3770.143 overwrote all of the PE files from build 75.0.3770.142, which complicated the investigation of (restricted view, sorry) crbug.com/964273. This probably happened many other times. The PDB files were never overwritten which is why this was not noticed earlier. When the 5 am quantization was added we were using the current time for the build timestamps. Now that we are using the last commit time it is less important to quantize to 5 am so this change removes that quantization, for official builds only. An increased number of days where we do multiple builds of one channel means that this issue is hit more frequently than when the quantization was initially added. Bug: 993509 Change-Id: Ibfac95569b713ede056d3ff070db0c05b4a38c77 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1754527 Reviewed-by: Nico Weber <thakis@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Commit-Queue: Bruce Dawson <brucedawson@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#687349} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 02f0edc3fb60b55db652fce1c298beda75e7d1d9
2019-08-15 21:23:05 +03:00
>>> GetUnofficialBuildDate(datetime.datetime(2016, 2, 8, 5))
Reland "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time" This reverts commit 2df0c83aee0a00bc0f7c26a22b68b4ea3eefef80. Reason for revert: official android should be fine with this after #583420 Original change's description: > Revert "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time" > > This reverts commit ef36dc19986a90f49d0d31520c88d310d72bd038. > > Reason for revert: This turns out to break the official android build, which apparently was relying on a broken aspect of the build that this fixed :(. See https://crbug.com/871173. > > Original change's description: > > win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time > > > > We used to set the timestamp to a hash of the binary, similar to > > https://blogs.msdn.microsoft.com/oldnewthing/20180103-00/?p=97705 > > However, that caused an appcompat warning on Windows 7 to appear, which > > interpreted the hash as a timestamp. (It's possible that https://llvm.org/PR38429 > > could help with that, but my guess it won't have an effect on Windows 7, > > which likely always believes that the the coff timestamp field always stores > > a timestamp). > > > > So currently we write the current time during linking in that field, but that's > > bad for build determinism and that in turn is bad for swarming test result cachability. > > > > build/write_build_date_header.py already creates a deterministic BUILD_DATE > > with several tradeoffs. Cachability wants this to change infrequently, but > > things like HSTS need a "real" build date and want this to change frequently. > > The compromise is: The date changes once per day in official builds, and > > once a month in regular builds. > > > > (We could use /Brepro in ldflags instead of /TIMESTAMP for unofficial builds to get > > the binary hash in the timestamp, but having the header timestamp match the BUILD_DATE > > define seems nice.) > > > > So let's use that same time as timestamp in the PE/COFF header. lld-link has a > > /TIMESTAMP: flag we can use to pass in an explicit timestamp. > > > > Since tools can't have deps, we need to compute the timestamp at gn time, > > so split write_build_date_header.py in two pieces: build/compute_build_timestamp.py > > that just prints the timestamp we want to use, and the old write_build_date_header.py, which > > now takes that timestamp and writes the header file. > > > > Call compute_build_timestamp.py at gn time so that we can pass it in ldflags, and > > pass the resultl to write_build_date_header.py which keeps running as an action > > during build time (so that we at least don't need to write a file at gn time). > > > > An additional wrinkle here is that the PE/COFF timestamp is used as one of just two > > keys per binary for uploading PE binaries to the symbol server, the other being file size. > > https://bugs.llvm.org/show_bug.cgi?id=35914#c0 has a good description of this, and > > tools/symsrc/img_fingerprint.py's GetImgFingerprint() is our implementation of it. > > But since we only upload binaries with symbols for official chrome builds to the symbol server, > > a timestamp that changes once a day should be still enough. (32-bit and 64-bit chromes > > have the same filename, and we might rarely build canary and beta and stable all on the > > same day, but them all being the same size seems highly unlikely.) > > > > Bug: 843199,804926,330260 > > Change-Id: I1d4193cc537ae0c4b2d6ac9281fad29de754dd6c > > Reviewed-on: https://chromium-review.googlesource.com/1161104 > > Reviewed-by: Dirk Pranke <dpranke@chromium.org> > > Reviewed-by: Hans Wennborg <hans@chromium.org> > > Commit-Queue: Nico Weber <thakis@chromium.org> > > Cr-Commit-Position: refs/heads/master@{#580585} > > TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org > NOTRY=true > > # Not skipping CQ checks because original CL landed > 1 day ago. > > Bug: 843199, 804926, 330260 > Change-Id: Ib93697a82f8a9d3fb303b763609e82e0612887cd > Reviewed-on: https://chromium-review.googlesource.com/1166203 > Commit-Queue: Hans Wennborg <hans@chromium.org> > Reviewed-by: Dirk Pranke <dpranke@chromium.org> > Reviewed-by: Nico Weber <thakis@chromium.org> > Cr-Commit-Position: refs/heads/master@{#581485} TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org # Not skipping CQ checks because original CL landed > 1 day ago. Bug: 843199, 804926, 330260 Change-Id: I136e405c84eba3f61a4ac96b2017a34ade0cfba6 Reviewed-on: https://chromium-review.googlesource.com/1179281 Commit-Queue: Nico Weber <thakis@chromium.org> Reviewed-by: Nico Weber <thakis@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#583945} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 967d1f1e6adc9b7641a5b0174c48f0883e293880
2018-08-17 05:42:02 +03:00
datetime.datetime(2016, 2, 7, 5, 0)
"""
Don't quantize build timestamps on official builds Official builds - the PDBs and the PE files - often get published to a symbol server. For PE files the path name in the symbol server is: "%s\%08X%s\%s" % (peName, timeStamp, imageSize, peName) Since the peName for a particular DLL/EXE never changes and since the size often doesn't change this means that the timeStamp is the only differentiator between nearby builds. With the strategy of setting the build timestamp to 5 am of the last commit time it is easy to get a build that overwrites the previous build on the symbol server. This happened when build 75.0.3770.143 overwrote all of the PE files from build 75.0.3770.142, which complicated the investigation of (restricted view, sorry) crbug.com/964273. This probably happened many other times. The PDB files were never overwritten which is why this was not noticed earlier. When the 5 am quantization was added we were using the current time for the build timestamps. Now that we are using the last commit time it is less important to quantize to 5 am so this change removes that quantization, for official builds only. An increased number of days where we do multiple builds of one channel means that this issue is hit more frequently than when the quantization was initially added. Bug: 993509 Change-Id: Ibfac95569b713ede056d3ff070db0c05b4a38c77 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1754527 Reviewed-by: Nico Weber <thakis@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Commit-Queue: Bruce Dawson <brucedawson@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#687349} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 02f0edc3fb60b55db652fce1c298beda75e7d1d9
2019-08-15 21:23:05 +03:00
if build_date.hour < 5:
# The time is locked at 5:00 am in UTC to cause the build cache
# invalidation to not happen exactly at midnight. Use the same calculation
# as the day before.
# See //base/build_time.cc.
build_date = build_date - datetime.timedelta(days=1)
build_date = datetime.datetime(build_date.year, build_date.month,
build_date.day, 5, 0, 0)
day = build_date.day
month = build_date.month
year = build_date.year
first_sunday = GetFirstSundayOfMonth(year, month)
# If our build is after the first Sunday, we've already refreshed our build
# cache on a quiet day, so just use that day.
# Otherwise, take the first Sunday of the previous month.
if day >= first_sunday:
day = first_sunday
else:
month -= 1
if month == 0:
month = 12
year -= 1
day = GetFirstSundayOfMonth(year, month)
Reland "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time" This reverts commit 2df0c83aee0a00bc0f7c26a22b68b4ea3eefef80. Reason for revert: official android should be fine with this after #583420 Original change's description: > Revert "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time" > > This reverts commit ef36dc19986a90f49d0d31520c88d310d72bd038. > > Reason for revert: This turns out to break the official android build, which apparently was relying on a broken aspect of the build that this fixed :(. See https://crbug.com/871173. > > Original change's description: > > win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time > > > > We used to set the timestamp to a hash of the binary, similar to > > https://blogs.msdn.microsoft.com/oldnewthing/20180103-00/?p=97705 > > However, that caused an appcompat warning on Windows 7 to appear, which > > interpreted the hash as a timestamp. (It's possible that https://llvm.org/PR38429 > > could help with that, but my guess it won't have an effect on Windows 7, > > which likely always believes that the the coff timestamp field always stores > > a timestamp). > > > > So currently we write the current time during linking in that field, but that's > > bad for build determinism and that in turn is bad for swarming test result cachability. > > > > build/write_build_date_header.py already creates a deterministic BUILD_DATE > > with several tradeoffs. Cachability wants this to change infrequently, but > > things like HSTS need a "real" build date and want this to change frequently. > > The compromise is: The date changes once per day in official builds, and > > once a month in regular builds. > > > > (We could use /Brepro in ldflags instead of /TIMESTAMP for unofficial builds to get > > the binary hash in the timestamp, but having the header timestamp match the BUILD_DATE > > define seems nice.) > > > > So let's use that same time as timestamp in the PE/COFF header. lld-link has a > > /TIMESTAMP: flag we can use to pass in an explicit timestamp. > > > > Since tools can't have deps, we need to compute the timestamp at gn time, > > so split write_build_date_header.py in two pieces: build/compute_build_timestamp.py > > that just prints the timestamp we want to use, and the old write_build_date_header.py, which > > now takes that timestamp and writes the header file. > > > > Call compute_build_timestamp.py at gn time so that we can pass it in ldflags, and > > pass the resultl to write_build_date_header.py which keeps running as an action > > during build time (so that we at least don't need to write a file at gn time). > > > > An additional wrinkle here is that the PE/COFF timestamp is used as one of just two > > keys per binary for uploading PE binaries to the symbol server, the other being file size. > > https://bugs.llvm.org/show_bug.cgi?id=35914#c0 has a good description of this, and > > tools/symsrc/img_fingerprint.py's GetImgFingerprint() is our implementation of it. > > But since we only upload binaries with symbols for official chrome builds to the symbol server, > > a timestamp that changes once a day should be still enough. (32-bit and 64-bit chromes > > have the same filename, and we might rarely build canary and beta and stable all on the > > same day, but them all being the same size seems highly unlikely.) > > > > Bug: 843199,804926,330260 > > Change-Id: I1d4193cc537ae0c4b2d6ac9281fad29de754dd6c > > Reviewed-on: https://chromium-review.googlesource.com/1161104 > > Reviewed-by: Dirk Pranke <dpranke@chromium.org> > > Reviewed-by: Hans Wennborg <hans@chromium.org> > > Commit-Queue: Nico Weber <thakis@chromium.org> > > Cr-Commit-Position: refs/heads/master@{#580585} > > TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org > NOTRY=true > > # Not skipping CQ checks because original CL landed > 1 day ago. > > Bug: 843199, 804926, 330260 > Change-Id: Ib93697a82f8a9d3fb303b763609e82e0612887cd > Reviewed-on: https://chromium-review.googlesource.com/1166203 > Commit-Queue: Hans Wennborg <hans@chromium.org> > Reviewed-by: Dirk Pranke <dpranke@chromium.org> > Reviewed-by: Nico Weber <thakis@chromium.org> > Cr-Commit-Position: refs/heads/master@{#581485} TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org # Not skipping CQ checks because original CL landed > 1 day ago. Bug: 843199, 804926, 330260 Change-Id: I136e405c84eba3f61a4ac96b2017a34ade0cfba6 Reviewed-on: https://chromium-review.googlesource.com/1179281 Commit-Queue: Nico Weber <thakis@chromium.org> Reviewed-by: Nico Weber <thakis@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#583945} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 967d1f1e6adc9b7641a5b0174c48f0883e293880
2018-08-17 05:42:02 +03:00
return datetime.datetime(
Don't quantize build timestamps on official builds Official builds - the PDBs and the PE files - often get published to a symbol server. For PE files the path name in the symbol server is: "%s\%08X%s\%s" % (peName, timeStamp, imageSize, peName) Since the peName for a particular DLL/EXE never changes and since the size often doesn't change this means that the timeStamp is the only differentiator between nearby builds. With the strategy of setting the build timestamp to 5 am of the last commit time it is easy to get a build that overwrites the previous build on the symbol server. This happened when build 75.0.3770.143 overwrote all of the PE files from build 75.0.3770.142, which complicated the investigation of (restricted view, sorry) crbug.com/964273. This probably happened many other times. The PDB files were never overwritten which is why this was not noticed earlier. When the 5 am quantization was added we were using the current time for the build timestamps. Now that we are using the last commit time it is less important to quantize to 5 am so this change removes that quantization, for official builds only. An increased number of days where we do multiple builds of one channel means that this issue is hit more frequently than when the quantization was initially added. Bug: 993509 Change-Id: Ibfac95569b713ede056d3ff070db0c05b4a38c77 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1754527 Reviewed-by: Nico Weber <thakis@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Commit-Queue: Bruce Dawson <brucedawson@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#687349} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 02f0edc3fb60b55db652fce1c298beda75e7d1d9
2019-08-15 21:23:05 +03:00
year, month, day, build_date.hour, build_date.minute, build_date.second)
Reland "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time" This reverts commit 2df0c83aee0a00bc0f7c26a22b68b4ea3eefef80. Reason for revert: official android should be fine with this after #583420 Original change's description: > Revert "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time" > > This reverts commit ef36dc19986a90f49d0d31520c88d310d72bd038. > > Reason for revert: This turns out to break the official android build, which apparently was relying on a broken aspect of the build that this fixed :(. See https://crbug.com/871173. > > Original change's description: > > win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time > > > > We used to set the timestamp to a hash of the binary, similar to > > https://blogs.msdn.microsoft.com/oldnewthing/20180103-00/?p=97705 > > However, that caused an appcompat warning on Windows 7 to appear, which > > interpreted the hash as a timestamp. (It's possible that https://llvm.org/PR38429 > > could help with that, but my guess it won't have an effect on Windows 7, > > which likely always believes that the the coff timestamp field always stores > > a timestamp). > > > > So currently we write the current time during linking in that field, but that's > > bad for build determinism and that in turn is bad for swarming test result cachability. > > > > build/write_build_date_header.py already creates a deterministic BUILD_DATE > > with several tradeoffs. Cachability wants this to change infrequently, but > > things like HSTS need a "real" build date and want this to change frequently. > > The compromise is: The date changes once per day in official builds, and > > once a month in regular builds. > > > > (We could use /Brepro in ldflags instead of /TIMESTAMP for unofficial builds to get > > the binary hash in the timestamp, but having the header timestamp match the BUILD_DATE > > define seems nice.) > > > > So let's use that same time as timestamp in the PE/COFF header. lld-link has a > > /TIMESTAMP: flag we can use to pass in an explicit timestamp. > > > > Since tools can't have deps, we need to compute the timestamp at gn time, > > so split write_build_date_header.py in two pieces: build/compute_build_timestamp.py > > that just prints the timestamp we want to use, and the old write_build_date_header.py, which > > now takes that timestamp and writes the header file. > > > > Call compute_build_timestamp.py at gn time so that we can pass it in ldflags, and > > pass the resultl to write_build_date_header.py which keeps running as an action > > during build time (so that we at least don't need to write a file at gn time). > > > > An additional wrinkle here is that the PE/COFF timestamp is used as one of just two > > keys per binary for uploading PE binaries to the symbol server, the other being file size. > > https://bugs.llvm.org/show_bug.cgi?id=35914#c0 has a good description of this, and > > tools/symsrc/img_fingerprint.py's GetImgFingerprint() is our implementation of it. > > But since we only upload binaries with symbols for official chrome builds to the symbol server, > > a timestamp that changes once a day should be still enough. (32-bit and 64-bit chromes > > have the same filename, and we might rarely build canary and beta and stable all on the > > same day, but them all being the same size seems highly unlikely.) > > > > Bug: 843199,804926,330260 > > Change-Id: I1d4193cc537ae0c4b2d6ac9281fad29de754dd6c > > Reviewed-on: https://chromium-review.googlesource.com/1161104 > > Reviewed-by: Dirk Pranke <dpranke@chromium.org> > > Reviewed-by: Hans Wennborg <hans@chromium.org> > > Commit-Queue: Nico Weber <thakis@chromium.org> > > Cr-Commit-Position: refs/heads/master@{#580585} > > TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org > NOTRY=true > > # Not skipping CQ checks because original CL landed > 1 day ago. > > Bug: 843199, 804926, 330260 > Change-Id: Ib93697a82f8a9d3fb303b763609e82e0612887cd > Reviewed-on: https://chromium-review.googlesource.com/1166203 > Commit-Queue: Hans Wennborg <hans@chromium.org> > Reviewed-by: Dirk Pranke <dpranke@chromium.org> > Reviewed-by: Nico Weber <thakis@chromium.org> > Cr-Commit-Position: refs/heads/master@{#581485} TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org # Not skipping CQ checks because original CL landed > 1 day ago. Bug: 843199, 804926, 330260 Change-Id: I136e405c84eba3f61a4ac96b2017a34ade0cfba6 Reviewed-on: https://chromium-review.googlesource.com/1179281 Commit-Queue: Nico Weber <thakis@chromium.org> Reviewed-by: Nico Weber <thakis@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#583945} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 967d1f1e6adc9b7641a5b0174c48f0883e293880
2018-08-17 05:42:02 +03:00
def main():
if doctest.testmod()[0]:
return 1
argument_parser = argparse.ArgumentParser()
argument_parser.add_argument(
'build_type', help='The type of build', choices=('official', 'default'))
args = argument_parser.parse_args()
# The mtime of the revision in build/util/LASTCHANGE is stored in a file
# next to it. Read it, to get a deterministic time close to "now".
# That date is then modified as described at the top of the file so that
# it changes less frequently than with every commit.
# This intentionally always uses build/util/LASTCHANGE's commit time even if
# use_dummy_lastchange is set.
lastchange_file = os.path.join(THIS_DIR, 'util', 'LASTCHANGE.committime')
last_commit_timestamp = int(open(lastchange_file).read())
Don't quantize build timestamps on official builds Official builds - the PDBs and the PE files - often get published to a symbol server. For PE files the path name in the symbol server is: "%s\%08X%s\%s" % (peName, timeStamp, imageSize, peName) Since the peName for a particular DLL/EXE never changes and since the size often doesn't change this means that the timeStamp is the only differentiator between nearby builds. With the strategy of setting the build timestamp to 5 am of the last commit time it is easy to get a build that overwrites the previous build on the symbol server. This happened when build 75.0.3770.143 overwrote all of the PE files from build 75.0.3770.142, which complicated the investigation of (restricted view, sorry) crbug.com/964273. This probably happened many other times. The PDB files were never overwritten which is why this was not noticed earlier. When the 5 am quantization was added we were using the current time for the build timestamps. Now that we are using the last commit time it is less important to quantize to 5 am so this change removes that quantization, for official builds only. An increased number of days where we do multiple builds of one channel means that this issue is hit more frequently than when the quantization was initially added. Bug: 993509 Change-Id: Ibfac95569b713ede056d3ff070db0c05b4a38c77 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1754527 Reviewed-by: Nico Weber <thakis@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Commit-Queue: Bruce Dawson <brucedawson@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#687349} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 02f0edc3fb60b55db652fce1c298beda75e7d1d9
2019-08-15 21:23:05 +03:00
build_date = datetime.datetime.utcfromtimestamp(last_commit_timestamp)
# For official builds we want full fidelity time stamps because official
# builds are typically added to symbol servers and Windows symbol servers
# use the link timestamp as the prime differentiator, but for unofficial
# builds we do lots of quantization to avoid churn.
if args.build_type != 'official':
build_date = GetUnofficialBuildDate(build_date)
print(int(calendar.timegm(build_date.utctimetuple())))
Reland "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time" This reverts commit 2df0c83aee0a00bc0f7c26a22b68b4ea3eefef80. Reason for revert: official android should be fine with this after #583420 Original change's description: > Revert "win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time" > > This reverts commit ef36dc19986a90f49d0d31520c88d310d72bd038. > > Reason for revert: This turns out to break the official android build, which apparently was relying on a broken aspect of the build that this fixed :(. See https://crbug.com/871173. > > Original change's description: > > win: write a deterministic-ish timestamp into the PE/COFF header instead of the current time > > > > We used to set the timestamp to a hash of the binary, similar to > > https://blogs.msdn.microsoft.com/oldnewthing/20180103-00/?p=97705 > > However, that caused an appcompat warning on Windows 7 to appear, which > > interpreted the hash as a timestamp. (It's possible that https://llvm.org/PR38429 > > could help with that, but my guess it won't have an effect on Windows 7, > > which likely always believes that the the coff timestamp field always stores > > a timestamp). > > > > So currently we write the current time during linking in that field, but that's > > bad for build determinism and that in turn is bad for swarming test result cachability. > > > > build/write_build_date_header.py already creates a deterministic BUILD_DATE > > with several tradeoffs. Cachability wants this to change infrequently, but > > things like HSTS need a "real" build date and want this to change frequently. > > The compromise is: The date changes once per day in official builds, and > > once a month in regular builds. > > > > (We could use /Brepro in ldflags instead of /TIMESTAMP for unofficial builds to get > > the binary hash in the timestamp, but having the header timestamp match the BUILD_DATE > > define seems nice.) > > > > So let's use that same time as timestamp in the PE/COFF header. lld-link has a > > /TIMESTAMP: flag we can use to pass in an explicit timestamp. > > > > Since tools can't have deps, we need to compute the timestamp at gn time, > > so split write_build_date_header.py in two pieces: build/compute_build_timestamp.py > > that just prints the timestamp we want to use, and the old write_build_date_header.py, which > > now takes that timestamp and writes the header file. > > > > Call compute_build_timestamp.py at gn time so that we can pass it in ldflags, and > > pass the resultl to write_build_date_header.py which keeps running as an action > > during build time (so that we at least don't need to write a file at gn time). > > > > An additional wrinkle here is that the PE/COFF timestamp is used as one of just two > > keys per binary for uploading PE binaries to the symbol server, the other being file size. > > https://bugs.llvm.org/show_bug.cgi?id=35914#c0 has a good description of this, and > > tools/symsrc/img_fingerprint.py's GetImgFingerprint() is our implementation of it. > > But since we only upload binaries with symbols for official chrome builds to the symbol server, > > a timestamp that changes once a day should be still enough. (32-bit and 64-bit chromes > > have the same filename, and we might rarely build canary and beta and stable all on the > > same day, but them all being the same size seems highly unlikely.) > > > > Bug: 843199,804926,330260 > > Change-Id: I1d4193cc537ae0c4b2d6ac9281fad29de754dd6c > > Reviewed-on: https://chromium-review.googlesource.com/1161104 > > Reviewed-by: Dirk Pranke <dpranke@chromium.org> > > Reviewed-by: Hans Wennborg <hans@chromium.org> > > Commit-Queue: Nico Weber <thakis@chromium.org> > > Cr-Commit-Position: refs/heads/master@{#580585} > > TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org > NOTRY=true > > # Not skipping CQ checks because original CL landed > 1 day ago. > > Bug: 843199, 804926, 330260 > Change-Id: Ib93697a82f8a9d3fb303b763609e82e0612887cd > Reviewed-on: https://chromium-review.googlesource.com/1166203 > Commit-Queue: Hans Wennborg <hans@chromium.org> > Reviewed-by: Dirk Pranke <dpranke@chromium.org> > Reviewed-by: Nico Weber <thakis@chromium.org> > Cr-Commit-Position: refs/heads/master@{#581485} TBR=thakis@chromium.org,hans@chromium.org,dpranke@chromium.org # Not skipping CQ checks because original CL landed > 1 day ago. Bug: 843199, 804926, 330260 Change-Id: I136e405c84eba3f61a4ac96b2017a34ade0cfba6 Reviewed-on: https://chromium-review.googlesource.com/1179281 Commit-Queue: Nico Weber <thakis@chromium.org> Reviewed-by: Nico Weber <thakis@chromium.org> Reviewed-by: Dirk Pranke <dpranke@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#583945} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 967d1f1e6adc9b7641a5b0174c48f0883e293880
2018-08-17 05:42:02 +03:00
return 0
if __name__ == '__main__':
sys.exit(main())