gecko-dev/tools/rb/fix_stacks.py

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

102 строки
3.2 KiB
Python
Исходник Обычный вид История

#!/usr/bin/python
# vim:sw=4:ts=4:et:
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
# This script uses `fix-stacks` to post-process the entries produced by
# MozFormatCodeAddress().
from __future__ import absolute_import, print_function
from subprocess import Popen, PIPE
import atexit
import os
import platform
import re
import sys
import six
# Matches lines produced by MozFormatCodeAddress(), e.g.
# `#01: ???[tests/example +0x43a0]`.
line_re = re.compile("#\d+: .+\[.+ \+0x[0-9A-Fa-f]+\]")
fix_stacks = None
def fixSymbols(
line, jsonMode=False, slowWarning=False, breakpadSymsDir=None, hide_errors=False
):
global fix_stacks
line = six.ensure_str(line)
result = line_re.search(line)
if result is None:
return line
if not fix_stacks:
# Look in MOZ_FETCHES_DIR (for automation), then in MOZBUILD_STATE_PATH
# (for a local build where the user has that set), then in ~/.mozbuild
# (for a local build with default settings).
base = os.environ.get(
"MOZ_FETCHES_DIR",
os.environ.get("MOZBUILD_STATE_PATH", os.path.expanduser("~/.mozbuild")),
)
fix_stacks_exe = base + "/fix-stacks/fix-stacks"
if platform.system() == "Windows":
fix_stacks_exe = fix_stacks_exe + ".exe"
if not (os.path.isfile(fix_stacks_exe) and os.access(fix_stacks_exe, os.X_OK)):
raise Exception("cannot find `fix-stacks`; please run `./mach bootstrap`")
args = [fix_stacks_exe]
if jsonMode:
args.append("-j")
if breakpadSymsDir:
args.append("-b")
args.append(breakpadSymsDir)
Bug 1628527 - Introduce explicit initialization and finalization of `fix-stacks`. r=erahm,perftest-reviewers,sparky,gbrown Currently AWSY-with-DMD doesn't work on Windows. This is because `fix-stacks` is initialized lazily, and by the time the initialization happens some file descriptors for files are open, and that leads to some major Python2-on-Windows sadness as described in the big comment in the commit. To fix the problem, this commit adds an `init` function to `fix_stacks.py` so that `fix-stacks` can be initialized eagerly, hopefully before any file descriptors for files are open. For `dmd.py`, other than fixing the AWSY problems, this has little effect, because `fix-stacks` is always initialized. For `utils.py`, which is used to process the output of most tests, this has a more noticeable effect: the `fix-stacks` process is always spawned, rather than being spawned only when needed. If no stack traces appear in the test output, this means that `fix-stacks` is spawned unnecessarily. But it's cheap to spawn; the expensive part only happens when stack traces start getting fixed. So I think this change in behaviour is acceptable. Furthermore, the commit adds a `finish` function to `fix_stacks.py`, so that the `fix-stacks` process can be explicitly shut down. This has never been done for processes spawned for any of the stack fixing scripts. It's never caused problems on Linux/Mac, but it seems to be necessary on Windows to avoid similar "this file is locked" problems with the test_dmd.js test. The commit also renames some things to more standard Python style, e.g. `json_mode` instead of `jsonMode`. Finally, Android tests use `utils.py` from the repository but `fix_stacks.py` from the Android host utils. Because the two scripts must be updated in tandem, this commit also updates the Android host utils to a version that contains the updated `fix_stacks.py`. Thanks to aerickson for packaging up the new Android host utils and providing the change to the `hostutils.manifest` file. Differential Revision: https://phabricator.services.mozilla.com/D69478
2020-04-21 01:43:25 +03:00
# Sometimes we need to prevent errors from going to stderr.
stderr = open(os.devnull) if hide_errors else None
fix_stacks = Popen(
args, stdin=PIPE, stdout=PIPE, stderr=stderr, universal_newlines=True
)
Bug 1628527 - Introduce explicit initialization and finalization of `fix-stacks`. r=erahm,perftest-reviewers,sparky,gbrown Currently AWSY-with-DMD doesn't work on Windows. This is because `fix-stacks` is initialized lazily, and by the time the initialization happens some file descriptors for files are open, and that leads to some major Python2-on-Windows sadness as described in the big comment in the commit. To fix the problem, this commit adds an `init` function to `fix_stacks.py` so that `fix-stacks` can be initialized eagerly, hopefully before any file descriptors for files are open. For `dmd.py`, other than fixing the AWSY problems, this has little effect, because `fix-stacks` is always initialized. For `utils.py`, which is used to process the output of most tests, this has a more noticeable effect: the `fix-stacks` process is always spawned, rather than being spawned only when needed. If no stack traces appear in the test output, this means that `fix-stacks` is spawned unnecessarily. But it's cheap to spawn; the expensive part only happens when stack traces start getting fixed. So I think this change in behaviour is acceptable. Furthermore, the commit adds a `finish` function to `fix_stacks.py`, so that the `fix-stacks` process can be explicitly shut down. This has never been done for processes spawned for any of the stack fixing scripts. It's never caused problems on Linux/Mac, but it seems to be necessary on Windows to avoid similar "this file is locked" problems with the test_dmd.js test. The commit also renames some things to more standard Python style, e.g. `json_mode` instead of `jsonMode`. Finally, Android tests use `utils.py` from the repository but `fix_stacks.py` from the Android host utils. Because the two scripts must be updated in tandem, this commit also updates the Android host utils to a version that contains the updated `fix_stacks.py`. Thanks to aerickson for packaging up the new Android host utils and providing the change to the `hostutils.manifest` file. Differential Revision: https://phabricator.services.mozilla.com/D69478
2020-04-21 01:43:25 +03:00
# Shut down the fix_stacks process on exit. We use `terminate()`
# because it is more forceful than `wait()`, and the Python docs warn
# about possible deadlocks with `wait()`.
def cleanup(fix_stacks):
for fn in [fix_stacks.stdin.close, fix_stacks.terminate]:
try:
fn()
except OSError:
pass
atexit.register(cleanup, fix_stacks)
if slowWarning:
print(
"Initializing stack-fixing for the first stack frame, this may take a while..."
)
# Sometimes `line` is lacking a trailing newline. If we pass such a `line`
# to `fix-stacks` it will wait until it receives a newline, causing this
# script to hang. So we add a newline if one is missing and then remove it
# from the output.
is_missing_newline = not line.endswith("\n")
if is_missing_newline:
line = line + "\n"
fix_stacks.stdin.write(line)
fix_stacks.stdin.flush()
out = fix_stacks.stdout.readline()
if is_missing_newline:
out = out[:-1]
return out
if __name__ == "__main__":
for line in sys.stdin:
sys.stdout.write(fixSymbols(line))