chromium-src-build/toolchain/get_cpu_count.py

24 строки
474 B
Python
Исходник Обычный вид История

Use action pool for non-goma tasks Invoking cpu intensive python processes more than machine cores has some overhead on Mac(crbug.com/695864) and Win. This change introduces pool mainly for python generator to restrict the number of running process when we specify large parallelism with goma. I took 3 time build stats using target generate_bindings_modules_v8_interfaces and generate_bindings_core_v8_interfaces which have 1148 python tasks. With this CL on z840 windows10 TotalSeconds : 18.2953436 TotalSeconds : 18.6283626 TotalSeconds : 19.2731436 Without this CL on Z840 windows10 TotalSeconds : 23.8277797 TotalSeconds : 23.6952018 TotalSeconds : 23.0853999 Linux looks to have good task scheduler. With this CL on z840 linux 0m9.067s 0m8.771s 0m8.953s Without this CL on Z840 linux 0m8.998s 0m9.022s 0m8.958s Also this improves UI's responsiveness when we are building chrome on windows. Stats of clean chrome build in each major OS is like below. 5 time clean build of chrome on Z840 windows 10 with -j1000 and warm goma backend cache is like below. With this CL 333.3425057 317.4724857 305.0217898 317.8907203 305.1031952 Avg: 315.76613934 Without this CL 369.9731363 331.296758 329.0041556 329.1472297 333.3883952 Avg: 338.56193496 5 time clean build of chrome on Z840 linux with -j1000 and warm goma backend cache is like below. With this CL 90.42 87.91 90.45 90.50 89.02 avg: 89.66 Without this CL 89.52 86.34 86.08 85.67 85.89 avg: 86.7 3 time clean build of chrome on 24 thread Mac Pro with -j500 and warm goma backend cache is like below. With this CL 638.28 627.28 624.69 avg: 630.083 Without this CL 667.52 663.83 655.95 avg: 662.433 Bug: 695864 Change-Id: I6838c0f71b8d8030e6eab58b2990810aaa997dfa Reviewed-on: https://chromium-review.googlesource.com/882581 Reviewed-by: Dirk Pranke <dpranke@chromium.org> Commit-Queue: Takuto Ikuta <tikuta@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#535589} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 2c098797567226115653c2567dbcfc72ee5af5ae
2018-02-09 03:21:48 +03:00
# Copyright 2018 The Chromium Authors. All rights reserved.
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.
# This script shows cpu count to specify capacity of action pool.
from __future__ import print_function
Use action pool for non-goma tasks Invoking cpu intensive python processes more than machine cores has some overhead on Mac(crbug.com/695864) and Win. This change introduces pool mainly for python generator to restrict the number of running process when we specify large parallelism with goma. I took 3 time build stats using target generate_bindings_modules_v8_interfaces and generate_bindings_core_v8_interfaces which have 1148 python tasks. With this CL on z840 windows10 TotalSeconds : 18.2953436 TotalSeconds : 18.6283626 TotalSeconds : 19.2731436 Without this CL on Z840 windows10 TotalSeconds : 23.8277797 TotalSeconds : 23.6952018 TotalSeconds : 23.0853999 Linux looks to have good task scheduler. With this CL on z840 linux 0m9.067s 0m8.771s 0m8.953s Without this CL on Z840 linux 0m8.998s 0m9.022s 0m8.958s Also this improves UI's responsiveness when we are building chrome on windows. Stats of clean chrome build in each major OS is like below. 5 time clean build of chrome on Z840 windows 10 with -j1000 and warm goma backend cache is like below. With this CL 333.3425057 317.4724857 305.0217898 317.8907203 305.1031952 Avg: 315.76613934 Without this CL 369.9731363 331.296758 329.0041556 329.1472297 333.3883952 Avg: 338.56193496 5 time clean build of chrome on Z840 linux with -j1000 and warm goma backend cache is like below. With this CL 90.42 87.91 90.45 90.50 89.02 avg: 89.66 Without this CL 89.52 86.34 86.08 85.67 85.89 avg: 86.7 3 time clean build of chrome on 24 thread Mac Pro with -j500 and warm goma backend cache is like below. With this CL 638.28 627.28 624.69 avg: 630.083 Without this CL 667.52 663.83 655.95 avg: 662.433 Bug: 695864 Change-Id: I6838c0f71b8d8030e6eab58b2990810aaa997dfa Reviewed-on: https://chromium-review.googlesource.com/882581 Reviewed-by: Dirk Pranke <dpranke@chromium.org> Commit-Queue: Takuto Ikuta <tikuta@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#535589} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 2c098797567226115653c2567dbcfc72ee5af5ae
2018-02-09 03:21:48 +03:00
import multiprocessing
import sys
def main():
try:
cpu_count = multiprocessing.cpu_count()
except:
cpu_count = 1
print(cpu_count)
Use action pool for non-goma tasks Invoking cpu intensive python processes more than machine cores has some overhead on Mac(crbug.com/695864) and Win. This change introduces pool mainly for python generator to restrict the number of running process when we specify large parallelism with goma. I took 3 time build stats using target generate_bindings_modules_v8_interfaces and generate_bindings_core_v8_interfaces which have 1148 python tasks. With this CL on z840 windows10 TotalSeconds : 18.2953436 TotalSeconds : 18.6283626 TotalSeconds : 19.2731436 Without this CL on Z840 windows10 TotalSeconds : 23.8277797 TotalSeconds : 23.6952018 TotalSeconds : 23.0853999 Linux looks to have good task scheduler. With this CL on z840 linux 0m9.067s 0m8.771s 0m8.953s Without this CL on Z840 linux 0m8.998s 0m9.022s 0m8.958s Also this improves UI's responsiveness when we are building chrome on windows. Stats of clean chrome build in each major OS is like below. 5 time clean build of chrome on Z840 windows 10 with -j1000 and warm goma backend cache is like below. With this CL 333.3425057 317.4724857 305.0217898 317.8907203 305.1031952 Avg: 315.76613934 Without this CL 369.9731363 331.296758 329.0041556 329.1472297 333.3883952 Avg: 338.56193496 5 time clean build of chrome on Z840 linux with -j1000 and warm goma backend cache is like below. With this CL 90.42 87.91 90.45 90.50 89.02 avg: 89.66 Without this CL 89.52 86.34 86.08 85.67 85.89 avg: 86.7 3 time clean build of chrome on 24 thread Mac Pro with -j500 and warm goma backend cache is like below. With this CL 638.28 627.28 624.69 avg: 630.083 Without this CL 667.52 663.83 655.95 avg: 662.433 Bug: 695864 Change-Id: I6838c0f71b8d8030e6eab58b2990810aaa997dfa Reviewed-on: https://chromium-review.googlesource.com/882581 Reviewed-by: Dirk Pranke <dpranke@chromium.org> Commit-Queue: Takuto Ikuta <tikuta@chromium.org> Cr-Original-Commit-Position: refs/heads/master@{#535589} Cr-Mirrored-From: https://chromium.googlesource.com/chromium/src Cr-Mirrored-Commit: 2c098797567226115653c2567dbcfc72ee5af5ae
2018-02-09 03:21:48 +03:00
return 0
if __name__ == '__main__':
sys.exit(main())