Bug 1105128 - Avoid excessive path normalization in FileCopier.copy(); r=glandium

FileCopier.copy() was performing a lot of os.path.normpath() operations.
Profiling revealed that os.path.normpath() was the function with the
most wall time CPU usage when processing the tests manifests. Upon
subsequent examination of the code in question, all the paths being used
were already normalized. So, os.path.normpath() wasn't accomplishing
anything.

This patch results in ~300ms reduction in wall time to process the tests
install manifest on a fully populated page cache. Execution time drops
from ~2.8s to ~2.5s.

Profiling reveals that after this patch os.stat() is the #1 wall time
consumer. However, os.path.{join,dirname,normpath} still account for
~1.5x the wall time of os.stat(). There is still room to optimize
this function.

--HG--
extra : rebase_source : b6f0862baa5168c609499fd95eb3517854bc8cce
extra : amend_source : 7e04c1eb74132bbbe86e721f0f209b19309a7a51
This commit is contained in:
Gregory Szorc 2014-11-25 18:16:22 -08:00
Родитель 548d773d4e
Коммит d5a6121790
1 изменённых файлов: 6 добавлений и 6 удалений

Просмотреть файл

@ -297,21 +297,21 @@ class FileCopier(FileRegistry):
# removed and a directory created above.
if remove_all_directory_symlinks:
os.remove(full)
result.removed_files.add(os.path.normpath(full))
result.removed_files.add(full)
else:
existing_files.add(os.path.normpath(full))
existing_files.add(full)
else:
filtered.append(d)
dirs[:] = filtered
existing_dirs.add(os.path.normpath(root))
existing_dirs.add(root)
for d in dirs:
existing_dirs.add(os.path.normpath(os.path.join(root, d)))
existing_dirs.add(os.path.join(root, d))
for f in files:
existing_files.add(os.path.normpath(os.path.join(root, f)))
existing_files.add(os.path.join(root, f))
# Now we reconcile the state of the world against what we want.
@ -329,7 +329,7 @@ class FileCopier(FileRegistry):
# Install files.
for p, f in self:
destfile = os.path.normpath(os.path.join(destination, p))
destfile = os.path.join(destination, p)
if f.copy(destfile, skip_if_older):
result.updated_files.add(destfile)
else: