зеркало из https://github.com/mozilla/gecko-dev.git
Bug 1438688, part 6 - Compile XPT information to C++ at build time. r=glandium,njn
This patch handles the actual generation of the static data structures used to represent XPT information. XPT files are generated in the same way as they are now, but they are used only as an intermediate representation to speed up incremental compilation rather than something used by Firefox itself. Instead of linking XPTs into a single big XPT file at packaging time, they are linked into a single big C++ file at build time, that defines the various static consts in XPTHeader. In xpt.py, every data structure that can get written to disk gets an additional code_gen() method that returns a representation of that data structure as C++ source code. CodeGenData aggregates this information together, handling deduplication and the final source code generation. The ctors are needed for XPTConstValue to statically initialize the different union cases without resorting to designated initializers, which are part of C99, not C++. Designated initializers appear to be supported in C++ code by Clang and GCC, but not MSVC. The ctors must be constexpr to ensure they are actually statically initialized so they can be shared between Firefox processes. I also removed an unnecessary "union" in XPTConstDescriptor. Together, these patches reduce the amount of memory reported by xpti-working-set from about 860,000 bytes to about 200,000 bytes. The remaining memory is used for xptiInterface and xptiTypelibGuts (which are thin wrappers around the XPT interfaces and header) and hash tables to speed up looking up interfaces by name or IID. That could potentially be eliminated from dynamic allocations in follow up work. These patches did not affect memory reporting because XPT arenas are still used by the remaining XPTI data structures. MozReview-Commit-ID: Jvi9ByCPa6H --HG-- extra : rebase_source : a9e48e7026aab4ad1b7f97e50424adf4e3f4142f
This commit is contained in:
Andrew McCreight
Родитель
da728cc408
Коммит
27f44d82d0
|
|
@ -30,6 +30,8 @@ dist_idl_dir := $(DIST)/idl
|
|||
dist_include_dir := $(DIST)/include
|
||||
dist_xpcrs_dir := $(DIST)/xpcrs
|
||||
process_py := $(topsrcdir)/python/mozbuild/mozbuild/action/xpidl-process.py
|
||||
generated_file := $(topobjdir)/xpcom/typelib/xpt/XPTInfo.cpp
|
||||
code_gen_py := $(topsrcdir)/xpcom/typelib/xpt/tools/xpt.py
|
||||
|
||||
# TODO we should use py_action, but that would require extra directories to be
|
||||
# in the virtualenv.
|
||||
|
|
@ -55,14 +57,20 @@ xpt_files := $(addsuffix .xpt,$(xpidl_modules))
|
|||
|
||||
depends_files := $(foreach root,$(xpidl_modules),$(idl_deps_dir)/$(root).pp)
|
||||
|
||||
GARBAGE += $(xpt_files) $(depends_files)
|
||||
GARBAGE += $(xpt_files) $(depends_files) $(generated_file)
|
||||
|
||||
ifdef COMPILE_ENVIRONMENT
|
||||
xpidl:: $(xpt_files)
|
||||
xpidl:: $(generated_file)
|
||||
endif
|
||||
|
||||
$(xpt_files): $(process_py) $(call mkdir_deps,$(idl_deps_dir) $(dist_include_dir) $(dist_xpcrs_dir))
|
||||
|
||||
$(generated_file): $(xpt_files) $(code_gen_py)
|
||||
$(REPORT_BUILD)
|
||||
$(PYTHON_PATH) $(PLY_INCLUDE) \
|
||||
$(code_gen_py) linkgen \
|
||||
$(generated_file) $(xpt_files)
|
||||
|
||||
-include $(depends_files)
|
||||
|
||||
define xpt_deps
|
||||
|
|
|
|||
|
|
@ -12,6 +12,10 @@ UNIFIED_SOURCES += [
|
|||
'xpt_arena.cpp',
|
||||
]
|
||||
|
||||
SOURCES += [
|
||||
'!XPTInfo.cpp',
|
||||
]
|
||||
|
||||
EXPORTS += [
|
||||
'xpt_arena.h',
|
||||
'xpt_struct.h',
|
||||
|
|
|
|||
|
|
@ -34,9 +34,9 @@
|
|||
A module for working with XPCOM Type Libraries.
|
||||
|
||||
The XPCOM Type Library File Format is described at:
|
||||
http://www.mozilla.org/scriptable/typelib_file.html . It is used
|
||||
to provide type information for calling methods on XPCOM objects
|
||||
from scripting languages such as JavaScript.
|
||||
https://www-archive.mozilla.org/scriptable/typelib_file.html
|
||||
It is used to provide type information for calling methods on XPCOM
|
||||
objects from scripting languages such as JavaScript.
|
||||
|
||||
This module provides a set of classes representing the parts of
|
||||
a typelib in a high-level manner, as well as methods for reading
|
||||
|
|
@ -134,6 +134,123 @@ class IndexedList(object):
|
|||
return len(self._list)
|
||||
|
||||
|
||||
class CodeGenData(object):
|
||||
"""
|
||||
This stores the top-level data needed to generate XPT information in C++.
|
||||
|methods| and |constants| are the top-level declarations in the module.
|
||||
These contain names, and so are not likely to benefit from deduplication.
|
||||
|params| are the lists of parameters for |methods|, stored concatenated.
|
||||
These are deduplicated if there are only a few. |types| and |strings| are
|
||||
side data stores for the other things, and are deduplicated.
|
||||
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self.interfaces = []
|
||||
|
||||
self.types = []
|
||||
self.type_indexes = {}
|
||||
|
||||
self.params = []
|
||||
self.params_indexes = {}
|
||||
|
||||
self.methods = []
|
||||
|
||||
self.constants = []
|
||||
|
||||
self.strings = []
|
||||
self.string_indexes = {}
|
||||
self.curr_string_index = 0
|
||||
|
||||
@staticmethod
|
||||
def write_array_body(fd, iterator):
|
||||
fd.write("{\n")
|
||||
for s in iterator:
|
||||
fd.write(" %s,\n" % s)
|
||||
fd.write("};\n\n")
|
||||
|
||||
def finish(self, fd):
|
||||
fd.write("const uint16_t XPTHeader::kNumInterfaces = %s;\n\n" % len(self.interfaces))
|
||||
|
||||
fd.write("const XPTInterfaceDescriptor XPTHeader::kInterfaces[] = ")
|
||||
CodeGenData.write_array_body(fd, self.interfaces)
|
||||
|
||||
fd.write("const XPTTypeDescriptor XPTHeader::kTypes[] = ")
|
||||
CodeGenData.write_array_body(fd, self.types)
|
||||
|
||||
fd.write("const XPTParamDescriptor XPTHeader::kParams[] = ")
|
||||
CodeGenData.write_array_body(fd, self.params)
|
||||
|
||||
fd.write("const XPTMethodDescriptor XPTHeader::kMethods[] = ")
|
||||
CodeGenData.write_array_body(fd, self.methods)
|
||||
|
||||
fd.write("const XPTConstDescriptor XPTHeader::kConsts[] = ")
|
||||
CodeGenData.write_array_body(fd, self.constants)
|
||||
|
||||
fd.write("const char XPTHeader::kStrings[] = {\n")
|
||||
if self.strings:
|
||||
for s in self.strings:
|
||||
# Store each string as individual characters to work around
|
||||
# MSVC's limit of 65k characters for a single string literal
|
||||
# (error C1091).
|
||||
s_index = self.string_indexes[s]
|
||||
fd.write(" '%s', '\\0', // %s %d\n" % ("', '".join(list(s)), s, s_index))
|
||||
else:
|
||||
fd.write('""')
|
||||
fd.write('};\n\n')
|
||||
|
||||
def add_interface(self, new_interface):
|
||||
assert new_interface
|
||||
self.interfaces.append(new_interface)
|
||||
|
||||
def add_type(self, new_type):
|
||||
assert isinstance(new_type, basestring)
|
||||
if new_type in self.type_indexes:
|
||||
return self.type_indexes[new_type]
|
||||
index = len(self.types)
|
||||
self.types.append(new_type)
|
||||
self.type_indexes[new_type] = index
|
||||
return index
|
||||
|
||||
def add_params(self, new_params):
|
||||
# Always represent empty parameter lists as being at 0, for no
|
||||
# particular reason beside it being nicer.
|
||||
if len(new_params) == 0:
|
||||
return 0
|
||||
|
||||
index = len(self.params)
|
||||
# The limit of 4 here is fairly arbitrary. The idea is to not
|
||||
# spend time adding large things to the cache that have little
|
||||
# chance of getting used again.
|
||||
if len(new_params) <= 4:
|
||||
params_key = "".join(new_params)
|
||||
if params_key in self.params_indexes:
|
||||
return self.params_indexes[params_key]
|
||||
else:
|
||||
self.params_indexes[params_key] = index
|
||||
self.params += new_params
|
||||
return index
|
||||
|
||||
def add_methods(self, new_methods):
|
||||
index = len(self.methods)
|
||||
self.methods += new_methods
|
||||
return index
|
||||
|
||||
def add_constants(self, new_constants):
|
||||
index = len(self.constants)
|
||||
self.constants += new_constants
|
||||
return index
|
||||
|
||||
def add_string(self, new_string):
|
||||
if new_string in self.string_indexes:
|
||||
return self.string_indexes[new_string]
|
||||
index = self.curr_string_index
|
||||
self.strings.append(new_string)
|
||||
self.string_indexes[new_string] = index
|
||||
self.curr_string_index += len(new_string) + 1
|
||||
return index
|
||||
|
||||
|
||||
# Descriptor types as described in the spec
|
||||
class Type(object):
|
||||
"""
|
||||
|
|
@ -263,6 +380,13 @@ class Type(object):
|
|||
"""
|
||||
file.write(Type._prefixdescriptor.pack(self.encodeflags() | self.tag))
|
||||
|
||||
def typeDescriptorPrefixString(self):
|
||||
"""
|
||||
Return a string for the C++ code to represent the XPTTypeDescriptorPrefix.
|
||||
|
||||
"""
|
||||
return "{0x%x}" % (self.encodeflags() | self.tag)
|
||||
|
||||
|
||||
class SimpleType(Type):
|
||||
"""
|
||||
|
|
@ -311,6 +435,9 @@ class SimpleType(Type):
|
|||
s += " *"
|
||||
return s
|
||||
|
||||
def code_gen(self, typelib, cd):
|
||||
return "{%s, 0, 0}" % self.typeDescriptorPrefixString()
|
||||
|
||||
|
||||
class InterfaceType(Type):
|
||||
"""
|
||||
|
|
@ -366,6 +493,12 @@ class InterfaceType(Type):
|
|||
# write out the interface index (1-based)
|
||||
file.write(InterfaceType._descriptor.pack(typelib.interfaces.index(self.iface) + 1))
|
||||
|
||||
def code_gen(self, typelib, cd):
|
||||
index = typelib.interfaces.index(self.iface) + 1
|
||||
hi = int(index / 256)
|
||||
lo = index - (hi * 256)
|
||||
return "{%s, %d, %d}" % (self.typeDescriptorPrefixString(), hi, lo)
|
||||
|
||||
def __str__(self):
|
||||
if self.iface:
|
||||
return self.iface.name
|
||||
|
|
@ -425,6 +558,10 @@ class InterfaceIsType(Type):
|
|||
Type.write(self, typelib, file)
|
||||
file.write(InterfaceIsType._descriptor.pack(self.param_index))
|
||||
|
||||
def code_gen(self, typelib, cd):
|
||||
return "{%s, %d, 0}" % (self.typeDescriptorPrefixString(),
|
||||
self.param_index)
|
||||
|
||||
def __str__(self):
|
||||
return "InterfaceIs *"
|
||||
|
||||
|
|
@ -485,6 +622,12 @@ class ArrayType(Type):
|
|||
self.length_is_arg_num))
|
||||
self.element_type.write(typelib, file)
|
||||
|
||||
def code_gen(self, typelib, cd):
|
||||
element_type_index = cd.add_type(self.element_type.code_gen(typelib, cd))
|
||||
return "{%s, %d, %d}" % (self.typeDescriptorPrefixString(),
|
||||
self.size_is_arg_num,
|
||||
element_type_index)
|
||||
|
||||
def __str__(self):
|
||||
return "%s []" % str(self.element_type)
|
||||
|
||||
|
|
@ -541,6 +684,10 @@ class StringWithSizeType(Type):
|
|||
file.write(StringWithSizeType._descriptor.pack(self.size_is_arg_num,
|
||||
self.length_is_arg_num))
|
||||
|
||||
def code_gen(self, typelib, cd):
|
||||
return "{%s, %d, 0}" % (self.typeDescriptorPrefixString(),
|
||||
self.size_is_arg_num)
|
||||
|
||||
def __str__(self):
|
||||
return "string_s"
|
||||
|
||||
|
|
@ -597,6 +744,10 @@ class WideStringWithSizeType(Type):
|
|||
file.write(WideStringWithSizeType._descriptor.pack(self.size_is_arg_num,
|
||||
self.length_is_arg_num))
|
||||
|
||||
def code_gen(self, typelib, cd):
|
||||
return "{%s, %d, 0}" % (self.typeDescriptorPrefixString(),
|
||||
self.size_is_arg_num)
|
||||
|
||||
def __str__(self):
|
||||
return "wstring_s"
|
||||
|
||||
|
|
@ -725,6 +876,9 @@ class Param(object):
|
|||
file.write(Param._descriptorstart.pack(self.encodeflags()))
|
||||
self.type.write(typelib, file)
|
||||
|
||||
def code_gen(self, typelib, cd):
|
||||
return "{0x%x, %s}" % (self.encodeflags(), self.type.code_gen(typelib, cd))
|
||||
|
||||
def prefix(self):
|
||||
"""
|
||||
Return a human-readable string representing the flags set
|
||||
|
|
@ -905,6 +1059,21 @@ class Method(object):
|
|||
"""
|
||||
self._name_offset = string_writer.write(self.name)
|
||||
|
||||
def code_gen(self, typelib, cd):
|
||||
# Don't store any extra info for methods that can't be called from JS.
|
||||
if self.notxpcom or self.hidden:
|
||||
string_index = 0
|
||||
param_index = 0
|
||||
num_params = 0
|
||||
else:
|
||||
string_index = cd.add_string(self.name)
|
||||
param_index = cd.add_params([p.code_gen(typelib, cd) for p in self.params])
|
||||
num_params = len(self.params)
|
||||
|
||||
return "{%d, %d, 0x%x, %d}" % (string_index,
|
||||
param_index,
|
||||
self.encodeflags(),
|
||||
num_params)
|
||||
|
||||
class Constant(object):
|
||||
"""
|
||||
|
|
@ -918,6 +1087,10 @@ class Constant(object):
|
|||
Type.Tags.uint16: '>H',
|
||||
Type.Tags.int32: '>i',
|
||||
Type.Tags.uint32: '>I'}
|
||||
memberTypeMap = {Type.Tags.int16: 'int16_t',
|
||||
Type.Tags.uint16: 'uint16_t',
|
||||
Type.Tags.int32: 'int32_t',
|
||||
Type.Tags.uint32: 'uint32_t'}
|
||||
|
||||
def __init__(self, name, type, value):
|
||||
self.name = name
|
||||
|
|
@ -976,6 +1149,15 @@ class Constant(object):
|
|||
"""
|
||||
self._name_offset = string_writer.write(self.name)
|
||||
|
||||
def code_gen(self, typelib, cd):
|
||||
string_index = cd.add_string(self.name)
|
||||
|
||||
# The static cast is needed for disambiguation.
|
||||
return "{%d, %s, XPTConstValue(static_cast<%s>(%d))}" % (string_index,
|
||||
self.type.code_gen(typelib, cd),
|
||||
Constant.memberTypeMap[self.type.tag],
|
||||
self.value)
|
||||
|
||||
def __repr__(self):
|
||||
return "Constant(%s, %s, %d)" % (self.name, str(self.type), self.value)
|
||||
|
||||
|
|
@ -1171,6 +1353,35 @@ class Interface(object):
|
|||
for c in self.constants:
|
||||
c.write_name(string_writer)
|
||||
|
||||
def code_gen_interface(self, typelib, cd):
|
||||
iid = Typelib.code_gen_iid(self.iid)
|
||||
string_index = cd.add_string(self.name)
|
||||
|
||||
parent_idx = 0
|
||||
if self.resolved:
|
||||
methods_index = cd.add_methods([m.code_gen(typelib, cd) for m in self.methods])
|
||||
constants_index = cd.add_constants([c.code_gen(typelib, cd) for c in self.constants])
|
||||
if self.parent:
|
||||
parent_idx = typelib.interfaces.index(self.parent) + 1
|
||||
else:
|
||||
# Unresolved interfaces only have their name and IID set to non-zero values.
|
||||
methods_index = 0
|
||||
constants_index = 0
|
||||
assert len(self.methods) == 0
|
||||
assert len(self.constants) == 0
|
||||
assert self.encodeflags() == 0
|
||||
|
||||
return "{%s, %s, %d, %d, %d, %d, %d, 0x%x} /* %s */" % (
|
||||
iid,
|
||||
string_index,
|
||||
methods_index,
|
||||
constants_index,
|
||||
parent_idx,
|
||||
len(self.methods),
|
||||
len(self.constants),
|
||||
self.encodeflags(),
|
||||
self.name)
|
||||
|
||||
|
||||
class Typelib(object):
|
||||
"""
|
||||
|
|
@ -1288,6 +1499,16 @@ class Typelib(object):
|
|||
iface.read_descriptor(xpt, data, data_pool_offset)
|
||||
return xpt
|
||||
|
||||
@staticmethod
|
||||
def code_gen_iid(iid):
|
||||
chunks = iid.split('-')
|
||||
return "{0x%s, 0x%s, 0x%s, {0x%02x, 0x%02x, 0x%02x, 0x%02x, 0x%02x, 0x%02x, 0x%02x, 0x%02x}}" % (
|
||||
chunks[0], chunks[1], chunks[2],
|
||||
int(chunks[3][0:2], 16), int(chunks[3][2:4], 16),
|
||||
int(chunks[4][0:2], 16), int(chunks[4][2:4], 16),
|
||||
int(chunks[4][4:6], 16), int(chunks[4][6:8], 16),
|
||||
int(chunks[4][8:10], 16), int(chunks[4][10:12], 16))
|
||||
|
||||
def __repr__(self):
|
||||
return "<Typelib with %d interfaces>" % len(self.interfaces)
|
||||
|
||||
|
|
@ -1362,6 +1583,39 @@ class Typelib(object):
|
|||
else:
|
||||
self.writefd(output_file)
|
||||
|
||||
def code_gen_writefd(self, fd):
|
||||
cd = CodeGenData()
|
||||
|
||||
for i in self.interfaces:
|
||||
cd.add_interface(i.code_gen_interface(self, cd))
|
||||
|
||||
fd.write("""/* -*- Mode: C++; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
|
||||
/* vim: set ts=8 sts=2 et sw=2 tw=80: */
|
||||
/* This Source Code Form is subject to the terms of the Mozilla Public
|
||||
* License, v. 2.0. If a copy of the MPL was not distributed with this
|
||||
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */
|
||||
|
||||
/* THIS IS AN AUTOGENERATED FILE. DO NOT EDIT. */
|
||||
|
||||
#include "xpt_struct.h"
|
||||
|
||||
""")
|
||||
cd.finish(fd)
|
||||
|
||||
def code_gen_write(self, output_file):
|
||||
"""
|
||||
Write the contents of this typelib to |output_file|,
|
||||
which can be either a filename or a file-like object.
|
||||
|
||||
"""
|
||||
self._sanityCheck()
|
||||
|
||||
if isinstance(output_file, basestring):
|
||||
with open(output_file, "wb") as f:
|
||||
self.code_gen_writefd(f)
|
||||
else:
|
||||
self.code_gen_writefd(output_file)
|
||||
|
||||
def dump(self, out):
|
||||
"""
|
||||
Print a human-readable listing of the contents of this typelib
|
||||
|
|
@ -1579,9 +1833,11 @@ def xpt_link(inputs):
|
|||
|
||||
if __name__ == '__main__':
|
||||
if len(sys.argv) < 3:
|
||||
print >>sys.stderr, "xpt <dump|link> <files>"
|
||||
print >>sys.stderr, "xpt <dump|link|linkgen> <files>"
|
||||
sys.exit(1)
|
||||
if sys.argv[1] == 'dump':
|
||||
xpt_dump(sys.argv[2])
|
||||
elif sys.argv[1] == 'link':
|
||||
xpt_link(sys.argv[3:]).write(sys.argv[2])
|
||||
elif sys.argv[1] == 'linkgen':
|
||||
xpt_link(sys.argv[3:]).code_gen_write(sys.argv[2])
|
||||
|
|
|
|||
|
|
@ -178,6 +178,15 @@ union XPTConstValue {
|
|||
uint16_t ui16;
|
||||
int32_t i32;
|
||||
uint32_t ui32;
|
||||
|
||||
// These constructors are needed to statically initialize different cases of
|
||||
// the union because MSVC does not support the use of designated initializers
|
||||
// in C++ code. They need to be constexpr to ensure that no initialization code
|
||||
// is run at startup, to enable sharing of this memory between Firefox processes.
|
||||
explicit constexpr XPTConstValue(int16_t aInt) : i16(aInt) {}
|
||||
explicit constexpr XPTConstValue(uint16_t aInt) : ui16(aInt) {}
|
||||
explicit constexpr XPTConstValue(int32_t aInt) : i32(aInt) {}
|
||||
explicit constexpr XPTConstValue(uint32_t aInt) : ui32(aInt) {}
|
||||
};
|
||||
|
||||
struct XPTConstDescriptor {
|
||||
|
|
@ -187,7 +196,7 @@ struct XPTConstDescriptor {
|
|||
|
||||
uint32_t mName; // Index into XPTHeader::mStrings.
|
||||
XPTTypeDescriptor mType;
|
||||
union XPTConstValue mValue;
|
||||
XPTConstValue mValue;
|
||||
};
|
||||
|
||||
/*
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче