Bug 1444745 - Part 4: Rewrite xptinfo, and write a new xptcodegen.py to generate the required datastructures, r=mccr8

This patch contains the meat of the changes here. The following summarize the changes:
1. xptinfo.h is rewritten to expose the new interface for reading the XPT data,

The nsXPTInterfaceInfo object exposes methods with the same signatures as
the methods on nsIInterfaceInfo, to make converting code which used
nsIInterfaceInfo as easy as possible, even when those methods don't have
signatures which make a ton of sense anymore. There are also a few methods
which are unnecessary (they return `true` or similar), which should be
removed over time.

Members of the data structures are made private in order to prevent reading
them directly. Code should instead call the getter methods. This should make
it easier to change their memory representation in the future. Constructing
these structs is made possible by making the structs `friend class` with the
XPTConstruct class, which is implemented by the code generator, and is able
to access the private fields.

In addition, rather than using integers with flag constants, I opted for
using C++ bitfields to store individual flags, as I found it made it easier
to both write the code generator, and reason about the layouts of the types.

I was able to shave a byte off of each nsXPTParamInfo (4 bytes -> 3 bytes)
by shoving the flags into spare bits in the nsXPTType. Unfortunately there
was not enough room for the retval flag. Fortunately, we already depend in
our code on the retval parameter being the last parameter, so I worked
around this by removing the retval flag and instead having a `hasretval`
flag on the method itself.

2. An xptinfo.cpp file is added for out-of-line definitions of more complex
methods, and the internal implementation details of the perfect hash.

Notable is the handling of xptshim interfaces. As the type is uniform, a
flag is checked when trying to read constant information, and a different
table with pointers into webidl data structures is checked when the type is
determined to be a shim.

Ideally we could remove this once we remove the remaining consumers of the
existing shim interfaces.

3. A python code generator which takes in the json XPT files generated in the
previous part, and emits a xptdata.cpp file with the data structures. I did
my best to heavily comment the code.

This code uses the friend class trick to construct the private fields of the
structs, and avoid a dependency on the ordering of fields in xptinfo.h.

The sInterfaces array's order is determined by a generated perfect hash
which is also written into the binary. This should allow for fast lookups by
IID or name of interfaces in memory. The hash function used for the perfect
hash is a simple FNV hash, as they're pretty fast.

For perfect hashing of names, another table is created which contains
indexes into the sInterfaces table. Lookup by name is less common, and this
form of lookup should still be very fast.

4. The necessary Makefiles are updated to use the new code generator, and
generate the file correctly.

config/makefiles/xpidl/Makefile.in

@@ -30,8 +30,9 @@ dist_idl_dir := $(DIST)/idl
 dist_include_dir := $(DIST)/include
 dist_xpcrs_dir := $(DIST)/xpcrs
 process_py := $(topsrcdir)/python/mozbuild/mozbuild/action/xpidl-process.py
-generated_file := $(topobjdir)/xpcom/typelib/xpt/XPTInfo.cpp
+generated_file := $(topobjdir)/xpcom/reflect/xptinfo/xptdata.cpp
-code_gen_py := $(topsrcdir)/xpcom/typelib/xpt/tools/xpt.py
+code_gen_py := $(topsrcdir)/xpcom/reflect/xptinfo/xptcodegen.py
+code_gen_deps := $(topsrcdir)/xpcom/reflect/xptinfo/perfecthash.py
 
 # TODO we should use py_action, but that would require extra directories to be
 # in the virtualenv.
@@ -65,11 +66,9 @@ endif
 
 $(xpt_files): $(process_py) $(call mkdir_deps,$(idl_deps_dir) $(dist_include_dir) $(dist_xpcrs_dir))
 
-$(generated_file): $(xpt_files) $(code_gen_py)
+$(generated_file): $(xpt_files) $(code_gen_py) $(code_gen_deps)
 	$(REPORT_BUILD)
-	$(PYTHON_PATH) $(PLY_INCLUDE) \
+	$(PYTHON_PATH) $(PLY_INCLUDE) $(code_gen_py) $(generated_file) $(xpt_files)
-		$(code_gen_py) linkgen \
-		$(generated_file) $(xpt_files)
 
 -include $(depends_files)
 

xpcom/idl-parser/xpidl/jsonxpt.py

@@ -185,7 +185,6 @@ def build_interface(iface):
             methods.append(mk_method(a.name, [param], setter=1, hidden=a.noscript,
                                      context=a.implicit_jscontext))
 
-    implicit_builtinclass = False
     for member in iface.members:
         if isinstance(member, xpidl.ConstMember):
             build_const(member)
@@ -193,13 +192,6 @@ def build_interface(iface):
             build_attr(member)
         elif isinstance(member, xpidl.Method):
             build_method(member)
-
-            # XXX(hacky): If we have a notxpcom method (other than
-            # nsISupports::{AddRef,Release}), we need to implicitly mark
-            # ourselves as builtinclass, as we cannot be implemented in JS.
-            if member.notxpcom and iface.name != "nsISupports":
-                implicit_builtinclass = True
-
         elif isinstance(member, xpidl.CDATA):
             pass
         else:
@@ -218,7 +210,7 @@ def build_interface(iface):
         'flags': flags(
             ('scriptable', iface.attributes.scriptable),
             ('function', iface.attributes.function),
-            ('builtinclass', iface.attributes.builtinclass or implicit_builtinclass),
+            ('builtinclass', iface.attributes.builtinclass or iface.implicit_builtinclass),
             ('main_process_only', iface.attributes.main_process_scriptable_only),
         )
     }

xpcom/idl-parser/xpidl/xpidl.py

@@ -576,11 +576,20 @@ class Interface(object):
         self.namemap = NameMap()
         self.doccomments = doccomments
         self.nativename = name
+        self.implicit_builtinclass = False
 
         for m in members:
             if not isinstance(m, CDATA):
                 self.namemap.set(m)
 
+            if m.kind == 'method' and m.notxpcom and name != 'nsISupports':
+                # An interface cannot be implemented by JS if it has a
+                # notxpcom method. Such a type is an "implicit builtinclass".
+                #
+                # XXX(nika): Why does nostdcall not imply builtinclass?
+                # It could screw up the shims as well...
+                self.implicit_builtinclass = True
+
     def __eq__(self, other):
         return self.name == other.name and self.location == other.location
 
@@ -617,6 +626,9 @@ class Interface(object):
             if self.attributes.scriptable and realbase.attributes.builtinclass and not self.attributes.builtinclass:
                 raise IDLError("interface '%s' is not builtinclass but derives from builtinclass '%s'" % (self.name, self.base), self.location)
 
+            if realbase.implicit_builtinclass:
+                self.implicit_builtinclass = True # Inherit implicit builtinclass from base
+
         for member in self.members:
             member.resolve(self)
 

xpcom/reflect/xptinfo/moz.build

@@ -5,8 +5,12 @@
 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
 
 UNIFIED_SOURCES += [
+    'xptinfo.cpp',
 ]
 
+SOURCES += [
+    '!xptdata.cpp',
+]
 
 EXPORTS += [
     'xptinfo.h',

xpcom/reflect/xptinfo/perfecthash.py

@@ -0,0 +1,121 @@
+#!/usr/bin/env python
+# perfecthash.py - Helper for generating perfect hash functions for xptcodegen.py
+#
+# This Source Code Form is subject to the terms of the Mozilla Public
+# License, v. 2.0. If a copy of the MPL was not distributed with this
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
+
+
+# A perfect hash function (PHF) is a function which maps distinct elements from
+# a source set to a set of integers with no collisions. Perfect hash functions
+# created by perfecthash.py take in a mapping from a key bytearray to an output
+# value. The generated PHF uses a 32-bit FNV hash to index into an intermediate
+# table. The value from that table is then used to feed into a second 32-bit FNV
+# hash to get the index into the final table.
+#
+# This is done by starting with the largest set of conflicts, and guessing
+# intermediate table values such that we generate no conflicts. This then allows
+# us to do a constant-time lookup at runtime into these tables.
+
+from collections import namedtuple
+
+# 32-bit FNV offset basis and prime value.
+FNV_OFFSET_BASIS = 0x811C9DC5
+FNV_PRIME = 16777619
+
+# We use uint32_ts for our arrays in PerfectHash. 0x80000000 is the high bit
+# which we sometimes use as a flag.
+U32_HIGH_BIT = 0x80000000
+
+# A basic FNV-based hash function. bytes is the bytearray to hash. 32-bit FNV is
+# used for indexing into the first table, and the value stored in that table is
+# used as the offset basis for indexing into the values table.
+#
+# NOTE: C++ implementation is in xptinfo.cpp
+def hash(bytes, h=FNV_OFFSET_BASIS):
+    for byte in bytes:
+        h ^= byte       # xor-in the byte
+        h *= FNV_PRIME  # Multiply by the FNV prime
+        h &= 0xffffffff # clamp to 32-bits
+    return h
+
+IntermediateBucket = namedtuple('IntermediateBucket', ['index', 'entries'])
+HashEntry = namedtuple('HashEntry', ['key', 'value'])
+
+class PerfectHash(object):
+    """An object representing a perfect hash function"""
+
+    def __init__(self, intermediate_table_size, data):
+        # data should be a list of (bytearray, value) pairs
+
+        self.intermediate = [0] * intermediate_table_size
+        self.values = [None] * len(data)
+
+        assert len(self.values) < U32_HIGH_BIT, \
+            "Not enough space in uint32_t to index %d values" % len(self.values)
+
+        # Buckets contains a set of IntermediateBucket values. Each bucket
+        # contains the index into the intermediate table, and the set of entries
+        # which map into that table.
+        buckets = [IntermediateBucket(index=idx, entries=[])
+                   for idx in range(len(self.intermediate))]
+
+        # Determine which input strings map to which buckets in the intermediate
+        # array.
+        for key, value in data:
+            assert isinstance(key, bytearray), \
+                "data should be a list of (bytearray, value) pairs"
+            assert value is not None, "cannot handle a None value"
+
+            buckets[hash(key) % len(self.intermediate)].entries \
+                .append(HashEntry(key=key, value=value))
+
+        # Sort the buckets such that the largest one first.
+        buckets.sort(key=lambda b: len(b.entries), reverse=True)
+
+        freecursor = 0
+        for bucket in buckets:
+            # If we've reached buckets with no conflicts, we can just start
+            # storing direct indices into the final array. Once we reach an
+            # empty bucket, we're done. The high bit is set to identify direct
+            # indices into the final array.
+            if len(bucket.entries) == 0:
+                break
+            elif len(bucket.entries) == 1:
+                while freecursor < len(self.values):
+                    if self.values[freecursor] is None:
+                        self.intermediate[bucket.index] = freecursor | U32_HIGH_BIT
+                        self.values[freecursor] = bucket.entries[0].value
+                        break
+                    freecursor += 1
+                continue
+
+            # Try values for the basis until we find one with no conflicts.
+            idx = 0
+            basis = 1
+            slots = []
+            while idx < len(bucket.entries):
+                slot = hash(bucket.entries[idx].key, basis) % len(self.values)
+                if self.values[slot] is not None or slot in slots:
+                    # There was a conflict, try the next basis.
+                    basis += 1
+                    idx = 0
+                    del slots[:]
+                else:
+                    slots.append(slot)
+                    idx += 1
+
+            assert basis < U32_HIGH_BIT, \
+                "not enough space in uint32_t to store basis %d" % basis
+
+            # We've found a basis which doesn't conflict
+            self.intermediate[bucket.index] = basis
+            for slot, entry in zip(slots, bucket.entries):
+                self.values[slot] = entry.value
+
+    def lookup(self, key):
+        mid = self.intermediate[hash(key) % len(self.intermediate)]
+        if mid & U32_HIGH_BIT:  # direct index
+            return self.values[mid & ~U32_HIGH_BIT]
+        else:
+            return self.values[hash(key, mid) % len(self.values)]

xpcom/reflect/xptinfo/xptcodegen.py

@@ -0,0 +1,470 @@
+#!/usr/bin/env python
+# jsonlink.py - Merge JSON typelib files into a .cpp file
+#
+# This Source Code Form is subject to the terms of the Mozilla Public
+# License, v. 2.0. If a copy of the MPL was not distributed with this
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
+
+# NOTE: Once shims are removed, this code can be cleaned up, removing all
+# reference to them.
+
+import json
+from perfecthash import PerfectHash
+import time
+from collections import OrderedDict
+
+# We fix the number of entries in our intermediate table used by the perfect
+# hashes to 256. This number is constant in xptinfo, allowing the compiler to
+# generate a more efficient modulo due to it being a power of 2.
+PHFSIZE = 256
+
+def indented(s):
+    return s.replace('\n', '\n  ')
+
+def cpp(v):
+    if type(v) == bool:
+        return "true" if v else "false"
+    return str(v)
+
+def mkstruct(*fields):
+    def mk(comment, **vals):
+        assert len(fields) == len(vals)
+        r = "{ // " + comment
+        r += indented(','.join(
+            "\n/* %s */ %s" % (k, cpp(vals[k])) for k in fields))
+        r += "\n}"
+        return r
+    return mk
+
+##########################################################
+# Ensure these fields are in the same order as xptinfo.h #
+##########################################################
+nsXPTInterfaceInfo = mkstruct(
+    "mIID",
+    "mName",
+    "mParent",
+    "mBuiltinClass",
+    "mMainProcessScriptableOnly",
+    "mMethods",
+    "mConsts",
+    "mIsShim",
+    "mFunction",
+    "mNumMethods",
+    "mNumConsts",
+)
+
+##########################################################
+# Ensure these fields are in the same order as xptinfo.h #
+##########################################################
+nsXPTType = mkstruct(
+    "mTag",
+    "mInParam",
+    "mOutParam",
+    "mOptionalParam",
+    "mData1",
+    "mData2",
+)
+
+##########################################################
+# Ensure these fields are in the same order as xptinfo.h #
+##########################################################
+nsXPTParamInfo = mkstruct(
+    "mType",
+)
+
+##########################################################
+# Ensure these fields are in the same order as xptinfo.h #
+##########################################################
+nsXPTMethodInfo = mkstruct(
+    "mName",
+    "mParams",
+    "mNumParams",
+    "mGetter",
+    "mSetter",
+    "mNotXPCOM",
+    "mHidden",
+    "mOptArgc",
+    "mContext",
+    "mHasRetval",
+)
+
+##########################################################
+# Ensure these fields are in the same order as xptinfo.h #
+##########################################################
+ConstInfo = mkstruct(
+    "mName",
+    "mSigned",
+    "mValue",
+)
+
+
+# Helper functions for dealing with IIDs.
+#
+# Unfortunately, the way we represent IIDs in memory depends on the endianness
+# of the target architecture. We store an nsIID as a 16-byte, 4-tuple of:
+#
+#   (uint32_t, uint16_t, uint16_t, [uint8_t; 8])
+#
+# Unfortunately, this means that when we hash the bytes of the nsIID on a
+# little-endian target system, we need to hash them in little-endian order.
+# These functions let us split the input hexadecimal string into components,
+# encoding each as a little-endian value, and producing an accurate bytearray.
+#
+# It would be nice to have a consistent representation of IIDs in memory such
+# that we don't have to do these gymnastics to get an accurate hash.
+
+def split_at_idxs(s, lengths):
+    idx = 0
+    for length in lengths:
+        yield s[idx:idx+length]
+        idx += length
+    assert idx == len(s)
+
+def split_iid(iid): # Get the individual components out of an IID string.
+    iid = iid.replace('-', '') # Strip any '-' delimiters
+    return tuple(split_at_idxs(iid, (8, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2)))
+
+def iid_bytes(iid): # Get the byte representation of the IID for hashing.
+    bs = bytearray()
+    for num in split_iid(iid):
+        b = bytearray.fromhex(num)
+        # We store the bytes in little-endian. On big-endian systems, the C++
+        # code will flip the bytes to little-endian before hashing in order to
+        # keep the tables consistent.
+        b.reverse()
+        bs += b
+    return bs
+
+# Split a 16-bit integer into its high and low 8 bits
+def splitint(i):
+    assert i < 2**16
+    return (i >> 8, i & 0xff)
+
+
+# Core of the code generator. Takes a list of raw JSON XPT interfaces, and
+# writes out a file containing the necessary static declarations into fd.
+def link_to_cpp(interfaces, fd):
+    # Perfect Hash from IID into the ifaces array.
+    iid_phf = PerfectHash(PHFSIZE, [
+        (iid_bytes(iface['uuid']), iface)
+        for iface in interfaces
+    ])
+    # Perfect Hash from name to index in the ifaces array.
+    name_phf = PerfectHash(PHFSIZE, [
+        (bytearray(iface['name'], 'ascii'), idx)
+        for idx, iface in enumerate(iid_phf.values)
+    ])
+
+    def interface_idx(name):
+        if name is not None:
+            idx = name_phf.lookup(bytearray(name, 'ascii'))
+            if iid_phf.values[idx]['name'] == name:
+                return idx + 1 # One-based, so we can use 0 as a sentinel.
+        return 0
+
+    # NOTE: State used while linking. This is done with closures rather than a
+    # class due to how this file's code evolved.
+    includes = set()
+    types = []
+    type_cache = {}
+    ifaces = []
+    params = []
+    param_cache = {}
+    methods = []
+    consts = []
+    prophooks = []
+    strings = OrderedDict()
+
+    def lower_uuid(uuid):
+        return "{0x%s, 0x%s, 0x%s, {0x%s, 0x%s, 0x%s, 0x%s, 0x%s, 0x%s, 0x%s, 0x%s}}" % split_iid(uuid)
+
+    def lower_string(s):
+        if s in strings:
+            # We've already seen this string.
+            return strings[s]
+        elif len(strings):
+            # Get the last string we inserted (should be O(1) on OrderedDict).
+            last_s = next(reversed(strings))
+            strings[s] = strings[last_s] + len(last_s) + 1
+        else:
+            strings[s] = 0
+        return strings[s]
+
+    def describe_type(type): # Create the type's documentation comment.
+        tag = type['tag'][3:].lower()
+        if tag == 'array':
+            return '%s[size_is=%d]' % (
+                describe_type(type['element']), type['size_is'])
+        elif tag == 'interface_type':
+            return type['name']
+        elif tag == 'interface_is_type':
+            return 'iid_is(%d)' % type['iid_is']
+        elif tag.endswith('_size_is'):
+            return '%s(size_is=%d)' % (tag, type['size_is'])
+        return tag
+
+    def lower_type(type, in_=False, out=False, optional=False):
+        tag = type['tag']
+        d1 = d2 = 0
+
+        if tag == 'TD_ARRAY':
+            d1 = type['size_is']
+
+            # index of element in extra types list
+            key = describe_type(type['element'])
+            d2 = type_cache.get(key)
+            if d2 is None:
+                d2 = type_cache[key] = len(types)
+                types.append(lower_type(type['element']))
+
+        elif tag == 'TD_INTERFACE_TYPE':
+            d1, d2 = splitint(interface_idx(type['name']))
+
+        elif tag == 'TD_INTERFACE_IS_TYPE':
+            d1 = type['iid_is']
+
+        elif tag.endswith('_SIZE_IS'):
+            d1 = type['size_is']
+
+        assert d1 < 256 and d2 < 256, "Data values too large"
+        return nsXPTType(
+            describe_type(type),
+            mTag=tag,
+            mData1=d1,
+            mData2=d2,
+            mInParam=in_,
+            mOutParam=out,
+            mOptionalParam=optional,
+        )
+
+    def lower_param(param, paramname):
+        params.append(nsXPTParamInfo(
+            "%d = %s" % (len(params), paramname),
+            mType=lower_type(param['type'],
+                             in_='in' in param['flags'],
+                             out='out' in param['flags'],
+                             optional='optional' in param['flags'])
+        ))
+
+    def lower_method(method, ifacename):
+        methodname = "%s::%s" % (ifacename, method['name'])
+
+        if 'notxpcom' in method['flags'] or 'hidden' in method['flags']:
+            paramidx = name = numparams = 0 # hide parameters
+        else:
+            name = lower_string(method['name'])
+            numparams = len(method['params'])
+
+            # Check cache for parameters
+            cachekey = json.dumps(method['params'])
+            paramidx = param_cache.get(cachekey)
+            if paramidx is None:
+                paramidx = param_cache[cachekey] = len(params)
+                for idx, param in enumerate(method['params']):
+                    lower_param(param, "%s[%d]" % (methodname, idx))
+
+        methods.append(nsXPTMethodInfo(
+            "%d = %s" % (len(methods), methodname),
+
+            # If our method is hidden, we can save some memory by not
+            # generating parameter info about it.
+            mName=name,
+            mParams=paramidx,
+            mNumParams=numparams,
+
+            # Flags
+            mGetter='getter' in method['flags'],
+            mSetter='setter' in method['flags'],
+            mNotXPCOM='notxpcom' in method['flags'],
+            mHidden='hidden' in method['flags'],
+            mOptArgc='optargc' in method['flags'],
+            mContext='jscontext' in method['flags'],
+            mHasRetval='hasretval' in method['flags'],
+        ))
+
+    def lower_const(const, ifacename):
+        assert const['type']['tag'] in \
+            ['TD_INT16', 'TD_INT32', 'TD_UINT16', 'TD_UINT32']
+        is_signed = const['type']['tag'] in ['TD_INT16', 'TD_INT32']
+
+        # Constants are always either signed or unsigned 16 or 32 bit integers,
+        # which we will only need to convert to JS values. To save on space,
+        # don't bother storing the type, and instead just store a 32-bit
+        # unsigned integer, and stash whether to interpret it as signed.
+        consts.append(ConstInfo(
+            "%d = %s::%s" % (len(consts), ifacename, const['name']),
+
+            mName=lower_string(const['name']),
+            mSigned=is_signed,
+            mValue="(uint32_t)%d" % const['value'],
+        ))
+
+    def lower_prop_hooks(iface): # XXX: Used by xpt shims
+        assert iface['shim'] is not None
+
+        # Add an include for the Binding file for the shim.
+        includes.add("mozilla/dom/%sBinding.h" %
+            (iface['shimfile'] or iface['shim']))
+
+        # Add the property hook reference to the sPropHooks table.
+        prophooks.append(
+            "mozilla::dom::%sBinding::sNativePropertyHooks, // %d = %s(%s)" % \
+                (iface['shim'], len(prophooks), iface['name'], iface['shim']))
+
+    def collect_base_info(iface):
+        methods = 0
+        consts = 0
+        while iface is not None:
+            methods += len(iface['methods'])
+            consts += len(iface['consts'])
+            idx = interface_idx(iface['parent'])
+            if idx == 0:
+                break
+            iface = iid_phf.values[idx - 1]
+
+        return methods, consts
+
+    def lower_iface(iface):
+        isshim = iface['shim'] is not None
+        assert isshim or 'scriptable' in iface['flags']
+
+        method_off = len(methods)
+        consts_off = len(consts)
+        method_cnt = const_cnt = 0
+        if isshim:
+            # If we are looking at a shim, don't lower any methods or constants,
+            # as they will be pulled from the WebIDL binding instead. Instead,
+            # we use the constants offset field to store the index into the prop
+            # hooks table.
+            consts_off = len(prophooks)
+        else:
+            method_cnt, const_cnt = collect_base_info(iface)
+
+        # The number of maximum methods is not arbitrary. It is the same value
+        # as in xpcom/reflect/xptcall/genstubs.pl; do not change this value
+        # without changing that one or you WILL see problems.
+        #
+        # In addition, mNumMethods and mNumConsts are stored as a 8-bit ints,
+        # meaning we cannot exceed 255 methods/consts on any interface.
+        assert method_cnt < 250, "%s has too many methods" % iface['name']
+        assert const_cnt < 256, "%s has too many constants" % iface['name']
+
+        ifaces.append(nsXPTInterfaceInfo(
+            "%d = %s" % (len(ifaces), iface['name']),
+
+            mIID=lower_uuid(iface['uuid']),
+            mName=lower_string(iface['name']),
+            mParent=interface_idx(iface['parent']),
+
+            mMethods=method_off,
+            mNumMethods=method_cnt,
+            mConsts=consts_off,
+            mNumConsts=const_cnt,
+
+            # Flags
+            mIsShim=isshim,
+            mBuiltinClass='builtinclass' in iface['flags'],
+            mMainProcessScriptableOnly='main_process_only' in iface['flags'],
+            mFunction='function' in iface['flags'],
+        ))
+
+        if isshim:
+            lower_prop_hooks(iface)
+            return
+
+        # Lower the methods and constants used by this interface
+        for method in iface['methods']:
+            lower_method(method, iface['name'])
+        for const in iface['consts']:
+            lower_const(const, iface['name'])
+
+    # Lower interfaces in the order of the IID phf's values lookup.
+    for iface in iid_phf.values:
+        lower_iface(iface)
+
+    # Write out the final output file
+    fd.write("/* THIS FILE WAS GENERATED BY xptcodegen.py - DO NOT EDIT */\n\n")
+
+    # Include any bindings files which we need to include due to XPT shims.
+    for include in includes:
+        fd.write('#include "%s"\n' % include)
+
+    # Write out our header
+    fd.write("""
+#include "xptinfo.h"
+#include "mozilla/TypeTraits.h"
+
+namespace xpt {
+namespace detail {
+
+""")
+
+    # Static data arrays
+    def array(ty, name, els):
+        fd.write("const %s %s[] = {%s\n};\n\n" %
+            (ty, name, ','.join(indented('\n' + str(e)) for e in els)))
+    array("nsXPTInterfaceInfo", "sInterfaces", ifaces)
+    array("nsXPTType", "sTypes", types)
+    array("nsXPTParamInfo", "sParams", params)
+    array("nsXPTMethodInfo", "sMethods", methods)
+    array("ConstInfo", "sConsts", consts)
+    array("mozilla::dom::NativePropertyHooks*", "sPropHooks", prophooks)
+
+    # The strings array. We write out individual characters to avoid MSVC restrictions.
+    fd.write("const char sStrings[] = {\n")
+    for s, off in strings.iteritems():
+        fd.write("  // %d = %s\n  '%s','\\0',\n" % (off, s, "','".join(s)))
+    fd.write("};\n\n")
+
+    # Record the information required for perfect hashing.
+    # NOTE: Intermediates stored as 32-bit for safety. Shouldn't need >16-bit.
+    def phfarr(name, ty, it):
+        fd.write("const %s %s[] = {" % (ty, name))
+        for idx, v in enumerate(it):
+            if idx % 8 == 0:
+                fd.write('\n ')
+            fd.write(" 0x%04x," % v)
+        fd.write("\n};\n\n")
+    phfarr("sPHF_IIDs", "uint32_t", iid_phf.intermediate)
+    phfarr("sPHF_Names", "uint32_t", name_phf.intermediate)
+    phfarr("sPHF_NamesIdxs", "uint16_t", name_phf.values)
+
+    # The footer contains some checks re: the size of the generated arrays.
+    fd.write("""\
+const uint16_t sInterfacesSize = mozilla::ArrayLength(sInterfaces);
+static_assert(sInterfacesSize == mozilla::ArrayLength(sPHF_NamesIdxs),
+              "sPHF_NamesIdxs must have same size as sInterfaces");
+
+static_assert(kPHFSize == mozilla::ArrayLength(sPHF_Names),
+              "sPHF_IIDs must have size kPHFSize");
+static_assert(kPHFSize == mozilla::ArrayLength(sPHF_IIDs),
+              "sPHF_Names must have size kPHFSize");
+
+} // namespace detail
+} // namespace xpt
+""")
+
+
+def link_and_write(files, outfile):
+    interfaces = []
+    for file in files:
+        with open(file, 'r') as fd:
+            interfaces += json.load(fd)
+
+    link_to_cpp(interfaces, outfile)
+
+
+def main():
+    from argparse import ArgumentParser
+    import sys
+
+    parser = ArgumentParser()
+    parser.add_argument('outfile', help='Output C++ file to generate')
+    parser.add_argument('xpts', nargs='*', help='source xpt files')
+
+    args = parser.parse_args(sys.argv[1:])
+    with open(args.outfile, 'w') as fd:
+        link_and_write(args.xpts, fd)
+
+if __name__ == '__main__':
+    main()

xpcom/reflect/xptinfo/xptinfo.cpp

@@ -0,0 +1,388 @@
+/* -*- Mode: C++; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
+/* vim: set ts=8 sts=2 et sw=2 tw=80: */
+/* This Source Code Form is subject to the terms of the Mozilla Public
+ * License, v. 2.0. If a copy of the MPL was not distributed with this
+ * file, You can obtain one at https://mozilla.org/MPL/2.0/. */
+
+#include "xptinfo.h"
+#include "nsISupports.h"
+#include "mozilla/dom/DOMJSClass.h"
+#include "mozilla/ArrayUtils.h"
+
+using namespace mozilla;
+using namespace mozilla::dom;
+using namespace xpt::detail;
+
+///////////////////////////////////////
+// C++ Perfect Hash Helper Functions //
+///////////////////////////////////////
+
+// WARNING: This must match phf.py's implementation of the hash functions etc.
+static const uint32_t FNV_OFFSET_BASIS = 0x811C9DC5;
+static const uint32_t FNV_PRIME = 16777619;
+static const uint32_t U32_HIGH_BIT = 0x80000000;
+
+static uint32_t
+Phf_DoHash(const void* bytes, uint32_t len, uint32_t h=FNV_OFFSET_BASIS)
+{
+  for (uint32_t i = 0; i < len; ++i) {
+    h ^= reinterpret_cast<const uint8_t*>(bytes)[i];
+    h *= FNV_PRIME;
+  }
+  return h;
+}
+
+static uint16_t
+Phf_DoLookup(const void* aBytes, uint32_t aLen, const uint32_t* aIntr)
+{
+  uint32_t mid = aIntr[Phf_DoHash(aBytes, aLen) % kPHFSize];
+  if (mid & U32_HIGH_BIT) {
+    return mid & ~U32_HIGH_BIT;
+  }
+  return Phf_DoHash(aBytes, aLen, mid) % sInterfacesSize;
+}
+static_assert(kPHFSize == 256, "wrong phf size?");
+
+
+////////////////////////////////////////
+// PHF-based interface lookup methods //
+////////////////////////////////////////
+
+/* static */ const nsXPTInterfaceInfo*
+nsXPTInterfaceInfo::ByIID(const nsIID& aIID)
+{
+  // Make sure the bytes in the IID are all little-endian, as xptcodegen.py
+  // generates code assuming that nsIID is encoded in little-endian.
+  nsIID iid = aIID;
+  iid.m0 = NativeEndian::swapToLittleEndian(aIID.m0);
+  iid.m1 = NativeEndian::swapToLittleEndian(aIID.m1);
+  iid.m2 = NativeEndian::swapToLittleEndian(aIID.m2);
+
+  uint16_t idx = Phf_DoLookup(&iid, sizeof(nsIID), sPHF_IIDs);
+  MOZ_ASSERT(idx < sInterfacesSize, "index out of range");
+
+  const nsXPTInterfaceInfo* found = &sInterfaces[idx];
+  return found->IID() == aIID ? found : nullptr;
+}
+static_assert(sizeof(nsIID) == 16, "IIDs have the wrong size?");
+
+/* static */ const nsXPTInterfaceInfo*
+nsXPTInterfaceInfo::ByName(const char* aName)
+{
+  uint16_t idx = Phf_DoLookup(aName, strlen(aName), sPHF_Names);
+  MOZ_ASSERT(idx < sInterfacesSize, "index out of range");
+
+  idx = sPHF_NamesIdxs[idx];
+  MOZ_ASSERT(idx < sInterfacesSize, "index out of range");
+
+  const nsXPTInterfaceInfo* found = &sInterfaces[idx];
+  return strcmp(found->Name(), aName) ? nullptr : found;
+}
+
+
+////////////////////////////////////
+// Constant Lookup Helper Methods //
+////////////////////////////////////
+
+// XXX: Remove when shims are gone.
+// This method either looks for the ConstantSpec at aIndex, or counts the
+// number of constants for a given shim.
+// NOTE: Only one of the aSpec and aCount outparameters should be provided.
+// NOTE: If aSpec is not passed, aIndex is ignored.
+// NOTE: aIndex must be in range if aSpec is passed.
+static void
+GetWebIDLConst(uint16_t aHookIdx, uint16_t aIndex,
+               const ConstantSpec** aSpec, uint16_t* aCount)
+{
+  MOZ_ASSERT((aSpec && !aCount) || (aCount && !aSpec),
+             "Only one of aSpec and aCount should be provided");
+
+  const NativePropertyHooks* propHooks = sPropHooks[aHookIdx];
+
+  uint16_t idx = 0;
+  do {
+    const NativeProperties* props[] = {
+      propHooks->mNativeProperties.regular,
+      propHooks->mNativeProperties.chromeOnly
+    };
+    for (size_t i = 0; i < ArrayLength(props); ++i) {
+      auto prop = props[i];
+      if (prop && prop->HasConstants()) {
+        for (auto cs = prop->Constants()->specs; cs->name; ++cs) {
+          // We have found one constant here.  We explicitly do not bother
+          // calling isEnabled() here because it's OK to define potentially
+          // extra constants on these shim interfaces.
+          if (aSpec && idx == aIndex) {
+            *aSpec = cs;
+            return;
+          }
+          ++idx;
+        }
+      }
+    }
+  } while ((propHooks = propHooks->mProtoHooks));
+
+  MOZ_ASSERT(aCount, "aIndex is out of bounds!");
+  *aCount = idx;
+}
+
+bool
+nsXPTInterfaceInfo::HasAncestor(const nsIID& aIID) const
+{
+  for (const auto* info = this; info; info = info->GetParent()) {
+    if (info->IID() == aIID) {
+      return true;
+    }
+  }
+  return false;
+}
+
+uint16_t
+nsXPTInterfaceInfo::ConstantCount() const
+{
+  if (!mIsShim) {
+    return mNumConsts;
+  }
+
+  // Get the number of WebIDL constants.
+  uint16_t num = 0;
+  GetWebIDLConst(mConsts, 0, nullptr, &num);
+  return num;
+}
+
+const char*
+nsXPTInterfaceInfo::Constant(uint16_t aIndex, JS::MutableHandleValue aValue) const
+{
+  if (!mIsShim) {
+    MOZ_ASSERT(aIndex < mNumConsts);
+
+    if (const nsXPTInterfaceInfo* pi = GetParent()) {
+      MOZ_ASSERT(!pi->mIsShim);
+      if (aIndex < pi->mNumConsts) {
+        return pi->Constant(aIndex, aValue);
+      }
+      aIndex -= pi->mNumConsts;
+    }
+
+    // Extract the value and name from the Constant Info.
+    const ConstInfo& info = sConsts[mConsts + aIndex];
+    if (info.mSigned || info.mValue <= (uint32_t)INT32_MAX) {
+      aValue.set(JS::Int32Value((int32_t)info.mValue));
+    } else {
+      aValue.set(JS::DoubleValue(info.mValue));
+    }
+    return GetString(info.mName);
+  }
+
+  // Get a single WebIDL constant.
+  const ConstantSpec* spec;
+  GetWebIDLConst(mConsts, aIndex, &spec, nullptr);
+  aValue.set(spec->value);
+  return spec->name;
+}
+
+const nsXPTMethodInfo&
+nsXPTInterfaceInfo::Method(uint16_t aIndex) const
+{
+  MOZ_ASSERT(aIndex < MethodCount());
+
+  if (const nsXPTInterfaceInfo* pi = GetParent()) {
+    if (aIndex < pi->MethodCount()) {
+      return pi->Method(aIndex);
+    }
+    aIndex -= pi->MethodCount();
+  }
+
+  return xpt::detail::GetMethod(mMethods + aIndex);
+}
+
+
+////////////////////////////////////////////////
+// nsIInterfaceInfo backcompat implementation //
+////////////////////////////////////////////////
+
+nsresult
+nsXPTInterfaceInfo::GetName(char** aName) const
+{
+  *aName = moz_xstrdup(Name());
+  return NS_OK;
+}
+
+nsresult
+nsXPTInterfaceInfo::IsScriptable(bool* aRes) const
+{
+  *aRes = IsScriptable();
+  return NS_OK;
+}
+
+nsresult
+nsXPTInterfaceInfo::IsBuiltinClass(bool* aRes) const
+{
+  *aRes = IsBuiltinClass();
+  return NS_OK;
+}
+
+nsresult
+nsXPTInterfaceInfo::GetParent(const nsXPTInterfaceInfo** aParent) const
+{
+  *aParent = GetParent();
+  return NS_OK;
+}
+
+nsresult
+nsXPTInterfaceInfo::GetMethodCount(uint16_t* aMethodCount) const
+{
+  *aMethodCount = MethodCount();
+  return NS_OK;
+}
+
+nsresult
+nsXPTInterfaceInfo::GetConstantCount(uint16_t* aConstantCount) const
+{
+  *aConstantCount = ConstantCount();
+  return NS_OK;
+}
+
+nsresult
+nsXPTInterfaceInfo::GetMethodInfo(uint16_t aIndex, const nsXPTMethodInfo** aInfo) const
+{
+  *aInfo = aIndex < MethodCount() ? &Method(aIndex) : nullptr;
+  return *aInfo ? NS_OK : NS_ERROR_FAILURE;
+}
+
+nsresult
+nsXPTInterfaceInfo::GetConstant(uint16_t aIndex,
+                                JS::MutableHandleValue aConstant,
+                                char** aName) const
+{
+  *aName = aIndex < ConstantCount()
+    ? moz_xstrdup(Constant(aIndex, aConstant))
+    : nullptr;
+  return *aName ? NS_OK : NS_ERROR_FAILURE;
+}
+
+nsresult
+nsXPTInterfaceInfo::GetTypeForParam(uint16_t /* UNUSED aMethodIndex */,
+                                    const nsXPTParamInfo* aParam,
+                                    uint16_t aDimension,
+                                    nsXPTType* aRetval) const
+{
+  const nsXPTType* type = &aParam->Type();
+  for (uint16_t i = 0; i < aDimension; ++i) {
+    if (type->Tag() != TD_ARRAY) {
+      NS_ERROR("bad dimension");
+      return NS_ERROR_INVALID_ARG;
+    }
+    type = &type->ArrayElementType();
+  }
+
+  *aRetval = *type; // NOTE: This copies the type, which is fine I guess?
+  return NS_OK;
+}
+
+nsresult
+nsXPTInterfaceInfo::GetSizeIsArgNumberForParam(uint16_t /* UNUSED aMethodIndex */,
+                                               const nsXPTParamInfo* aParam,
+                                               uint16_t aDimension,
+                                               uint8_t* aRetval) const
+{
+  const nsXPTType* type = &aParam->Type();
+  for (uint16_t i = 0; i < aDimension; ++i) {
+    if (type->Tag() != TD_ARRAY) {
+      NS_ERROR("bad dimension");
+      return NS_ERROR_INVALID_ARG;
+    }
+    type = &type->ArrayElementType();
+  }
+
+  if (type->Tag() != TD_ARRAY &&
+      type->Tag() != TD_PSTRING_SIZE_IS &&
+      type->Tag() != TD_PWSTRING_SIZE_IS) {
+    NS_ERROR("not a size_is");
+    return NS_ERROR_INVALID_ARG;
+  }
+
+  *aRetval = type->ArgNum();
+  return NS_OK;
+}
+
+nsresult
+nsXPTInterfaceInfo::GetInterfaceIsArgNumberForParam(uint16_t /* UNUSED aMethodIndex */,
+                                                    const nsXPTParamInfo* aParam,
+                                                    uint8_t* aRetval) const
+{
+  const nsXPTType* type = &aParam->Type();
+  while (type->Tag() == TD_ARRAY) {
+    type = &type->ArrayElementType();
+  }
+
+  if (type->Tag() != TD_INTERFACE_IS_TYPE) {
+    NS_ERROR("not an iid_is");
+    return NS_ERROR_INVALID_ARG;
+  }
+
+  *aRetval = type->ArgNum();
+  return NS_OK;
+}
+
+nsresult
+nsXPTInterfaceInfo::IsIID(const nsIID* aIID, bool* aIs) const
+{
+  *aIs = mIID == *aIID;
+  return NS_OK;
+}
+
+nsresult
+nsXPTInterfaceInfo::GetNameShared(const char** aName) const
+{
+  *aName = Name();
+  return NS_OK;
+}
+
+nsresult
+nsXPTInterfaceInfo::GetIIDShared(const nsIID** aIID) const
+{
+  *aIID = &IID();
+  return NS_OK;
+}
+
+nsresult
+nsXPTInterfaceInfo::IsFunction(bool* aRetval) const
+{
+  *aRetval = IsFunction();
+  return NS_OK;
+}
+
+nsresult
+nsXPTInterfaceInfo::HasAncestor(const nsIID* aIID, bool* aRetval) const
+{
+  *aRetval = HasAncestor(*aIID);
+  return NS_OK;
+}
+
+nsresult
+nsXPTInterfaceInfo::GetIIDForParamNoAlloc(uint16_t aMethodIndex,
+                                          const nsXPTParamInfo* aParam,
+                                          nsIID* aIID) const
+{
+  const nsXPTType* type = &aParam->Type();
+  while (type->Tag() == TD_ARRAY) {
+    type = &type->ArrayElementType();
+  }
+
+  if (type->Tag() == TD_INTERFACE_TYPE) {
+    const nsXPTInterfaceInfo* info = type->GetInterface();
+    if (info) {
+      *aIID = info->IID();
+      return NS_OK;
+    }
+  }
+  return NS_ERROR_FAILURE;
+}
+
+nsresult
+nsXPTInterfaceInfo::IsMainProcessScriptableOnly(bool* aRetval) const
+{
+  *aRetval = IsMainProcessScriptableOnly();
+  return NS_OK;
+}

xpcom/reflect/xptinfo/xptinfo.h

@@ -1,216 +1,518 @@
-/* -*- Mode: C++; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 4 -*- */
+/* -*- Mode: C++; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
+/* vim: set ts=8 sts=2 et sw=2 tw=80: */
 /* This Source Code Form is subject to the terms of the Mozilla Public
  * License, v. 2.0. If a copy of the MPL was not distributed with this
- * file, You can obtain one at http://mozilla.org/MPL/2.0/. */
+ * file, You can obtain one at https://mozilla.org/MPL/2.0/. */
 
-/* XPTI_PUBLIC_API and XPTI_GetInterfaceInfoManager declarations. */
+/**
+ * Structures and methods with information about XPCOM interfaces for use by
+ * XPConnect. The static backing data structures used by this file are generated
+ * from xpidl interfaces by the jsonxpt.py and xptcodegen.py scripts.
+ */
 
-#ifndef xptiinfo_h___
+#ifndef xptinfo_h
-#define xptiinfo_h___
+#define xptinfo_h
 
-#include "nscore.h"
+#include <stdint.h>
-#include "xpt_struct.h"
+#include "nsID.h"
+#include "mozilla/Assertions.h"
+#include "js/Value.h"
 
-// Flyweight wrapper classes for xpt_struct.h structs.
+// Forward Declarations
-// Everything here is dependent upon - and sensitive to changes in -
+namespace mozilla {
-// xpcom/typelib/xpt/xpt_struct.h!
+namespace dom {
+struct NativePropertyHooks;
+} // namespace dom
+} // namespace mozilla
 
-class nsXPTType : public XPTTypeDescriptorPrefix
+struct nsXPTInterfaceInfo;
+struct nsXPTType;
+struct nsXPTParamInfo;
+struct nsXPTMethodInfo;
+
+// Internal helper methods.
+namespace xpt {
+namespace detail {
+
+inline const nsXPTInterfaceInfo* GetInterface(uint16_t aIndex);
+inline const nsXPTType& GetType(uint16_t aIndex);
+inline const nsXPTParamInfo& GetParam(uint16_t aIndex);
+inline const nsXPTMethodInfo& GetMethod(uint16_t aIndex);
+inline const char* GetString(uint32_t aIndex);
+
+extern const uint16_t sInterfacesSize;
+
+} // namespace detail
+} // namespace xpt
+
+
+/*
+ * An Interface describes a single XPCOM interface, including all of its
+ * methods. We don't record non-scriptable interfaces.
+ */
+struct nsXPTInterfaceInfo
 {
-// NO DATA - this a flyweight wrapper
+  // High efficiency getters for Interfaces based on perfect hashes.
-public:
+  static const nsXPTInterfaceInfo* ByIID(const nsIID& aIID);
-    nsXPTType()
+  static const nsXPTInterfaceInfo* ByName(const char* aName);
-        {}    // random contents
-    MOZ_IMPLICIT nsXPTType(const XPTTypeDescriptorPrefix& prefix)
-        {*(XPTTypeDescriptorPrefix*)this = prefix;}
 
-    MOZ_IMPLICIT nsXPTType(const uint8_t& prefix)
+  // These are only needed for Components_interfaces's enumerator.
-        {*(uint8_t*)this = prefix;}
+  static const nsXPTInterfaceInfo* ByIndex(uint16_t aIndex) {
+    // NOTE: We add 1 here, as the internal index 0 is reserved for null.
+    return xpt::detail::GetInterface(aIndex + 1);
+  }
+  static uint16_t InterfaceCount() { return xpt::detail::sInterfacesSize; }
 
-    nsXPTType& operator=(uint8_t val)
-        {mFlags = val; return *this;}
 
-    nsXPTType& operator=(const nsXPTType& other)
+  // Interface flag getters
-        {mFlags = other.mFlags; return *this;}
+  bool IsScriptable() const { return true; } // XXX remove (backcompat)
+  bool IsFunction() const { return mFunction; }
+  bool IsBuiltinClass() const { return mBuiltinClass; }
+  bool IsMainProcessScriptableOnly() const { return mMainProcessScriptableOnly; }
 
-    operator uint8_t() const
+  const char* Name() const { return xpt::detail::GetString(mName); }
-        {return mFlags;}
+  const nsIID& IID() const { return mIID; }
 
-    // 'Arithmetic' here roughly means that the value is self-contained and
+  // Get the parent interface, or null if this interface doesn't have a parent.
-    // doesn't depend on anything else in memory (ie: not a pointer, not an
+  const nsXPTInterfaceInfo* GetParent() const {
-    // XPCOM object, not a jsval, etc).
+    return xpt::detail::GetInterface(mParent);
-    //
+  }
-    // Supposedly this terminology comes from Harbison/Steele, but it's still
-    // a rather crappy name. We'd change it if it wasn't used all over the
-    // place in xptcall. :-(
-    bool IsArithmetic() const
-        {return mFlags <= T_WCHAR;}
 
-    // We used to abuse 'pointer' flag bit in typelib format quite extensively.
+  // Do we have an ancestor interface with the given IID?
-    // We've gotten rid of most of the cases, but there's still a fair amount
+  bool HasAncestor(const nsIID& aIID) const;
-    // of refactoring to be done in XPCWrappedJSClass before we can safely stop
-    // asking about this. In the mean time, we've got a temporary version of
-    // IsPointer() that should be equivalent to what's in the typelib.
-    bool deprecated_IsPointer() const
-        {return !IsArithmetic() && TagPart() != T_JSVAL;}
 
-    bool IsInterfacePointer() const
+  // Constant Getters and Setters.
-        {  switch (TagPart()) {
+  uint16_t ConstantCount() const;
-             default:
+  const char* Constant(uint16_t aIndex, JS::MutableHandleValue aConst) const;
-               return false;
-             case T_INTERFACE:
-             case T_INTERFACE_IS:
-               return true;
-           }
-        }
 
-    bool IsArray() const
+  // Method Getters and Setters.
-        {return TagPart() == T_ARRAY;}
+  uint16_t MethodCount() const { return mNumMethods; }
+  const nsXPTMethodInfo& Method(uint16_t aIndex) const;
 
-    // 'Dependent' means that params of this type are dependent upon other
-    // params. e.g. an T_INTERFACE_IS is dependent upon some other param at
-    // runtime to say what the interface type of this param really is.
-    bool IsDependent() const
-        {  switch (TagPart()) {
-             default:
-               return false;
-             case T_INTERFACE_IS:
-             case TD_ARRAY:
-             case T_PSTRING_SIZE_IS:
-             case T_PWSTRING_SIZE_IS:
-               return true;
-           }
-        }
 
-    enum
+  //////////////////////////////////////////////
-    {
+  // nsIInterfaceInfo backwards compatibility //
-        T_I8                = TD_INT8             ,
+  //////////////////////////////////////////////
-        T_I16               = TD_INT16            ,
+
-        T_I32               = TD_INT32            ,
+  nsresult GetName(char** aName) const;
-        T_I64               = TD_INT64            ,
+  nsresult IsScriptable(bool* aRes) const;
-        T_U8                = TD_UINT8            ,
+  nsresult IsBuiltinClass(bool* aRes) const;
-        T_U16               = TD_UINT16           ,
+  nsresult GetParent(const nsXPTInterfaceInfo** aParent) const;
-        T_U32               = TD_UINT32           ,
+  nsresult GetMethodCount(uint16_t* aMethodCount) const;
-        T_U64               = TD_UINT64           ,
+  nsresult GetConstantCount(uint16_t* aConstantCount) const;
-        T_FLOAT             = TD_FLOAT            ,
+  nsresult GetMethodInfo(uint16_t aIndex, const nsXPTMethodInfo** aInfo) const;
-        T_DOUBLE            = TD_DOUBLE           ,
+  nsresult GetConstant(uint16_t aIndex,
-        T_BOOL              = TD_BOOL             ,
+                       JS::MutableHandleValue constant,
-        T_CHAR              = TD_CHAR             ,
+                       char** aName) const;
-        T_WCHAR             = TD_WCHAR            ,
+  nsresult GetTypeForParam(uint16_t aMethodIndex, const nsXPTParamInfo* aParam,
-        T_VOID              = TD_VOID             ,
+                           uint16_t aDimension, nsXPTType* aRetval) const;
-        T_IID               = TD_PNSIID           ,
+  nsresult GetSizeIsArgNumberForParam(uint16_t aMethodIndex,
-        T_DOMSTRING         = TD_DOMSTRING        ,
+                                      const nsXPTParamInfo* aParam,
-        T_CHAR_STR          = TD_PSTRING          ,
+                                      uint16_t aDimension,
-        T_WCHAR_STR         = TD_PWSTRING         ,
+                                      uint8_t* aRetval) const;
-        T_INTERFACE         = TD_INTERFACE_TYPE   ,
+  nsresult GetInterfaceIsArgNumberForParam(uint16_t aMethodIndex,
-        T_INTERFACE_IS      = TD_INTERFACE_IS_TYPE,
+                                           const nsXPTParamInfo* aParam,
-        T_ARRAY             = TD_ARRAY            ,
+                                           uint8_t* aRetval) const;
-        T_PSTRING_SIZE_IS   = TD_PSTRING_SIZE_IS  ,
+  nsresult IsIID(const nsIID* aIID, bool* aIs) const;
-        T_PWSTRING_SIZE_IS  = TD_PWSTRING_SIZE_IS ,
+  nsresult GetNameShared(const char** aName) const;
-        T_UTF8STRING        = TD_UTF8STRING       ,
+  nsresult GetIIDShared(const nsIID** aIID) const;
-        T_CSTRING           = TD_CSTRING          ,
+  nsresult IsFunction(bool* aRetval) const;
-        T_ASTRING           = TD_ASTRING          ,
+  nsresult HasAncestor(const nsIID* aIID, bool* aRetval) const;
-        T_JSVAL             = TD_JSVAL
+  nsresult GetIIDForParamNoAlloc(uint16_t aMethodIndex,
-    };
+                                 const nsXPTParamInfo* aParam,
-// NO DATA - this a flyweight wrapper
+                                 nsIID* aIID) const;
+  nsresult IsMainProcessScriptableOnly(bool* aRetval) const;
+
+  // XXX: We can probably get away with removing this method. A shim interface
+  // _should_ never show up in code which calls EnsureResolved().
+  bool EnsureResolved() const { return !mIsShim; }
+
+  ////////////////////////////////////////////////////////////////
+  // Ensure these fields are in the same order as xptcodegen.py //
+  ////////////////////////////////////////////////////////////////
+
+  nsID mIID;
+  uint32_t mName; // Index into xpt::detail::sStrings
+
+  uint16_t mParent : 14;
+  uint16_t mBuiltinClass : 1;
+  // XXX(nika): Do we need this if we don't have addons anymore?
+  uint16_t mMainProcessScriptableOnly : 1;
+
+  uint16_t mMethods; // Index into xpt::detail::sMethods
+
+  uint16_t mConsts : 14; // Index into xpt::detail::sConsts
+  uint16_t mIsShim : 1; // Is this interface a WebIDL shim?
+  uint16_t mFunction : 1;
+
+  uint8_t mNumMethods; // NOTE(24/04/18): largest=nsIDocShell (193)
+  uint8_t mNumConsts; // NOTE(24/04/18): largest=nsIAccessibleRole (175)
 };
 
-class nsXPTParamInfo : public XPTParamDescriptor
+// The fields in nsXPTInterfaceInfo were carefully ordered to minimize size.
+static_assert(sizeof(nsXPTInterfaceInfo) == 28, "wrong size?");
+
+
+/*
+ * The following enum represents contains the different tag types which
+ * can be found in nsXPTTypeInfo::mTag.
+ *
+ * WARNING: mTag is 5 bits wide, supporting at most 32 tags.
+ */
+enum nsXPTTypeTag : uint8_t
 {
-// NO DATA - this a flyweight wrapper
+  TD_INT8              = 0,
-public:
+  TD_INT16             = 1,
-    MOZ_IMPLICIT nsXPTParamInfo(const XPTParamDescriptor& desc)
+  TD_INT32             = 2,
-        {*(XPTParamDescriptor*)this = desc;}
+  TD_INT64             = 3,
-
+  TD_UINT8             = 4,
-
+  TD_UINT16            = 5,
-    bool IsIn() const {return !!(mFlags & kInMask);}
+  TD_UINT32            = 6,
-    bool IsOut() const {return !!(mFlags & kOutMask);}
+  TD_UINT64            = 7,
-    bool IsRetval() const {return !!(mFlags & kRetvalMask);}
+  TD_FLOAT             = 8,
-    bool IsShared() const {return !!(mFlags & kSharedMask);}
+  TD_DOUBLE            = 9,
-
+  TD_BOOL              = 10,
-    // Dipper types are one of the more inscrutable aspects of xpidl. In a
+  TD_CHAR              = 11,
-    // nutshell, dippers are empty container objects, created and passed by
+  TD_WCHAR             = 12,
-    // the caller, and filled by the callee. The callee receives a fully-
+  TD_VOID              = 13,
-    // formed object, and thus does not have to construct anything. But
+  TD_PNSIID            = 14,
-    // the object is functionally empty, and the callee is responsible for
+  TD_DOMSTRING         = 15,
-    // putting something useful inside of it.
+  TD_PSTRING           = 16,
-    //
+  TD_PWSTRING          = 17,
-    // XPIDL decides which types to make dippers. The list of these types
+  TD_INTERFACE_TYPE    = 18,
-    // is given in the isDipperType() function in typelib.py, and is currently
+  TD_INTERFACE_IS_TYPE = 19,
-    // limited to 4 string types.
+  TD_ARRAY             = 20,
-    //
+  TD_PSTRING_SIZE_IS   = 21,
-    // When a dipper type is declared as an 'out' parameter, xpidl internally
+  TD_PWSTRING_SIZE_IS  = 22,
-    // converts it to an 'in', and sets the XPT_PD_DIPPER flag on it. For this
+  TD_UTF8STRING        = 23,
-    // reason, dipper types are sometimes referred to as 'out parameters
+  TD_CSTRING           = 24,
-    // masquerading as in'. The burden of maintaining this illusion falls mostly
+  TD_ASTRING           = 25,
-    // on XPConnect, which creates the empty containers, and harvest the results
+  TD_JSVAL             = 26
-    // after the call.
-    bool IsDipper() const {return !!(mFlags & kDipperMask);}
-    bool IsOptional() const {return !!(mFlags & kOptionalMask);}
-    const nsXPTType GetType() const {return mType.mPrefix;}
-
-    bool IsStringClass() const {
-      switch (GetType().TagPart()) {
-        case nsXPTType::T_ASTRING:
-        case nsXPTType::T_DOMSTRING:
-        case nsXPTType::T_UTF8STRING:
-        case nsXPTType::T_CSTRING:
-          return true;
-        default:
-          return false;
-      }
-    }
-
-    // Whether this parameter is passed indirectly on the stack. This mainly
-    // applies to out/inout params, but we use it unconditionally for certain
-    // types.
-    bool IsIndirect() const {return IsOut() ||
-                               GetType().TagPart() == nsXPTType::T_JSVAL;}
-
-    // NOTE: other activities on types are done via methods on nsIInterfaceInfo
-
-private:
-    static const uint8_t kInMask =       0x80;
-    static const uint8_t kOutMask =      0x40;
-    static const uint8_t kRetvalMask =   0x20;
-    static const uint8_t kSharedMask =   0x10;
-    static const uint8_t kDipperMask =   0x08;
-    static const uint8_t kOptionalMask = 0x04;
-
-    nsXPTParamInfo() = delete;
-// NO DATA - this a flyweight wrapper
 };
 
-class nsXPTMethodInfo : public XPTMethodDescriptor
+
+/*
+ * A nsXPTType is a union used to identify the type of a method argument or
+ * return value. The internal data is stored as an 5-bit tag, and two 8-bit
+ * integers, to keep alignment requirements low.
+ *
+ * nsXPTType contains 3 extra bits, reserved for use by nsXPTParamInfo.
+ */
+struct nsXPTType
 {
-// NO DATA - this a flyweight wrapper
+  nsXPTTypeTag Tag() const { return static_cast<nsXPTTypeTag>(mTag); }
-public:
-    MOZ_IMPLICIT nsXPTMethodInfo(const XPTMethodDescriptor& desc)
-        {*(XPTMethodDescriptor*)this = desc;}
 
-    bool IsGetter() const { return !!(mFlags & kGetterMask); }
+  uint8_t ArgNum() const {
-    bool IsSetter() const { return !!(mFlags & kSetterMask); }
+    MOZ_ASSERT(Tag() == TD_INTERFACE_IS_TYPE ||
-    bool IsNotXPCOM() const { return !!(mFlags & kNotXPCOMMask); }
+               Tag() == TD_PSTRING_SIZE_IS ||
-    bool IsHidden() const { return !!(mFlags & kHiddenMask); }
+               Tag() == TD_PWSTRING_SIZE_IS ||
-    bool WantsOptArgc() const { return !!(mFlags & kOptArgcMask); }
+               Tag() == TD_ARRAY);
-    bool WantsContext() const { return !!(mFlags & kContextMask); }
+    return mData1;
-    const char* GetName() const { return Name(); }
+  }
-    uint8_t GetParamCount() const { return mNumArgs; }
-    const nsXPTParamInfo GetParam(uint8_t idx) const {
-        MOZ_ASSERT(idx < GetParamCount(), "bad arg");
-        return Param(idx);
-    }
 
-private:
+  const nsXPTType& ArrayElementType() const {
-    static const uint8_t kGetterMask =   0x80;
+    MOZ_ASSERT(Tag() == TD_ARRAY);
-    static const uint8_t kSetterMask =   0x40;
+    return xpt::detail::GetType(mData2);
-    static const uint8_t kNotXPCOMMask = 0x20;
+  }
-    static const uint8_t kHiddenMask =   0x08;
-    static const uint8_t kOptArgcMask =  0x04;
-    static const uint8_t kContextMask =  0x02;
 
-    nsXPTMethodInfo() = delete;
+  // We store the 16-bit iface value as two 8-bit values in order to
-// NO DATA - this a flyweight wrapper
+  // avoid 16-bit alignment requirements for XPTTypeDescriptor, which
+  // reduces its size and also the size of XPTParamDescriptor.
+  const nsXPTInterfaceInfo* GetInterface() const {
+    MOZ_ASSERT(Tag() == TD_INTERFACE_TYPE);
+    uint16_t index = ((uint16_t)mData1 << 8) | mData2;
+    return xpt::detail::GetInterface(index);
+  }
+
+  // 'Arithmetic' here roughly means that the value is self-contained and
+  // doesn't depend on anything else in memory (ie: not a pointer, not an
+  // XPCOM object, not a jsval, etc).
+  //
+  // Supposedly this terminology comes from Harbison/Steele, but it's still
+  // a rather crappy name. We'd change it if it wasn't used all over the
+  // place in xptcall. :-(
+  bool IsArithmetic() const { return Tag() <= TD_WCHAR; }
+
+  // We used to abuse 'pointer' flag bit in typelib format quite extensively.
+  // We've gotten rid of most of the cases, but there's still a fair amount
+  // of refactoring to be done in XPCWrappedJSClass before we can safely stop
+  // asking about this. In the mean time, we've got a temporary version of
+  // IsPointer() that should do the right thing.
+  bool deprecated_IsPointer() const {
+    return !IsArithmetic() && Tag() != TD_JSVAL;
+  }
+
+  bool IsInterfacePointer() const {
+    return Tag() == TD_INTERFACE_TYPE || Tag() == TD_INTERFACE_IS_TYPE;
+  }
+
+  bool IsArray() const { return Tag() == TD_ARRAY; }
+
+  bool IsDependent() const {
+    return Tag() == TD_INTERFACE_IS_TYPE || Tag() == TD_ARRAY ||
+           Tag() == TD_PSTRING_SIZE_IS || Tag() == TD_PWSTRING_SIZE_IS;
+  }
+
+  bool IsStringClass() const {
+    return Tag() == TD_DOMSTRING || Tag() == TD_ASTRING ||
+           Tag() == TD_CSTRING || Tag() == TD_UTF8STRING;
+  }
+
+  ///////////////////////////////////////
+  // nsXPTType backwards compatibility //
+  ///////////////////////////////////////
+
+  nsXPTType& operator=(uint8_t aPrefix) { mTag = aPrefix; return *this; }
+  operator uint8_t() const { return TagPart(); };
+  uint8_t TagPart() const { return mTag; };
+
+  enum // Re-export TD_ interfaces from nsXPTType
+  {
+    T_I8                = TD_INT8             ,
+    T_I16               = TD_INT16            ,
+    T_I32               = TD_INT32            ,
+    T_I64               = TD_INT64            ,
+    T_U8                = TD_UINT8            ,
+    T_U16               = TD_UINT16           ,
+    T_U32               = TD_UINT32           ,
+    T_U64               = TD_UINT64           ,
+    T_FLOAT             = TD_FLOAT            ,
+    T_DOUBLE            = TD_DOUBLE           ,
+    T_BOOL              = TD_BOOL             ,
+    T_CHAR              = TD_CHAR             ,
+    T_WCHAR             = TD_WCHAR            ,
+    T_VOID              = TD_VOID             ,
+    T_IID               = TD_PNSIID           ,
+    T_DOMSTRING         = TD_DOMSTRING        ,
+    T_CHAR_STR          = TD_PSTRING          ,
+    T_WCHAR_STR         = TD_PWSTRING         ,
+    T_INTERFACE         = TD_INTERFACE_TYPE   ,
+    T_INTERFACE_IS      = TD_INTERFACE_IS_TYPE,
+    T_ARRAY             = TD_ARRAY            ,
+    T_PSTRING_SIZE_IS   = TD_PSTRING_SIZE_IS  ,
+    T_PWSTRING_SIZE_IS  = TD_PWSTRING_SIZE_IS ,
+    T_UTF8STRING        = TD_UTF8STRING       ,
+    T_CSTRING           = TD_CSTRING          ,
+    T_ASTRING           = TD_ASTRING          ,
+    T_JSVAL             = TD_JSVAL
+  };
+
+  ////////////////////////////////////////////////////////////////
+  // Ensure these fields are in the same order as xptcodegen.py //
+  ////////////////////////////////////////////////////////////////
+
+  uint8_t mTag : 5;
+
+  // Parameter bitflags are packed into the XPTTypeDescriptor to save space.
+  // When the TypeDescriptor is not in a parameter, these flags are ignored.
+  uint8_t mInParam : 1;
+  uint8_t mOutParam : 1;
+  uint8_t mOptionalParam : 1;
+
+  // The data for the different variants is stored in these two data fields.
+  // These should only be accessed via the getter methods above, which will
+  // assert if the tag is invalid.
+  uint8_t mData1;
+  uint8_t mData2;
 };
 
-#endif /* xptiinfo_h___ */
+// The fields in nsXPTType were carefully ordered to minimize size.
+static_assert(sizeof(nsXPTType) == 3, "wrong size");
+
+
+/*
+ * A nsXPTParamInfo is used to describe either a single argument to a method or
+ * a method's result. It stores its flags in the type descriptor to save space.
+ */
+struct nsXPTParamInfo
+{
+  bool IsIn() const { return mType.mInParam; }
+  bool IsOut() const { return mType.mOutParam && !IsDipper(); }
+  bool IsOptional() const { return mType.mOptionalParam; }
+  bool IsShared() const { return false; } // XXX remove (backcompat)
+
+  // Get the type of this parameter.
+  const nsXPTType& Type() const { return mType; }
+  const nsXPTType& GetType() const { return Type(); } // XXX remove (backcompat)
+
+  // Dipper types are one of the more inscrutable aspects of xpidl. In a
+  // nutshell, dippers are empty container objects, created and passed by the
+  // caller, and filled by the callee. The callee receives a fully- formed
+  // object, and thus does not have to construct anything. But the object is
+  // functionally empty, and the callee is responsible for putting something
+  // useful inside of it.
+  //
+  // Dipper types are treated as `in` parameters when declared as an `out`
+  // parameter. For this reason, dipper types are sometimes referred to as 'out
+  // parameters masquerading as in'. The burden of maintaining this illusion
+  // falls mostly on XPConnect, which creates the empty containers, and harvest
+  // the results after the call.
+  //
+  // Currently, the only dipper types are the string classes.
+  //
+  // XXX: Dipper types may be able to go away? (bug 677784)
+  bool IsDipper() const { return mType.mOutParam && IsStringClass(); }
+
+  // Whether this parameter is passed indirectly on the stack. This mainly
+  // applies to out/inout params, but we use it unconditionally for certain
+  // types.
+  bool IsIndirect() const { return IsOut() || mType.Tag() == TD_JSVAL; }
+
+  bool IsStringClass() const { return mType.IsStringClass(); }
+
+  ////////////////////////////////////////////////////////////////
+  // Ensure these fields are in the same order as xptcodegen.py //
+  ////////////////////////////////////////////////////////////////
+
+  nsXPTType mType;
+};
+
+// The fields in nsXPTParamInfo were carefully ordered to minimize size.
+static_assert(sizeof(nsXPTParamInfo) == 3, "wrong size");
+
+/*
+ * A nsXPTMethodInfo is used to describe a single interface method.
+ */
+struct nsXPTMethodInfo
+{
+  bool IsGetter() const { return mGetter; }
+  bool IsSetter() const { return mSetter; }
+  bool IsNotXPCOM() const { return mNotXPCOM; }
+  bool IsHidden() const { return mHidden; }
+  bool WantsOptArgc() const { return mOptArgc; }
+  bool WantsContext() const { return mContext; }
+  uint8_t ParamCount() const { return mNumParams; }
+
+  const char* Name() const {
+    return xpt::detail::GetString(mName);
+  }
+  const nsXPTParamInfo& Param(uint8_t aIndex) const {
+    MOZ_ASSERT(aIndex < mNumParams);
+    return xpt::detail::GetParam(mParams + aIndex);
+  }
+
+  bool HasRetval() const { return mHasRetval; }
+  const nsXPTParamInfo* GetRetval() const {
+    return mHasRetval ? &Param(mNumParams - 1) : nullptr;
+  }
+
+  /////////////////////////////////////////////
+  // nsXPTMethodInfo backwards compatibility //
+  /////////////////////////////////////////////
+
+  const char* GetName() const { return Name(); }
+  uint8_t GetParamCount() const { return ParamCount(); }
+  const nsXPTParamInfo& GetParam(uint8_t aIndex) const {
+    return Param(aIndex);
+  }
+
+  ////////////////////////////////////////////////////////////////
+  // Ensure these fields are in the same order as xptcodegen.py //
+  ////////////////////////////////////////////////////////////////
+
+  uint32_t mName; // Index into xpt::detail::sStrings.
+  uint16_t mParams; // Index into xpt::detail::sParams.
+  uint8_t mNumParams;
+
+  uint8_t mGetter : 1;
+  uint8_t mSetter : 1;
+  uint8_t mNotXPCOM : 1;
+  uint8_t mHidden : 1;
+  uint8_t mOptArgc : 1;
+  uint8_t mContext : 1;
+  uint8_t mHasRetval : 1;
+  // uint8_t unused : 1;
+};
+
+// The fields in nsXPTMethodInfo were carefully ordered to minimize size.
+static_assert(sizeof(nsXPTMethodInfo) == 8, "wrong size");
+
+
+namespace xpt {
+namespace detail {
+
+/**
+ * The compressed representation of constants from XPT. Not part of the public
+ * interface, as we also need to support Shim interfaces.
+ */
+struct ConstInfo
+{
+  ////////////////////////////////////////////////////////////////
+  // Ensure these fields are in the same order as xptcodegen.py //
+  ////////////////////////////////////////////////////////////////
+
+  uint32_t mName : 31; // Index into xpt::detail::mStrings.
+
+  // Whether the value should be interpreted as a int32_t or uint32_t.
+  uint32_t mSigned: 1;
+  uint32_t mValue; // The value stored as a u32
+};
+
+// The fields in ConstInfo were carefully ordered to minimize size.
+static_assert(sizeof(ConstInfo) == 8, "wrong size");
+
+
+//////////////////////////////////////////////
+// Raw typelib data stored in const statics //
+//////////////////////////////////////////////
+
+// XPIDL information
+extern const nsXPTInterfaceInfo sInterfaces[];
+extern const nsXPTType sTypes[];
+extern const nsXPTParamInfo sParams[];
+extern const nsXPTMethodInfo sMethods[];
+
+extern const char sStrings[];
+extern const ConstInfo sConsts[];
+
+// shim constant information
+extern const mozilla::dom::NativePropertyHooks* sPropHooks[];
+
+// Perfect Hash Function backing data
+static const uint16_t kPHFSize = 256;
+extern const uint32_t sPHF_IIDs[]; // Length == kPHFSize
+extern const uint32_t sPHF_Names[]; // Length == kPHFSize
+extern const uint16_t sPHF_NamesIdxs[]; // Length == sInterfacesSize
+
+
+//////////////////////////////////////
+// Helper Methods for fetching data //
+//////////////////////////////////////
+
+inline const nsXPTInterfaceInfo*
+GetInterface(uint16_t aIndex)
+{
+  if (aIndex > 0 && aIndex <= sInterfacesSize) {
+    return &sInterfaces[aIndex - 1]; // 1-based as 0 is a marker.
+  }
+  return nullptr;
+}
+
+inline const nsXPTType&
+GetType(uint16_t aIndex)
+{
+  return sTypes[aIndex];
+}
+
+inline const nsXPTParamInfo&
+GetParam(uint16_t aIndex)
+{
+  return sParams[aIndex];
+}
+
+inline const nsXPTMethodInfo&
+GetMethod(uint16_t aIndex)
+{
+  return sMethods[aIndex];
+}
+
+inline const char*
+GetString(uint32_t aIndex)
+{
+  return &sStrings[aIndex];
+}
+
+} // namespace detail
+} // namespace xpt
+
+#endif /* xptinfo_h */