Currently XPCVariant has some code for working with arrays of a series of basic
types. I want to unify and simplify code which works with nsXPTTypes to always
take the topmost level type (rather than passing in an array element type when
working with an array).
This is pretty easy for most of XPConnect, but XPCVariant occasionally needs to
perform calls on made-up array types, which isn't compatible with the current
implementation. Fortunately, it only needs a very small set of array types. This
patch adds a set of simple types (mostly the arithmetic types and
TD_INTERFACE_IS_TYPE for interfaces) to the extra types array unconditionally
with a known index, for XPCVariant.
An other option I was considering was to consider the value `0xff` in the data
byte on nsXPTType to be a flag which indicates that the array element type is
actually the type immediately following the current nsXPTType object in memory,
but that was incompatible with many of the existing nsXPTType consumers which
copy the nsXPTType objects around (e.g. onto the stack), rather than always
using them by reference, so I decided it was not a good approach to take.
In XPConnect, native values are passed around within nsXPTCMiniVariant objects.
an [nsXPTCMiniVariant] contains 64-bits of data in a union type.
nsXPTCMiniVariant values are created by the platform-specific glue code and
passed into XPConnect proper when calling from C++ into JS.
When calling from JS into C++, we instead create nsXPTCVariant objects and pass
them into the glue code. These objects have extra information in addition to the
nsXPTCMiniVariant: namely they also have a `type` field with the type stored in
the variant, space for flags, and a `ptr` field which is passed over IPC instead
of the inner nsXPTCMiniVariant when a flag (`PTR_IS_DATA`) is set.
The JSValue type in XPConnect is always passed as a pointer to a JSValue object,
both for in parameters and out parameters. This is handled by making the JSValue
type be unconditionally flagged as [`IsIndirect()`] This flag is also used for
all types of outparameters.
When the `IsIndirect()` flag is set, it means that the actual data is stored in
the nsXPTCVariant's val field, and it sets the flag to tell the glue code to
instead pass the `ptr` field (which is always pointing to the `val` field for
[legacy reasons]) into the C++ code.
In contrast "dipper" is a different and super weird flag. Currently only the
string class types (nsACString & nsAString) are marked as "dipper". A "dipper"
type is always passed as an "in" type (and thus always passed "directly"), even
when it's actually an out parameter.
XPConnect treats these types as though they are pointer types (nsAString*). This
means that there is no space in the nsXPTCVariant to store the actual nsAString
types when passing from JS into C++, so these values have to be allocated by a
different mechanism (in the current code, there is a size 2 buffer for each
string type in the context and once that buffer is exceeded, we heap allocate
the nsString values).
In effect, the current state looks something like this:
+------------+---------------------+---------------------+
| type | out (xpt/native) | in (xpt/native) |
+------------+---------------------+---------------------+
| TD_INT32 | indirect (int32_t*) | direct (int32_t) |
+------------+---------------------+---------------------+
| TD_JSVAL | indirect (JSValue*) | indirect (JSValue*) |
+------------+---------------------+---------------------+
| TD_ASTRING | direct (nsAString*) | direct (nsAString*) |
+------------+---------------------+---------------------+
This patch ensures there is enough space in the nsXPTCVariant to fit the
nsString value, and switches string class types to being unconditionally
indirect instead of using the dipper system. This allows us to delete a ton of
dipper-specific code, and unify the indirect and string class codepaths.
This only affects the size of nsXPTCVariant objects, and does not affect
nsXPTCMiniVariant objects. nsXPTCVariant objects are never allocated by the
platform-specific binding code, rather, they are allocated in an AutoTArray on
the stack as part of the CallMethodHelper object.
The size increase is a total of 1 word, so 4 bytes in 32-bit builds, and 8 bytes
in 64-bit builds, which is ignorable for stack allocated objects.
[nsXPTCMiniVariant]: https://searchfox.org/mozilla-central/rev/eb6c5214a63e20a3fff455e92c876287a8d2e188/xpcom/reflect/xptcall/xptcall.h#20-47
[`IsIndirect()`]: https://searchfox.org/mozilla-central/rev/c0d81882c7941c4ff13a50603e37095cdab0d1ea/xpcom/reflect/xptinfo/xptinfo.h#371
[legacy reasons]: https://searchfox.org/mozilla-central/rev/eb6c5214a63e20a3fff455e92c876287a8d2e188/xpcom/reflect/xptcall/xptcall.h#66-79
It used to be that in XPConnect there were many different pieces of code for
each place where we may need to clean up some untyped values based on their
nsXPTType information. This was a mess, and meant that every time you needed to
add a new data type you'd have to find every one of these places and add support
for your new type to them.
In fact, this was bad enough that it appears that I missed some places when
adding my webidl support! Which means that in some edge cases we may clean up
one of these values incorrectly D:!
This patch adds a new unified method which performs the cleanup by looking at a
nsXPTType object. The idea is that this function takes a void* which is actually
a T* where T is a value of the nsXPTType parmaeter. It clears the value behind
the pointer to a valid state such that free-ing the memory would not cause any
leaks. e.g. it free(...)s owned pointers and sets the pointer to `nullptr`, and
truncates nsA[C]String values such that they reference the static empty string.
I also modify every one of these custom cleanup codepaths to instead call into
this unified cleanup method.
This also involved some simplification of helper methods in order to make the
implementation cleaner.
This type is fairly simple on the idl parsing side of things. I handle it in the
same way that special types such as ns[C]String, nsid, and jsval are handled, by
using a special native type.
The logic for converting the value between C++ and JS follows the existing logic
from the nsISupports <=> JS Promise conversions.
This patch goes through the XPConnect conversion methods, and adds cases for
T_DOMOBJECT which call the Wrap, Unwrap, and Cleanup methods from the
nsXPTDOMObjectInfo objects created in the last part.
For consistency with normal interface pointers, and because it wasn't too
complex, I also added support for including T_DOMOBJECTs in XPCOM arrays.
Unlike the other lists in xptinfo, this list contains relocations. Each
DOMObject has 3 functions generated for it, `Wrap`, `Unwrap` and `Cleanup`,
which perform the necessary actions. These are stored as function pointers.
Wrap gets the DOMObject wrapper using the DOM binding code, Unwrap gets the
underlying C++ object, and addrefs it (as XPCOM methods return native types
via getter_AddRefs), and Cleanup releases a reference to the underlying
C++ object, for when the unwrapped object is used as a temporary during a call.
To generate the code, we need to have the declaration of the native C++ type
in scope, so we also emit #include-s for the headerFiles.
Due to the decision to keep the old API on nsXPTInterfaceInfo in part 4, this is
a fairly straightforward patch.
1. I had to change a couple of consumers of `IsRetval()` due to the movement of
that flag.
2. I changed all code which held a nsIInterfaceInfo to hold an `const
nsXPTInterfaceInfo*` instead.
3. I changed code which used the nsIInterfaceInfoManager to instead call the
static methods on nsXPTInterfaceInfo.
This patch contains the meat of the changes here. The following summarize the changes:
1. xptinfo.h is rewritten to expose the new interface for reading the XPT data,
The nsXPTInterfaceInfo object exposes methods with the same signatures as
the methods on nsIInterfaceInfo, to make converting code which used
nsIInterfaceInfo as easy as possible, even when those methods don't have
signatures which make a ton of sense anymore. There are also a few methods
which are unnecessary (they return `true` or similar), which should be
removed over time.
Members of the data structures are made private in order to prevent reading
them directly. Code should instead call the getter methods. This should make
it easier to change their memory representation in the future. Constructing
these structs is made possible by making the structs `friend class` with the
XPTConstruct class, which is implemented by the code generator, and is able
to access the private fields.
In addition, rather than using integers with flag constants, I opted for
using C++ bitfields to store individual flags, as I found it made it easier
to both write the code generator, and reason about the layouts of the types.
I was able to shave a byte off of each nsXPTParamInfo (4 bytes -> 3 bytes)
by shoving the flags into spare bits in the nsXPTType. Unfortunately there
was not enough room for the retval flag. Fortunately, we already depend in
our code on the retval parameter being the last parameter, so I worked
around this by removing the retval flag and instead having a `hasretval`
flag on the method itself.
2. An xptinfo.cpp file is added for out-of-line definitions of more complex
methods, and the internal implementation details of the perfect hash.
Notable is the handling of xptshim interfaces. As the type is uniform, a
flag is checked when trying to read constant information, and a different
table with pointers into webidl data structures is checked when the type is
determined to be a shim.
Ideally we could remove this once we remove the remaining consumers of the
existing shim interfaces.
3. A python code generator which takes in the json XPT files generated in the
previous part, and emits a xptdata.cpp file with the data structures. I did
my best to heavily comment the code.
This code uses the friend class trick to construct the private fields of the
structs, and avoid a dependency on the ordering of fields in xptinfo.h.
The sInterfaces array's order is determined by a generated perfect hash
which is also written into the binary. This should allow for fast lookups by
IID or name of interfaces in memory. The hash function used for the perfect
hash is a simple FNV hash, as they're pretty fast.
For perfect hashing of names, another table is created which contains
indexes into the sInterfaces table. Lookup by name is less common, and this
form of lookup should still be very fast.
4. The necessary Makefiles are updated to use the new code generator, and
generate the file correctly.
Unfortunately, I wasn't able to figure out a way to make firefox build & run in
the intermediate stages of these commits. Because of this, I am going to just
delete most of the code which I am deleting in the first patch, as I figure that
those are somewhat uninteresting changes, and then make the other changes in the
following patches.
In total, the following things are deleted:
1. All of xpcom/typelib, except for `xpt/tools` - this directory is being
subsumed entirely into xpcom/reflect/xptinfo.
2. Most of the code in xpcom/reflect/xptinfo, it is being rewritten to avoid
allocating and contain all of the necessary data structures.
3. idl-parser's typelib.py XPT generator, as it will be replaced.
4. Most includes of files which have been deleted.
NOTE: xpcom/typelib/xpt/tools/xpt.py was not removed, as it is used by bundling
code & bundling tests, which we don't want to remove yet.
Many of these fields have accessors, and can only be read indirectly
by going through the XPTHeader data structure anyways, so they should
be marked private. This makes the generated XPT data file noisier due
to the need for constexpr constructors.
I had to fix the ctors for the classes in xptinfo.h to be less weird
because there was a compiler error.
Members in two of the classes need to be marked protected because they
have subclasses in xptinfo.h. Ideally those classes would be merged
in.
MozReview-Commit-ID: 70IdFAhp5je
--HG--
extra : rebase_source : 48cb1ec84c872b8fd6bb8c07f8ad2b47d0173f28
The things in xptiWorkingSet are neither used nor defined.
MozReview-Commit-ID: 1ZPVe38OVcH
--HG--
extra : rebase_source : 6f74c70758cb66bfdc6ec0a2a27d8cead1b37ed6
With fully linked XPT data, there is exactly one directory entry per
descriptor, plus one per unresolved interface. There are 1200 of the
former and 40 of the latter. By merging them, we save a 32 bit int per
directory entry, at the cost of 11 bytes per unresolved interface.
This will make VerifyAndAddEntryIfNew slightly slower because it has
to do an nsID equality check rather than a null check, but I can't
imagine that will matter.
My main goal for this patch is to reduce the size of the executable,
to avoid a regression with my static XPT information patches, but it
should reduce memory a little bit, too.
MozReview-Commit-ID: L35YdOuAyF4
--HG--
extra : rebase_source : 8358a73a0f8f81081661538d4a7c9a31b1aa7a56
To allow XPT information to be shared between processes, it needs to
not contain any pointers, because they cause relocations. I've
eliminated pointers by hoisting all of the variable length data
structures to XPTHeader, into a single array for each type of
data. These data structures now use an index into these arrays to find
their first element. Strings are similar, but they are mashed into a
single giant string, including embedded null terminators. Modifying
the accessor methods to support this is easy, because there is only a
single global instance of each of these arrays in XPTHeader.
MozReview-Commit-ID: 5rgwaEBvDYl
--HG--
extra : rebase_source : 2e423f088d662920a89f3b66c70d26fe340b5fce
Now that there is only one XPTHeader, we can devolve the fields in it
to avoid some indirection. The biggest part here is getting rid of the
mHeader field on xptiTypelibGuts.
The array is [] instead of * to avoid a relocation, by ensuring that
XPTHeader::kInterfaceDirectory as well as the data it points to cannot
be changed.
MozReview-Commit-ID: AzvNTNZKkfi
--HG--
extra : rebase_source : d911a54b3db1f9f57b538d13ae86f28965ab33b3
This patch removes C++ code related to reading in XPT information from
files. (Code related to packaging XPT files will be removed in the
next patch.) This includes code in the manifest parser, in addition to
the actual code for parsing files.
XPT information is now loaded directly from a single static data
structure, XPTHeader::kHeader, which will be automatically generated
at compile time from .idl files (via .xpt files). Note that the script
to do that is not added until part 6 of this patch series, so linking
will fail on parts 2 through 5.
I inlined XPTInterfaceInfoManager::RegisterXPTHeader into the ctor,
because that is the only caller. It feels like the lock there should
not be needed any more, but I left it alone for now.
The forward declaration of XPTArena in xptiprivate.h is needed because
it was being bootlegged via xpt_xdr.h. Some of the data structures in
reflect/xptinfo/ (which wrap the xpt_struct.h data structures) are
still allocated using XPTArena. Hopefully we can get rid of that in
followup work.
I also deleted a lot of comments in xpt_struct.h that talk about the
on-disk format. I also deleted checking of the major version number,
because that should not matter when the XPT information is baked into
the executable.
MozReview-Commit-ID: 6NJdaCWRBhU
--HG--
extra : rebase_source : 6512a05f2a8bee1e6e6b0423d2cb376d8c34728b
This lets us hide later changes to how these arrays are stored. There
should be no behavioral change. Some methods in xpt_struct.h are
declared inline at the end of the header due to the order that classes
are declared in the header.
MozReview-Commit-ID: KAxUKn3sDOD
--HG--
extra : rebase_source : 867ce100e5178c85485e6c3bac5d6bfca098f78b
With fully linked XPT data, there is exactly one directory entry per
descriptor, plus one per unresolved interface. There are 1200 of the
former and 40 of the latter. By merging them, we save a 32 bit int per
directory entry, at the cost of 11 bytes per unresolved interface.
This will make VerifyAndAddEntryIfNew slightly slower because it has
to do an nsID equality check rather than a null check, but I can't
imagine that will matter.
My main goal for this patch is to reduce the size of the executable,
to avoid a regression with my static XPT information patches, but it
should reduce memory a little bit, too.
MozReview-Commit-ID: L35YdOuAyF4
--HG--
extra : rebase_source : b3d0780808f9fc708b5b2dffeda45030974dc0bf
To allow XPT information to be shared between processes, it needs to
not contain any pointers, because they cause relocations. I've
eliminated pointers by hoisting all of the variable length data
structures to XPTHeader, into a single array for each type of
data. These data structures now use an index into these arrays to find
their first element. Strings are similar, but they are mashed into a
single giant string, including embedded null terminators. Modifying
the accessor methods to support this is easy, because there is only a
single global instance of each of these arrays in XPTHeader.
MozReview-Commit-ID: 5rgwaEBvDYl
--HG--
extra : rebase_source : 6c35fd1034b727443bb87dfde0a4af4799ed480a
Now that there is only one XPTHeader, we can devolve the fields in it
to avoid some indirection. The biggest part here is getting rid of the
mHeader field on xptiTypelibGuts.
The array is [] instead of * to avoid a relocation, by ensuring that
XPTHeader::kInterfaceDirectory as well as the data it points to cannot
be changed.
MozReview-Commit-ID: AzvNTNZKkfi
--HG--
extra : rebase_source : 7b2043cecee5e8d063ea4d0b4a2223b2261c62e7
This patch removes C++ code related to reading in XPT information from
files. (Code related to packaging XPT files will be removed in the
next patch.) This includes code in the manifest parser, in addition to
the actual code for parsing files.
XPT information is now loaded directly from a single static data
structure, XPTHeader::kHeader, which will be automatically generated
at compile time from .idl files (via .xpt files). Note that the script
to do that is not added until part 6 of this patch series, so linking
will fail on parts 2 through 5.
I inlined XPTInterfaceInfoManager::RegisterXPTHeader into the ctor,
because that is the only caller. It feels like the lock there should
not be needed any more, but I left it alone for now.
The forward declaration of XPTArena in xptiprivate.h is needed because
it was being bootlegged via xpt_xdr.h. Some of the data structures in
reflect/xptinfo/ (which wrap the xpt_struct.h data structures) are
still allocated using XPTArena. Hopefully we can get rid of that in
followup work.
I also deleted a lot of comments in xpt_struct.h that talk about the
on-disk format. I also deleted checking of the major version number,
because that should not matter when the XPT information is baked into
the executable.
MozReview-Commit-ID: 6NJdaCWRBhU
--HG--
extra : rebase_source : e169b827a64bad5dde15de6be0b2ff5e7aa35e3f
This lets us hide later changes to how these arrays are stored. There
should be no behavioral change. Some methods in xpt_struct.h are
declared inline at the end of the header due to the order that classes
are declared in the header.
MozReview-Commit-ID: KAxUKn3sDOD
--HG--
extra : rebase_source : 782fddc7bbf29c8ec73a0c6dbcca0621739e6c6e
Many of these fields have accessors, and can only be read indirectly
by going through the XPTHeader data structure anyways, so they should
be marked private. This makes the generated XPT data file noisier due
to the need for constexpr constructors.
I had to fix the ctors for the classes in xptinfo.h to be less weird
because there was a compiler error.
Members in two of the classes need to be marked protected because they
have subclasses in xptinfo.h. Ideally those classes would be merged
in.
MozReview-Commit-ID: 70IdFAhp5je
--HG--
extra : rebase_source : a2670a65ace291defc47845c4058477f0ae5360a
For bug 1438688, I need to statically initialize all of the data types
in xpt_struct.h. It turns out that the approach I was using for unions
is C99 and not C++. While for some reason Clang and GCC are okay with
that, MSVC is not.
One of the unions is a gnarly anonymous union in
XPTTypeDescriptor. However, every arm of this union is either 1 or 2
uint8_t, so I think it is reasonable to eliminate the union and
replace it with two fields, mData1 and mData2. I've fixed up the
places that read this data to use accessor methods with nice asserts
about the tag on the variant, so if anything I think that part is
nicer.
The code in xpt_struct.cpp that writes to this data is maybe less
nice, but I'm deleting it in bug 1438688 so I think that's okay. We
also lose some detail in the comments about the relationship between
what is stored in memory compared to what is stored on disk, but I
think that's okay, too.
MozReview-Commit-ID: DGi19f8HnMi
--HG--
extra : rebase_source : e3535ecb3d7f02bdb643df153e8aba3e106d9104
Nika Layzell
Jeff Walden
Jorg K
Andrew McCreight
Boris Zbarsky
Narcis Beleuzu
arthur.iakab
Dorel Luca
Andreea Pavel
Peter Van der Beken