Jon Thysell 4c64b059c1 Merge facebook/hermes version RN 0.74.3 (#196) ... * Add JSI method for setting external memory size Summary: X-link: https://github.com/facebook/react-native/pull/41436 Add a JSI API for associating some native memory with a JS object. This is intended to provide a mechanism to trigger more frequent garbage collection when JS retains large external memory allocations, in order to avoid memory buildup. This diff just adds the JSI method, without any implementations. Changelog: [General][Added] - Added JSI method for reporting native memory to the GC. Reviewed By: tmikov Differential Revision: D50524912 fbshipit-source-id: c8df0e18b0415d9523e0a00f6d0ed2faa648ac68 * Deploy 0.223.0 to fbsource (#41725) Summary: Pull Request resolved: https://github.com/facebook/react-native/pull/41725 Reviewed By: SamChou19815 Differential Revision: D51713020 fbshipit-source-id: 84dc8c882f7603d642cb4ccb735fad0fb7e25d4d * Pass additional parameters to NativeState finalizer Summary: Pass the GC and `NativeState*` to the finalizer to make it more flexible. This will allow it to be used to manage external memory in the next diff. Reviewed By: tmikov Differential Revision: D50531361 fbshipit-source-id: f5483f0e25d92156bc4f593ffa0f19c6de88a1c0 * Remove asserts in objectFromPropertyDescriptor Summary: It turned out that these assertions hit with test262 tests. These test cases are, for example, passing property descriptor with incomplete attributes. For some tests, this doesn't matter and they are just testing the behavior of trap functions. Few test cases are testing with incorrect attribute set up and checking that the runtime is handling as spec says. Essentially, `objectFromPropertyDescriptor` should create the descriptor object as specified by the code, and Proxy should pass it to the `defineProperty` trap function as is. Each of the case handling is already well implemented in the Proxy code. Reviewed By: tmikov Differential Revision: D51692787 fbshipit-source-id: 340b49007ca7a35eacab07be1c1e40a6e5b7ec7d * Implement setExternalMemoryPressure in Hermes Summary: Add an implementation of `adjustExternalMemoryPressure` in Hermes. It uses an internal property pointing to a `NativeState` to maintain the external memory size. The finalizer of the `NativeState` informs the GC that the memory has been freed. Reviewed By: tmikov Differential Revision: D51241265 fbshipit-source-id: d9be03ee71d34a75fc6a2544e34ca98d08dfa70d * Optimize UnicodeICU localeCompare (#1186) Summary: This change eliminates repeated calls to `ucol_open` and `ucol_close` in `hermes::platform_unicode::localeCompare` for HERMES_PLATFORM_UNICODE_ICU. On a sample sort.js(that sorted 30k strings using localeCompare), the runtime goes down from ~500 ms to ~120ms. **Before:** I observed in my trace that ucol_open and ucol_close was called in every localeCompare invocation and each ucol_open took around 500-900 ns. ucol_close is tiny ~100ish ns in each call. Note that the first call to ucol_open, it takes around 100us even after applying this diff.  Here is localeCompare call stack before the patch:  **After** After making UCollator construction static, we avoid repeated ucol_open and ucol_close(except during static initialization) thus making the sort.js sorting fast. Here is the localeCompare stack after the patch:  Looking at the code, i found out that HERMES_PLATFORM_UNICODE_ICU is not used in Android/iOS but can make desktop usage faster (it it useful??). Here is the sort.js script which i got from one of the issues/posts in hermes. ``` print("gen..."); const randInt = (end, start = 0) => Math.floor(Math.random() * (end - start) + start); const alphabet = 'abcdefghijklmnopqrstuvwxyz'; const names = Array.from({length: 30000}, () => Array.from({length: 8}, () => alphabet[randInt(alphabet.length)]).join(''), ); for(var i = 0; i < 5; ++i){ print(names[i]) } print('sorting...'); const s = Date.now(); names.sort(localeCompare); print(`...done, took ${Date.now() - s}ms`); function localeCompare(a, b) { const s = Date.now(); const result = a.localeCompare(b); const e = Date.now(); if (e - s > 1) { print(`slow localeCompare: ${e - s}ms for '${a}' vs '${b}'`); } return result; } for(var i = 0; i < 5; ++i){ print(names[i]) } ``` Pull Request resolved: https://github.com/facebook/hermes/pull/1186 Test Plan: **Execute sort.js before applying this patch** ``` hermes (main)$ ../build_release/bin/hermesc -emit-binary -out sort.hbc samples/sort.js hermes (main)$ ../build_release/bin/hvm ./sort.hbc gen... sorting... ...done, took 466ms hermes (main)$ ../build_release/bin/hvm ./sort.hbc gen... sorting... ...done, took 486ms ``` **Execute sort.js after applying this patch** ``` $ ../build_release/bin/hermesc -emit-binary -out sort.hbc samples/sort.js hermes (icu_localecompare_optimize)$ ../build_release/bin/hvm ./sort.hbc gen... sorting... ...done, took 119ms hermes (icu_localecompare_optimize)$ ../build_release/bin/hvm ./sort.hbc gen... sorting... ...done, took 127ms ``` **Run check-hermes:** ``` hermes (icu_localecompare_optimize)$ cmake --build ../build_release/ --target check-hermes [0/1] Running the Hermes regression tests Testing Time: 15.91s Expected Passes : 1773 Unsupported Tests : 66 hermes (icu_localecompare_optimize)$ echo $? 0 ``` Reviewed By: avp Differential Revision: D51429400 Pulled By: neildhar fbshipit-source-id: f5a2f56952ec9bcbaad441df8f2e9c49c8a0b09a * InstSimplify: fix handling of undefined in relational comparisons (#1202) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1202 Relational comparisons (<= and >=) are simplified to a true/false when it is known that the operands are the same value with a primitive type. If the value is "undefined", it is converted to NaN, which never equals itself. The optimization was not correctly handling the case where the operand could be "undefined" or another type - it was only checking for "undefined". Example: try {} catch { var v0 = "m"; } return v0 <= v0; The try/catch is necessary to make the optimizer believe `v0` is not a constant. Reported in https://github.com/facebook/hermes/issues/1200. Reviewed By: neildhar Differential Revision: D51555324 fbshipit-source-id: 87a4285166a9e11e96014feae45497d1a30d8307 * Deploy 0.223.2 to xplat Summary: Changelog: [Internal] Reviewed By: pieterv Differential Revision: D51768863 fbshipit-source-id: 716add02d82cdfa56f2d3fc4d4560fcb26e8f426 * Do not apply the plugin for TS files Summary: Hermes parser currently cannot handle TS files as good as Flow ones. However, the plugin will unconditionally override the parser for all files. This will break if we introduce the plugin in a config that must work for both TS and JS. This diff skips the parser override if the filename ends in .ts or .tsx. According to babel's implementation, we need to return undefined to ignore the parserOverride: 85e649203b/packages/babel-core/src/parser/index.ts (L9-L25) Reviewed By: pieterv Differential Revision: D51771100 fbshipit-source-id: 7564856952c02d5707536c68a835603aaee77de2 * Fix binop side effects with BigInt Summary: `getBinarySideEffect()` wasn't considering that arithmetic operations between BigInt and non-BigInt (except string addition) throw a TypeError. Refactor the code to be explicit about all operators. Add new tests. Curiously, this change revealed a bug in an existing test: t5: string|number|bigint = t1:any + t2:any t6: string|number|bigint = t3:any + t4:any t7: string|number|bigint = t5 + t6 t7 is unused We were incorrectly removing the calculation of t7, because both operands are primitive types. However it might happen, for example, that t5 is biging and t6 is a number, which would throw an exception. Reported in https://github.com/facebook/hermes/issues/1199. This diff was ported from Static Hermes (since this is a bug). Had to fix BigInt dumping to make the Optimizer test pass. Reviewed By: avp Differential Revision: D51555323 fbshipit-source-id: 927f5e6041899f0800c5a641c62d32e18c3d9f18 * Always write result last in the interpreter (#1203) Summary: If an instruction is the last user of one of its operands, the register allocator may reuse one of its operands as the result register. This is typically fine since most operands are only input parameters, and they are always read before the output is written. However, when the operand is both an input and an output, this requires care to ensure that the operand register (whose live range ends with the given instruction) is written before the result. This ensures that even if the register allocator assigns them the same register, the value held in that register after the instruction is the result. Closes #1203 Reviewed By: avp Differential Revision: D51597086 fbshipit-source-id: 14a4f27887d11cbe21bae5b27712b498b48c3f20 * Skip low-signal assert while fuzzing Summary: In debug mode, this assertion is used to catch potential unbounded handle allocations. It offers however very poor signal when fuzzing. Skipping it to unblock stress-testing large GCScopes. Reviewed By: neildhar Differential Revision: D51587303 fbshipit-source-id: 5d373ecee8535bf38ba8f56055ff9efa03cadec3 * Avoid FMA contraction in makeDate Summary: Newer versions of clang will automatically produce FMA instead of separate multiply and add when the multiplication and addition are in the same expression. This is undesirable because it results in non-standard behaviour in makeDate, and breaks an existing test262 test. We can do this in two ways: 1. We can split them into separate expressions 2. We can use `#pragma STDC FP_CONTRACT OFF` Unfortunately, neither of these address the issue with GCC, which uses `-ffp-contract=fast` by default, and cannot be overridden except by using `volatile`. To fix this, add `-ffp-contract=on` to our build flags, and split the expressions. Reviewed By: tmikov Differential Revision: D51725454 fbshipit-source-id: 3d4c91bb32c619c05c53fd8b1962adbc862554aa * Deploy 0.223.3 to xplat Summary: X-link: https://github.com/facebook/react-native/pull/41797 Changelog: [Internal] Reviewed By: gkz Differential Revision: D51839509 fbshipit-source-id: 961fc4e8fc0f7a45c3882753301d027f319b9ea6 * Revert D51839509: Deploy 0.223.3 to xplat Differential Revision: D51839509 Original commit changeset: 961fc4e8fc0f Original Phabricator Diff: D51839509 fbshipit-source-id: 71a97ee1971d5c882cb704f4f9ff42a95e6d52f6 * Deploy 0.223.3 to xplat Summary: X-link: https://github.com/facebook/react-native/pull/41809 Changelog: [Internal] Reviewed By: samwgoldman Differential Revision: D51855821 fbshipit-source-id: 4fe9c937aff925150fb015be2ea93fa120e7f462 * Added support for ES6 classes (#1176) Summary: We are currently experimenting with a Hermes integration as a potential replacement for the QuickJS engine. As part of the migration, we've realized that the lack of support for ES6 classes and block scoped variables within Hermes is a bigger deal than expected for us. We have 3 engine integrations which all support ES2017+, and without support for ES6 classes and block scoped variables we have to either downgrade the ES target to ES5, or integrate Babel in our compilation pipeline which are both invasive and risky changes for us. In order to fix this, I've taken a stab at supporting ES6 classes somewhat natively within Hermes. I've went with the simplest and the less risky approach that could get the job done. This PR doesn't change the IR, it instead transforms the AST at compile time to convert ES6 classes into plain ES5 classes, very much like what the TypeScript and Babel compiler do. A more advanced implementation would likely add support for classes within the IR. This PR ended up to be fairly straightforward to do, as the JS parser implemented within Hermes already supports most of the ES6 syntax, it's only the IR Gen pass which doesn't support it. ## Supported ES6 class features - Define a class with a parent class (`class Child extends Parent {}`) - Method and static method definitions (`method() {}` and `static method() {}`) - Property and static property definitions (`get prop() {}`, `static get prop() {}`, `set prop(v) {}`, `static set prop(p) {}` - Super call expression in method and static method definitions (`super.method()`) - Super member expression in property and static property definitions (`super.property`) - User defined constructor (`constructor() {}`) - Properties that are immediately initialized. (`property = someValue`) - Class expressions (`const MyClass = class {}`) - Class expressions with private named identifier (`const MyClass = class Self { constructor() { Self.hello = 'world' }})`) ## Unsupported ES6 class features - static block initializers (`static { // code to eval at class load time }`). The existing parser is not able to parse these blocks yet. - Private methods and properties (`#privateMethod() {}`) - Inheritance from built-in types like `Error` or `Map` (`class CustomError extends Error {}`) ## Changes - Added a `enableES6Classes` compile flag. When enabled, a new post processing step will occur on the AST which will transform ES6 classes into ES5 functions, similar to what Babel or the TypeScript compiler does. - Added `lib/AST/ES6Class`, an AST visitor which visits the ClassNodeDeclaration and ClassExpressionNode and will turn them into plain ES5 functions. The generated AST relies on a new helper lib that is meant to be available on the global object as `HermesES6Internal`. - Added the `HermesES6Internal` helper module, which implements `defineClass`, `defineClassProperty`, ` defineStaticClassProperty`, `defineClassMethod` and `defineStaticClassMethod`. - Added a few tests Pull Request resolved: https://github.com/facebook/hermes/pull/1176 Test Plan: - All Hermes tests are passing Reviewed By: tmikov Differential Revision: D51713522 Pulled By: avp fbshipit-source-id: 15e5f3cd0762f7ffddb357f1ec9edce9713bbafe * handle Literal properties in flow-to-flowdif conversion Summary: the StringLiteral comparison here doesn't actually work. also NumericLiterals are valid as property keys, so it adds that as well this change effects object and class properties and methods. Reviewed By: pieterv Differential Revision: D51867940 fbshipit-source-id: ad546b7c539d29c80fc83393c2b5b66678d51b43 * Hermes ABI definition skeleton Summary: The initial boilerplate for the WIP Hermes stable ABI. This just creates a directory with the header defining the new interface, which will be used by both the wrapper and the actual implementation of the ABI. Reviewed By: tmikov Differential Revision: D47563699 fbshipit-source-id: b6da43b3f8fdd168d8bda3adee53277d17275325 * ABI implementation skeleton Summary: Create a skeleton for the concrete implementation of our new C-API on top of the Hermes internal APIs. Set up a build that will build either a static or dynamic library depending on the setting of `BUILD_SHARED_LIBS`. Reviewed By: dannysu Differential Revision: D50153971 fbshipit-source-id: e8462f914a076efc575d6ad9cec13cc9c4627e09 * Hermes ABI wrapper skeleton Summary: Add the boilerplate for the C++ wrapper that implements JSI on top of the C-API. Currently, each function is just implemented to throw. Reviewed By: tmikov, avp Differential Revision: D50153972 fbshipit-source-id: bea12505e75549c13a281fe9153875471ccd6ee5 * Remove unnecessary HostObject trackers in TraceInterpreter Summary: It turned out that these maps are holding the HostObjects and HostFunctions created during replay, and never be released until the runtime instance is destroyed. This changes the behavior of GC for the host object during replay compared to tracing time as there objects can never be collected. These maps are used only for assertions to make sure the created IDs are unique, but they are created by the real runtime's function, so such assumptions should be able to be made already. Hence, simply removing these members and usages. Reviewed By: neildhar Differential Revision: D51874351 fbshipit-source-id: 1ca93c35b4abbc78b95e0a24419f820e16946fac * Add fuzzilli profile for Hermes W2C fuzzer Summary: Additional Fuzzilli profile for sandbox memory fuzzing. Differential Revision: D51352212 fbshipit-source-id: 106a4c89704284c0c6c3f99e96f52869f225e3dc * Trace setExternalMemoryPressure Summary: We've added `jsi::Runtime::setExternalMemoryPressure()`. Let's add tracing and replaying for SynthTrace. Note that function call to `setExternalMemoryPressure()` won't affect correctness of the inputs and outputs during replay, but we want to replay for performance effects mostly for GC. Reviewed By: neildhar Differential Revision: D51835433 fbshipit-source-id: 0701bc2d83fe0f724ed3a215ef53ff016b9f5a58 * Minor fixes in HermesABIRuntimeWrapper Summary: Capitalise the macro and fix the include guard. Reviewed By: tmikov, ftanuma Differential Revision: D51923607 fbshipit-source-id: f7f978fb04ee566af2ffce05d29b04a1c1ef2945 * API for getting the list of loaded scripts Summary: CDPHandler needs to know the list of loaded scripts so that when `Debugger.enable` message comes, CDPHandler can emit `Debugger.scriptParsed` notifications. Currently CDPHandler obtains the loaded scripts by requiring that it be instantiated along with HermesRuntime, so that CDPHandler can monitor `PauseReason::ScriptLoaded` and cache the list of scripts. However, this is not a desirable design because it introduces the unnecessary requirement that CDPHandler must be created along with HermesRuntime even though there is no CDP connection. It introduces the question of "what does the msgCallback do when there is no connection?" Further examination revealed that there is actually no need for CDPHandler to be doing this caching. This is a sensible API to expose from DebuggerAPI. By exposing this functionality, we also save memory spent on the cache by not duplicating information already exists in the VM. I also made a change to align CMake with BUCK. The CMake build emits debug info for InternalBytecode whereas the BUCK build doesn't. `hermes_compile` by default builds with `-g0`. When debug info is present, iterating through the RuntimeModules would get InternalBytecode.js included as one of the loaded scripts. That by itself isn't a problem. I just thought it'd be good to be building things the same way. Reviewed By: mattbfb Differential Revision: D51684983 fbshipit-source-id: b838f71674320bd641ce47f43d7bff14a1312ff5 * Fix HostObject tracing for setProperty Summary: I noticed that we don't have test case nor trace record that exercises `HostObject`'s `setProperty`` calls. Adding a test case revealed that thereplay part isn't working, so fixing it here. Here are short description of bugs - `FakeHostObject::set()` was not passing `PropNameID *nativePropNameToConsumeAsDef` to `interpreter.execFunction`, which was actually needed. - `RecordType::GetPropertyNativeReturn` case was using `getObjForUse` when calling `traceValueToJSIValue` even though value can be String, BigInt, etc (non Object). `getObjForUse()` internally calls `jsi::Value::getObject()` without type check so it asserts in debug builds. - `RecordType::SetPropertyNativeReturn` was doing unnecessary operations copied from `RecordType::SetPropertyNative`. Reviewed By: neildhar Differential Revision: D51966162 fbshipit-source-id: 5f4cec2b4db4bd125b36e06ae1d5ec40000b05da * Move hostObjectsCallCount_ to FakeHostObject from TraceInterpreter Summary: There is no good reason to have HostObject's call count as TraceInterpreter's member. We can have it in each FakeHostObject's instance. Reviewed By: neildhar Differential Revision: D51920392 fbshipit-source-id: 50573ae5142efcba90b7c35467cbd268820fa0d4 * Fix build break in SynthTraceTest (#1217) Summary: Previous Diff broke the builds on CircleCI. It was missing include for variant. Pull Request resolved: https://github.com/facebook/hermes/pull/1217 Reviewed By: neildhar Differential Revision: D52045636 fbshipit-source-id: 930f1b3af05b3a80281f6050d36489390a939391 * Move hostObjectsPropertyNamesCallCount_ to FakeHostObject from TraceInterpreter Summary: There is no good reason to have HostObject's getPropertyNames() call count as TraceInterpreter's member. We can have it in each FakeHostObject's instance. Reviewed By: neildhar Differential Revision: D51966363 fbshipit-source-id: fd4155c4c15b18ce94d38570f28d1c614d2835d1 * Move hostFunctionsCallCount_ to HostFunction wrapper Summary: There is no good reason to have HostFunction's call count as TraceInterpreter's member. We can have it in each HostFunction wrapper lambda as a captured variable. Reviewed By: neildhar Differential Revision: D51966364 fbshipit-source-id: 93049829651afebe96b8951f36b9be2622c6c913 * Merge HostObjectInfo's propNameToCalls callsToGetPropertyNames Summary: We do not need to store set/get calls and getPropertyNames calls for a HostObject separately. Merge them into a Call vector simplifies the logic. Reviewed By: neildhar Differential Revision: D51997003 fbshipit-source-id: f0d1297e7a6de54b198066459d8a26f3e1a084e3 * Simplify TraceInterpreter getCalls() Summary: With the previous change, we can simplify the logic inside getCalls() that is using stack. The stack used to contain ObjectId or FunctionId to get Calls vector for each case, but we just need to stack the reference to the Call vector itself. Only for the case of GetNativePropertyNamesRecord, we need ObjectID. Reviewed By: neildhar Differential Revision: D51998012 fbshipit-source-id: b9c43b5e3ec2e0d8e1f79f36cfc4e5cf41239bab * Release version to 0.18.1 Summary: Release version to 0.18.1 Reviewed By: noahlemen, alexmckenley Differential Revision: D52164018 fbshipit-source-id: b819f0935e539d4eaa321b05a81a848819c2a895 * Fix local to utc conversion Summary: Use the same trick as in V8 to fix local to UTC time conversion. Reviewed By: neildhar Differential Revision: D51728232 fbshipit-source-id: 537b57bee65511b2eebfa92f0941ee2b3d64f59d * Replace localtime with localtime_r for thread safety Summary: Fix two things: 1. `localtime()` uses an internal static buffer, so not thread safe, use `localtime_r()` instead. 2. `localtime_r()` internally calls`tzset()`, but only for the first time. Add a wrapper in the unit tests to always explicitly call `tzset()` to update time zone. Reviewed By: neildhar Differential Revision: D51728257 fbshipit-source-id: f05745e01555f3d06ca888a64d65e57e51c0f16f * Deploy 0.224.0 to xplat Summary: X-link: https://github.com/facebook/react-native/pull/41949 Changelog: [internal] Reviewed By: gkz Differential Revision: D52177280 fbshipit-source-id: 37fe478645ad30b40f4fb4e81d7fc467ac971928 * Create yarn script for prettier plugin development Summary: I've added a script to make development of the prettier plugin easier. This script can be run with the following: ``` yarn build-prettier ``` It will do the following things: 1. Clone the remote fork to your devserver if necessary (inside a .gitignored directory) 1. Determine if there are any upstream commits you need to pull to ensure we dont accidentally lose any changes 1. run `yarn install` 1. copy the `dist` folder from local `hermes-parser` dir into node_modules 1. run `yarn build --no-minify` 1. copy the necessary assets to the proper directory in our source tree NOTE: Developers will still need to ensure that they commit any prettier changes to the upstream repo and create and merge a pull request. Reviewed By: SamChou19815 Differential Revision: D52149102 fbshipit-source-id: 5f800835957181fc0837edea8dd5d562ace6101e * Fix bigint side effects for >>>, /, % (#1214) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1214 BigInt differs from numbers - several bigint operations can throw an exception, specifically: - >>> is not allowed at all and throws - / and % can result in division by zero Improve the side effect model and add a test. This diff was ported from SH. Closes https://github.com/facebook/hermes/issues/1212 Reviewed By: avp Differential Revision: D52013223 fbshipit-source-id: 746f1180abfbdfff63928a248c05731e782ed0ad * Retain prop types when lowering component syntax Summary: Previously with the component syntax transform we would drop prop type annotations and just output a readonly inexact object. This change ensures all prop types are preserved through the transformation. Reviewed By: alexmckenley Differential Revision: D52208500 fbshipit-source-id: 30fc7c0664ef1a3a1be19224188ad101d78acb78 * Back out "Replace localtime with localtime_r for thread safety" Summary: Backing out the localtime thread safety change since it is broken on Windows. Original commit changeset: f05745e01555 Original Phabricator Diff: D51728257 Reviewed By: lavenzg Differential Revision: D52210979 fbshipit-source-id: 17e0ac1513eb23cc02c091940327d93a69ce06e0 * Build hdb release with static libraries (#1197) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1197 `hdb` links against `libhermes` and `libjsi`, so make sure they are built as static libraries when we are building `hdb` for distribution. Reviewed By: dannysu Differential Revision: D51520865 fbshipit-source-id: e123afcffc46873c4b492590262ef0d756a06693 * Fixes for ES6 class support (#1221) Summary: This change provides a few fixes to the ES6 class support which was added in this PR: https://github.com/facebook/hermes/pull/1176 - Added support for class methods and properties being declared using the element expression syntax: ```js const propKey = Symbol(); class Test { [propKey]() { } } ``` This change required that properties are no longer indexed at compile time into a map, as the property keys themselves might be provided from external values. - Fixed class expressions not being processed in array literal expressions: ```js const entries = [0, 1, 2, new (class extends Parent {})] ``` - Methods can now be overwritten as regular properties: ```js class Test { constructor(getter) { if (getter) { this.getMyNumber = getter; } } getMyNumber() { return 42; } } ``` - Method names are now preserved at runtime and can be read through `object.method.name()` - Added more tests Pull Request resolved: https://github.com/facebook/hermes/pull/1221 Test Plan: All tests run Reviewed By: tmikov Differential Revision: D52273463 Pulled By: avp fbshipit-source-id: 273aedb1922ed3dc4a6a5863ae9ce9c7a7ab8da0 * Release version to 0.18.2 Reviewed By: alexmckenley Differential Revision: D52223566 fbshipit-source-id: 5406af0d4ea83676a4929394a45989174f0e56fc * Deploy 0.225.0 to xplat Summary: X-link: https://github.com/facebook/react-native/pull/42004 Changelog: [Internal] Reviewed By: SamChou19815 Differential Revision: D52305312 fbshipit-source-id: b18045c450dc3204b08452ede17b76d1a43cce50 * Deploy 0.225.1 to xplat Summary: X-link: https://github.com/facebook/react-native/pull/42027 Changelog: [Internal] Reviewed By: SamChou19815 Differential Revision: D52353283 fbshipit-source-id: 4f6cefdcfe38d34c8629178823df6b39c5aebbfb * EASY: silence memory leak with GCC ASAN (#1227) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1227 The leak was caused by the allocation of the alt signal stack. To prevent leak reporting, it is stored into a global symbol. However, that symbol is unused and can be optimized out. Force it to be considered "used". Closes https://github.com/facebook/hermes/issues/1225 Reviewed By: avp Differential Revision: D52356667 fbshipit-source-id: 1bd7098218f7d3c155f669e975c26ad491fa23bd * Fix date-local-to-utc.js failure Summary: 1. Windows has different TZ name, so allow it to fail. 2. On some old Android devices (e.g. Nexus), the system timezone data is old, so can't handle America/Santiago correctly, use America/New_York instead. Reviewed By: neildhar Differential Revision: D52213926 fbshipit-source-id: 1597e014893274e02d37eb54a2020be29bd73de9 * Revive "Replace localtime with localtime_r for thread safety" (#1231) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1231 Fix two things: 1. localtime() uses an internal static buffer, so is not thread safe, use localtime_r() instead. 2. localtime_r() internally calls tzset(), but only for the first time. Add a wrapper in the unit tests to always explicitly call tzset() to update time zone. These changes were committed in 5fdb47a and caused crashes on Windows due to wrong error code check, and were reverted. Now commit again with fix for that. Reviewed By: neildhar Differential Revision: D52358023 fbshipit-source-id: 2529e874ee631887d93edcb807e93c712e0feab5 * Special-casing `int32`-convertible nums in `fromDouble` Summary: Adds a special-case to check for numbers which are storable in an `int32`, and adds a short-circuit if so. It looks like a lot of time is spent in the `abstractEqualityTest_RJS` method, of which the lion's share is in `BigIntPrimitive::fromDouble` This function allocates a big buffer capable of storing 1024-bit values, since doubles can represent ints (though very few) up to that size. We then spend cycles trimming extra bytes that aren't needed until this buffer is shrunk-to-fit. This is a separate thing, and can be optimized separately and probably get big wins for `fromDouble` in general as a follow-up. Instead, if our number fits in a 32-bit int, let's just allocate enough bits for the particular number we're setting, set the value specifically to that number, and return (rather than over-allocating and shrinking). Reviewed By: avp Differential Revision: D52234978 fbshipit-source-id: a097151a71b88e6221bfc5323d38c3fe26dcf0b1 * Passing num digits to represent a double into its BigInt `fromDouble` parsing Summary: Currently, we'll always allocate a buffer of 1024 digits in order to cast a double to a BigInt. This makes sense in the limit, as doubles can represent 1024-bit numbers. In lots of current paths, though, we have access to the number of digits before we actually allocate this buffer. This is *really* expensive in practice. As such, let's just allocate enough space to actually represent the number we're converting rather than the largest possible representible number. Reviewed By: mshamis85 Differential Revision: D52262602 fbshipit-source-id: a02d11f85e39496e159ed5c32a61a278a3ebe4e7 * SimplifyCFG: fix strict equality for literals (#1229) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1229 This diff was imported from Static Hermes D52403620. Strict equality for literals was incorrect - it was performing a "bit comparison" for numbers (by comparing pointers to the uniqued literals), while it should be performing a numeric comparison. This caused incorrect switch case optimization. Closes https://github.com/facebook/hermes/issues/1215 Reviewed By: neildhar Differential Revision: D52403687 fbshipit-source-id: a1496a79dd95e333d54ee307086dc9189e09932f * EASY:API/hermes.cpp: enable ES6-classes if enabled in config Summary: The ES6Classes flag wasn't passed to BCProviderFromSrc in the JSI API. Reviewed By: neildhar Differential Revision: D52458224 fbshipit-source-id: 4e1010414940c1a396e0a72f80078cbb7852d6dd * JSLexer: don't auto-register magic comment URLs Summary: These changes were imported from Static Hermes D51795387. ## The problem with auto-registering magic URLs Until now, JSLex would always automatically register `sourceURL=` and `sourceMappingURL=` magic comments in the associated `SourceErrorManager` as soon as detected. That is problematic for a few reasons: - The comment could be encountered at any time or multiple times, leading to inconsistent state, where it is registered for half of the file, or different values are registered for different parts. - Violates separation of concerns: the lexer shouldn't really be concerned with populating state in unrelated components. - The registered sourceMappingURL is usually recorded in the debug information for later use by the debugger. However we might just want to fetch the source map and use it. This decision shouldn't belong to the lexer. - Finally, it is inefficient because SourceErrorManager creates copies of the URLs. In many scenarios we know that the URLs point to the existing source buffer, so we don't need to make a copy. TODO: look into whether SourceErrorManager needs to make copies or it could just hold references. ## Change This diff removes the auto-registration in JSLexer and adds a new function `JSParser::registerMagicURLs()`, which performs the same function, but is invoked only at well defined points. ## Complications with pre-parsing Under certain conditions, which are not important, pre-parsing is invoked first to parses the entire source and record locations of all functions. As such, it encounters all magic comments and other things that we need to keep track of, like "use static builtins". The existing pre-parser API does not expose a parser/lexer instance, it just performs it in one go and discards the parser, which prevents us from querying the magic URLs or calling `registerMagicURLs()`. To solve this, we change pre-parsing to return an instance of the parser, after parsing has finished. This allows users of the API to query the returned parser. For practical reasons the returned instance has to use `std::shared_ptr<>`, which necessitated a change from `std::unique_ptr<>` to the former in one place. It should make no difference - parsers are not created that often. ## Misc other changes Simplify the API of JSParserImpl a bit - a few methods are simply forwarded to the lexer, so they can be implemented directly in JSParser, if the lexer is exported by JSParserImpl. Improve the registration of the magic URLs in SourceErrorManager, so that an empty URL deletes the record instead of storing the empty string. This makes a difference in `getSourceUrl()`, which is treated like an override on top of `getBufferFileName()`. Lastly, I am aware of the difference in case between the old functions which used `Url`, while new functions use `URL`. The latter is correct, but I don't want to perform unrelated stylistic changes in this diff. ## Portability to Hermes The URL registration changes are only enabled in Static Hermes by relying on -DSTATIC_HERMES. Reviewed By: avp Differential Revision: D52190480 fbshipit-source-id: 871c1fd7d889779439d00eda63abce95e77be4fd * Add a new type of WeakRefSlot Summary: Add a new type of WeakRefSlot, which has an additional pinned value field. This value will be marked if the WeakRef object is alive (or it's reached from other objects). This new primitive is intended to be used in new WeakMap implementation. Reviewed By: neildhar Differential Revision: D51435175 fbshipit-source-id: bfe8e2dff98f76c8f5de502281dae6ccd55b2086 * Add weakRefReadBarrier function Summary: Add weakRefReadBarrier for HermesValue Reviewed By: neildhar Differential Revision: D52336308 fbshipit-source-id: 9ef84303e71d02dc21ec2a4df4b27be490b41e24 * Add new WeakRef type Summary: Add a new WeakRef type to manage the new WeakRefSlot. Reviewed By: neildhar Differential Revision: D51605891 fbshipit-source-id: 10bf14bdfb2fcb0fad5e85a8d725ffb0e3215e65 * Mark values in WeakMapRefSlot list Summary: Add a marking function for WeakMapRefSlots in the ManagedChunkedList. Reviewed By: neildhar Differential Revision: D51435851 fbshipit-source-id: b13b11e32c6400e329a4e0f5142425304a8ec0d0 * Re-implement JSWeakMap (#1232) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1232 Reimplement JSWeakMap. Each key/value pair is now stored in a WeakMapRefSlot, which is managed by a WeakMapRef, so we no longer need a separate ValueStorage. These slots are freed at three places: 1. of JSWeakMap::deleteValue() (if the key exists). 2. In completeWeakMapMarking(), when the slot has no value or the object is not marked. 3. In the finalizer of JSWeakMap. Currently, WeakRefLock is still used in setValue(), since WeakRef marking is not removed yet. We will do that in separate diffs. Add a lookup key type for this JSWeakMap, and use it for finding matching key. This eliminates allocating temporary WeakMapRefSlots for querying. Reviewed By: neildhar Differential Revision: D51584024 fbshipit-source-id: 34dd65e4e729753a3d279b32f5ee380d2f979a9d * Remove TracingRuntime configs (#1233) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1233 Attempt to remove unnecessary configs for TraceRuntime from RuntimeConfig. Naively moved up three params to `makeTracingHermesRuntime` and `HermesExecutorFactory`. Reviewed By: neildhar Differential Revision: D51904121 fbshipit-source-id: 9ac8be2dde48fa4d49dbbb2d6b43e66691aff879 * Implement the ABI value representation Summary: Define the layout of values that will be passed across the C-API. These are largely analagous to `jsi::Value` and the subclasses of `jsi::Pointer`, with the addition of also carrying error information. Reviewed By: tmikov Differential Revision: D47563696 fbshipit-source-id: 921122d2f8705318ea35dd67c6ea64c26f6c1445 * Add pointer types and convenience helpers to ABI implementation Summary: Implement the infrastructure needed to support the types specified in the C-API in `hermes_vtable.cpp`. This includes helper functions that will be needed in upcoming diffs to make it easier to manipulate values. Reviewed By: tmikov Differential Revision: D50396059 fbshipit-source-id: bcca6d615ef5479c178286e8fac438962158b10e * Implement pointer types in wrapper Summary: Implement conversions to and from the ABI and JSI value types. Reviewed By: avp Differential Revision: D50396060 fbshipit-source-id: a2fbf0a6b860e4d773dcfae0259bd03ba2172617 * Implement exception reporting and retrieval Summary: Implement exceptions in both the implementation and wrapper, allowing Hermes VM errors to be converted to C++ JSI exceptions. Reviewed By: tmikov Differential Revision: D50212959 fbshipit-source-id: 1538bba118cae179b23e98ed5ebd606f76cecd3a * Implement cloning values Summary: Add support for cloning references end-to-end. Reviewed By: avp Differential Revision: D50212960 fbshipit-source-id: ea822617309aae7f3a7ec041493e92d91730492c * Implement evaluateJavaScript in the ABI (#1060) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1060 Add an API to pass buffers and implement `evaluateJavaScript`. Reviewed By: avp Differential Revision: D47563700 fbshipit-source-id: 85d4f00cbaa1ee441b582fd9272565f38fdad7f3 * Implement basic object and string creation Summary: Implement functions to create objects and strings. Reviewed By: avp Differential Revision: D50798237 fbshipit-source-id: 1aca53e647b01e2e07400beba21336d45f4ea407 * Implement object property manipulation Summary: Implement has/get/set for object properties. Reviewed By: avp Differential Revision: D50798238 fbshipit-source-id: be5620f2a5ffee209d2bbbc14b4cf7d605c2b76f * Implement setting external memory pressure Summary: Implement the new setExternalMemoryPressure API in the C-API. Reviewed By: avp Differential Revision: D51970917 fbshipit-source-id: 9582d0cd418eca8cb515b5683cc0ed05a212a26d * Implement array creation and manipulation Summary: Implement array creation and get/set operations as they exist in JSI. Reviewed By: avp Differential Revision: D51516039 fbshipit-source-id: 9695c514bb016e433a3316ba47f1c9c22fec69a7 * Implement ArrayBuffer manipulation Summary: Implement ArrayBuffer creation from external buffers, as well as functions to obtain the size and underlying data for an ArrayBuffer. Reviewed By: avp Differential Revision: D51535577 fbshipit-source-id: 0fa853958afa77d387f6d2b7c6d1266fe3dd2204 * Implement PropNameID creation Summary: Implement creation of PropNameIDs from Symbol and String. Skip creation of PropNameID from ASCII and UTF-8 buffers for now, since they can be easily implemented on top of other APIs without incurring significant cost. Reviewed By: avp Differential Revision: D51535575 fbshipit-source-id: 175a460815c9466e8c9841b1b2404eeb644a0659 * Implement calls Summary: Implement calling JS functions from native code through the ABI. Reviewed By: avp Differential Revision: D51541197 fbshipit-source-id: fb5df2d3ce6d6c0c19422c511f507d1b29d9c454 * Implement HostFunction Summary: Implement HostFunction in the ABI and JSI. Reviewed By: avp Differential Revision: D51551527 fbshipit-source-id: dd77825de2ae35f53edd1a6c2f53798ac90f118a * Implement HostObject Summary: Implement HostObject in the ABI and JSI. This includes implementing a new `HermesABIPropNameIDList` to pass the result of `HostObject::getPropertyNames`. The ABI method for getting properties from a `HostObject` is named slightly differently than its JSI counterpart (`get_own_keys` vs `getPropertyNames`). This is to align it with `Reflect.ownKeys` and the Proxy `ownKeys` trap, and permit `Symbol`s to be returned from the function. Our implementation doesn't currently deal with such symbols properly, but that is a separate concern. JSI is currently ambiguous on whether this is permissible, but this is something we should address and align the two on before we stabilise the ABI. Reviewed By: avp Differential Revision: D51551526 fbshipit-source-id: ff6e399794af2084d34d5333b4ce21f04a1ed824 * Implement NativeState Summary: Implement NativeState in the ABI and wrapper. Reviewed By: avp Differential Revision: D51551528 fbshipit-source-id: 9d26743e9e11ea2cb77dc9e67fb05f3270ba2af2 * Implement object type checks Summary: Implement type checks for objects in the ABI and wrapper. Reviewed By: avp Differential Revision: D51551529 fbshipit-source-id: 2454644033eb3ee475131e3af2edc0ec48c400f8 * Implement WeakObject Reviewed By: avp Differential Revision: D48052542 fbshipit-source-id: 4f9bb1f982781eaaae7cf42dea0800e9383b787b * Implement getting UTF-8 strings from references Reviewed By: avp Differential Revision: D51535573 fbshipit-source-id: ba7f4631717e648e6491bfba4967dcffa5278eb5 * Implement instance_of and strict equality Reviewed By: avp Differential Revision: D50797413 fbshipit-source-id: 1c5cb6847f838cf9c2dc40b39862d42400edcf28 * Implement drainMicrotasks in the ABI Summary: Implement drainMicrotasks, in the ABI and the wrapper, with the same behaviour as in JSI. Reviewed By: avp Differential Revision: D50797412 fbshipit-source-id: e232c010e331d62454cdf2954af40d8c64f5b709 * Implement BigInt manipulation in the ABI Summary: Implement JSI's BigInt manipulation functionality in the ABI and wrapper. Reviewed By: avp Differential Revision: D50797414 fbshipit-source-id: c34feffe8ac1d3c103b5d4b3af249fc0db60a081 * Run tests against HermesABIRuntime (#1061) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1061 Set up the new `HermesABIRuntimeWrapper` to run against our JSI tests, and API tests that use the APITestFactory. We should extend this to cover more of our APITests, since most of them should run correctly against the ABI runtime. Reviewed By: avp Differential Revision: D47563698 fbshipit-source-id: 5a994c708627c19c30000597e4b6068a52de71b1 * Implement native stack checking for Emscripten Reviewed By: avp Differential Revision: D49900200 fbshipit-source-id: cd1785e13a161027aed52808272d576e9b348d71 * Update the Github issue template Summary: The issue template was misleading by implying that we might be able to support non-current versions of Hermes, which is definitely not the case. React Native occasionally backports fixes, but that is only after the fix has been applied to the current version. The stable ABI will improve this, but we are not there yet. I also attempted to address a couple of things that bugged me: - Asking for a Hermes version is nonsensical and has been for a long time. - It made it seem like reporting the OS is optional. Frankly, I am still not very happy with the template, but I hope that at least these are changes in the right direction. Reviewed By: neildhar Differential Revision: D52580926 fbshipit-source-id: dfcd266fbe5a30216018781350026fb62cb887ab * Parse typeof with type arguments Summary: This diff implements the parsing support for typeof with type arguments. Changelog: [Internal] Reviewed By: avp Differential Revision: D52527139 fbshipit-source-id: ca2e4255620fe3d28d1cd5eda3bd89e541140108 * Parse typeof with type arguments JS changes Summary: This is the companion change of the previous diff. Most of the stuff are generated, the only notable change is that I make the babel adapter strip away the type arguments. Reviewed By: alexmckenley Differential Revision: D52527138 fbshipit-source-id: 26affce6151ca40a4aca95e4ea1b9ca3282acb22 * format_code_in_doc_comments=true Summary: D52632085 Reviewed By: zertosh, dtolnay Differential Revision: D52635896 fbshipit-source-id: a0faaa9a24dbc7216c05581e9af4028866bd454c * Fix _snapshotAddEdgesImpl for WeakRef edge Summary: Before adding edges, we should check if WeakRef is freed or empty. Reviewed By: neildhar Differential Revision: D52613241 fbshipit-source-id: 91988b80e12e11eb642bac4b6f3cfaf5f9fb4f8b * Fix NumberFormat SetNumberFormatDigitOptions logic (#1246) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1246 The old Android logic was the 2020 version of the [spec](https://402.ecma-international.org/7.0/index.html#sec-setnfdigitoptions). The old logic passes `mnfdDefault` as the fallback to `DefaultNumberOption()` at step 12b. This causes the `mnfd` passed to step 12d to be whatever the default for 'currency' style is. And since `maximumFractionDigits` is set to 0 and 0 is less than `mnfd = 2`, a RangeError was thrown. This problem was already fixed by the 2021 version of the spec. `PlatformIntlApple.mm` already implements the newer spec, so that's why it's an Android-only issue. Since the future plan is to move Intl to use ICU4C on Android, that means much of the Java code becomes obsolete. Still, it's relatively easy to just update the logic to match Apple platform [spec](https://402.ecma-international.org/8.0/index.html#sec-setnfdigitoptions), so I did. In the process, I added tests to catch this bug. The tests are probably the more valuable part as we implement on top of ICU4C. Reviewed By: neildhar Differential Revision: D52641902 fbshipit-source-id: d85a415c3440b2fe81c899d8ad0fd3c6eefc8031 * Deploy 0.226.0 to xplat Summary: X-link: https://github.com/facebook/react-native/pull/42252 Changelog: [Internal] Reviewed By: SamChou19815 Differential Revision: D52708681 fbshipit-source-id: a864d2f5654ce011a4658564ec140aca03fb9ac8 * Doing single-digit fast-path for abstract equality testing Summary: Lots of common Hermes use-cases perform equality checks using `abstractEqualityTest_RJS`. This method compares a `number` against a `BigInt`, which makes total sense. The current implementation creates a `BigInt` from the `double`, which also makes sense (since the BigInt might need to store a value bigger than the max double size). However, lots of use-cases need to compare *specifically* against small numbers. Since we store numbers in our BigInts one digit at a time, we can pretty cheaply compare doubles directly against BigInts in the case that they both represent single digits. This diff adds a fast-path to do that check before performing a BigInt creation from our double. This fast-path *does* incur an extra couple checks on comparisons where the BigInt actually *is* too big to store in an `int64`, but the cost looks negligible next to the BigInt construction cost (plus real-world workloads seem overwhelmingly likely to have more sub-2^64 comparisons). NOTE: The Hermes codebase refers to each `int64` as a "digit" instead of a "word". This is not a big deal, but just wanted to call out that a `digit` can store values up to 2^63 today. Reviewed By: jpporto Differential Revision: D52551370 fbshipit-source-id: 29dd6752dc702c8257daaca5494f51268a93740b * Add wording on security bugs in GitHub issue template Summary: Add a message to the GitHub issue template, reminding users that if they believe they have discovered a security vulnerability in Hermes, they should submit it through the Meta Bug Bounty program rather than filing a public issue. This is important to avoid publicly disclosing security vulnerabilities before a fix. Reviewed By: tmikov Differential Revision: D52626730 fbshipit-source-id: 18e323051daff9a6def50a05444be6c930157b14 * Use builtin yarn version in yarn build-prettier script Summary: I've updated our `yarn build-prettier` build script to use the builtin version of yarn included with prettier to ensure the yarn lockfile is respected. I've also added logic to run `yarn build` if none of the hermes-parser `dist` assets exist to make the development flow easier from ondemand machines. NOTE: My changes to .yarnrc.cjs are needed in order to ensure the fwdproxy config is set correctly. See https://github.com/pieterv/prettier/pull/1/files Reviewed By: SamChou19815 Differential Revision: D52739505 fbshipit-source-id: 4ae67a3ef352b069627b19968fb4ba986f6ad8a1 * Add Hermes wasm2c compilation Summary: Add the ability to compile Hermes with the C-API to wasm, and the resulting wasm to C. Check in the generated artefact for a release and debug build of Hermes, making it easier to build and develop with. Reviewed By: avp Differential Revision: D47065258 fbshipit-source-id: 0de6bc047a6ebc1fdac91a460fefcbf880c4e4b3 * Sandbox runtime skeleton Summary: Create a new JSI runtime that will consume the sandbox's exported interface, and wire up the build. Reviewed By: avp Differential Revision: D52531413 fbshipit-source-id: 58a161591f46d0b7ab1ab795cf4633149beb7fcb * Mirror ABI types and helpers in the sandbox Summary: The layout of data structures in the sandbox is different than that on the host. In order to make it more convenient to consume and create these data structures, create a full mirror of all the `HermesABI*` types such that their layout matches what it would have in the sandbox. When passing/returning a value from the sandbox, it just needs to be cast to one of these types to access its fields. Reviewed By: avp Differential Revision: D52558676 fbshipit-source-id: 2e88e294c7c827959e304381cc135d354bc0dc51 * Store WeakRefSlot in ManagedChunkList Summary: ManagedChunkList can automatically manage free slots, and shrink more effectively than std::deque. It help reduce the necessary work in GC. Reviewed By: neildhar Differential Revision: D51048229 fbshipit-source-id: 10f8c0059fe32dd98eda67db93eeba6f58ea0c6a * Enforce unique ownership of WeakRefs Summary: Each WeakRef uniquely owns a WeakRefSlot, and frees the slot only when destroying the WeakRef. Reviewed By: neildhar Differential Revision: D38065776 fbshipit-source-id: 8283dc22d03e653ccf9d11a9f713cc7ca808b309 * Remove concurrent marking of WeakRefs (#1243) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1243 Remove all WeakRef marking functions and weakRefMutex_. Reviewed By: neildhar Differential Revision: D38077265 fbshipit-source-id: 90693c81ce4f5ca168097a3157ccb6ba6da77dd0 * Implement a helper for function type signatures Summary: Function pointers in the sandbox are just indices into a function pointer table. The table also includes a type signature encoded as a `wasm_rt_func_type_t`, which is used to verify that the function being retrieved has the expected type (since it would be unsafe to call it if the types do not match). So whenever we add a function to the table, or we try to retrieve a function, it is necessary to construct its corresponding `wasm_rt_func_type_t` to populate/check the stored type. This results in a lot of boilerplate, and is error prone since we need to manually ensure that the constructed `wasm_rt_func_type_t` matches the actual signature of the function. This diff defines a template that performs the conversion from a function signature to a `wasm_rt_func_type_t` automatically, by decomposing the function type and constructing the right call to `wasm2c_hermes_get_func_type`. Reviewed By: avp Differential Revision: D52558681 fbshipit-source-id: 84f13bdf5a1cbde4a49f89fb70ad4c861892ec87 * Implement utilities for pointers and allocations in sandbox memory Summary: Implement some utilities to make it easier to interact with pointers into the sandbox. Since pointers are just `uint32`s, they convey no type information and are annoying to dereference. This diff defines a `Ptr` template, and subclasses, that allow you to treat the pointer like a reference to the underlying object. Note that this is safer than directly holding a pointer to the underlying memory, because the sandbox heap can move when it grows. The subclasses make it convenient to allocate data on the stack/heap for the duration of the implementation of JSI functions. Reviewed By: avp Differential Revision: D52558679 fbshipit-source-id: 3e189388861a517897288cc90a31f483e25335c6 * Create a sandbox module tied to each runtime Summary: Instantiate the w2c_hermes module for each instance of a `HermesSandboxRuntime`. This is the execution context for the sandbox, providing things like the heap, stack, and tables. While it is technically possible to instantiate multiple runtimes in a single module, for now, just maintain one module per runtime. This uses a base class instead of a field to make it convenient and efficient to obtain the module from a `HermesSandboxRuntime *` and vice versa. Reviewed By: avp Differential Revision: D52558675 fbshipit-source-id: c880f7f4700e1963e0ce3de8a3beff09378b6449 * Create and destroy the sandboxed runtime Summary: Instantiate a runtime object through the ABI by retrieving the vtable and calling `make_hermes_runtime`. The object is owned by the JSI runtime and is released on destruction. Read out the vtable fields as correctly typed function pointers to populate the mirrored vtable `vt_`. These will be the primary mechanism for calling into the sandbox. Reviewed By: avp Differential Revision: D52558677 fbshipit-source-id: 770d97291a6985d2b62f747b0c3b804caef5b56c * Implement GrowableBuffer Summary: Implement `GrowableBuffer`, allowing the sandbox to return variable length data (for now it is just strings). This also requires some general utilities and patterns that are used to perform allocations and expose functionality to the runtime. Reviewed By: avp Differential Revision: D52558678 fbshipit-source-id: 6e1790eb2a6010e10b65a240960ac64f17eb98ba * Implement values and exceptions Summary: Implement passing values to/from the sandbox and retrieving exceptions. This mostly mirrors the equivalent implementation in `HermesABIRuntimeWrapper`. Reviewed By: avp Differential Revision: D52558680 fbshipit-source-id: 834c15604d46458708a696ba54479f4e5e10493f * Expose the time and randomness to the sandbox Summary: Add implementations for the imported time and randomness functions. Reviewed By: avp Differential Revision: D52558753 fbshipit-source-id: fdc8e4c757c24596f18a2e04c6847018e055022b * Implement evaluateJavaScript in the sandbox Summary: Implement the buffer API, and use it to implement `evaluateJavaScript`. Reviewed By: ftanuma Differential Revision: D48086020 fbshipit-source-id: ee9472b7622f5602c5dd42ce11533c08f7c19489 * Implement most JSI functionality Summary: Add basic implementations for most of JSI on top of the sandbox. This excludes things like HostObject, HostFunction, and NativeState which require a little more work. Reviewed By: ftanuma Differential Revision: D52558754 fbshipit-source-id: b323d5d30f3c72f10d636038ed4cff3770a2d420 * Implement NativeState, HostFunction, and HostObject Summary: Add implementations for exposing NativeState, HostFunction, and HostObject to the sandbox. This also adds an indirection mechanism for accessing them with an index, to avoid storing the native pointers in the sandbox heap (which may get corrupted). Reviewed By: avp Differential Revision: D52558752 fbshipit-source-id: fb5be97bccde96374def8594559e78f48dbca84a * Run API tests against the sandbox runtime (#1248) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1248 X-link: https://github.com/facebook/react-native/pull/42264 Add a buck build for the sandbox, and run our existing API tests against it. Reviewed By: avp Differential Revision: D48086019 fbshipit-source-id: 2ee2eca93e27557d2609eaaec2fce3218d8b873f * Fix $ReadOnlyMap arity bug, add tests Summary: `flow-api-translator` currently expects `$ReadOnlyMap` to have only one type argument, but it actually takes two. Here we fix that and also add tests for `$ReadOnlyMap` and `$ReadOnlySet`. Reviewed By: pieterv Differential Revision: D52784895 fbshipit-source-id: fa8a3018dfc58e4ca07c797c990c80c275c2407c * Introduce AsyncDebuggerAPI Summary: Adding a new class AsyncDebuggerAPI that wraps the existing DebuggerAPI. This class extracts some core concepts from the existing CDPHandler so that we have a class that's not tied to CDP. The main things that this class enables are: 1) Exposes an interrupt API and guarantees those interrupt callbacks will be called exactly once. 2) Turns the didPause interaction into an asynchronous one. Consumers of this class will register DebuggerEventCallback but isn't expected to decide on a next debugger::Command while handling that callback. 3) Adds an API for unpausing the JavaScript program after a DebuggerEventCallback has been invoked. Together with #2, this allows someone to implement a debugging solution like CDP that's asynchronous in nature. The reason to introduce this class is so that we can contain most of the multithread complexity in one place. Rather than having every function in CDPHandler having to think about multithreading, CDPHandler can be simplified. Reviewed By: mattbfb Differential Revision: D51595285 fbshipit-source-id: f4bfaa9daf13339016e2ee0ac347272eb67e3c54 * Add tests for AsyncDebuggerAPI Summary: Adding tests for AsyncDebuggerAPI. Reviewed By: mattbfb Differential Revision: D51768909 fbshipit-source-id: 17243cca4d4ae0a82337ccb6498b54f4c44628fa * Fix build when HERMES_ENABLE_DEBUGGER=0 (#1254) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1254 Taking the same strategy as DebuggerAPI by stubbing out all functionality in AsyncDebuggerAPI when `HERMES_ENABLE_DEBUGGER=0`. Reviewed By: mattbfb Differential Revision: D52812969 fbshipit-source-id: 7fb05dbe3b92738b715e10f66d2e28d59dc42cf5 * btoa implementation (#1255) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1255 Implement [btoa](https://html.spec.whatwg.org/multipage/webappapis.html#atob) utility function for encoding a string to base64. This implementation doesn't follow the HTML spec 100% in that for error cases, the code doesn't throw DOMException. Existing alternatives people use with Hermes simply throw Error, which is what this code throws as well. Reviewed By: avp Differential Revision: D51876325 fbshipit-source-id: 085aa069a761d093fd9e504c0478ee18a36e8d34 * atob implementation (#1256) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1256 Implement [atob](https://html.spec.whatwg.org/multipage/webappapis.html#atob) utility function for decoding a base64 string. This implementation doesn't follow the HTML spec 100% in that for error cases, the code doesn't throw DOMException. Existing alternatives people use with Hermes simply throw Error, which is what this code throws as well. Reviewed By: avp Differential Revision: D52181353 fbshipit-source-id: c90ec95e1ed3b44a7668a6ae4071df536bb31a71 * Regenerate sandbox code Summary: Re-generate the sandbox from the following: emsdk 3.1.39 wasm2c 1.0.33 git revision: 005897de9a Reviewed By: tmikov Differential Revision: D52807947 fbshipit-source-id: 39b6af111ab2104ece6c60a019c115aa33f644ba * Check bounds on data pointer in GrowableBufferImpl::getString Summary: This was missing the third parameter to the `Ptr` constructor, which specifies the size of the memory region being accessed. Adding it ensures that the bounds check applies to the entire region. Reviewed By: tmikov Differential Revision: D52807948 fbshipit-source-id: f4629e001e7bba49ee1670f204d30b93faab2d41 * Remove regex breakpoints Summary: Remove handling of URL regexes. This functionality is not not used in our current configurations. The only reference to regex breakpoints in Chrome DevTools appears to be for debugging a NodeJS target on Windows, when running scripts from local filepaths (it's used to work around the case insensitivity of windows paths). Our use cases only make use of setting breakpoints by URL or by script ID. This change makes the subsequent stack simpler, and removes use of `std::regex`, which is considered problematic. Reviewed By: dannysu Differential Revision: D51674836 fbshipit-source-id: e547738cbe5b0f571f87d8a206002ff2852231f2 * Add end-to-end CI for sandbox (#1252) Summary: Add a CI job that recreates the sandbox from source and tests that it works. Pull Request resolved: https://github.com/facebook/hermes/pull/1252 Reviewed By: avp Differential Revision: D52823512 Pulled By: neildhar fbshipit-source-id: 7c9ef620a78a95db4e7b3608372e513ddb134f3c * Avoid potential overflow in pointer bounds check Summary: This check could overflow on 32 bit platforms if `n` is large. Cast it to `u64` before checking to ensure that the math is done with 64 bit numbers to avoid overflow. Reviewed By: tmikov Differential Revision: D52807946 fbshipit-source-id: 3adc2f898588b046915966f4d97c8377d1a3d75b * Fixed symbol conflicts when using identical function and method names in an ES6 class (#1257) Summary: This change fixes an issue that could occur when calling a function from a method that has the same name. Consider this example: ```js function myMethod() { console.log('Hello from function'); } class MyClass { myMethod() { myMethod(); } } ``` In this scenario, `Hello from function` would not be printed when `instance.myMethod()` is called as the method would be called recursively instead of the function that is available in the outer scope. ## Changes - Added test showcasing the issue - Always create an anonymous function for methods (maybe there is a way to make the identifier private instead?), so that the method body cannot recursively refers to itself by its name. - Insert `method.name` at the JS level when the method is about to be registered into the class. Pull Request resolved: https://github.com/facebook/hermes/pull/1257 Test Plan: All tests passes. <!-- Demonstrate the code is solid. Example: The exact commands you ran and their output, screenshots / videos if the pull request changes the user interface. --> Reviewed By: avp Differential Revision: D52824378 Pulled By: tmikov fbshipit-source-id: e224732fea670680250dee50e964e8e91c1ab1f0 * Split PauseReason::AsyncTrigger into Implicit vs Explicit (#1258) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1258 `PauseReason::AsyncTrigger` was used to represent both Implicit and Explicit cases of `triggerAsyncPause()`. However, using a single value to represent multiple cases makes it harder for `didPause()` to figure out what to do. Furthermore, the old code doesn't trigger one `didPause()` for each `triggerAsyncPause()`. If both Implicit and Explicit where requested, the old code only triggers one `didPause()`. Implementors of `didPause()` could try to keep track of what was requested. However, there's no good reason to not just tell `didPause()` exactly what happened inside the Interpreter that led to the pause. This diff does two things: 1) Split `AsyncTrigger` into `AsyncTriggerImplicit` and `AsyncTriggerExplicit` 2) When both Implicit and Explicit are requested, `didPause()` will trigger twice. One time for each reason. Reviewed By: mattbfb Differential Revision: D51779538 fbshipit-source-id: 6551f0ac4a2e1286531bb23d82f2aa5ee87c357d * Add edges to WeakMap values (#1250) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1250 In new WeakMap implementation, we removed the value storage for values, instead, they are stored in a native set. So the edges to them aren't added automatically. This diff adds them back in `_snapshotAddEdgesImpl()`. Reviewed By: neildhar Differential Revision: D52740140 fbshipit-source-id: e5ca972ef3383f6bd8d230c62b2f7c72bb646536 * Refactor WeakMapEntryRef into WeakRefKey (#1251) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1251 The only user of WeakMapEntryRef is WeakRefKey. Reviewed By: neildhar Differential Revision: D52795638 fbshipit-source-id: 9aac2e017a331cc321b08c16a1c55187db1134cc * Fix signed shift error in ../xplat/hermes/include/hermes/Support/StringTableEntry.h Summary: Bit-shifting signed variables is scary because the bit-shift can result in the signed variable becoming negative, which is often unwanted. This diff fixes such a situation, which is often as a simple as changing `1 << 31` to `1ul << 31` or an equivalent construction. - If you approve of this diff, please use the "Accept & Ship" button :-) Reviewed By: palmje, dmm-fb Differential Revision: D52847056 fbshipit-source-id: 1c39da4949ab692724eae4ccf01341fd3e238b61 * Deploy 0.227.0 to xplat Summary: Changelog: [Internal] Reviewed By: gkz Differential Revision: D52893857 fbshipit-source-id: 2a4edd77d91d4d6206d59f57afe6feee6fffc8a8 * ci: Use GITHUB_OUTPUT envvar instead of set-output command (#1263) Summary: `save-state` and `set-output` commands used in GitHub Actions are deprecated and [GitHub recommends using environment files](https://github.blog/changelog/2023-07-24-github-actions-update-on-save-state-and-set-output-commands/). This PR updates the usage of `set-output` to `$GITHUB_OUTPUT` Instructions for envvar usage from GitHub docs: https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-an-output-parameter Pull Request resolved: https://github.com/facebook/hermes/pull/1263 Reviewed By: cipolleschi Differential Revision: D52904530 Pulled By: cortinico fbshipit-source-id: a98ec75d0251ab8bbc10d279bbff8d22c7e00159 * Move fuzzer references to new location Summary: Move fuzzer references from the alpha version of Hermes Sandbox to its new location. Reviewed By: hmflash Differential Revision: D52836183 fbshipit-source-id: 08c478fbd04d1550dad7ac88350ec7b461cfa93f * Initial support for Hooks Syntax Summary: I've added initial support for hooks syntax to the parser. This diff contains only the parser code itself. The next diff in the stack includes the hermes-parser js changes. NOTE: This is my first contribution to the hermes parser implementation, so please take a close look and let me know if I'm missing anything! Reviewed By: avp Differential Revision: D51875900 fbshipit-source-id: 75650b7fa070916a54b882728cfb9912812c4e6d * Support for declared hooks and hook types Summary: Builds hermes parser support for hook types and for hook declarations. Follows function types and declared functions in most ways and shares the implementation, threading through a bool flag to indicate that we're parsing a hook where necessary. Reviewed By: avp Differential Revision: D51956489 fbshipit-source-id: 84ca2a83e31b549d3ae8fb6e1eab5d5e3b4a00cd * Support hooks syntax in hermes-parser js Summary: I've added support for hooks syntax hermes-parser and the related JS packages. * Added new AST nodes to hermes-parser estree types: - `HookDeclaration` - `HookTypeAnnotation` - `DeclareHook` * **flow-api-translator:** Added translation logic for new node types * **babel-plugin-syntax-hermes-parser** Implemented logic to strip hooks syntax. * **prettier-plugin-hermes-parser:** Updated prettier plugin to support formatting new nodes * Also added several tests for component syntax formatting (migrated from prettier backport repo) NOTE: Please merge these changes into the prettier backport repo: https://github.com/pieterv/prettier/pull/1/files Reviewed By: pieterv, SamChou19815 Differential Revision: D52129315 fbshipit-source-id: 152d6226ca2a02c86da86f36c40d758a0239e76a * Implement vm_uncommit on Windows (#1265) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1265 Pull in Microsoft's patch made in: 94a8bd49e9 which adds vm_uncommit in OSCompat. This ensures that when we are using a contiguous heap, compacted segments are returned to the OS. Reviewed By: lavenzg Differential Revision: D52919351 fbshipit-source-id: 3fba67c8ccca3ae6ec07ed14ef1c0887f019658b * Added support for providing bytecode generation option to BytecodeProviderFromSrc (#1262) Summary: This change adds support for providing the bytecode generation options to the `BytecodeProviderFromSrc` API. This makes it possible to strip the function names and debug information when using the high level `BytecodeProviderFromSrc` API without having to re-write what `BytecodeProviderFromSrc` or `CompilerDriver` are already doing. For context, we currently statically embed Hermes into one of our own CLI binary that is able to do a lot of different compile related tasks for us, as such we are not using the Hermes CLI API. We still would like to be able to compile modules as byte code, and the only thing missing from `BytecodeProviderFromSrc` to be at parity with the `CompilerDriver` CLI option that does bytecode generation was the ability to configure the bytecode generation phase. Pull Request resolved: https://github.com/facebook/hermes/pull/1262 Test Plan: <!-- Demonstrate the code is solid. Example: The exact commands you ran and their output, screenshots / videos if the pull request changes the user interface. --> Tests pass Reviewed By: tmikov Differential Revision: D52918720 Pulled By: neildhar fbshipit-source-id: 0836a8f67a8c3c8783f346b9ee625cba3d207b6e * Add runtime task runner Summary: Add a helper for running tasks that require access to the runtime. Reviewed By: dannysu Differential Revision: D52800390 fbshipit-source-id: 52cbd2d5a893ae4118f5cc0f467fc050adc74402 * Add CDP Agent Summary: Add an agent that provides an CDP-based interface to interact with a runtime being debugged. It is currently a framework for setting up handlers for each CDP domain, and does not process any messages yet. Reviewed By: dannysu Differential Revision: D52802495 fbshipit-source-id: f9bbf82aca4c780762e786afb4f3915447d41b38 * Add CDP Agent test Summary: Add a basic test framework for CDP Agent, and some basic tests that check whether runtime tasks are being emitted. Reviewed By: dannysu Differential Revision: D52803105 fbshipit-source-id: 38716faff1c5b5100ee75cce7c4e988f7c1f26f3 * Add base DomainAgent class Summary: Add a base class for commonly used functions across all domains Reviewed By: mattbfb Differential Revision: D52930098 fbshipit-source-id: fd35cb9cfa0fae9aa2cc97b9ad16f9d258503694 * Clarify the "no observer" semantic in AsyncDebuggerAPI Summary: Adding a comment to clarify that AsyncDebuggerAPI currently doesn't support a caller that adds an event callback and just observes the events. The problematic scenario is as follows: - One "controller" caller adds an event callback - One "observer" caller adds an event callback - JS program hits a `debugger;` statement and pauses - The "controller" caller gets destroyed and removes its event callback while the program is still paused - The "observer" is the only caller left and won't ever issue a `setNextCommand()`, so the program is now forever paused until the "observer" caller removes its event callback Reviewed By: mattbfb Differential Revision: D52880206 fbshipit-source-id: 845378b039e101312ad667cbbbfd482c00c9d4c1 * Enable AsyncBreakCheckInEval by default Summary: We currently default `AsyncBreakCheckInEval` to false, which means that source evaluated through `eval` and `evaluateJavaScript` cannot be interrupted. This leads to some odd practical problems: 1. The new `asyncTriggerTimeout` method only works if a `RuntimeConfig` is correctly set. 2. Code evaluated before a call to `watchTimeLimit` will not be instrumented, but code evaluated after it will, since calling `watchTimeLimit` enables async break checks. So if code evaluated after `watchTimeLimit` is called then calls into a function evaluated before it, the timeout may fail to trigger. It also seems that code generated by `eval` will not be instrumented, even when `watchTimeLimit` is called, leading to pretty confusing behaviour. 3. When the debugger is enabled, we set the debug compile flag to true, which also results in the emission of async breaks. So an async break may correctly trigger in dev mode, but not in release mode. Switching it to be true by default results in simpler and more predictable behaviour, while keeping the option to disable it for users who are absolutely sure it is not needed. Reviewed By: avp Differential Revision: D52947242 fbshipit-source-id: 9901eee3a121f11cf13e5bc530a523cf7c3bca0b * Add asyncTriggerTimeout to sandbox Summary: Most sandbox users would also benefit from having the ability to interrupt execution of untrusted JS when it is taking too long. It would be slightly more complicated (but still possible) to implement async timeouts in the way that they are implemented in Hermes, since that would involve reaching into sandbox memory from another thread and writing to it. To avoid needing any special synchronisation with the sandbox and exposing the internal flag, this diff instead has the runtime call a host provided callback on every `AsyncBreakCheck` to check if a timeout was requested. Empirically, this call has relatively low performance overhead, and under LTO, it may even be inlined since the target of the call is known. allow-large-files Reviewed By: avp Differential Revision: D52947241 fbshipit-source-id: 1e1d777754af7c0cc378aa0df3a1f4cd84b23a91 * Fix test build Summary: Isolate use of JSON to files built without RTTI. Reviewed By: fbmal7 Differential Revision: D52968852 fbshipit-source-id: 7822e8f0a1f78d71a3b82b6d50e89e08ec055dc0 * Fix failing test Summary: The CDP Agent impl was being kept alive too long by queued tasks that held a shared pointer to it, causing the shutdown task for the domain handlers to be queued after the task runner had disappeared. Remove the shared pointer so the CDP Agent impl can shut down during destruction of the CDP Agent. This change is a blatant rip-off of dannysu's D52937432. I'm happy to have it merged instead of this diff if that works. Reviewed By: fbmal7 Differential Revision: D52978446 fbshipit-source-id: 3a1ab66db217133a8abd26be80b72d18c0043f46 * Disable -Xes6-class in internal builds Reviewed By: avp Differential Revision: D52883219 fbshipit-source-id: 01be2e38c62942e6a24e4249ec561698f07dd1ac * ES6Class: add stack overflow checking Summary: . Reviewed By: avp Differential Revision: D52943699 fbshipit-source-id: 9d1645bb45d28cc42e2a20c66091513d75e4557a * Split the sandbox file into 8 for parallel compilation (#1271) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1271 Reviewed By: lavenzg Differential Revision: D53029684 fbshipit-source-id: ca9daa9fd6026382fd27a62c809e3e1e63939f4c * Fix sending Runtime.compileScript response on errors Summary: Hermes was not responding to CDP `Runtime.compileScript` requests that encountered parse errors. (It was in fact constructing a response, but then not sending it.) This diff fixes that. Reviewed By: fbmal7 Differential Revision: D53009376 fbshipit-source-id: 3b0ed5dbf7fd343adee64b7b73129a78540e855d * Allow disabling console API logic for CDPHandler Summary: Adding a parameter so that RN can disable the console API logic while building up their new architecture. Reviewed By: mattbfb Differential Revision: D52971652 fbshipit-source-id: 828d1d060409ea8020838b5ac52fb1f4b914af20 * Annotate HookDeclaration nodes while lowering for babel Summary: Add annotations to lowered hook declaration functions to preserve this info for downstream transforms. This uses the same approach we use for `ComponentDeclarations`. Reviewed By: SamChou19815 Differential Revision: D53090734 fbshipit-source-id: 1e54c9a7f8a5ff40c17a8368645411b0ff39dfa7 * Fallback to slowpath when element is empty in JSArray::at Summary: When an element is empty in JSArray, returning `undefined` is incorrect- fall through to the slowpath instead. Reviewed By: avp Differential Revision: D53065628 fbshipit-source-id: 05ce3f59b806bb4b0e49e449bc2fdd849857e6ab * Stable bytecode encoding for qNaN (#1110) Summary: The default bit encoding of quiet NaN is platform dependent. For example, on x86_64 the sign bit is 1, while on arm64 it is 0. Both encodings are "understood" by the hardware, but which one is generated may differ. Make sure we always emit the x86_64 bit pattern emit serializing bytecode. This was tested with the following input: ``` let x = 0/0; print(x); ``` Closes #1110 Reviewed By: avp Differential Revision: D53098568 fbshipit-source-id: bcab736d20ebffde25948db5633cdd3dacaa13f8 * Add DebuggerDomainAgent Summary: Initial implementation of DebuggerDomainAgent. Currently only handles `Debugger.enable` and `Debugger.disable`. Reviewed By: mattbfb Differential Revision: D52803668 fbshipit-source-id: 7bc398ae0a89d681556ae22c798ecee3dd8324db * Make OutboundMessageFunc usage multithread safe Summary: `EnqueueRuntimeTaskFunc` and `OutboundMessageFunc` are arguments passed to CDPAgent. From the caller's perspective, they might not expect those arguments to still be used after CDPAgent is destroyed. However, the old implementation passes `OutboundMessageFunc` to domain agents and it can still be invoked from the runtime thread after CDPAgent is destroyed. That makes things confusing thus adding the `SynchronizedCallback` wrapper class to be able to stop `OutboundMessageFunc` from being used. Reviewed By: mattbfb Differential Revision: D52952597 fbshipit-source-id: e262cc9a0d0682de3d5e3938c8655b1d6d621730 * Have AsyncDebuggerAPI manage DebuggerAPI's isDebuggerAttached state Summary: Hermes DebuggerAPI has a `setIsDebuggerAttached()` function whose purpose is so that in JavaScript one could query whether debugger is attached via `DebuggerInternal.isDebuggerAttached`. What it means for debugger to be attached is left up to the caller. It is not correlated to whether there is an EventObserver. For AsyncDebuggerAPI, there could be multiple debug clients that add DebuggerEventCallback. It doesn't quite make sense for individual debug clients to be calling `setIsDebuggerAttached()` and potentially run into conflict with each other. Therefore, adding code to manage that state centrally in AsyncDebuggerAPI. The meaning of whether "debugger is attached" is thus "whether any clients registered DebuggerEventCallback". Reviewed By: mattbfb Differential Revision: D52887165 fbshipit-source-id: 5f407833efa704e900f4bab6c639949511b21d11 * Add initial tests for DebuggerDomainAgent Summary: Migrated some Debugger domain tests from ConnectionTests related to enable/disable. NOTE: In the `CDPHandler` implementation we seem to allow `Debugger.enable` to be received when the domain is already enabled. I changed this such that an error will be sent. This means there are slight differences in the `TestScriptsOnEnable` and `TestScriptsOrdering` tests. Reviewed By: mattbfb Differential Revision: D52803852 fbshipit-source-id: 98f73c1ddf1034b489e3f8bb812b0fbdc8d82e8f * Fix AsyncDebuggerAPI::processInterruptWhilePaused() getting stuck (#1280) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1280 Due to the inner while-loop not checking for `isWaitingForCommand_`, if there is a next command AND if the reason `signal_` gets notified is due to changes to `eventCallbacks_`, then `processInterruptWhilePaused()` doesn't properly exit even though there is a next command. Reviewed By: mattbfb Differential Revision: D53073819 fbshipit-source-id: f3fbb2ad208caaa5053b17e25f0ba7397335d4be * Fix compilation warning for uninitialized event variable Summary: Fixes compilation warning about potentially uninitialized `event` variable. This code path is expected to never occur, so switching to a fatal. Reviewed By: fbmal7 Differential Revision: D53113916 fbshipit-source-id: f0d77ee6dfabf76db2aec3d89b7895cc3845d1f9 * Bump Android build tool versions (#1282) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1282 Bump the versions of some of our Android build tools so they automatically use a newer version of the NDK. While here, also enable LTO. With the newer NDK, LLD is the default (and possibly only) available linker, so remove the LLD flag to avoid warnings about unused arguments. Reviewed By: cortinico Differential Revision: D53113612 fbshipit-source-id: f2d9db37578c1be0b8c3f01ceb01503c4849dc58 * Track CDP breakpoints Summary: CDP supports setting an exact breakpoint on a specific script instance (identified by script ID), which does not persist across reloads. CDP also supports a logical breakpoint that describes scripts by some criteria (e.g. by filename), and this description should be applied to all matches over time: in current scripts, in scripts that are added to this runtime later, or even in a future runtime (e.g. a "page reload"). Thus, a single CDP breakpoint can imply multiple break locations, each of which requires a Hermes breakpoint. This diff models CDP breakpoints, and applies CDP logical breakpoints to all matching locations at the time of CDP breakpoint creation, returning the list of locations via the `setBreakpointByUrl` response. Similarly, upon removing a CDP breakpoint, all corresponding Hermes breakpoints are be removed. Future diffs will handle the application of CDP breakpoints to future scripts (i.e. where scripts arrive after the logical breakpoint is set). Reviewed By: dannysu Differential Revision: D49954924 fbshipit-source-id: d3b764bdaad7953c722f83a9ce469b24f0615cf2 * Apply breakpoints to later scripts Summary: Previous diffs store descriptions of where breakpoints should be applied (specified by CDP breakpoint details). Bow, apply CDP breakpoints to newly-loaded scripts and send breakpoint-creation notifications after sending the script-loaded notification. Reviewed By: dannysu Differential Revision: D49965655 fbshipit-source-id: c7223f3157e42cc03821eba102bd77a1a518c097 * Preserve breakpoints across instances Summary: Allow state to be exported from a CDP Handler instance and imported into another. The criteria for when state should be persisted are up to the integrator (e.g. if they know the same app is being reloaded in a new runtime). The state object is setup to contain arbitrary state we want to persist. Exporting the state (`getState`) copies the data to be persisted (breakpoint descriptions, currently) into the state object. Importing the state (via the CDPHandler constructor) loads the relevant data from the state into the new CDP handler. Currently, this involves creating a breakpoint for each loaded description, and adjusting the breakpoint ID counter. Reviewed By: dannysu Differential Revision: D50204914 fbshipit-source-id: f95d87d9e648110aeb41824f8a7c67ed89f0933d * Fix ulimit on array-flat test Summary: 512 is too low to run InternalBytecode on asan/handlesan builds, but 1024 is too high to cause an overflow in release builds. Increase the number of iterations and the stack size to handle both these problems. Reviewed By: fbmal7 Differential Revision: D53107206 fbshipit-source-id: 071c4ba445fd97cd837ae26ef4fe6decdd8318d2 * Update Sandcastle Windows test build script (#1268) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1268 Cmake `--parallel` flag on Windows actually reduces parallelism, so upgrade to VS2019 and use MTT. Also update CircleCI to use the same flag. Build time changes (on a Sandcastle Windows VM): ``` # Run command Before: ~17min Removing --parallel: ~6min With MTT flag: ~3min ``` [Facebook] for details, please check our post: https://fburl.com/workplace/a6mjnirn. Reviewed By: neildhar Differential Revision: D52878323 fbshipit-source-id: 06c71701d6b57be7b44d1a3c1e1cc10af2a89669 * Fix end-to-end Android tests (#1284) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1284 Intl tests in CircleCI are broken because using a newer version of the NDK with an older version of fbjni appears to cause problems. Bump the version of fbjni. While here, skip a test that I observed is failing on my computer, possibly because of a change in the default handling for some localisation functionality in different Android versions. Reviewed By: lavenzg Differential Revision: D53153403 fbshipit-source-id: dc88df5d83c063c4ca2f25f246298286fdd4ae51 * Add `AsExpression` to `isExpression` predicate Summary: There are many automatically generated predicates, but `isExpression` is not one of them. Add `AsExpression` to this check. Reviewed By: SamChou19815 Differential Revision: D53142937 fbshipit-source-id: 0963b3f55c948d9fb62055f914f297e14312ed28 * Implement Debugger.pause and Debugger.resume (#1283) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1283 Implements pause and resume functionality in the new DebuggerDomainAgent. Also port over the `TestAsyncPauseWhileRunning` test from ConnectionTests. Reviewed By: mattbfb Differential Revision: D52932780 fbshipit-source-id: 0b5a5090a62a7fc5b08a75f436cba50845c5a2c4 * Handle debugger; statement in DebuggerDomainAgent Summary: Implement support for `debugger;` statement. Note that the behavior is different than in CDPHandler. In CDPHandler, it registers for `didPause` on construction of `CDPHandlerImpl` before receiving any CDP messages. That's been changed in DebuggerDomainAgent because the new code only registers for DebuggerEventCallback after receiving `Debugger.enable`. Reviewed By: mattbfb Differential Revision: D52976377 fbshipit-source-id: 551fb4f14eeee8c72763bb9b50f3cc75dde88c46 * Implement step into, step out, and step over Summary: `Debugger.stepInto`, `Debugger.stepOut`, and `Debugger.stepOver` implementation. Note that the test is slightly different than what's in ConnectionTests due to the new code only registering for DebuggerEventCallback after receiving `Debugger.enable`. Reviewed By: mattbfb Differential Revision: D52978925 fbshipit-source-id: bee127ab45acff118df27597218cc19f852ab842 * Disable unreliable tests Summary: These two tests have been intermittently failing. Disable them for now until root cause is figured out. Reviewed By: fbmal7 Differential Revision: D53203967 fbshipit-source-id: 1c87cffb024c4697dc0e41e1bb48498429d4862a * Simplify importing host compiler Summary: Simplify the logic to import a host compiler so that we can do it even when not cross-compiling. This allows us to set up a host build that imports the compiler from a different host build, so that we can use different configurations for the compiler and VM builds. Reviewed By: tmikov Differential Revision: D53189094 fbshipit-source-id: 169226e0ea8e54bdf44fad3227a066ed1f8fbdd0 * Rust crates for estree representation/parsing/analysis (#1276) Summary: (discussed internally with pieterv but sharing for broader context) This PR adds several crates designed to expose HermesParser to Rust to support parsing JS source code into an ergonomic Rust representation of the AST, serializing that format to/from estree json, and performing semantic analysis of the AST. The crates are structured as follows: * hermes_estree is an ergonomic Rust representation of a JavaScript AST using a structure that aligns as closely as possible to the estree spec, and which is designed for accurate serialization to/from estree-compatible JSON. * hermes_estree_codegen is the code generator for the above crate, driven from an easy to edit JSON-based description of the AST. * hermes_parser wraps the lower-level hermes parser crate, and converts raw parsed nodes into the above estree format. * hermes_semantic_analysis does name resolution for identifiers and labels, and checks other properties such as syntactic breaks/continue. * hermes_diagnostic is used for representing invalid syntax, todos, compiler invariants, etc. * hermes_utils is for utils. Note that the only crate here which specifically depends on Hermes code is `hermes_parser`, which bridges the raw Hermes parser result into the estree format. The estree format is Hermes-agnostic, the semantic analysis is written against estree rather than the raw hermes parse tree, etc. So we could put most or all of this code somewhere else, but it isn't clear where. Our motivation for landing this code now is that we'd like to use this within Relay Compiler. In the future, it may also be used for the upcoming React Compiler (which was the original motivation for writing this code). Pull Request resolved: https://github.com/facebook/hermes/pull/1276 Test Plan: Todo. I've tested all this code locally, but have to recreate the fixtures. Reviewed By: tyao1 Differential Revision: D53071562 Pulled By: josephsavona fbshipit-source-id: 3023b2e193dd6942e93226a77f617aa8c0508aca * Increase stack size for worker (runtime) thread Summary: Running AsyncDebuggerAPITest with ASAN enabled causes a bunch of "call stack size exceeded" errors. Fixing this by switching to use pthread and set the stack size of the runtime thread to be the same as the main test runner thread. The runtime thread can be thought of as HermesRuntime's main thread and needs more stack space. ``` ******************** TEST 'Hermes-Unit :: API/./APITests/AsyncDebuggerAPITest.NoDebuggerEventCallbackTest' FAILED ******************** Note: Google Test filter = AsyncDebuggerAPITest.NoDebuggerEventCallbackTest [==========] Running 1 test from 1 test case. [----------] Global test environment set-up. [----------] 1 test from AsyncDebuggerAPITest [ RUN ] AsyncDebuggerAPITest.NoDebuggerEventCallbackTest libc++abi: terminating due to uncaught exception of type facebook::jsi::JSError: Maximum call stack size exceeded (native stack depth) RangeError: Maximum call stack size exceeded (native stack depth) ``` Reviewed By: mattbfb Differential Revision: D53145701 fbshipit-source-id: 378c4ce7580d786522bf63f904f2acf088b169dd * Implement Debugger.setPauseOnExceptions (#1278) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1278 Implement `Debugger.setPauseOnExceptions` in the new architecture. Reviewed By: mattbfb Differential Revision: D53097497 fbshipit-source-id: 5021974290b65a38c823d173e2b12ae1eaf0547c * Fix missing sys/resource.h include (#1288) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1288 This built fine on my MacBook, but failed in CI. Reviewed By: neildhar Differential Revision: D53247117 fbshipit-source-id: e808176caeeffe4f0d4f5b62e1cfb0f0422774d7 * Add RuntimeDomainAgent Summary: Add initial version of the `Runtime` domain agent. Reviewed By: dannysu Differential Revision: D53018991 fbshipit-source-id: 50418b04089804c72d90cc3fc9a3a1a2909ce608 * Fix unreliable tests (#1287) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1287 There were a few CI test failures due to recently landed diffs: - https://app.circleci.com/pipelines/github/facebook/hermes/5538/workflows/c993258e-511e-4622-b932-973d41f75fd1/jobs/55809 - https://app.circleci.com/pipelines/github/facebook/hermes/5536/workflows/09c1b07c-b12e-4cc7-9a66-97ee30bc7058/jobs/55789 Both failures are due to: ``` C++ exception with description "triggerInterrupt didn't get executed" thrown in the test body. ``` I'm not able to reproduce the test failure locally, but suspect that this is due to the `waitFor` of explicit pause timinig out because there isn't JavaScript to run. If true, the failure is intermittent based on timing of the VM executing the pre-fix JavaScript code. Reviewed By: mattbfb Differential Revision: D53204781 fbshipit-source-id: 145d25c6181ef236961c490171e5e912a577926f * Sync latest flow parser tests Summary: Changelog: [Internal] Reviewed By: alexmckenley Differential Revision: D53286196 fbshipit-source-id: cab8c7ef39216249ba64c41c1f3f6473de6243de * Allow pre-enabling the Runtime domain in CDPHandler Summary: When integrating Hermes's `CDPHandler` into React Native, the Hermes runtime doesn't necessarily exist for the entire CDP session, and in particular can be destroyed and recreated due to a reload. For similar reasons, `CDPHandler` can be destroyed and recreated. If `CDPHandler` is recreated after a `Runtime.enable` message has already been received, the client will be expecting a `Runtime.executionContextCreated` notification, but `CDPHandler` will not send it unless the client sends an additional `Runtime.enable`. Based on Chrome's behaviour, this is incorrect: `Runtime.enable` should effectively be Session-scoped rather than Agent-scoped. Therefore, we need a way to pre-enable the Runtime domain when instantiating `CDPHandler`. This diff proposes one possible API: passing an (optional) session config object to the constructor. We also need to send the notification as a side effect of `registerCallbacks` in the case where the Runtime domain is pre-enabled. Reviewed By: dannysu Differential Revision: D53006914 fbshipit-source-id: e89405d9cdb380188649e6847d59bef40cb7bb6b * Fix GrammarContext for 'as' expression type Summary: The RHS of the `as` should be parsed with `Type` GrammarContext, which can be done by moving the `advance()` call after the check. Tested on a large Fb4a bundle and this doesn't appear to regress overall parser performance at all (no extra work is done, just some reordering). Reviewed By: tmikov Differential Revision: D53135103 fbshipit-source-id: f24ad7482aa743ff5826460cded4df81d97e7db7 * Add RuntimeDomainAgent tests (#1292) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1292 Add the initial tests for Runtime enable, disable, and execution context creation. Reviewed By: dannysu Differential Revision: D53018992 fbshipit-source-id: 3cd746cd73c4cef304f4309ca10e1f3d91f22282 * Implement Runtime.getHeapUsage Summary: Implement `Runtime.getHeapUsage` Reviewed By: dannysu Differential Revision: D53021357 fbshipit-source-id: 86c19dd49891052844d428d2205f10584308d3e4 * Fix CDPAgentTest failures when running with ASAN Summary: Same as D53145701. Increase SerialExecutor's stack size to avoid ASAN test failures. Reviewed By: mattbfb Differential Revision: D53298974 fbshipit-source-id: f321135c74690eb1d9f52e1fac1d57b28f2566c2 * Stop running interrupts if next command is set Summary: If multiple interrupts set the next command, it becomes ambiguous what to do next. Therefore, immediately stop processing interrupts if a new command has been set. The remaining interrupts can be processed in the next go around. Reviewed By: mattbfb Differential Revision: D53256646 fbshipit-source-id: c2f1bc4ad9996a7b81ac0e5013ad512eaceaabcf * Fixing calculation of frame size when debugging (#1275) Summary: I'm encountering a crash when the Hermes Engine hits a break point in certain code configurations, and tries to return the Call Frames through the CDPHandler. This is reproducible 100% of the time for me. When a breakpoint is hit, `Debugger::getLexicalInfoInFrame` is called and the following code runs: ``` uint32_t currFrame = 0; while (auto idx = getScopeDescIndexForFrame(scopeDescs, currFrame++)) { result.variableCountsByScope_.push_back(getFrameSize(scopeDescs, *idx)); } ``` `getScopeDescIndexForFrame`, will just return the inner-most scope index for the frame, so that's the 0th scope for frame 0. However when `getFrameSize` is called in certain cases, it will crash. For example with the following code: ``` function testFunction(num: number) { if (num > 1) { try { console.log('HelloWorld2' + num); } catch (err: any) { console.log('HelloError'); } } } ``` The scopeDesc flags will look like: `scopeDesc[0].isInnerScope = true` (The try/catch scope) `scopeDesc[1].isInnerScope = true` (The if scope) `scopeDesc[2].isInnerScope = false` (The function scope) `getScopeDescIndexForFrame(scopeDescs, 0)` will return *idx=0 This then gets passed to `getFrameSize(scopeDescs, 0)` which contains the following code: ``` do { frameSize += scopeDescs[i].names.size(); } while (scopeDescs[i--].flags.isInnerScope); ``` Which will crash on the 2nd iteration since `scopeDescs[0].flags.isInnerScope == true` so it will then try to access `scopeDescs[-1]`. I believe this should be `i++`, to count all of the sizes of scopes from inner-most to outer-most, stopping after a non-innerScope, indicating that the next scope is a new frame. In this case, it should be adding up the `names.size()` of scopes, 0, 1 and 2, instead of starting at 0 and counting down to -1. Pull Request resolved: https://github.com/facebook/hermes/pull/1275 Test Plan: Just CI Tests. Reviewed By: dannysu Differential Revision: D53137420 Pulled By: mattbfb fbshipit-source-id: 70f8c3474fc9cf32e1a925bc73292e44c1c530ac * Add scope test Summary: Add a test that exercises frame-size calculation with block scoping enabled. Reviewed By: dannysu Differential Revision: D53144330 fbshipit-source-id: 9e6439a2ff84714ce01a3a452c8ac3beef9f4636 * Release version to 0.19.0 Summary: Release version 0.19.0 Reviewed By: mvitousek Differential Revision: D53319577 fbshipit-source-id: 913f3b5d8a906c2d0ca4b9526acde81d0212ebcf * Align hermes parser and flow parser estree output Summary: The Flow and hermes parsers had some mismatches around hook syntax, specifically with Flow including additional function attributes e.g. predicates that are banned on hooks, and producing a DeclareFunction node with a HookTypeAnnotation type for declared hooks rather than a DeclareHook node. This diff aligns the parsers by having Flow match Hermes. Reviewed By: SamChou19815 Differential Revision: D53321496 fbshipit-source-id: d6ed7c3bd679bf1e221edb401d6d968c64d9cd0d * Align declare module parsing with flow parser Summary: Recently, flow parser no longer errors on unsupported statements in `declare module`, and these errors are moved into type checking stage. In addition, "kind" is removed from declare module AST. This diff implements the same behavior in hermes parser. Changelog: [Internal] Reviewed By: avp Differential Revision: D53290189 fbshipit-source-id: 56659c62079888191037df4f87448cce7bdea8af * Align ASI behavior of typeof with type arguments with flow parser Summary: For ``` type A = typeof b <foo> ``` Both Flow and TS will not treat `<foo>` as type argument of the `typeof` type. This diff makes hermes parser behave the same. Changelog: [Internal] Reviewed By: avp Differential Revision: D53290191 fbshipit-source-id: db5cb619beb762a9ae94cbd24fe612eb36acb6be * Accept execution context ID in CDPAgent Summary: There is no need for Hermes CDPAgent to come up with an arbitrary ID that we use as the `executionContextId`. If integrators simply tell us what ID to use to associate with the given HermesRuntime, then for future notifications CDPAgent can just use that ID. Reviewed By: mattbfb Differential Revision: D53192516 fbshipit-source-id: 72dc9dcb8016a7e2715227de10d449be0c71ca7e * Reject bytecode in evaluateJavaScript Summary: All callsites use evaluateHermesBytecode for bytecode, so we should be able to reject it in evaluateJavaScript Reviewed By: avp Differential Revision: D53031500 fbshipit-source-id: 068595ac597c920c03138b9ae13bcb477ababde8 * `declare namespace` parsing support Summary: This diff adds `declare namespace` parsing support. It's very similar to `declare module`, except that id is always identifier. Changelog: [Internal] Reviewed By: avp Differential Revision: D53318806 fbshipit-source-id: 959f2ee76da755f9b6d8842886de3b72e3747f47 * `declare namespace` parsing support JS changes Summary: This is the companion change of the previous diff. We will replace declare namespace statement with declare variable, similar to how to handle declare enum. Changelog: [Internal] Reviewed By: alexmckenley Differential Revision: D53324233 fbshipit-source-id: d924707cad38280e2a2abef8c3846feb0a6b0c91 * Hook syntax eslint scope support Summary: Add scope analysis support for `HookDeclaration`, `HookTypeAnnotation` and `DeclareHook` Reviewed By: alexmckenley Differential Revision: D53371319 fbshipit-source-id: 38fd50d8cd5cdab6a46ab9110ee2f37d22cdad4f * Back out "Reject bytecode in evaluateJavaScript" Summary: This broke some tests, so back it out for now to unblock them. Reviewed By: avp Differential Revision: D53377783 fbshipit-source-id: 4e2687fbc98b8d11574cf1ef4da256612ad52b57 * Declare namespace eslint scope support Summary: Add scope analysis support for `DeclareNamespace`, mostly following the example of `DeclareModule`. Reviewed By: pieterv Differential Revision: D53373642 fbshipit-source-id: c7fe3e45d8d25aacd649c6304d7cb8fffac8643c * Update integration docs for React Native (#1297) Summary: Fixes https://github.com/facebook/hermes/issues/1241 Pull Request resolved: https://github.com/facebook/hermes/pull/1297 Test Plan: N/A Reviewed By: cipolleschi Differential Revision: D53398699 Pulled By: cortinico fbshipit-source-id: a1794ca747cf235fe0f40ee62fa01cfeeb2ebe9b * Release version to 0.19.1 Summary: Release 0.19.1 Reviewed By: SamChou19815 Differential Revision: D53446578 fbshipit-source-id: 219356413216f84d35eadb36b7906ffcf739bcb2 * Deploy 0.228.0 to xplat Summary: X-link: https://github.com/facebook/react-native/pull/42802 Changelog: [Internal] Reviewed By: SamChou19815 Differential Revision: D53337590 fbshipit-source-id: 31e40d692495fc795d3d69b20ba12fcb420b28e7 * Store and clear debugger callback id Summary: Store the callback ID when subscribing to debugger events so it can be used later to unsubscribe. Clear the id after unsubscribing. Reviewed By: dannysu Differential Revision: D53446188 fbshipit-source-id: 69c0e51e2993a04074e34bf0b19f479bbfd597ce * Handle eval command & EvalComplete differently Summary: It does not make sense to broadcast the `EvalComplete` event to all subscribers because the EvalComplete event is for a specific request. Performing an eval as the next command from didPause is actually a bit special. It's different because `lib/VM/Debugger/Debugger.cpp` will remain in the debugger loop and any expression that gets evaluated will not trigger recursive `didPause`. So the result of running an eval command and getting the EvalComplete back can be seen as if those 2 things are just a synchronous function and the PauseReason has not changed. Reviewed By: mattbfb Differential Revision: D53257099 fbshipit-source-id: f76cd63b8adea459f25b776872e7b2ab96d04102 * Implement Debugger.evaluateOnCallFrame Summary: Implement `Debugger.evaluateOnCallFrame` in the new architecture. Reviewed By: mattbfb Differential Revision: D53151007 fbshipit-source-id: f9f5985986aa7c5603456be8a23c6f23a29ec675 * Implement Debugger.setBreakpoint and removeBreakpoint Summary: Implement set and remove breakpoint messages. Reviewed By: mattbfb Differential Revision: D53340313 fbshipit-source-id: f1eb0b407dfc50140b1eb01a3631d3121316072c * Use OUT_DIR for hermes_estree build script Summary: Our infra isn't happy about the hardcoded path, and throws an IO error when writing to `src`. This diff - updates it to use the OUT_DIR env variable for the generated source file. - Add a new mod output so it can directly be included and does not fail to compile on `an inner attribute is not permitted in this context` error. Reviewed By: alunyov Differential Revision: D53502298 fbshipit-source-id: e1398ac86c982687a9ac7dabc382a2e019271161 * Update generated_ffi.rs Summary: The current hermes_parser crate fails on node type assertions. One suspision is the Rust wrapper is outdated, this diff updates the ffi code by running `hermes/unsupported/tools/rustgen/gen.sh`. Reviewed By: monicatang Differential Revision: D53502989 fbshipit-source-id: facaef95ea5f01705f1016d22d5d7bd57c85edc8 * Use OUT_DIR for hermes_parser build script Summary: Same as D53502298, we need to use the OUT_DIR for generated files in build.rs to make internal infra happy. Reviewed By: alunyov Differential Revision: D53545692 fbshipit-source-id: b8af97d6ff6c7ef363e88f30019b51ae27239c30 * Implement Runtime.globalLexicalScopeNames (#1300) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1300 Implement Runtime.globalLexicalScopeNames Reviewed By: dannysu Differential Revision: D53059570 fbshipit-source-id: 5009eb7004d59898ff788b7d527c948afa06412e * Implement Runtime.compileScript Summary: Implement `Runtime.compileScript` Reviewed By: dannysu Differential Revision: D53314067 fbshipit-source-id: bb3c87a08af92404c34b6901f529d11811f07f0c * Add support for ES2023 `Array` non-mutating methods (#1286) Summary: ### Overview This PR implements partial support for new `Array.prototype` non-mutating methods introduced in ES2023. These are: - [`Array.prototype.toReversed()`](https://262.ecma-international.org/14.0/#sec-array.prototype.toreversed) - [`Array.prototype.toSpliced(start, skipCount, ...items)`](https://262.ecma-international.org/14.0/#sec-array.prototype.tospliced) - [`Array.prototype.with(index, value)`](https://262.ecma-international.org/14.0/#sec-array.prototype.with) `Array.prototype.toSorted()` is implemented in separate PR https://github.com/facebook/hermes/issues/1298 as per request. Other Array methods introduced in ES14 seem to already be implemented (`findLast`, `findLastIndex`). Implementation for `TypedArray` methods are not included in this PR and will be provided in another one. ### Motivation Aforementioned methods see support in all major browsers for quite some time [^toReversed][^toSorted][^toSpliced][^with]. Adding support brings hermes closer to full ES2023 coverage and will improve interoperability with browsers. It also should provide slight performance gains across all applications upon adoption. <img width="1438" alt="obraz" src="https://github.com/facebook/hermes/assets/681837/e6f405b7-0645-4d27-8ca7-54f2c0aaf5d6"> [^toReversed]: https://caniuse.com/?search=toReversed [^toSorted]: https://caniuse.com/?search=toSorted [^toSpliced]: https://caniuse.com/?search=toSpliced [^with]: https://caniuse.com/?search=array.with - [x] Implementation - [x] Add tests - [x] Format changes - [ ] Add documentation somewhere? ### Implementation/review notes This is my first contribution to this project. While I have some experience in C++ and have spent some time studying hermes codebase, there might be some slight misusages of your APIs (most notably handles and memory management) and breaking code-style. If so, please let me know and I will try fix them promptly 🙂 Most of the code was inspired/taken from already existing mutating functions. I also tried to apply optimisations that I've noticed in some of the methods (most notably fast path for array element accessing). Pull Request resolved: https://github.com/facebook/hermes/pull/1286 Test Plan: In the description Reviewed By: avp Differential Revision: D53206784 Pulled By: tmikov fbshipit-source-id: abe00e8143e6adf57e575fe0e189038c21787bf4 * Implement Debugger.setBreakpointsActive Summary: Implement `Debugger.setBreakpointsActive` Reviewed By: mattbfb Differential Revision: D53458761 fbshipit-source-id: 8c6fb333ab09dfdf36c0a0f70c821dba3295caa0 * Implement Debugger.setBreakpointByUrl Summary: Port over the `Debugger.setBreakpointByUrl` implementation from CDPHandler. The code in CDPHandler breaks the processing of new script and sending `breakpointResolved` notification into two steps: `processCurrentScriptLoaded()` and `processPendingScriptLoads()`. In the new code, this is handled in a more intuitive way and completely contained in `processNewLoadedScript()`. The new code no longer maintains a list of loaded scripts because it can simply obtain it from DebuggerAPI via the `getLoadedScripts()` added by D51684983. That's why the new code doesn't have the equivalent of `processPendingScriptLoads()`. Reviewed By: mattbfb Differential Revision: D53550169 fbshipit-source-id: 08e554b2d0d4938db12419da9d29d850db1ce290 * Check Debugger enabled status Summary: Forgot to validate Debugger domain enabled status for some of the messages. Reviewed By: mattbfb Differential Revision: D53580139 fbshipit-source-id: a77b948d8b83a35719270a1fd4a54e196b75dbab * Fix windows circleci (#1302) Summary: Pull Request resolved: https://github.com/facebook/hermes/pull/1302 Windows builds started failing to install from chocolatey, and failed to link to python. https://app.circleci.com/pipelines/github/facebook/hermes/5622/workflows/883d6ccc-938a-4d9f-928a-e401330b3434/jobs/56969 This diff accepts the prompt to install (`-y`), and updates the link. Reviewed By: neildhar Differential Revision: D53575163 fbshipit-source-id: 5db4f28733fae8438bf4bc84d51ec1eeaf5b0285 * Enable Thread-Safety Analysis by Default Summary: With the new architecture of `AsyncDebuggerAPI`, we've limited the multi-thread complexity to just that one class. Unlike the old `CDPHandler`, we no longer have recursive mutex that can't be checked with Thread-Safety Analysis (TSA), so all the new code can be validated at compile time using TSA. Reviewed By: fbmal7 Differential Revision: D53622570 fbshipit-source-id: 05d8cce28578dbf41233131b4db4b010017b7a7f * Avoid repeated branches while iterating StringView Summary: Original Author: avp@meta.com Original Git: 8fd20f29af83ab59c5e21d7de8212bcc904dcc4a Original Reviewed By: neildhar Original Revision: D46117526 `quoteStringForJSON` was operating via indexed access on `StringView`, performing an `isASCII()` check every iteration and leading to slow quoting during `JSON.stringify`. Modify the function to take `UTF16Ref` or `ASCIIRef` directly, so that it knows the type. This increases code size slightly while resulting in significant speedups in `JSON.stringify`. Reviewed By: avp Differential Revision: D53618494 fbshipit-source-id: 48ed47cc0d2deb6f9291c7c944f8addb07fffce0 * Implement Runtime.getProperties Summary: Migrate `Runtime.getProperties` into the new `RuntimeDomainHandler`. Reviewed By: dannysu Differential Revision: D53434398 fbshipit-source-id: 71e1ac1c9f887dc2ac27e7f9f7fd9e5f84f8e6d7 * Fix ThreadSanitizer problem with debugger tests Summary: Run with ThreadSanitizer gives us additional confidence on thread safety on top of the compile time check of Thread-Safety Analysis. Reviewed By: mattbfb Differential Revision: D53628862 fbshipit-source-id: 5657f00b0c6d40b1c3701338ffa7b0d089ef7b0f * Allow stripping of file cleanup code Summary: The linker currently cannot strip the file cleanup code because it is called on fatal errors and signals (when a the LLVM signal handler is installed). To allow it to be stripped, add a layer of indirection through a function pointer, which is only populated by `RemoveFileOnSignal`. This way, the cleanup code is only considered reachable when `RemoveFileOnSignal` is actually used, and it can be stripped otherwise. Reviewed By: tmikov Differential Revision: D53535800 fbshipit-source-id: ae26197a66bc154dd83eb304b4c731eabac10873 * Removing API usage not applicable on iOS (stat and fstat) in libhermes Summary: This diff just removes references to functions that use (either directly or transitively) those functions, and relies on linker DCE to clean up the rest. Trying to extricate all uses of these functions would otherwise result in very significant changes. This diff makes two changes: 1. In raw_fd_ostream, fall back to a default buffer size when building for iOS. 2. Update a function in LLVM that was marked as "used" to suppress warnings, to instead use the new `[[maybe_unused]]` attribute. This allows the function, and the calls to stat it retains, to be stripped. Reviewed By: tmikov, TheSavior Differential Revision: D53133166 fbshipit-source-id: f425762bc6b7072cbd8235c65e9cf652cecb981c * Implement Runtime.evaluate Summary: Implement `Runtime.evaluate`. Unlike the previous implementation, this version uses `evaluateJavaScript`, which more closely matches the logical operation (to just run some JavaScript in the runtime, not evaluate it against a particular pause location), and allows the evaluation to be done from any interrupt, rather than requiring a full debugger pause. Reviewed By: dannysu Differential Revision: D53588361 fbshipit-source-id: 50069de903da080dda0fae59d7832dade1796a76 * Migrate Profiler domain Summary: Migrate the `Profiler` domain. Also noticed that `DomainAgent` was missing virtual destructor and fixed it. Reviewed By: mattbfb Differential Revision: D53619190 fbshipit-source-id: bd179a27c3eb2cf871e63eadf65531930a9fd0d6 * Fix Windows builds in CircleCI Summary: For locating cmake: `refreshenv` for chocolatey requires extra setup for powershell: https://docs.chocolatey.org/en-us/troubleshooting#refreshenv-has-no-effect Since we know where it gets installed to, we can just update the env ourselves. For python3: It looks like somehow the machines have 3.11 installed instead of 3.12. Don't know how that happened. In any case, unifying the setup between the "windows" and "test-windows" jobs is probably a good idea. And doing a choco install of python3 seems allow test-windows to pass. Reviewed By: neildhar Differential Revision: D53682818 fbshipit-source-id: 0eb267ffc18219895de4b7f7edf4d0b5a9fe8ecf * Fix trace_normalize.py to normalize thisArg Summary: "trace_normalize.py" supposed to normalize ObjectID throughout the trace file. But it is missing to normalize "thisArg" field of the trace file. Reviewed By: neildhar Differential Revision: D53736900 fbshipit-source-id: 4b2f93eee53a5df355ec13db0c073ddeb73f8e84 * Disable new profiler domain test Summary: Something about the new Profiler domain test is making things time out. Disabling it for now to unbreak tests on GitHub. Will investigate how to fix proper afterwards. Reviewed By: mattbfb Differential Revision: D53691277 fbshipit-source-id: baeb2d95dd559636686726caa7bb568d8bfa70e4 * Trim unused functionality in the sandbox Summary: Given that `raw_fd_ostream` doesn't actually work in the sandbox, remove its dependency on fstat when building for the sandbox. Also switch the sandbox to building with `HERMES_IS_MOBILE_BUILD` to trim other unused functionality. allow-large-files Reviewed By: dannysu Differential Revision: D53668399 fbshipit-source-id: a3fa8511bc888d9ee2919e8284b75adcc13ffc6b * Remove Thread-Safety Analysis from CDPHandler Summary: Last year when we moved to use mega lock, things became more complex with recursive_mutex and Thread-Safety Analysis (TSA) wasn't able to deal with it anymore. However, with the new AsyncDebuggerAPI, we again simplified the locking such that TSA can be applied. The macros used in CDPHandler.cpp weren't actually being used anymore, so removing. Reviewed By: mattbfb Differential Revision: D53785755 fbshipit-source-id: 493ffcce40c5b43a5a8fd126cd02da5abb80aa77 * Add missing 'override' to AsyncDebuggerAPI destructor Summary: Getting error in buck build for this Reviewed By: mattbfb Differential Revision: D53786286 fbshipit-source-id: a4376a9a131d9ec5c5439ecb152cd301d323182b * Include AsyncDebuggerAPI in BUCK build Summary: Adding `AsyncDebuggerAPI` to our BUCK targets. Needed to add additional `#ifdef` to avoid including `ThreadSafetyAnalysis.h` so we don't need to expose that header. Reviewed By: mattbfb Differential Revision: D53780915 fbshipit-source-id: 29f719a6d14e7f2b668daf178815dfc67c33c77a * Allow configuring execution context in CDPHandler constructor Summary: A rudimentary way of allowing users of CDPHandler to: 1. Control the execution context ID that CDPHandler will respond to. 2. Set the execution context description CDPHandler will report in [`Runtime.executionContextCreated`](https://cdpstatus.reactnative.dev/devtools-protocol/tot/Runtime#event-executionContextCreated) and other messages. 3. Control whether CDPHandler will emit executionContextCreated events at all. The defaults are unchanged, but embedders of CDPHandler can use the extended API to opt into managing execution contexts outside of Hermes. Reviewed By: dannysu Differential Revision: D53759777 fbshipit-source-id: 93cddbd99938d4d973f367ab8031a8e83d6cc281 * Extract console message caching into reusable class Summary: Extracted existing console cache logic from CDPHandler into its own class Reviewed By: mattbfb Differential Revision: D53740456 fbshipit-source-id: d4eaf9066ddbc5e07c6b7dae445b80520679eea5 * Expose RemoteObjectsTable from HermesAPI target & cdp directory Summary: Changing our buck build to be done in a similar way as our cmake build. The HermesAPI target is updated to include `RemoteObjectsTable.cpp`. This way integrators just need to include HermesAPI (or the lean version), just like how cmake build works. I'm doing `RemoteObjectsTable.cpp` first to prove out the idea. If this is good, then I'll do the new CDPAgent the same way. I also had to move `RemoteObjectsTable.cpp` from the `inspector/chrome` directory to avoid buck warning that it should belong to a target inside `inspector/chrome` because there's a BUCK file there. Reviewed By: mattbfb Differential Revision: D53839508 fbshipit-source-id: abf2813fc545d9f2eac05a8982466c20578e8a12 * Add TextEncoder class Summary: This diff adds just the TextEncoder class and constructor. Other functions are added with subsequent diffs. Reviewed By: fbmal7 Differential Revision: D53212825 fbshipit-source-id: c38c1c8cee880c49f62ced238d7f2a2a3650d941 * Add TextEncoder.prototype.encoding property Summary: Adds the read-only `encoding` property to TextEncoder Reviewed By: avp Differential Revision: D53212867 fbshipit-source-id: 010875ff359aaaa8e84a185c8d716bc4689a1538 * Rename decodeSurrogatePair to match ECMAScript spec naming Summary: The ECMAScript spec has a function that does exactly the same thing, so renaming `decodeSurrogatePair` to indicate that it's spec compliant. Reviewed By: avp Differential Revision: D53456339 fbshipit-source-id: 293822f5f7bad832cc2b7a57941ef67872320ebb * Clarify convertUTF16ToUTF8WithReplacements is spec compliant Summary: The `convertUTF16ToUTF8WithReplacements()` is actually a spec compliant implementation of the UTF-8 Encoder: https://encoding.spec.whatwg.org/#utf-8-encoder Added comments to clarify what we pass to `encodeUTF8()` is always a scalar value. Reviewed By: avp Differential Revision: D53456530 fbshipit-source-id: 1160b0907523c0a53c07ee825c9124ec16f6f831 * Add TextEncoder.prototype.encode() Summary: Implement TextEncoder's `encode()` function. Reviewed By: avp Differential Revision: D53213625 fbshipit-source-id: 706d57c9bd155872e07b2547028db7bdb045609f * Add TextEncoder.prototype.encodeInto() Summary: Implement TextEncoder's `encodeInto()` function. Reviewed By: avp Differential Revision: D53216139 fbshipit-source-id: eb4f5a1461084d22c77a7c0723de624f50468785 * Reject Runtime.evaluate messages with the wrong contextId Summary: Changelog: [Internal] Hermes: Adds the missing `validateExecutionContext` call to `Runtime.evaluate`. React Native: Adds an integration test case to cover the expected behaviour around targeting `Runtime.evaluate` by execution context. bypass-github-export-checks Reviewed By: huntie Differential Revision: D53776532 fbshipit-source-id: 66676383ba5b373fdbf2deb8c75f22791b07e300 * Fix incorrect use of GetProcessMemoryInfo's return value. (#1315) Summary: According to the API documentation [GetProcessMemoryInfo function (psapi.h) ](https://learn.microsoft.com/en-us/windows/win32/api/psapi/nf-psapi-getprocessmemoryinfo) > If the function succeeds, the return value is nonzero. > If the function fails, the return value is zero. To get extended error information, call [GetLastError](https://learn.microsoft.com/en-us/windows/desktop/api/errhandlingapi/nf-errhandlingapi-getlasterror). Pull Request resolved: https://github.com/facebook/hermes/pull/1315 Test Plan: - Turn on ShouldRecordStats ( GCConfig.h ) - Be able to get the actual value instead of 0 Reviewed By: neildhar Differential Revision: D53901267 Pulled By: tmikov fbshipit-source-id: 9053e0e94eddee30a9bcf6b50e32bd01a0172eb5 * Save exported state across RuntimeAgent instances Summary: X-link: https://github.com/facebook/react-native/pull/43098 Changelog: [Internal] Wraps Hermes's `CDPHandler::getState()` API in an engine-agnostic abstraction (`RuntimeAgentDelegate::getExportedState`). An Agent's lifetime ends when its Target is destroyed, but it can occasionally be useful to persist some state for the "next" Target+Agent of the same type (in the same session) to read. `RuntimeAgentDelegate` is polymorphic and can't just write arbitrary data to SessionState. Instead, it can now *export* a state object that we'll store and pass to the next `RuntimeTargetDelegate::createAgentDelegate` call. Reviewed By: huntie Differential Revision: D53919696 fbshipit-source-id: a8e9b921bc8fc2d195c5dddea9537e6ead3d0358 * JSON encode remote object strings Summary: Backports D56031659 from `CDPAgent` to `CDPHandler` in order to fix a visible regression from RN 0.72 --> 0.73 (which introduced the breakage in `CDPHandler`). Reviewed By: mattbfb Differential Revision: D56417968 fbshipit-source-id: 04e9ae17c83b1acf1960767efb81aa8b1ac34345 * Fix atob not accepting encoded strings without padding Summary: The forgiving-base64 decode algorithm (https://infra.spec.whatwg.org/#forgiving-base64) gives specific instructions for if remainder is 0 and if remainder is 1. The only error case is when the remainder is 1: > If data’s code point length divides by 4 leaving a remainder of 1, then return failure. When the remainder is 2 or 3, it should still be considered valid base64 data. Fixing `base64DecodeOutputLength()` so we're not returning error incorrectly. Reviewed By: avp Differential Revision: D56480630 fbshipit-source-id: ef500ddd6ed1a128150a79f6d7d8ae4ef8be4f2d * Revert "Disable DateTimeFormat::formatToParts for Apple platform (#1155)" This reverts commit c5a633f645. * feat: add visionOS support * Add queueMicrotask method to JSI Summary: Changelog: [internal] ## Context Microtasks are an important aspect of JavaScript and they will become increasingly important in the hosts where we're currently using JSI. For example, React Native is going to adopt an event loop processing model similar to the one on the Web, which means it would need the ability to schedule and execute microtasks in every iteration of the loop. See https://github.com/react-native-community/discussions-and-proposals/pull/744 for details. JSI already has a method to execute all pending microtasks (`drainMicrotasks`) but without a method to schedule microtasks this is incomplete. We're currently testing microtasks with Hermes using an internal method to schedule microtasks (`HermesInternal.enqueueJob`) but we need a method in JSI so this also works in other runtimes like JSC and V8. ## Changes This adds the `queueMicrotask` to the Runtime API in JSI so we have symmetric API for microtasks and we can implement the necessary functionality. The expectation for JSI implementations is to queue microtasks from this method and from built-ins like Promises and async functions in the same queue, and not drain that queue until explicitly done via `drainMicrotasks` in JSI. This also modifies Hermes and JSC to provide stubs for those methods, and the actual implementation will be done in following diffs. Reviewed By: neildhar Differential Revision: D54302536 fbshipit-source-id: 25f52f91d7ef1a51687c431d2c7562c373dc72a5 * Implement queueMicrotask in Hermes Summary: This adds a basic implementation for `jsi::Runtime::queueMicrotask` in Hermes that adds the callbacks to the existing microtask queue in Hermes (used in `drainMicrotasks`). Reviewed By: neildhar Differential Revision: D54302535 fbshipit-source-id: f62b83b972f8610699380368e15b0db253ba2324 * Make queueMicrotask pure virtual (#1337) Summary: X-link: https://github.com/facebook/react-native/pull/43311 Pull Request resolved: https://github.com/facebook/hermes/pull/1337 Changelog: [internal] We've done this in a separate diff because the changes in Hermes don't propagate immediately to the React Native repository. We need to land the changes in JSI and Hermes first (in a backwards-compatible way) and then land this in a separate commit to make the method mandatory. Reviewed By: neildhar Differential Revision: D54413830 fbshipit-source-id: 3b89fe0e6697b0019544b73daa89d932db97b63a * Fix up code after merge to hermes-2024-06-28-RNv0.74.3 This commit is to clean-up issues introduced during the merge to `hermes-2024-06-28-RNv0.74.3-7bda0c267e76d11b68a585f84cfdd65000babf85`. --------- Co-authored-by: Neil Dhar <neildhar@meta.com> Co-authored-by: Pieter Vanderwerff <pieterv@meta.com> Co-authored-by: Fumihiko Tanuma <fumi@meta.com> Co-authored-by: Sujan Khadka <sujan.khadka@live.com> Co-authored-by: Tzvetan Mikov <tmikov@meta.com> Co-authored-by: Sam Zhou <samzhou19815@meta.com> Co-authored-by: Luigi Coniglio <luigiconiglio@meta.com> Co-authored-by: Ji An Yang <yangjian@fb.com> Co-authored-by: Simon Corsin <simon@corsin.me> Co-authored-by: Noah Lemen <noahlemen@meta.com> Co-authored-by: Danny Su <dsc@meta.com> Co-authored-by: Alex Hunt <huntie@meta.com> Co-authored-by: Gang Zhao (Hermes) <zhaogang@meta.com> Co-authored-by: Alex Taylor (alta) <alta@meta.com> Co-authored-by: Panos Vekris <pvekris@meta.com> Co-authored-by: Paul Jewell <pjewell@meta.com> Co-authored-by: Stiopa Koltsov <nga@meta.com> Co-authored-by: Moti Zilberman <moti@meta.com> Co-authored-by: Matt Blagden <mattblagden@meta.com> Co-authored-by: Neil Dhar <neildhar@fb.com> Co-authored-by: Richard Barnes <rbarnes@meta.com> Co-authored-by: Arun Sathiya <arun@arun.blog> Co-authored-by: Michael Leon <fbmal7@meta.com> Co-authored-by: Aakash Patel <avp@meta.com> Co-authored-by: George Zahariev <gkz@meta.com> Co-authored-by: Joe Savona <joesavona@fb.com> Co-authored-by: Beanyy <edward.leekim.koon@gmail.com> Co-authored-by: Mike Vitousek <mvitousek@meta.com> Co-authored-by: Nicola Corti <ncor@meta.com> Co-authored-by: Tianyu Yao <skyyao@meta.com> Co-authored-by: Robert Pasiński <robert.pasinski@outlook.com> Co-authored-by: mylhyz <mylhyz@gmail.com> Co-authored-by: Riccardo Cipolleschi <cipolleschi@meta.com> Co-authored-by: Oskar Kwaśniewski <oskarkwasniewski@icloud.com> Co-authored-by: Rubén Norte <rubennorte@meta.com>		2024-08-13 13:58:19 -07:00
..
ADT	…
API	…
AST	…
BCGen	…
IR	…
Optimizer	…
Parser	…
PlatformIntl	…
PlatformUnicode	…
SourceMap	…
Support	…
VMRuntime	…
dtoa	…
node-api	…
CMakeLists.txt	…
lit.cfg	…