diff --git a/.gitignore b/.gitignore index 04f653c..4dca5f4 100644 --- a/.gitignore +++ b/.gitignore @@ -15,6 +15,7 @@ test/module/test-dir test/module/test-file !test/module/node_modules !test/module/**/*.js +!test/transport-builtin-test.js !unittest/module/test-files/node_modules diff --git a/docs/design/transport-js-builtins.md b/docs/design/transport-js-builtins.md new file mode 100644 index 0000000..84e1fd4 --- /dev/null +++ b/docs/design/transport-js-builtins.md @@ -0,0 +1,135 @@ +# Transport JavaScript standard built-in objects + +## Incentives +The abstraction of 'Transportable' lies in the center of napa.js to efficiently share objects between JavaScript VMs (napa workers). Except JavaScript primitive types, an object needs to implement 'Transportable' interface to make it transportable. It means [Javascript standard built-in objects](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects) are not transportable unless wrappers or equivalent implementations for them are implemented by extending 'Transportable' interface. Developing cost for all those objects is not trivial, and new abstraction layer (wrappers or equivalent implementations) will bring barriers for users to learn and adopt these new stuffs. Moreover, developers also need to deal with the interaction between JavaScript standards objects and those wrappers or equivalent implementations. + +The incentive of this design is to provide a solution to make JavaScript standard built-in objects transportable with requirements listed in the Goals section. + +At the first stage, we will focus on an efficient solution to share data between napa workers. Basically, it is about making SharedArrayBuffer / TypedArray / DataView transportable. + +## Goals +Make Javascript standard built-in objects transportable with +- an efficient way to share structured data, like SharedArrayBuffer, among napa workers +- consistent APIs with ECMA standards +- no new abstraction layers for the simplest usage +- the least new concepts for advanced usage +- a scalable solution to make all Javascript standard built-in objects transportable, avoiding to make them transportable one by one. + +## Example +The below example shows how SharedArrayBuffer object is transported across multiple napa workers. It will print the TypedArray 'ta' created from a SharedArrayBuffer, with all its elements set to 100 from different napa workers. +```js +var napa = require("napajs"); +var zone = napa.zone.create('zone', { workers: 4 }); + +function foo(sab, i) { + var ta = new Uint8Array(sab); + ta[i] = 100; + return i; +} + +function run() { + var promises = []; + var sab = new SharedArrayBuffer(4); + for (var i = 0; i < 4; i++) { + promises[i] = zone.execute(foo, [sab, i]); + } + + return Promise.all(promises).then(values => { + var ta = new Uint8Array(sab); + console.log(ta); + }); +} + +run(); + +``` + +## Solution +Here we just give a high level description of the solution. Its api will go to docs/api/transport. +- V8 provides its value-serialization mechanism by ValueSerializer and ValueDeserializer, which is compatible with the HTML structured clone algorithm. It is a horizontal solution to serialize / deserialize JavaScript objects. ValueSerializer::Delegate and ValueDeserializer::Delegate are their inner class. They work as base classes from which developers can deprive to customize some special handling of external / shared resources, like memory used by a SharedArrayBuffer object. + +- napa::v8_extensions::ExternalizedContents +> 1. It holds the externalized contents (memory) of a SharedArrayBuffer instance once it is serialized via napa::v8_extensions::Utils::SerializeValue(). +> 2. Only 1 instance of ExternalizedContents wil be generated for each SharedArrayBuffer. If a SharedArrayBuffer had been externalized, it will reuse the ExternalizedContents instance created before in napa::v8_extensions::Utils::SerializeValue() + +- napa::v8_extensions::SerializedData +> 1. It is generated by napa::v8_extensions::Utils::SerializeValue(). It holds the serialized data of a JavaScript object, which is required during its deserialization. + +- BuiltInObjectTransporter +> 1. napa::v8_extensions::Serializer, derived from v8::ValueSerializer::Delegate +> 2. napa::v8_extensions::Deserializer, derived from v8::ValueDeserializer::Delegate +> 3. static std::shared_ptr v8_extensions::Utils::SerializeValue(Isolate* isolate, Local value); +>>> Generate the SerializedData instance given an input value. +>>> If any SharedArrayBuffer instances exist in the input value, their ExternalizedContents instances will be generated and attached to the ShareArrayBuffer instances respectively. +> 4. static MaybeLocal v8_extensions::Utils::DeserializeValue(Isolate* isolate, std::shared_ptr data); +>>> Restore a JavaScript value from its SerializedData instance generated by v8_extensions::Utils::SerializeValue() before. + +- Currently, napa relies on Transportable API and a registered constructor to make an object transportable. In [marshallTransform](https://github.com/Microsoft/napajs/blob/master/lib/transport/transport.ts), when a JavaScript object is detected to have a registered constructor, it will go with napa way to marshall this object with the help of a TransportContext object, otherwise a non-transportable error is thrown. + +- Instead of throwing an Error when no registered constructor is detected, the above mentioned BuiltInObjectTransporter can jump in to help marshall this object. We can use a whitelist of object types to restrict this solution to those verified types at first. +```js +export function marshallTransform(jsValue: any, context: transportable.TransportContext): any { + if (jsValue != null && typeof jsValue === 'object' && !Array.isArray(jsValue)) { + let constructorName = Object.getPrototypeOf(jsValue).constructor.name; + if (constructorName !== 'Object') { + if (typeof jsValue['cid'] === 'function') { + return (jsValue).marshall(context); + } else if (_builtInTypeWhitelist.has(constructorName)) { + let serializedData = builtinObjectTransporter.serializeValue(jsValue); + if (serializedData) { + return { _serialized : serializedData }; + } else { + throw new Error(`Failed to serialize object with type of \"${constructorName}\".`); + } + } else { + throw new Error(`Object type \"${constructorName}\" is not transportable.`); + } + } + } + return jsValue; +} +``` +- The reverse process will be invoked in [unmarshallTransform](https://github.com/Microsoft/napajs/edit/master/lib/transport/transport.ts) if the payload is detected to have '_serialized' property. +```js +function unmarshallTransform(payload: any, context: transportable.TransportContext): any { + if (payload != null && payload._cid !== undefined) { + let cid = payload._cid; + if (cid === 'function') { + return functionTransporter.load(payload.hash); + } + let subClass = _registry.get(cid); + if (subClass == null) { + throw new Error(`Unrecognized Constructor ID (cid) "${cid}". Please ensure @cid is applied on the class or transport.register is called on the class.`); + } + let object = new subClass(); + object.unmarshall(payload, context); + return object; + } else if (payload.hasOwnProperty('_serialized')) { + return builtinObjectTransporter.deserializeValue(payload['_serialized']); + } + return payload; +} +``` + +- Life cycle of SharedArrayBuffer (SAB) +> 1. When a SAB participates transportation among napa workers, its life cycle will be extended till the last reference this SAB. The reference of a SAB could be +>>> 1) a SAB object in its original isolate. + +>>> 2) a received SAB transported from another napa workers, including node zone of napa. + +>>> 3) a TypedArray or DataView created from the original SAB or a received SAB. + +> 2. The life cycle extension during transportation is achieved through the ExternalizedContents SharedPtrWrap of the SAB. +>>> 1) When a SAB is transported for the first time, it will be externalized and its ExternalizedContents will be stored in its SerializedData. At the same time, the SharedPtrWrap of the ExternalizedContents will be set to the '_externalized' property of the original SAB. + +>>> 2) When a SAB is transported for the second time or later, it wil skip externalization and find its ExternalizedContents from its '_externalized' property, and store it to its SerializedData. + +>>> 3) When a napa worker try to restored a transported SAB, it will find the pre-stored ExternalizedContents, and create a SharedPtrWrap for it, then set it to the to-be-restored SAB. + +>>> 4) The life cycle of the SharedArrayBuffer is extended by the SharedPtrWrap of its ExternalizedContents. + + +## Constraints +The above solution is based on the serialization / deserialization mechanism of V8. It may have the following constraints. +- Not all [JavaScripts standard built-in objects](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects) are supported by Node (as a dependency of Napa in node mode) or V8 of a given version. We only provide transporting solution for those mature object types. +- Up to present, Node does not explicitly support multiple V8 isolates. There might be inconsistency to transport objects between node zone and napa zones. Extra effort might be required to make it consistent. diff --git a/inc/napa/module/binding/basic-wraps.h b/inc/napa/module/binding/basic-wraps.h new file mode 100644 index 0000000..5c8853b --- /dev/null +++ b/inc/napa/module/binding/basic-wraps.h @@ -0,0 +1,27 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. + +#pragma once + +#include +#include + +#include + +namespace napa { +namespace module { +namespace binding { + /// It creates a new instance of wrapType with a shared_ptr. + /// shared_ptr of object. + /// wrap type from napa-binding, which extends napa::module::Sharable. + /// V8 object of wrapType. + + template + inline v8::Local CreateShareableWrap(std::shared_ptr object, const char* wrapType = "SharedPtrWrap") { + auto instance = NewInstance(wrapType, 0, nullptr).ToLocalChecked(); + ShareableWrap::Set(instance, std::move(object)); + return instance; + } +} +} +} \ No newline at end of file diff --git a/inc/napa/module/binding/wraps.h b/inc/napa/module/binding/wraps.h index 817a542..0ce2f02 100644 --- a/inc/napa/module/binding/wraps.h +++ b/inc/napa/module/binding/wraps.h @@ -3,8 +3,7 @@ #pragma once -#include -#include +#include #include #include @@ -13,16 +12,6 @@ namespace napa { namespace module { namespace binding { - /// It creates a new instance of wrapType with a shared_ptr. - /// shared_ptr of object. - /// wrap type from napa-binding, which extends napa::module::Sharable. - /// V8 object of wrapType. - template - inline v8::Local CreateShareableWrap(std::shared_ptr object, const char* wrapType = "SharedPtrWrap") { - auto instance = NewInstance(wrapType, 0, nullptr).ToLocalChecked(); - ShareableWrap::Set(instance, std::move(object)); - return instance; - } /// It creates a new instance of AllocatorWrap. /// shared_ptr of allocator. diff --git a/lib/transport/builtin-object-transporter.ts b/lib/transport/builtin-object-transporter.ts new file mode 100644 index 0000000..3998b49 --- /dev/null +++ b/lib/transport/builtin-object-transporter.ts @@ -0,0 +1,18 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. + +import { Shareable } from '../memory/shareable'; + +/// +/// ShareableWrap of SerializedData. +/// +export interface SerializedData extends Shareable { +} + +export function serializeValue(jsValue: any): SerializedData { + return require('../binding').serializeValue(jsValue); +} + +export function deserializeValue(serializedData: SerializedData): any { + return require('../binding').deserializeValue(serializedData); +} diff --git a/lib/transport/transport.ts b/lib/transport/transport.ts index d896e2a..0e136e4 100644 --- a/lib/transport/transport.ts +++ b/lib/transport/transport.ts @@ -3,11 +3,26 @@ import * as transportable from './transportable'; import * as functionTransporter from './function-transporter'; +import * as builtinObjectTransporter from './builtin-object-transporter'; import * as path from 'path'; /// Per-isolate cid => constructor registry. let _registry: Map transportable.Transportable> - = new Map transportable.Transportable>(); + = new Map transportable.Transportable>(); + +let _builtInTypeWhitelist = new Set(); +[ + 'ArrayBuffer', + 'Float32Array', + 'Float64Array', + 'Int16Array', + 'Int32Array', + 'Int8Array', + 'SharedArrayBuffer', + 'Uint16Array', + 'Uint32Array', + 'Uint8Array' +].forEach((type) => { _builtInTypeWhitelist.add(type); }); /// Register a TransportableObject sub-class with a Constructor ID (cid). export function register(subClass: new(...args: any[]) => any) { @@ -33,6 +48,13 @@ export function marshallTransform(jsValue: any, context: transportable.Transport if (constructorName !== 'Object') { if (typeof jsValue['cid'] === 'function') { return (jsValue).marshall(context); + } else if (_builtInTypeWhitelist.has(constructorName)) { + let serializedData = builtinObjectTransporter.serializeValue(jsValue); + if (serializedData) { + return { _serialized : serializedData }; + } else { + throw new Error(`Failed to serialize object with type of \"${constructorName}\".`); + } } else { throw new Error(`Object type \"${constructorName}\" is not transportable.`); } @@ -58,6 +80,8 @@ function unmarshallTransform(payload: any, context: transportable.TransportConte let object = new subClass(); object.unmarshall(payload, context); return object; + } else if (payload.hasOwnProperty('_serialized')) { + return builtinObjectTransporter.deserializeValue(payload['_serialized']); } return payload; } diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index 9035780..fc7d577 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -1,5 +1,26 @@ +set(CMAKE_CXX_FLAGS_ORG ${CMAKE_CXX_FLAGS}) + +# Use -fno-rtti and -fPIC to build v8-extensions as a static library for compatibility with node and v8. +set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-rtti -fPIC") +add_subdirectory(v8-extensions) + +# Restore CMAKE_CXX_FLAGS from CMAKE_CXX_FLAGS_ORG. +set(CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS_ORG}) + # Files to compile -file(GLOB_RECURSE SOURCE_FILES ${CMAKE_CURRENT_SOURCE_DIR}/*.cpp) +file(GLOB SOURCE_FILES_0 ${CMAKE_CURRENT_SOURCE_DIR}/*.cpp) +file(GLOB_RECURSE SOURCE_FILES_1 + ${CMAKE_CURRENT_SOURCE_DIR}/api/*.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/memory/*.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/module/*.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/platform/*.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/providers/*.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/settings/*.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/store/*.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/utils/*.cpp + ${CMAKE_CURRENT_SOURCE_DIR}/zone/*.cpp +) +set(SOURCE_FILES ${SOURCE_FILES_0} ${SOURCE_FILES_1}) # The target name set(TARGET_NAME ${PROJECT_NAME}) @@ -19,7 +40,7 @@ target_include_directories(${TARGET_NAME} target_compile_definitions(${TARGET_NAME} PRIVATE NAPA_EXPORTS NAPA_BINDING_EXPORTS BUILDING_NAPA_EXTENSION) # Link libraries -target_link_libraries(${TARGET_NAME} PRIVATE) +target_link_libraries(${TARGET_NAME} PRIVATE "v8-extensions") if(CMAKE_JS_VERSION) # Building Napa as an npm package for node usage (using exported v8 from node.exe) diff --git a/src/api/capi.cpp b/src/api/capi.cpp index cc38eca..e19023f 100644 --- a/src/api/capi.cpp +++ b/src/api/capi.cpp @@ -6,7 +6,7 @@ #include #include #include -#include +#include #include #include #include diff --git a/src/module/core-modules/napa/napa-binding.cpp b/src/module/core-modules/napa/napa-binding.cpp index 93ad70b..b200f32 100644 --- a/src/module/core-modules/napa/napa-binding.cpp +++ b/src/module/core-modules/napa/napa-binding.cpp @@ -22,6 +22,11 @@ #include #include +#include +#if V8_VERSION_CHECK_FOR_BUILT_IN_TYPE_TRANSPORTER + #include +#endif + using namespace napa; using namespace napa::module; @@ -65,8 +70,10 @@ static void CreateZone(const v8::FunctionCallbackInfo& args) { try { auto zoneProxy = std::make_unique(*zoneId, ss.str()); args.GetReturnValue().Set(ZoneWrap::NewInstance(std::move(zoneProxy))); - } catch (const std::exception& ex) { + } catch (const std::runtime_error& ex) { JS_FAIL(isolate, ex.what()); + } catch (...) { + JS_FAIL(isolate, "Failed to initialize zone with id ", *zoneId); } } @@ -81,8 +88,10 @@ static void GetZone(const v8::FunctionCallbackInfo& args) { auto zoneProxy = napa::Zone::Get(*zoneId); args.GetReturnValue().Set(ZoneWrap::NewInstance(std::move(zoneProxy))); } - catch (const std::exception &ex) { - JS_ASSERT(isolate, false, ex.what()); + catch (const std::runtime_error &ex) { + JS_FAIL(isolate, ex.what()); + } catch (...) { + JS_FAIL(isolate, "No zone exists with id ", *zoneId); } } @@ -213,6 +222,55 @@ static void Log(const v8::FunctionCallbackInfo& args) { logger.LogMessage(section, level, traceId, *file, line, *message); } +void SerializeValue(const v8::FunctionCallbackInfo& args) { + auto isolate = v8::Isolate::GetCurrent(); + v8::HandleScope scope(isolate); + + #if V8_VERSION_CHECK_FOR_BUILT_IN_TYPE_TRANSPORTER + + CHECK_ARG(isolate, args.Length() == 1, "1 argument is required for \"serializeValue\"."); + + auto serializedData = v8_extensions::Utils::SerializeValue(isolate, args[0]); + if (serializedData) { + args.GetReturnValue().Set(binding::CreateShareableWrap(serializedData)); + } + + #else + + isolate->ThrowException(v8::Exception::TypeError(napa::v8_helpers::MakeV8String( + isolate, + "It requires v8 newer than 6.2.x to transport builtin types. \ + If run in node mode, please make sure the node version is v9.0.0 or above."))); + + #endif +} + +void DeserializeValue(const v8::FunctionCallbackInfo& args) { + auto isolate = v8::Isolate::GetCurrent(); + v8::HandleScope scope(isolate); + + #if V8_VERSION_CHECK_FOR_BUILT_IN_TYPE_TRANSPORTER + + CHECK_ARG(isolate, args.Length() == 1, "1 argument is required for \"deserializeValue\"."); + CHECK_ARG(isolate, args[0]->IsObject(), "Argument \"object\" shall be 'SharedPtrWrap' type."); + auto shareableWrap = NAPA_OBJECTWRAP::Unwrap(v8::Local::Cast(args[0])); + auto serializedData = shareableWrap->Get(); + + v8::Local value; + if (v8_extensions::Utils::DeserializeValue(isolate, serializedData).ToLocal(&value)) { + args.GetReturnValue().Set(value); + } + + #else + + isolate->ThrowException(v8::Exception::TypeError(napa::v8_helpers::MakeV8String( + isolate, + "It requires v8 newer than 6.2.x to transport builtin types. \ + If run in node mode, please make sure the node version is v9.0.0 or above."))); + + #endif +} + void binding::Init(v8::Local exports, v8::Local module) { // Register napa binding in worker context. RegisterBinding(module); @@ -250,4 +308,7 @@ void binding::Init(v8::Local exports, v8::Local module) NAPA_SET_METHOD(exports, "getDefaultAllocator", GetDefaultAllocator); NAPA_SET_METHOD(exports, "log", Log); + + NAPA_SET_METHOD(exports, "serializeValue", SerializeValue); + NAPA_SET_METHOD(exports, "deserializeValue", DeserializeValue); } diff --git a/src/v8-extensions/CMakeLists.txt b/src/v8-extensions/CMakeLists.txt new file mode 100644 index 0000000..a464490 --- /dev/null +++ b/src/v8-extensions/CMakeLists.txt @@ -0,0 +1,41 @@ +# Files to compile +file(GLOB SOURCE_FILES ${CMAKE_CURRENT_SOURCE_DIR}/*.cpp) + +# The target name +set(TARGET_NAME "v8-extensions") + +# The generated library +add_library(${TARGET_NAME} STATIC ${SOURCE_FILES}) + +# Include directories +target_include_directories(${TARGET_NAME} + PRIVATE + ${CMAKE_CURRENT_SOURCE_DIR} + ${PROJECT_SOURCE_DIR}/inc) + +# Compiler definitions +target_compile_definitions(${TARGET_NAME} PRIVATE NAPA_EXPORTS NAPA_BINDING_EXPORTS BUILDING_NAPA_EXTENSION) + +if(CMAKE_JS_VERSION) + # Building Napa as an npm package for node usage (using exported v8 from node.exe) + + target_include_directories(${TARGET_NAME} PRIVATE ${CMAKE_JS_INC}) + target_link_libraries(${TARGET_NAME} PRIVATE ${CMAKE_JS_LIB}) + + # Using the V8 functions exported from node.exe + target_compile_definitions(${TARGET_NAME} PRIVATE USING_V8_SHARED) +else() + # Building Napa for embed scenarios (static linking with v8) + if (NOT DEFINED NODE_ROOT) + message(FATAL_ERROR "NODE_ROOT must be set to the node sources root directory") + endif() + + set(NODE_BUILD_TYPE "Release") + if ((DEFINED CMAKE_BUILD_TYPE) AND (CMAKE_BUILD_TYPE STREQUAL "debug")) + set(NODE_BUILD_TYPE "Debug") + endif() + + # V8 header files + target_include_directories(${TARGET_NAME} PRIVATE ${NODE_ROOT}/deps/v8/include) + +endif() diff --git a/src/v8-extensions/array-buffer-allocator.cpp b/src/v8-extensions/array-buffer-allocator.cpp new file mode 100644 index 0000000..2592106 --- /dev/null +++ b/src/v8-extensions/array-buffer-allocator.cpp @@ -0,0 +1,29 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. + +#include "v8-extensions-macros.h" + +#if !(V8_VERSION_CHECK_FOR_ARRAY_BUFFER_ALLOCATOR) + +#include "array-buffer-allocator.h" + +#include +#include +#include + +using namespace napa::v8_extensions; + +void* ArrayBufferAllocator::Allocate(size_t length) { + void* data = AllocateUninitialized(length); + return std::memset(data, 0, length); +} + +void* ArrayBufferAllocator::AllocateUninitialized(size_t length) { + return malloc(length); +} + +void ArrayBufferAllocator::Free(void* data, size_t length) { + free(data); +} + +#endif \ No newline at end of file diff --git a/src/v8/array-buffer-allocator.h b/src/v8-extensions/array-buffer-allocator.h similarity index 64% rename from src/v8/array-buffer-allocator.h rename to src/v8-extensions/array-buffer-allocator.h index 3c3105b..7b3a535 100644 --- a/src/v8/array-buffer-allocator.h +++ b/src/v8-extensions/array-buffer-allocator.h @@ -4,7 +4,6 @@ #pragma once #include -#include namespace napa { namespace v8_extensions { @@ -14,20 +13,13 @@ namespace v8_extensions { public: /// v8::ArrayBuffer::Allocator::Allocate - virtual void* Allocate(size_t length) override { - void* data = AllocateUninitialized(length); - return std::memset(data, 0, length); - } + virtual void* Allocate(size_t length) override; /// v8::ArrayBuffer::Allocator::AllocateUninitialized - virtual void* AllocateUninitialized(size_t length) override { - return malloc(length); - } + virtual void* AllocateUninitialized(size_t length) override; /// v8::ArrayBuffer::Allocator::Free - virtual void Free(void* data, size_t length) override { - free(data); - } + virtual void Free(void* data, size_t length) override; }; } } diff --git a/src/v8-extensions/deserializer.cpp b/src/v8-extensions/deserializer.cpp new file mode 100644 index 0000000..c24071a --- /dev/null +++ b/src/v8-extensions/deserializer.cpp @@ -0,0 +1,50 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. + +#include "v8-extensions-macros.h" + +#if V8_VERSION_CHECK_FOR_BUILT_IN_TYPE_TRANSPORTER + +#include "deserializer.h" + +#include + +using namespace napa::v8_extensions; +using namespace v8; + +Deserializer::Deserializer(Isolate* isolate, std::shared_ptr data) : + _isolate(isolate), + _deserializer(isolate, data->GetData(), data->GetSize(), this), + _data(std::move(data)) { + _deserializer.SetSupportsLegacyWireFormat(true); +} + +MaybeLocal Deserializer::ReadValue() { + bool readHeader = false; + Local context = _isolate->GetCurrentContext(); + if (!_deserializer.ReadHeader(context).To(&readHeader)) { + return MaybeLocal(); + } + + uint32_t index = 0; + Local key = v8_helpers::MakeV8String(_isolate, "_externalized"); + for (const auto& contents : _data->GetExternalizedSharedArrayBufferContents()) { + Local sharedArrayBuffers = SharedArrayBuffer::New( + _isolate, contents.first.Data(), contents.first.ByteLength()); + auto shareableWrap = napa::module::binding::CreateShareableWrap(contents.second); + + // After deserialization of a SharedArrayBuffer from its SerializedData, + // set its '_externalized' property to a ShareableWrap of its ExternalizedContents. + // This extends the lifecycle of the ExternalizedContents by the lifetime of the restored SharedArrayBuffer object. + sharedArrayBuffers->CreateDataProperty(context, key, shareableWrap); + _deserializer.TransferSharedArrayBuffer(index++, sharedArrayBuffers); + } + + return _deserializer.ReadValue(context); +} + +Deserializer* Deserializer::NewDeserializer(v8::Isolate* isolate, std::shared_ptr data) { + return new Deserializer(isolate, data); +} + +#endif diff --git a/src/v8-extensions/deserializer.h b/src/v8-extensions/deserializer.h new file mode 100644 index 0000000..38e9c4e --- /dev/null +++ b/src/v8-extensions/deserializer.h @@ -0,0 +1,38 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. + +#pragma once + +#include "serialized-data.h" + +#include + +namespace napa { +namespace v8_extensions { + + /// + /// Deserializer is used to deserialize a SerializedData instance to a JavaScript object. + /// It is derived from v8::ValueDeserializer::Delegate, whose interface is implemented + /// to handle ShardArrayBuffer specially as below. + /// For each of the SharedArrayBuffer in the input SerializedData, + /// 1). create a SharedArrayBuffer instance from its SharedArrayBuffer::Contents stored in SerializedData. + /// 2). generate a ShareableWrap of ExternalizedContents, and attach it to the SharedArrayBuffer instance. + /// + class Deserializer : public v8::ValueDeserializer::Delegate { + public: + Deserializer(v8::Isolate* isolate, std::shared_ptr data); + + v8::MaybeLocal ReadValue(); + + static Deserializer* NewDeserializer(v8::Isolate* isolate, std::shared_ptr data); + + private: + v8::Isolate* _isolate; + v8::ValueDeserializer _deserializer; + std::shared_ptr _data; + + Deserializer(const Deserializer&) = delete; + Deserializer& operator=(const Deserializer&) = delete; + }; +} +} diff --git a/src/v8-extensions/externalized-contents.cpp b/src/v8-extensions/externalized-contents.cpp new file mode 100644 index 0000000..b066a4c --- /dev/null +++ b/src/v8-extensions/externalized-contents.cpp @@ -0,0 +1,41 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. + +#include "v8-extensions-macros.h" + +#if V8_VERSION_CHECK_FOR_BUILT_IN_TYPE_TRANSPORTER + +#include "externalized-contents.h" + +#include + +using namespace napa::v8_extensions; +using namespace v8; + +ExternalizedContents::ExternalizedContents(const SharedArrayBuffer::Contents& contents) : + _data(contents.Data()), + _size(contents.ByteLength()) {} + +ExternalizedContents::ExternalizedContents(ExternalizedContents&& other) : + _data(other._data), + _size(other._size) { + other._data = nullptr; + other._size = 0; +} + +ExternalizedContents& ExternalizedContents::operator=(ExternalizedContents&& other) { + if (this != &other) { + _data = other._data; + _size = other._size; + other._data = nullptr; + other._size = 0; + } + return *this; +} + +ExternalizedContents::~ExternalizedContents() { + // TODO #146: Get array_buffer_allocator to free ExternalizedContents. + free(_data); +} + +#endif diff --git a/src/v8-extensions/externalized-contents.h b/src/v8-extensions/externalized-contents.h new file mode 100644 index 0000000..c899ac4 --- /dev/null +++ b/src/v8-extensions/externalized-contents.h @@ -0,0 +1,35 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. + +#pragma once + +#include + +namespace napa { +namespace v8_extensions { + + /// + /// 1. ExternalizedContents holds the externalized memory of a SharedArrayBuffer once it is serialized. + /// 2. Only 1 instance of ExternalizedContents would be generated for each SharedArrayBuffer. + /// If a SharedArrayBuffer had been externalized, it will reuse the ExternalizedContents instance + /// created before in napa::v8_extensions::Utils::SerializeValue(). + /// + class ExternalizedContents { + public: + explicit ExternalizedContents(const v8::SharedArrayBuffer::Contents& contents); + + ExternalizedContents(ExternalizedContents&& other); + + ExternalizedContents& operator=(ExternalizedContents&& other); + + ~ExternalizedContents(); + + private: + void* _data; + size_t _size; + + ExternalizedContents(const ExternalizedContents&) = delete; + ExternalizedContents& operator=(const ExternalizedContents&) = delete; + }; +} +} diff --git a/src/v8-extensions/serialized-data.cpp b/src/v8-extensions/serialized-data.cpp new file mode 100644 index 0000000..83a23f5 --- /dev/null +++ b/src/v8-extensions/serialized-data.cpp @@ -0,0 +1,28 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. + +#include "v8-extensions-macros.h" + +#if V8_VERSION_CHECK_FOR_BUILT_IN_TYPE_TRANSPORTER + +#include "serialized-data.h" + +#include + +using namespace napa::v8_extensions; +using namespace v8; + +SerializedData::SerializedData() : _size(0) {} + +const uint8_t* SerializedData::GetData() const { return _data.get(); } + +size_t SerializedData::GetSize() const { return _size; } + +const std::vector& +SerializedData::GetExternalizedSharedArrayBufferContents() const { + return _externalizedSharedArrayBufferContents; +} + +void SerializedData::DataDeleter::operator()(uint8_t* p) const { free(p); } + +#endif diff --git a/src/v8-extensions/serialized-data.h b/src/v8-extensions/serialized-data.h new file mode 100644 index 0000000..148bb76 --- /dev/null +++ b/src/v8-extensions/serialized-data.h @@ -0,0 +1,46 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. + +#pragma once + +#include "externalized-contents.h" + +namespace napa { +namespace v8_extensions { + + using namespace v8; + + typedef std::pair> ExternalizedSharedArrayBufferContents; + + /// + /// SerializedData holds the serialized data of a JavaScript object, and it is required during its deserialization. + /// If the JavaScript object has properties or elements of SharedArrayBuffer or types based on SharedArrayBuffer, + /// like DataView and TypedArray, their ExternalizedContents will be stored in _externalizedSharedArrayBufferContents. + /// + class SerializedData { + public: + SerializedData(); + + const uint8_t* GetData() const; + + size_t GetSize() const; + + const std::vector& GetExternalizedSharedArrayBufferContents() const; + + private: + struct DataDeleter { + void operator()(uint8_t* p) const; + }; + + std::unique_ptr _data; + size_t _size; + std::vector _externalizedSharedArrayBufferContents; + + private: + friend class Serializer; + + SerializedData(const SerializedData&) = delete; + SerializedData& operator=(const SerializedData&) = delete; + }; +} +} diff --git a/src/v8-extensions/serializer.cpp b/src/v8-extensions/serializer.cpp new file mode 100644 index 0000000..c8500d3 --- /dev/null +++ b/src/v8-extensions/serializer.cpp @@ -0,0 +1,112 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. + +#include "v8-extensions-macros.h" + +#if V8_VERSION_CHECK_FOR_BUILT_IN_TYPE_TRANSPORTER + +#include "serializer.h" + +#include +#include + +using namespace napa::v8_extensions; +using namespace v8; + +Serializer::Serializer(Isolate* isolate) : + _isolate(isolate), + _serializer(isolate, this) {} + +Maybe Serializer::WriteValue(Local value) { + bool ok = false; + _data.reset(new SerializedData); + _serializer.WriteHeader(); + + Local context = _isolate->GetCurrentContext(); + if (!_serializer.WriteValue(context, value).To(&ok)) { + _data.reset(); + return Nothing(); + } + + if (!FinalizeTransfer().To(&ok)) { + return Nothing(); + } + + std::pair pair = _serializer.Release(); + _data->_data.reset(pair.first); + _data->_size = pair.second; + return Just(true); +} + +std::shared_ptr Serializer::Release() { + return _data; +} + +void Serializer::ThrowDataCloneError(Local message) { + _isolate->ThrowException(Exception::Error(message)); +} + +Maybe Serializer::GetSharedArrayBufferId( + Isolate* isolate, + Local sharedArrayBuffer +) { + for (size_t index = 0; index < _sharedArrayBuffers.size(); ++index) { + if (_sharedArrayBuffers[index] == sharedArrayBuffer) { + return Just(static_cast(index)); + } + } + + size_t index = _sharedArrayBuffers.size(); + _sharedArrayBuffers.emplace_back(_isolate, sharedArrayBuffer); + return Just(static_cast(index)); +} + +void* Serializer::ReallocateBufferMemory(void* oldBuffer, size_t size, size_t* actualSize) { + void* result = realloc(oldBuffer, size); + *actualSize = result ? size : 0; + return result; +} + +void Serializer::FreeBufferMemory(void* buffer) { free(buffer); } + +ExternalizedSharedArrayBufferContents +Serializer::MaybeExternalize(Local sharedArrayBuffer) { + Local context = _isolate->GetCurrentContext(); + Local key = v8_helpers::MakeV8String(_isolate, "_externalized"); + bool ok = false; + if (sharedArrayBuffer->IsExternal() + && sharedArrayBuffer->Has(context, key).To(&ok)) { + Local value; + // If the SharedArrayBuffer has been externalized, just get its Contents without externalizing it again, + // and get its ExternalizedContents which has been stored in the '_externalized' property of the SharedArrayBuffer. + if (sharedArrayBuffer->Get(context, key).ToLocal(&value)) { + auto shareableWrap = NAPA_OBJECTWRAP::Unwrap(Local::Cast(value)); + auto externalizedContents = shareableWrap->Get(); + return std::make_pair(sharedArrayBuffer->GetContents(), externalizedContents); + } + return std::make_pair(sharedArrayBuffer->GetContents(), nullptr); + } else { + // If the SharedArrayBuffer has not been externalized, + // externalize it and get its Contents and ExternalizedContents at first, + // then store its ExternalizedContents in the '_externalized' property of the original SharedArrayBuffer. + auto contents = sharedArrayBuffer->Externalize(); + auto externalizedContents = std::make_shared(contents); + auto shareableWrap = napa::module::binding::CreateShareableWrap(externalizedContents); + sharedArrayBuffer->CreateDataProperty(context, key, shareableWrap); + return std::make_pair(contents, externalizedContents); + } +} + +Maybe Serializer::FinalizeTransfer() { + for (const auto& globalSharedArrayBuffer : _sharedArrayBuffers) { + Local sharedArrayBuffer = + Local::New(_isolate, globalSharedArrayBuffer); + // Externalize the SharedArrayBuffer if it hasn't been done before, + // and store it's ExternalizedContents which will be used when deserializing it in deserializer. + _data->_externalizedSharedArrayBufferContents.push_back(MaybeExternalize(sharedArrayBuffer)); + } + + return Just(true); +} + +#endif diff --git a/src/v8-extensions/serializer.h b/src/v8-extensions/serializer.h new file mode 100644 index 0000000..45f6126 --- /dev/null +++ b/src/v8-extensions/serializer.h @@ -0,0 +1,57 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. + +#pragma once + +#include "serialized-data.h" + +#include + +namespace napa { +namespace v8_extensions { + + /// + /// Serializer is used to serialize a JavaScript object to a SerializedData instance. + /// It is derived from v8::ValueSerializer::Delegate, whose interface is implemented + /// to handle ShardArrayBuffer specially as below. + /// If a SharedArrayBuffer is being serialized for the first time, + /// 1). it will be externlized and the ExternalizedContents will be attached to its SerializedData. + /// 2). a ShareableWrap of the ExternalizedContents will be set to the input SharedArrayBuffer. + /// If a SharedArrayBuffer has been serialized, the externalization will be skipped, and its ExternalizedContents + /// will be retrieved from the input SharedArrayBuffer and attached to its SerializedData. + /// + class Serializer : public v8::ValueSerializer::Delegate { + public: + explicit Serializer(v8::Isolate* isolate); + + v8::Maybe WriteValue(v8::Local value); + + std::shared_ptr Release(); + + protected: + void ThrowDataCloneError(v8::Local message) override; + + v8::Maybe GetSharedArrayBufferId( + v8::Isolate* isolate, + v8::Local sharedArrayBuffer + ) override; + + void* ReallocateBufferMemory(void* oldBuffer, size_t size, size_t* actualSize) override; + + void FreeBufferMemory(void* buffer) override; + + private: + ExternalizedSharedArrayBufferContents MaybeExternalize(v8::Local sharedArrayBuffer); + + v8::Maybe FinalizeTransfer(); + + v8::Isolate* _isolate; + v8::ValueSerializer _serializer; + std::shared_ptr _data; + std::vector> _sharedArrayBuffers; + + Serializer(const Serializer&) = delete; + Serializer& operator=(const Serializer&) = delete; + }; +} +} diff --git a/src/v8/v8-common.cpp b/src/v8-extensions/v8-common.cpp similarity index 100% rename from src/v8/v8-common.cpp rename to src/v8-extensions/v8-common.cpp diff --git a/src/v8/v8-common.h b/src/v8-extensions/v8-common.h similarity index 100% rename from src/v8/v8-common.h rename to src/v8-extensions/v8-common.h diff --git a/src/v8-extensions/v8-extensions-macros.h b/src/v8-extensions/v8-extensions-macros.h new file mode 100644 index 0000000..7d55d10 --- /dev/null +++ b/src/v8-extensions/v8-extensions-macros.h @@ -0,0 +1,15 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. + +#pragma once + +#include + +#define V8_VERSION_EQUALS_TO_OR_NEWER_THAN(MAJOR, MINOR) \ + (V8_MAJOR_VERSION == (MAJOR) && V8_MINOR_VERSION >= (MINOR) || V8_MAJOR_VERSION > (MAJOR)) + +#define V8_VERSION_CHECK_FOR_ARRAY_BUFFER_ALLOCATOR \ + V8_VERSION_EQUALS_TO_OR_NEWER_THAN(5, 5) + +#define V8_VERSION_CHECK_FOR_BUILT_IN_TYPE_TRANSPORTER \ + V8_VERSION_EQUALS_TO_OR_NEWER_THAN(6, 2) diff --git a/src/v8-extensions/v8-extensions.cpp b/src/v8-extensions/v8-extensions.cpp new file mode 100644 index 0000000..c997153 --- /dev/null +++ b/src/v8-extensions/v8-extensions.cpp @@ -0,0 +1,32 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. + +#include "v8-extensions-macros.h" + +#if V8_VERSION_CHECK_FOR_BUILT_IN_TYPE_TRANSPORTER + +#include "deserializer.h" +#include "serializer.h" +#include "v8-extensions.h" + +using namespace napa; +using namespace v8; + +std::shared_ptr +v8_extensions::Utils::SerializeValue(Isolate* isolate, Local value) { + bool ok = false; + Serializer serializer(isolate); + if (serializer.WriteValue(value).To(&ok)) { + return serializer.Release(); + } + return nullptr; +} + +MaybeLocal +v8_extensions::Utils::DeserializeValue(Isolate* isolate, std::shared_ptr& data) { + Local value; + Deserializer deserializer(isolate, data); + return deserializer.ReadValue(); +} + +#endif diff --git a/src/v8-extensions/v8-extensions.h b/src/v8-extensions/v8-extensions.h new file mode 100644 index 0000000..9f34136 --- /dev/null +++ b/src/v8-extensions/v8-extensions.h @@ -0,0 +1,23 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. + +#pragma once + +#include +#include + +namespace napa { +namespace v8_extensions { + + class SerializedData; + + class NAPA_API Utils { + public: + static std::shared_ptr + SerializeValue(v8::Isolate* isolate, v8::Local value); + + static v8::MaybeLocal + DeserializeValue(v8::Isolate* isolate, std::shared_ptr& data); + }; +} +} diff --git a/src/zone/worker.cpp b/src/zone/worker.cpp index 329f791..72dc00f 100644 --- a/src/zone/worker.cpp +++ b/src/zone/worker.cpp @@ -4,7 +4,6 @@ #include "worker.h" #include -#include #include @@ -16,6 +15,11 @@ #include #include +#include +#if !(V8_VERSION_CHECK_FOR_ARRAY_BUFFER_ALLOCATOR) + #include +#endif + using namespace napa; using namespace napa::zone; @@ -156,7 +160,7 @@ static v8::Isolate* CreateIsolate(const settings::ZoneSettings& settings) { v8::Isolate::CreateParams createParams; // The allocator is a global V8 setting. -#if (V8_MAJOR_VERSION == 5 && V8_MINOR_VERSION >= 5) || V8_MAJOR_VERSION > 5 +#if V8_VERSION_CHECK_FOR_ARRAY_BUFFER_ALLOCATOR static std::unique_ptr defaultArrayBufferAllocator(v8::ArrayBuffer::Allocator::NewDefaultAllocator()); createParams.array_buffer_allocator = defaultArrayBufferAllocator.get(); #else diff --git a/test/transport-builtin-test.js b/test/transport-builtin-test.js new file mode 100644 index 0000000..bd99909 --- /dev/null +++ b/test/transport-builtin-test.js @@ -0,0 +1,246 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT license. + +const assert = require('assert'); +const napa = require('../lib/index'); +let zoneId = 'transport-test-zone'; +let zone = napa.zone.create(zoneId, { workers: 4 }); + +/// Construct an expected result string. +/// constructExpectedResult(5, 5, 255) returns '0,0,0,0,0' +/// constructExpectedResult(2, 5, 255) returns '0,0,255,255,255' +/// constructExpectedResult(0, 5, 255) returns '255,255,255,255,255' +global.constructExpectedResult = +function constructExpectedResult(i, length, value) { + const assert = require('assert'); + assert(i >= 0 && length >= i); + let expected = ''; + for (var t = 0; t < i; t++) { + if (t > 0) expected += ','; + expected += '0'; + } + for (var t = i; t < length; t++) { + if (t > 0) expected += ','; + expected += value.toString(); + } + return expected; +} +zone.broadcast("global.constructExpectedResult = " + constructExpectedResult.toString()); + +function transportSharedArrayBuffer() { + var promises = []; + var sab = new SharedArrayBuffer(4); + for (var i = 0; i < 4; i++) { + promises[i] = zone.execute((sab, i) => { + var ta = new Uint8Array(sab); + ta[i] = 100; + }, [sab, i]); + } + + return Promise.all(promises).then(values => { + var ta = new Uint8Array(sab); + assert.deepEqual(ta.toString(), '100,100,100,100'); + }); +} +exports.transportSharedArrayBuffer = transportSharedArrayBuffer; + +function transportCompositeObjectOfSharedArrayBuffer() { + let sab = new SharedArrayBuffer(4); + let ta1 = new Uint8Array(sab); + let ta2 = new Uint8Array(sab); + let obj = { sab: sab, tas: { ta1: ta1, ta2: ta2 }, ta22:ta2 }; + return zone.execute((obj) => { + var ta = new Uint8Array(obj.sab); + ta[0] = 99; + obj.tas.ta1[1] = 88; + obj.tas.ta2[2] = 77; + obj.ta22[3] = 66; + }, [obj]).then((result) => { + var ta_sab = new Uint8Array(sab); + assert.deepEqual(ta_sab.toString(), '99,88,77,66'); + }); +} +exports.transportCompositeObjectOfSharedArrayBuffer = transportCompositeObjectOfSharedArrayBuffer; + +function recursivelySetElementOfSharedArrayBuffer(zoneId, sab, i, value) { + if (i < 0) return; + let ta = new Uint8Array(sab); + ta[i] = value; + + const assert = require('assert'); + + // SharedArrayBuffer shares storage when it is transported, + // so elements with index > i have been set to {value} by those finished zone.executions. + let expected = global.constructExpectedResult(i, ta.length, value); + assert.equal(ta.toString(), expected); + + const napa = require('../lib/index'); + let zone = (i % 4 < 2) ? napa.zone.get(zoneId) : napa.zone.node; + zone.execute( + recursivelySetElementOfSharedArrayBuffer, + [zoneId, sab, i - 1, value] + ).then((result) => { + // SharedArrayBuffer shares storage when it is transported, + // if i > 0, ta[i - 1] has been set to {value} by the previous zone.execute, + // so ta.toString() should be larger than {expected} constructed before. + if (i > 0) assert(ta.toString() > expected); + else if (i === 0) assert.equal(ta.toString(), expected); + else assert(false); + }); +} + +function recursivelyTransportSharedArrayBuffer(length, timeout) { + let value = 255; + let sab = new SharedArrayBuffer(length); + let ta = new Uint8Array(sab); + recursivelySetElementOfSharedArrayBuffer(zoneId, sab, length - 1, value); + + return new Promise((resolve, reject) => { + setTimeout(() => { + // Because SharedArrayBuffer will share storage when it is transported, + // once the recursive process finished, all elements of + // the original TypeArray (based on SharedArrayBuffer) should have been set to {value}. + let expected = global.constructExpectedResult(0, ta.length, value); + assert.equal(ta.toString(), expected); + resolve(); + }, timeout); + }); +} +exports.recursivelyTransportSharedArrayBuffer = recursivelyTransportSharedArrayBuffer; + + +function recursivelySetElementOfTypedArray_SAB(zoneId, ta, i, value) { + if (i < 0) return; + ta[i] = value; + + const assert = require('assert'); + + // SharedArrayBuffer shares storage when it is transported, + // so elements with index > i have been set to {value} by those finished zone.executions. + let expected = global.constructExpectedResult(i, ta.length, value); + assert.equal(ta.toString(), expected); + + const napa = require('../lib/index'); + let zone = (i % 4 < 2) ? napa.zone.get(zoneId) : napa.zone.node; + zone.execute( + recursivelySetElementOfTypedArray_SAB, + [zoneId, ta, i - 1, value] + ).then((result) => { + // SharedArrayBuffer shares storage when it is transported, + // if i > 0, ta[i - 1] has been set to {value} by the previous zone.execute, + // so ta.toString() should be larger than {expected} constructed before. + if (i > 0) assert(ta.toString() > expected); + else if (i === 0) assert.equal(ta.toString(), expected); + else assert(false); + }); +} + +function recursivelyTransportTypedArray_SAB(length, timeout) { + let value = 255; + let sab = new SharedArrayBuffer(length); + let ta = new Uint8Array(sab); + recursivelySetElementOfTypedArray_SAB(zoneId, ta, length - 1, value); + + return new Promise((resolve, reject) => { + setTimeout(() => { + // Because SharedArrayBuffer will share storage when it is transported, + // once the recursive process finished, all elements of + // the original TypeArray (based on SharedArrayBuffer) should have been set to {value}. + let expected = global.constructExpectedResult(0, ta.length, value); + assert.equal(ta.toString(), expected); + resolve(); + }, timeout); + }); +} +exports.recursivelyTransportTypedArray_SAB = recursivelyTransportTypedArray_SAB; + + +function recursivelySetElementOfArrayBuffer(zoneId, ab, i, value) { + if (i < 0) { + return; + } + + let ta = new Uint8Array(ab); + ta[i] = value; + + const assert = require('assert'); + + // ArrayBuffer's storage will be copied when it is transported. + // Elements with index > i should all be {value}. + // They are copied from the previous zone.execution. + let expected = global.constructExpectedResult(i, ta.length, value); + assert.equal(ta.toString(), expected); + + const napa = require('../lib/index'); + let zone = (i % 4 < 2) ? napa.zone.get(zoneId) : napa.zone.node; + zone.execute( + recursivelySetElementOfArrayBuffer, + [zoneId, ab, i - 1, value] + ).then((result) => { + // The original TypeArray (based on ArrayBuffer) shouldn't been changed by the just-finished zone.execute. + assert.equal(ta.toString(), expected); + }); +} + +function recursivelyTransportArrayBuffer(length, timeout) { + let value = 255; + let ab = new ArrayBuffer(length); + let ta = new Uint8Array(ab); + recursivelySetElementOfArrayBuffer(zoneId, ab, length - 1, value); + + return new Promise((resolve, reject) => { + setTimeout(() => { + // Except ta[ta-length -1] was set to {value} before the 1st transportation, + // the original TypeArray (based on ArrayBuffer) shouldn't been changed by the recursive execution. + let expected = global.constructExpectedResult(ta.length - 1, ta.length, value); + assert.equal(ta.toString(), expected); + resolve(); + }, timeout); + }); +} +exports.recursivelyTransportArrayBuffer = recursivelyTransportArrayBuffer; + + +function recursivelySetElementOfTypeArray_AB(zoneId, ta, i, value) { + if (i < 0) { + return; + } + + ta[i] = value; + + const assert = require('assert'); + + // ArrayBuffer's storage will be copied when it is transported. + // Elements with index > i should all be {value}. + // They are copied from the previous zone.execution. + let expected = global.constructExpectedResult(i, ta.length, value); + assert.equal(ta.toString(), expected); + + const napa = require('../lib/index'); + let zone = (i % 4 < 2) ? napa.zone.get(zoneId) : napa.zone.node; + zone.execute( + recursivelySetElementOfTypeArray_AB, + [zoneId, ta, i - 1, value] + ).then((result) => { + // The original TypeArray (based on ArrayBuffer) shouldn't been changed by the just-finished zone.execute. + assert.equal(ta.toString(), expected); + }); +} + +function recursivelyTransportTypedArray_AB(length, timeout) { + let value = 255; + let ab = new ArrayBuffer(length); + let ta = new Uint8Array(ab); + recursivelySetElementOfTypeArray_AB(zoneId, ta, length - 1, value); + + return new Promise((resolve, reject) => { + setTimeout(() => { + // Except ta[ta-length -1] was set to {value} before the 1st transportation, + // the original TypeArray (based on ArrayBuffer) shouldn't been changed by the recursive execution. + let expected = global.constructExpectedResult(ta.length - 1, ta.length, value); + assert.equal(ta.toString(), expected); + resolve(); + }, timeout); + }); +} +exports.recursivelyTransportTypedArray_AB = recursivelyTransportTypedArray_AB; diff --git a/test/transport-test.ts b/test/transport-test.ts index a4be775..5c00b62 100644 --- a/test/transport-test.ts +++ b/test/transport-test.ts @@ -6,6 +6,9 @@ import * as assert from 'assert'; import * as path from 'path'; import * as t from './napa-zone/test'; +// TODO #150 (helloshuangzi): Update TypeScript version and convert './transport-builtin-test' to TypeScript. +var transportBuiltin = require('./transport-builtin-test'); + describe('napajs/transport', () => { let napaZone = napa.zone.create('zone10'); describe('TransportContext', () => { @@ -91,4 +94,42 @@ describe('napajs/transport', () => { napaZone.execute('./napa-zone/test', "nontransportableTest"); }); }); + + function transportBuiltinObjects() { + it('@node: transport SharedArrayBuffer (SAB)', () => { + return transportBuiltin.transportSharedArrayBuffer(); + }); + + it('@node: transport composite object of SharedArrayBuffer', () => { + return transportBuiltin.transportCompositeObjectOfSharedArrayBuffer(); + }); + + // @node: node -> napa -> napa -> node -> node -> napa -> napa + it('@node: recursively transport received SharedArrayBuffer (SAB)', () => { + return transportBuiltin.recursivelyTransportSharedArrayBuffer(8, 50); + }); + + // @node: node -> napa -> napa -> node -> node -> napa -> napa + it('@node: recursively transport received TypedArray based on SAB', () => { + return transportBuiltin.recursivelyTransportTypedArray_SAB(8, 50); + }); + + // @node: node -> napa -> napa -> node -> node -> napa -> napa + it('@node: recursively transport received ArrayBuffer (AB)', () => { + return transportBuiltin.recursivelyTransportArrayBuffer(8, 50); + }); + + // @node: node -> napa -> napa -> node -> node -> napa -> napa + it('@node: recursively transport received TypedArray based on AB', () => { + return transportBuiltin.recursivelyTransportTypedArray_AB(8, 50); + }); + } + + var builtinTestGroup = 'Transport built-in objects'; + if (process.version >= 'v9.0.0' || process.version.indexOf('.') > 2) { + describe(builtinTestGroup, transportBuiltinObjects); + } else { + describe.skip(builtinTestGroup, transportBuiltinObjects); + require('npmlog').warn(builtinTestGroup, 'This test group is skipped since it requires node newer than v9.0.0'); + } }); \ No newline at end of file diff --git a/test/zone-test.ts b/test/zone-test.ts index 6705883..3e3b82f 100644 --- a/test/zone-test.ts +++ b/test/zone-test.ts @@ -40,7 +40,7 @@ describe('napajs/zone', function () { }); it('@napa: zone id already exists', () => { - shouldFail(() => { + return shouldFail(() => { return napaZone1.execute(`${napaLibPath}/zone`, 'create', ['napa-zone1']); }); }); @@ -85,7 +85,7 @@ describe('napajs/zone', function () { }); it('@napa: zone not existed', () => { - shouldFail(() => { + return shouldFail(() => { return napaZone1.execute(`${napaLibPath}/zone`, 'get', ['zonex']); }); });