diff --git a/doc/CMakeLists.txt b/doc/CMakeLists.txt index 4b342e3d..c3ad6b0a 100644 --- a/doc/CMakeLists.txt +++ b/doc/CMakeLists.txt @@ -89,6 +89,14 @@ if (Haskell_PANDOC_EXECUTABLE AND Doxygen_EXECUTABLE) OPTIONS --self-contained --table-of-contents OUTPUT_DIR manual) + add_pandoc_markdown(src/bond_comm.md src/bond_comm_roadmap.md + OPTIONS --self-contained --table-of-contents + OUTPUT_DIR manual) + + add_pandoc_markdown(src/bond_comm_epoxy.md src/bond_comm_epoxy_wire.md + OPTIONS --self-contained --table-of-contents + OUTPUT_DIR manual) + add_pandoc_markdown (src/bond_cs.md CODE "cs,numberLines" OPTIONS --self-contained --table-of-contents diff --git a/doc/src/bond_comm.md b/doc/src/bond_comm.md new file mode 100644 index 00000000..28606ddb --- /dev/null +++ b/doc/src/bond_comm.md @@ -0,0 +1,59 @@ +% Bond Communications + +# About # + +The Bond Communications libraries allows participants to pass Bond messages +between themselves. + +# Features # + +## Defining Services ## + +The Bond IDL has been extended to support the definition of +[services](compiler.html#service-definition) and +[generic services](compiler.html#generic-service). These definitions are +used by the Bond compiler to generate: + +* service stubs that can be used as the basis for implementing service + methods +* proxy stubs that can be used by clients to invoke those methods + +## Transport Flexibility ## + +The Bond service stubs and proxy stubs are not aware of exactly how the +client and server get connected or how messages are exchanged between +clients and servers. These capabilities are encapsulated in a transport. + +Bond includes the following transports: + +* [Epoxy](bond_epoxy.html): asynchronous communication over TCP connections + with support for TLS using a binary protocol + +Not all transports support all messaging patterns. Check the documentation +for the respective transports. + +See the [roadmap](bond_comm_roadmap.html) for plans about future transports. + +## Messaging Patterns ## + +There are a number of conceptual messaging patterns that are supported: + +* request/response: a message is sent to the receiver and the receiver sends + back a different message (including the ability to send back an error + response) +* event: a one-way, best effort, message is sent to a receiver. There's no + way to know whether the message was delivered or processed successfully. + +### Future Messaging Patterns ### + +We expect to implement additional messaging patterns in the future. Patterns +that are intriguing include--but are not limited to: + +* sequences of requests/responses: similar to request/response, but with a + sequence of requests/responses of unknown count +* message bus/broadcast patterns: send the same request to a bunch of + listeners, with a variety of ways to collect responses--if any +* topics/subscriptions: publish a stream of events and receive a filtered + view of that stream +* aggregations: send requests to multiple receivers and combine their + responses in various ways diff --git a/doc/src/bond_comm_roadmap.md b/doc/src/bond_comm_roadmap.md new file mode 100644 index 00000000..ca1668a6 --- /dev/null +++ b/doc/src/bond_comm_roadmap.md @@ -0,0 +1,38 @@ +% Bond Comm Roadmap + +# Roadmap + +This is what we currently are working on for Bond Comm. + +Note that none of these feature are guaranteed to ship. + +If there's a feature you think is useful and should be on the roadmap or +would like to help implement a feature, please see the +[guidelines for contributing](https://github.com/Microsoft/bond/blob/master/CONTRIBUTING.md). + +## Sort term (next few weeks) + +* TLS support in C# Epoxy +* C# Epoxy performance improvements +* C++ support + +## Medium term (next few months) + +* additional languages, based on interest +* HTTP(S)-based transports (HTTP 1.1 or HTTP 2 TBD) + +## Long term (next few years) + +* additional messaging patterns +* additional transports + +## Never + +These are features that we never plan to implement in Bond Comm. + +* Automatic polymorphic deserialization of Error responses and conversion to + language-specific exceptions. To do so transparently and automatically, + type information would need to be included in payloads. This is not + something we want to require all users to include in their payloads. We + wouldn't be opposed to providing library support to make this easier to + opt in to, but we will not make it the default due to its cost. diff --git a/doc/src/bond_cs.md b/doc/src/bond_cs.md index 8c06f3ec..43d81e93 100644 --- a/doc/src/bond_cs.md +++ b/doc/src/bond_cs.md @@ -1134,50 +1134,47 @@ public constructors: Bond Comm ========= -The Bond communication framework makes it easy and efficent to construct services -and hook clients up to those services. Built on top of the Bond serialization framework, -Bond Comm aims for the same principles of high-performance and extensibility. +The [Bond Communication](bond_comm.html) C# framework makes it easy and efficent to +construct services and hook clients up to those services. Built on top of the Bond +serialization framework, Bond Comm aims for the same principles of high-performance and +extensibility. -Today the framework supports "request-response" -(roundtrip messages supporting response payloads or error return) and "event" -(one-way fire-and-forget messages with no response or error return) messaging patterns. +Today the framework supports two messaging patterns: +- request-response: roundtrip messages supporting either payload or error responses +- event: one-way, fire-and-forget messages with no responses -Bond Comm is naturally asyncronous, and the C# implementation takes advantage of the -idiomatic async/await facilities of .NET. +Bond Comm is naturally asynchronous, and the C# implementation takes advantage of the +idiomatic Task and async/await facilities of .NET. -The initial release of Bond Comm is designed to run on .NET 4.5 and Core CLR. Since -Core CLR is not 100% stable yet, we will update as needed. Mono support is forthcoming. +The initial release of Bond Comm is designed to run on Windows running .NET 4.5 or Core +CLR. Since Core CLR is not 100% stable yet, we will update as needed. Mono support is +forthcoming, as are Linux and Mac OS X. -### Defining Services ### +Only Windows 10 and Windows Server 2012 R2 have been tested to date. -The Bond IDL has been extended to support the definition of -[services](https://Microsoft.github.io/bond/manual/compiler.html#service-definition) -and [generic services](https://Microsoft.github.io/bond/manual/compiler.html#generic-services). -These definitions are used by the Bond compiler to generate: +See the following examples: -- service stubs that can be used as the basis for implementing service methods -- proxy stubs that can be used by clients to invoke those methods +- `examples/cs/comm/pingpong` +- `examples/cs/comm/notifyevent` -The service stub contains a service base class that the developer -should subclass to provide the business logic of their service. +### Defining services ### -The proxy stub provides a class with methods that the developer can call to +Cross-language services are defined in the [Bond IDL](bond_comm.html#defining-services). + +The generated C# service stub contains a service base class that the developer should subclass to +provide the business logic of their service. + +The generated proxy stub provides a class with methods that the developer can call to exhange messages with the service. -### Transports ### - -The Bond service stubs and proxy stubs are not aware of exactly how the client -and server get connected or how bytes are exchanged between client and server. -These capabilites are encapsulated in Transports and their related Connection -classes. - ### Epoxy Transport ### -The initial release of Bond Comm provides a TCP-based transport called EpoxyTransport. -This is the recommended default Transport. The EpoxyTransport is designed to be -Core CLR compliant which will allow for cross-platform support. +The initial release of Bond Comm provides a binary transport called +[Epoxy](bond_epoxy.html). This is the recommended default Transport. The +Epoxy transport is designed to be Core CLR compliant, which will allow for +cross-platform support. -TLS support in EpoxyTransport is forthcoming. +TLS support in Epoxy is forthcoming. ### Logging and Metrics Facades ### @@ -1185,27 +1182,37 @@ Bond Comm includes facades for logging and metrics. By writing simple handlers for logging and metrics, developers can supply their own logging and metrics facilities to gather log and metric data from Bond Comm-based services. +See the following examples: + +- `examples/cs/comm/logging` +- `examples/cs/comm/metrics` + ### Layers ### -Bond Comm includes extensibility points called "layers" that provides hooks for -capabilites that are not service specific. An example of this would be tracing -hooks like those mentioned in the [Dapper paper](http://research.google.com/pubs/pub36356.html). +Bond Comm includes extensibility points called "layers" that provides hooks +for capabilites that are not service- or method-specific. An example of this +would be tracing hooks like those mentioned in the [Dapper +paper](http://research.google.com/pubs/pub36356.html). A set of layers are encapsulated together in a "layer stack", which can be -provided to the Transport. When the Transport sends out a message, the layer stack -invokes the layers in forward order. On the receive side, the layer stack -invokes the layers in reverse order. +provided to the Transport. When the Transport sends out a message, the layer +stack invokes the layers in forward order. On the receive side, the layer +stack invokes the layers in reverse order. -The layers in a layer stack are expected to share a single Bond structure for -passing state from the send side to the receive side. This structure is serialized -by the Transport and sent alongside the actual message payload. +The layers in a layer stack are expected to share a single Bond structure +for passing state from the send side to the receive side. This structure is +serialized by the transport and sent alongside the actual message payload. -If any layer returns an error, the layer stack is halted and the error replaces -the current message; therefore, layers should only return errors for conditions -that are truly fatal for the current message. +If any layer returns an error, the layer stack is halted and the error +replaces the current message; therefore, layers should only return errors +for conditions that are truly fatal for the current message. Support for layers that contain per-instance state is forthcoming. +### Roadmap ### + +We have a [roadmap](bond_comm_roadmap.html) for Bond Comm. + References ========== @@ -1218,8 +1225,10 @@ References [Python User's Manual][bond_py] ---------------------------- -[compiler]: compiler.html - -[bond_py]: bond_py.html +[Bond Comm overview][bond_comm] +---------------------------- +[bond_comm]: bond_comm.html [bond_cpp]: bond_cpp.html +[bond_py]: bond_py.html +[compiler]: compiler.html diff --git a/doc/src/bond_epoxy.md b/doc/src/bond_epoxy.md new file mode 100644 index 00000000..385d9032 --- /dev/null +++ b/doc/src/bond_epoxy.md @@ -0,0 +1,13 @@ +% Bond Epoxy transport + +# About # + +The Bond Epoxy transport is a Bond [communication +transport](bond_comm.html#transport-flexibility) that uses a [custom binary +protocol over TCP](bond_epoxy_wire.html), and optionally TLS (in a +forthcoming release), to pass messages between participants. + +It supports the following messaging styles: + +* request/response +* event diff --git a/doc/src/bond_epoxy_wire.md b/doc/src/bond_epoxy_wire.md new file mode 100644 index 00000000..d4ffa376 --- /dev/null +++ b/doc/src/bond_epoxy_wire.md @@ -0,0 +1,599 @@ +% Bond Epoxy transport wire format + +# Epoxy transport wire format # + +This documents the [Bond Epoxy transport](bond_epoxy.html) wire format. For +API-level documentation, see the respective language guides. + +The intended audience for this is transport implementers so that compatible +clients/servers can be produced. + +# Requirements # + +Since Bond transports operate on messages, the Epoxy transport needs a way +to group sequences of bytes within the stream of bytes provided by a TCP +connection into individual messages. It does this by "framing" each message. +Thus, this wire format turns a stream of bytes into a sequence of _frames_. + +These frames are then used to transmit Bond messages between participants. +The messages are decomposed into simple tuples. These tuples are used to +implement the [messaging patterns](bond_comm.html#messaging-patterns) this +transport supports. + +The Epoxy transport wire format should be [simple](#example), so that it is +easy to implement--at least the basic feature set. + +The Epoxy transport defers to Bond serialization/marshalling when it can. +Instead of re-implementing backward/forward compatible serialization +schemes, it uses Bond. For any structures that are used as part of the +protocol itself, the [Fast Binary](bond_cpp.html#fast-binary) protocol +version 1 is used to serialize them, as this protocol is simple to +implement. Any other structures are marshalled so that applications do not +have to convert from their serialization format of choice. + +"Interesting" structures need to be able to be allocated and +serialized/deserialized independently. For example, a Bond Epoxy transport +proxy that only wants to inspect headers should not be required to +deserialize entire payloads. + +# Conventions # + +The following hold in this document, unless otherwise specified. + +* All sizes are given in bytes. +* All multi-byte quantities are little-endian. + +## Terminology ## + +* _**client**_: an entity that establishes connections to _servers_. In + Berkeley socket parlance, this entity calls `connect()` +* _**connection**_: an established Epoxy connection capable of + bi-directional communication +* _**conversation**_: one message-based interaction between two _endpoints_. + This includes things like the request/response and event + [messaging patterns](bond_comm.html#messaging-patterns). +* _**endpoint**_: either the _client_ or the _server_ +* _**frame**_: a collection of related data, all of which are transmitted + together +* _**framelet**_: a type, length, data tuple within a _frame_ +* _**initiator**_: the entity that starts a conversation +* _**layer data**_: opaque blob of data used by Bond Communication's layers + facility +* _**payload data**_: opaque blob of data that is the contents of the + message being sent +* _**receiver**_: the endpoint that is reading _frames_, regardless of the + higher-level messaging pattern being employed +* _**sender**_: the endpoint originating _frames_, regardless of the + higher-level messaging pattern being employed +* _**server**_: an entity listening for connections from _clients_, + typically on a well-known port. In Berkeley socket parlance, this entity + calls `bind()`, `listen()`, and `accept()` +* _**service**_: a Bond entity that exposes invokable methods that may take + and return Bond structures + +# The frame # + +A frame is the indivisible unit of transmission in the Epoxy transport. + +Each frame is an ordered collection of type, length, data tuples. These +tuples are called _framelets_. + +The basic structure of a frame is: + +* the count of the framelets in the frame (2 bytes) +* framelet 1 +* ... +* framelet _n_ + +The framelet count must be greater than 0 and less than 65,535. + +The first framelet in the frame determines the type of the frame. It +dictates which subsequent framelets (after the first) are allowed and in +which order. Thus, the order of the framelets within a frame must be +preserved by both the sender and the receiver. + +The possible frame types are: + +| Frame type | First framelet | +|------------|----------------| +| [Config frame](#establishing-a-connection) | `EpoxyConfig` | +| [Message frame](#exchanging-messages) | `EpoxyHeaders` | +| [Error frame](#protocol-error) | `ProtocolError` | + +Each frame type is described in more details in its own section. + +See also [Limits](#limits) for a discussion about things like maximum frame +size. + + +----0---+----1---+================+ + | framelet count | framelets ... | + +--------+--------+================+ + +## Framelets ## + +A framelet is an entity within a frame. It consists of: + +* the [framelet type](#framelet-types) (2 bytes) +* the framelet content size (4 bytes, but see [Limits](#limits).) +* the content itself + +How the content of a framelet is interpreted depends on the type. + + +----0---+----1---+----2---+----3---+----4---+----5---+============+ + | framelet type | framelet size | content | + +--------+--------+--------+--------+--------+--------+============+ + +## Framelet types ## + +Each framelet has a unique type ID, regardless of which frame its is permitted to occur in. + +| Framelet | Type ID | Parent frame | Required | Encoding | Contents | +|----------|---------|--------------|----------|----------|----------| +| `EpoxyConfig` | 0x47 0x43 ("GC") | Config | Yes | Fast Binary v1 serialized | [`EpoxyConfig`](#epoxy-config-structure) structure | +| `EpoxyHeaders` | 0x52 0x48 ("RH") | Message | Yes | Fast Binary v1 serialized | [`EpoxyHeaders`](#epoxy-headers-structure) structure | +| `LayerData` | 0x59 0x4C ("YL") | Message | No | Marshalled Bond data | Auxiliary data used by Bond Communications layers stack | +| `PayloadData` | 0x54 0x44 ("TD") | Message | Yes | Marshalled Bond data | Actual message payload | +| `ProtocolError` | 0x52 0x45 ("RE") | Error | Yes | Fast Binary v1 serialized | [`ProtocolError`](#protocol-error) structure | + +_Note_: The framelet type IDs were assigned such that they are mnemonic +strings for their contents. In this table, they are "reversed" from what +might be expected, as they are little-endian encoded. This reversal makes +them show up the "right way" when reading a frame in most hex editors. + +### Unknown framelets ### + +If an endpoint encounters an unknown framelet, it must send a +[protocol error](#protocol-error) with the error code `PROTOCOL_VIOLATED` +and close the connection. + +Encountering an unknown framelet indicates that the handshake +[connection handshake](#establishing-a-connection) has gone catastrophically +wrong, as newer features can only be used after both endpoints have agreed +to them. + +This complicates the implementation and deployment of Epoxy transport +proxies (not to be confused with the generated client-side proxy objects), +but in our experience, transport-specific proxies are extremely rare and are +only used in high constrained environments. + +# Connection life cycle # + +When a connection is initially established, a +[two frame handshake](#establishing-a-connection) occurs. Then the endpoints +[exchange message](#exchanging-messages) with each other. + +At any point, one of the endpoints may send a +[protocol error](#protocol-error). It may then either gracefully or forcibly +close its TCP connection, ending the Epoxy connection. + +When one endpoint has finished sending all its messages, it gracefully +closes its side of the TCP connection. It should then wait a reasonable +amount of time for the other side to also gracefully close its its side of +the TCP connection, ending the Epoxy connection. + +# Establishing a connection # + +When a connection is initially established, a two frame handshake occurs. + +The client sends a frame consisting of only one `EpoxyConfig` framelet with +the [various features proposes to use](#epoxy-config-structure). The client +then waits for a corresponding frame to be sent by the server. + +Upon accepting a connection, the server waits for the first frame from the +client. The server then inspects the `EpoxyConfig` structure and determines +whether and which features to use. The server then sends back the final +`EpoxyConfig` that will be used for this connection. If the server is +presented with a config it cannot or will not use, it must respond with a +frame containing only a `ProtocolError`framelet with error code +`CLIENT_BAD_HANDSHAKE` and close the connection. + +Application should be given a chance to accept/reject a connection when it +is being established. If the connection is rejected, a `ProtocolError` with +error code `CONNECTION_REJECTED` must be sent. Then, the connection must be +closed. + +If the client does not agree with the choices that the server made or if the +server committed a protocol violation (e.g., selecting an option not +proposed by the client), it may send a frame containing only a +`ProtocolError`framelet. Then it must close the connection. There is no way +to re-handshake. + +## Secure connections ## + +The Bond Epoxy wire format has no provisions for communications security. + +Implementations of the Epoxy transport are encouraged to offer both plain +TCP and TLS connections. Configuration of the TLS parameters should be +deferred to the application. + +When applications are given a chance to accept/reject a connection, +certificate details should be made available to them. + +Authorization is an application-level problem, and applications will need to +incorporate their authorization requirements into their service definitions. +Layer data is a good place to store information that is used across all +services and methods on the same connection. + +## Recommended port ## + +The recommended port for the Bond Epoxy transport is 25188 (ASCII "bd"). + +If TLS connections are being used, the recommended port is 25156 (ASCII +"bD"). + +# Exchanging messages # + +A Message frame is used to send a message between endpoints. It starts with +an `EpoxyHeaders` framelet. The required order of the framelets is: + +1. `EpoxyHeaders` +1. optional `LayerData` +1. `PayloadData` + +This order allows for clients and servers to start processing the frame as +it arrives. + +# Protocol Error # + +An Error frame consisting of just a `ProtocolError` framelet is used to signal +critical transport errors from which the transport cannot recover. Upon +receipt of a `ProtocolError` framelet, the connection transitions to an +error state and must be closed. All subsequent frames may be discarded. + +The contents of a `ProtocolError` framelet is a `ProtocolError` struct, +serialized in Bond Fast Binary v1. + +# Framelet structures # + +Within the Epoxy transport wire format, the following Bond structures are +used for schematized data, especially when backward/forward compatibility is +needed. Since Bond already has a rich system for dealing with such data, the +fields within these structures are not decomposed into framelets. + +## Epoxy Config Structure ## + +The `EpoxyConfig` structure is used to configure the Epoxy protocol during +the [initial handshake](#establishing-a-connection). + +It is currently empty, but as new features are added, fields to +enable/disable their use will be added to this structure. + + namespace bond.comm.epoxy; + + struct EpoxyConfig + { + } + +## Epoxy Headers Structure ## + +The `EpoxyHeaders` structure is used for metadata needed to implement the +various messaging patterns. + + namespace bond.comm.epoxy; + + enum PayloadType + { + Request = 1; + Response = 2; + Event = 3; + } + + struct EpoxyHeaders + { + 0: uint64 conversation_id; + 1: required PayloadType payload_type; + 2: string method_name; + 3: int32 error_code; + } + +### Conversation ID ### + +Conversation IDs are used to identify _conversations_. Messaging patterns +like request/response use conversation IDs to associate the request with the +response, as many requests and responses are simultaneously in flight over +the same connection. Other patterns, like event, just use the conversation +ID for diagnostic purposes (e.g., to correlate client- and server-side +logs.) + +When a sender initiates a new conversation, it must generate a new +conversation ID unique to the connection. If the receiver needs to refer to +the same conversation, it must use the same conversation ID the sender +assigned. + +Frames with unexpected conversation IDs may be dropped. (E.g., if a client +initiated a request/response but gave up before the server sent the +response, the client is permitted to ignore the response.) + +A client may only use odd-numbered conversation IDs; a server may only use +even-numbered conversation IDs. There are no reserved conversation IDs +otherwise. + +Conversation IDs should be assigned in increasing order to aid in debugging, +but endpoints are free to use whatever scheme they like. Conversation IDs +should not be reused for a different conversation during the lifetime of a +connection, as stated above, unexpected conversation IDs may be dropped. + +Endpoints must not rely on comparing IDs across conversations for causality. +That is, there is no guarantee that conversation 8 happened before +conversation 10. For some messaging patterns, there is causality within the +same conversation. For example, endpoints _can_ assume, that the response +for conversation 7 happened after the request for conversation 7 was sent, +received, and processed. + +#### Conversation ID exhaustion #### + +When either the client or the server has exhausted all of its conversation +IDs, it must send an [Error frame](#protocol-error) with the error code +`CONVERSATION_IDS_EXHAUSTED`. This will allow the connection to be +re-established, resetting the conversation IDs. + +**DESIGN NOTE**: Re-establishing the connection is a simple way to handle +conversation ID exhaustion. Rollover schemes are complicated to specify, +implement, debug, and test. Additionally, a single high-volume endpoint +sending 1,000,000,000 conversations per second will be able to use the same +connection for over 580 years before exhausting its conversation IDs. In +practice, the `CONVERSATION_IDS_EXHAUSTED` error should never be +encountered. + +### Method name ### + +The `method_name` field is the name of the service and it's method being +invoked. + +The `method_name` field must be set to a UTF-8 encoded string (without a +BOM) that is the concatenation of the fully-qualified service name with the +method name. Namespace elements are separated by the period character ('.', +ASCII 0x2E), as is the service name from the method name. + +Example: `root_namespace.child_namespace.some_service.some_method` + +### Payload type ### + +The payload type along with the [error code](#error-code) indicates what +kind of message the `PayloadData` framelet contains. + +* `Request`: initiates a conversation that expects a response. The + `PayloadData` is a Bond structure of the expected request type for the + method being invoked. +* `Response`: completes the conversation with a response to a previous + request. Based on the [`error_code`](#error-code) field, ``PayloadData` is + either a Bond structure of the expected response type for the method + invoked or error data. +* `Event`: initiates and completes a one-way, best-effort conversation. No + response is allowed, not even an error response. + +### Error code ### + +The `error_code` field must be set to 0 for all payload types other than +`Response`. For responses, the error code indicates the error/non-error +status of a response. + +If the error code is 0, the `PayloadData` framelet contains a Bond structure +of the expected response type for the method that was invoked. + +If the error code is non-zero, the `PayloadData` framelet contains a Bond +structure derived from `Error`. + +## `ProtocolError` struct ## + + struct ProtocolError + { + 0: ProtocolErrorCode error_code; + 1: nullable> details; + } + +The `error_code` field indicates the reason for the protocol error. + +Here is the current list of protocol errors. + + enum ProtocolErrorCode + { + // An unknown error has occurred. + GENERIC_ERROR = 0; + // The endpoint has suffered a catastrophic, unrecoverable error. + INTERNAL_ERROR = 1; + // The client attempted to communicate in a way the server does not support. + NOT_SUPPORTED = 2; + // The endpoint has detected a violation of the protocol and does + // not have a more specific error to use. + PROTOCOL_VIOLATED = 3; + // Some data that the Epoxy transport needed to process, like the + // EpoxyHeaders structure, was malformed. + MALFORMED_DATA = 4; + // Some limit has been exceeded. + LIMIT_EXCEEDED = 5; + // The set of config options given to the server were unacceptable. + CLIENT_BAD_HANDSHAKE = 6; + // The client detected that the server violated the handshake + // protocol (e.g., selected an option outside the client's set.) + SERVER_BAD_HANDSHAKE = 7; + // An invalid conversation ID was used (e.g., client used even) + BAD_CONVERSATION_ID = 8; + // The endpoint is out of conversation IDs. + CONVERSATION_IDS_EXHAUSTED = 9; + // The endpoint gave up waiting for data (e.g., initial handshake, + // reading a large framelet) + TIMEOUT = 10; + + // The connection was explicitly rejected. + CONNECTION_REJECTED = 11; + + // Indicates that the endpoint received a ProtocolError that was malformed. + // This should not be sent over the network, and should only be created by + // a Proxy/Service in place of the malformed ProtocolError it refers to. + ERROR_IN_ERROR = 0xffff; + } + +# Limits # + +In order to protect resources, implementations may impose reasonable limits +on: + +* the number of framelets in a frame (must support at least 4 and should + support up to 16) +* the size of an individual framelet (must support at least 512 bytes and + should support up to 32 MiB) +* the total size of a frame (must support at least 2 KiB and should support + up to 32 MiB) + +The `LIMIT_EXCEEDED` protocol error should be used when a limit has been +exceeded. + +Endpoints must accept the full range of +[conversation IDs](#conversation-id), but may use a limited range +themselves. + +# Example # + +A simple service that can perform basic arithmetic is used in this example. + + namespace examples.calc + + enum Operation + { + Add, + Div, + Sub, + Mul, + }; + + struct Params + { + 1: int32 x; + 2: int32 y; + 3: Operation operation = Add; + }; + + struct Result + { + 1: int32 z; + }; + + service Calc + { + Result Calculate(Params); + }; + +Say we have a client that just connected to a server hosting this service +and it would like to multiply the numbers 67 and 87: + + let doSomeMathRequest = make examples.calc.Params x:67 operation:Mul y:87. + +The `EpoxyHeaders` for this request would be + + let doSomeMathHeaders = make + EpoxyHeaders + conversation_id:3 + payload_type:Request + service_name:"examples.calc.Calc" + method_name:"Calculate". + +These two structures would then be marshalled and serialized, respectively, +and put into `PayloadData` and `EpoxyHeaders` frames. + +The request frame is: + + TODO: add hex dump after stabilizing representation + +The frame is presented in hex encoding here. Offsets within the frame are in +the left-most column, and the ASCII representation of each byte is in the +right-most column. + + +Let's break down the distinct entities in this frame. + +1. ... TODO ... + +The response is similar. + + let doSomeMathResponse = make examples.calc.Params z: 5829. + let doSomeMathResponseHeaders = make + EpoxyHeaders + conversation_id:3 + payload_type:Response + error_code:0. + +The response frame is: + + TODO: add hex dump after stabilizing representation + +It has a similar break down. + +1. ... TODO ... + +# Wire format evolution # + +_This section may need be expanded and clarified after we actually add a new +feature for the first time._ + +The wire format will need to be changed over time. Here's a sketch for how +that will be done: + +* Add a capability to the `EpoxyConfig` structure and use it as part of + negotiation. +* If the feature is negotiated successfully, then subsequent frames can use + that capability. This might mean that new framelet types are going to be + sent or that the client/server expects that a given field in a Bond struct + will be processed instead of ignored. + +# Rejected designs # + +The following designs were rejected. + +## Frame/framelet padding ## + +We don't expect to see any performance benefit from padding frames or +framelets to the next machine word boundary. (Also, which machine and which +word boundary?) + +If performance testing reveals otherwise, we can adding a padding framelet +easily. + +## Other framing schemes ## + +These framing schemes were rejected: + +* `(count) length length payload payload` - sender cannot stream framelets + as it produces them +* `(count) payload delimiter payload delimiter` - don't want to deal with + escaping of payloads +* omitting the size for fixed-width framelet types - there aren't any + fixed-width framelet types, so this optimization adds complexity with + little expected payoff + +## Explicit frame types ## + +Instead of tagging each frame with an explicit type, we use the first +framelet within each frame determine the type of the frame. Duplication +among frame types is handled by using the same structures in framelet +contents. + +## Omitting event conversation IDs ## + +Having a conversation ID for events is useful for debugging, and +conversation IDs are cheap to create and send. This also removes some +special casing to, say, always assign events ID 0/1. + +## Conversation ID framelet ## + +Conversation IDs are not in their own framelet. If they were, more errors, +like `LIMIT_EXCEEDED` and `MALFORMED_DATA`, could be turned into a normal +Bond Communication errors (`status_code` non-zero) and the connection could +remain open. + +However, promoting conversation IDs to a framelet adds stuff at the lower +framing layer that the framing layer doesn't need to know about most of the +time. Protocol errors are expected to be rare, and closing the connection +simplifies implementation significantly. + +## Ignoring unknown framelets ## + +Ignoring unknown framelets makes backward/forward compatibility simple. +There would also be no need to upgrade things like proxies when a new +framelet type is added. However, if the unknown framelet changes the +semantic value of a the frame or of a known framelet, the endpoint may not +do the right thing. + +We also expect to be able to add most features to the protocol by using Bond +backward/forward compatibility on the structures embedded within the +framelets.