azure-sdk-for-js/sdk/core
Timo van Veenendaal df9a34466b
[core] Make `request` and `response` properties on RestError non-enumerable (#30518)
### Packages impacted by this PR

- `@azure/core-rest-pipeline`
- `@typespec/ts-http-runtime`

### Issues associated with this PR

- See e.g. https://github.com/Azure/azure-sdk-for-js/issues/30247
- Also https://github.com/Azure/azure-sdk-for-js/issues/29910
- And #29630

### Describe the problem that is addressed by this PR

Taken from my last PR #30483:

> `PipelineRequest` and `PipelineResponse` objects may contain secrets
in the request URL or headers. This is problematic since `RestError`
objects are often logged. While we override `util.inspect.custom` to
sanitize log output in Node, this does not work in all situations and
environments. For example:
> - `console.log` in browser does not respect `util.inspect.custom`,
meaning secrets could be logged to the browser console. Other non-Node
environments also may not respect this Node-specific functionality.
> - JSON serialization of `RestError` objects. Calling `JSON.stringify`
on the `RestError` currently results in an object that contains
unsanitized secrets. We have encountered scenarios where the JSON
serialization is logged, for example with vitest.

This PR fixes this issue by making the `request` and `response`
properties **non-enumerable**. This means they are ignored by
JSON.stringify and `Object.entries`/`Object.keys`, but the properties
are still directly accessible. This should prevent any potential
sensitive information being accidentally logged in the majority of
scenarios. I think this is a better balance of not breaking folks versus
improving security than the approach in #30483.
2024-07-24 17:34:59 -07:00
..
abort-controller [core] Add react-native support at top level (#30521) 2024-07-24 15:07:46 -07:00
core-amqp [core] Add react-native support at top level (#30521) 2024-07-24 15:07:46 -07:00
core-auth [core] Add react-native support at top level (#30521) 2024-07-24 15:07:46 -07:00
core-client [core] Add react-native support at top level (#30521) 2024-07-24 15:07:46 -07:00
core-client-rest [core] Add react-native support at top level (#30521) 2024-07-24 15:07:46 -07:00
core-http-compat [core] Add react-native support at top level (#30521) 2024-07-24 15:07:46 -07:00
core-lro [core] Add react-native support at top level (#30521) 2024-07-24 15:07:46 -07:00
core-paging [core] Add react-native support at top level (#30521) 2024-07-24 15:07:46 -07:00
core-rest-pipeline [core] Make `request` and `response` properties on RestError non-enumerable (#30518) 2024-07-24 17:34:59 -07:00
core-sse [core] Add react-native support at top level (#30521) 2024-07-24 15:07:46 -07:00
core-tracing [core] Add react-native support at top level (#30521) 2024-07-24 15:07:46 -07:00
core-util [core] Add react-native support at top level (#30521) 2024-07-24 15:07:46 -07:00
core-xml [core-xml] Adding RN support at top level (#30493) 2024-07-24 14:16:05 -07:00
logger [core] Add react-native support at top level (#30521) 2024-07-24 15:07:46 -07:00
perf-tests/core-rest-pipeline [EngSys] remove tsconfig.package.json 2024-07-16 13:27:25 +00:00
ts-http-runtime [core] Make `request` and `response` properties on RestError non-enumerable (#30518) 2024-07-24 17:34:59 -07:00
README.md [core] Fix link in README (#21894) 2022-05-16 22:09:10 +00:00
ci.yml [core] remove core-http from main branch (#27994) 2023-12-05 15:40:11 -08:00

README.md

Azure Core Client Libraries

The core set of packages provide common functionality for interacting with Azure services in a way that follows our design guidelines.

These packages are generally not used directly by consumers, but are used as dependencies by service-specific packages. However, as many of the concepts implemented in core are exposed in service packages, so understanding these concepts will help in advanced scenarios of service interaction.

Core "v1" and Core "v2"

The package @azure/core-http is heavily based on @azure/ms-rest-js and inherited legacy API surface and concepts that sometimes conflicted with our design principles. A full explanation is available here: https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/core/core-rest-pipeline/documentation/core2.md

For the purposes of this document, understand that "core v1" refers to the package @azure/core-http and is considered legacy. "Core v2" refers to the packages @azure/core-rest-pipeline, @azure/core-client, and @azure/core-xml.

Common Patterns for REST

Many of the service packages interact with REST-based service APIs. This means they use standard HTTP verbs to communicate with Azure servers to perform operations against a particular service.

HTTP Request Pipeline

Many service operations require client libraries to make one or more HTTP calls to the service. While each request is unique, there are common behaviors that need to be applied to each call, such as serialization and retry logic.

The Pipeline is what manages these common behaviors, which are grouped into items called PipelinePolicys. Each client library configures its own Pipeline using a set of standard PipelineOptions.

For more information, refer to https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/core/core-rest-pipeline#key-concepts

ServiceClient

Client libraries come in two flavors: authored and generated. Generated clients are produced by AutoRest whereas authored clients are written by hand. Typically, authored clients wrap generated clients and extend them with custom API surface.

ServiceClient is the base class of all generated clients. It builds on top of the HTTP Pipeline in order to make requests to services.

For more information, refer to https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/core/core-client#key-concepts

Accessing raw HTTP responses

Usually all the information necessary to interact with a service is returned from each operation method on a client. However, sometimes developers may wish to look at additional information on the raw request object, such as headers, for debugging purposes.

This is done by passing a onResponse callback in the operation call:

function logResponseHeaders(response: FullOperationResponse) {
  console.log(response.parsedHeaders);
  // You can also access the original request inside response.request
}
const item = await client.getItemById(id, { onResponse: logResponseHeaders });

Legacy _response property

For packages that are still using @azure/core-http you can access the raw response by looking at a special non-enumerable _response property on the returned items:

const item = await client.getItemById(id);
console.log(item._response.parsedHeaders);

Authentication

Authentication is handled by @azure/identity. In most cases this is as simple as passing DefaultAzureCredential to a client that takes a TokenCredential as a means of authentication.

const { KeyClient } = require("@azure/keyvault-keys");
const { DefaultAzureCredential } = require("@azure/identity");

// Azure SDK clients accept the credential as a parameter
const credential = new DefaultAzureCredential();
const client = new KeyClient(vaultUrl, credential);

Note that DefaultAzureCredential does not work for applications that are running locally in a web browser. For such applications, consider using InteractiveBrowserCredential instead.

Pagination

Client libraries follow our design guidelines for pagination. This is largely hand-authored today in convenience clients, but code generation is improving to support this pattern inside generated clients as well.

The standard interfaces for pagination are provided by @azure/core-paging.

Basic code for iterating through all entries of a paged API looks something like:

for await (let secret of client.listSecrets()) {
  console.log("secret: ", secret);
}

In cases where more control is needed (or there are too many pages to iterate over), pagination can done explicitly:

for await (let page of client.listSecrets().byPage({ maxPageSize: 2 })) {
  for (const secret of page) {
    console.log("secret: ", secret);
  }
}

Long Running Operations

Client libraries follow our design guidelines for Long Running Operations (LROs). This ensures all LROs follow a similar pattern to remain consistent across clients.

To assist with implementing pollers correctly, primitives are provided by @azure/core-lro. These primitives help implement Poller objects which are used to manage PollOperations that contain PollOperationState.

In essence, a Poller handles the work of continously checking the server for updates to the LRO on a developer's behalf. Pollers are highly customizable, and consumers are able to decide when to poll manually if needed.

The simplest contract for a poller is to simply wait until it is finished:

const poller = await client.beginDeleteKey(keyName);
await poller.pollUntilDone();

Pollers are also capable of being serialized via the standard toString() method:

const poller = await client.beginDeleteKey(keyName);
const serializedPoller = poller.toString();
// some time later
const rehydratedPoller = await client.beginDeleteKey(keyName, { resumeFrom: serializedPoller });

Tracing

Azure Application Insights

Azure Application Insights, a feature of Azure Monitor, is an extensible Application Performance Management (APM) service for developers and DevOps professionals. Use it to monitor your live applications. It will automatically detect performance anomalies, and includes powerful analytics tools to help you diagnose issues and to understand what users actually do with your application.

If your application already uses ApplicationInsights, automatic collection of Azure SDK traces is supported in versions 1.8.0 and later.

To setup Application Insights tracking for your application follow the Start Monitoring your Node.js Application quickstart guide.

Open Telemetry

Client libraries have preliminary support for OpenTelemetry. This functionality is mostly managed by @azure/core-tracing

Each client library internally does the work to create a new OpenTelemetry Span for each service operation, making sure to end the Span after the result is returned back to the consumer. Many clients use a helper method called withSpan to manage the new Span and automatically handle closing it.

When tracingOptions.tracingContext is set on an operation, a default request policy will automatically create a span for each HTTP request that is issued.

Consumers are expected to pass in the Context of the parent Span when calling an operation, such as:

const result = await blobClient.download(undefined, undefined, {
  tracingOptions: {
    tracingContext: activeContext
  },
});

Logging

Logging in client libraries is provided by @azure/logger.

AzureLogger provides the ability to easily set a global log level (either programmatically or through an environment variable) and log output can be redirected by simply overriding the default log method:

const { AzureLogger, setLogLevel } = require("@azure/logger");

setLogLevel("verbose");

// override logging to output to console.log (default location is stderr)
AzureLogger.log = (...args) => {
  console.log(...args);
};

AutoRest and Generated Clients

AutoRest is a generation tool for creating a client library using an OpenAPI Specification (formerly known as "Swagger".)

AutoRest is used in conjunction with the autorest.typescript extension to generate client libraries for JS/TS consumers. While the generated code tries as much as possible to fulfill the TS design guidelines, it is often necessary to wrap the generated client classes in what are known as "convenience clients."

A convenience client extends the shape of a generated client in ways that make it more approachable to the consumer, such as simplifying the return shape of methods or adding helper functions for common operations.

AMQP and Message-based Clients

More information can be found in @azure/amqp