Instrumentation

All Node services and the Next.js web app instrument themselves through @repo/observability. The package owns the OTel SDK, resource detection, sampler, exception helper, global error handlers, and the graceful shutdown path. Apps only choose which preset of instrumentations to enable.

Package layout

packages/infrastructure/observability/src/
├── env.ts                   # Validates OTEL_* env vars via @repo/core/env createEnv
├── exception.ts             # recordException() — span event + OTel log record
├── node/
│   ├── instrumentations.ts  # Presets: base, web, graphqlServer, redis, postgres, worker
│   ├── sdk.ts               # NodeSDK + OTLP trace/log exporters + ParentBased sampler + shared Resource
│   ├── errors.ts            # process.on('unhandledRejection' | 'uncaughtException')
│   ├── shutdown.ts          # SIGTERM/SIGINT → flush traces + logs
│   └── register.ts          # Single bootstrap (idempotent)
└── nextjs/
    ├── register.ts          # Wraps node/register, gated to the nodejs runtime
    └── on-request-error.ts  # Next.js onRequestError → recordException

Subpath import	Use from
@repo/observability/node	Node services (presets, register)
@repo/observability/node/register	Side-effect import for tools that just bootstrap
@repo/observability/nextjs	Next.js instrumentation.ts
@repo/observability/exception	recordException() — usable from Node and Edge runtimes

Bootstrap

register() is the single entrypoint. It builds one shared Resource (so traces and logs both report the correct service.name), starts a NodeSDK with a ParentBased(TraceIdRatio(OTEL_TRACES_SAMPLER_ARG)) sampler, configures a single OTLP/gRPC log processor owned by that NodeSDK, disables metrics export, and installs handlers for unhandledRejection, uncaughtException, SIGTERM, and SIGINT. NodeSDK.shutdown() flushes traces and logs before exit; fatal process events record the exception, flush with a bounded timeout, and then terminate with exit code 1.

For the OTel instrumentations to actually patch pino at runtime, pino must be external to the application bundle and present as a real dependency in node_modules. See Bundling and runtime resolution below.

Presets

Apps pick a preset and optionally spread extra instrumentations on top.

Preset	Contains
presets.base()	HttpInstrumentation, UndiciInstrumentation, PinoInstrumentation (with log forwarding enabled)
presets.web()	base() — Next.js uses Undici under the hood
presets.graphqlServer()	base() + GraphQLInstrumentation
presets.redis()	RedisInstrumentation (node-redis v4) + IORedisInstrumentation
presets.postgres()	PgInstrumentation (pg driver — also covers Drizzle, postgres.js, Knex)
presets.worker()	base() + redis() — used by BullMQ workers

HttpInstrumentation ignores health probes, Next.js internals (/_next/), favicons, and OPTIONS requests by default. Pass ignorePaths to override.

Per-app wiring

Every Node service has a tiny src/instrumentation.ts that calls register() with the presets it needs:

// apps/api/src/instrumentation.ts
import { presets, register } from "@repo/observability/node";

register({
	instrumentations: [...presets.graphqlServer(), ...presets.redis(), ...presets.postgres()],
});

apps/identity and apps/notifications use presets.graphqlServer(); apps/match-gateway uses presets.worker(). Next.js uses a one-line re-export:

// apps/web/instrumentation.ts
export { onRequestError, register } from "@repo/observability/nextjs";

Bootstrap ordering

The instrumentation file MUST be the very first import of the entrypoint. OTel patches modules at require/import time; anything imported before register() runs is not instrumented.

Bundling and runtime resolution

OTel instrumentations work by patching require() / import calls at runtime via require-in-the-middle / import-in-the-middle. If a target module (notably pino) is inlined into the application bundle, the hook never fires and the instrumentation silently does nothing — traces and exceptions still work, but pino logs do not flow through the OTel pipeline and never reach App Insights traces.

To keep PinoInstrumentation working through tsup and Next.js bundling, the repo enforces two rules:

1. pino (and pino-pretty) must be external to the bundle, so the runtime require("pino") resolves to a real node_modules entry that import-in-the-middle can hook.
2. pino must be a direct runtime dependency of every Node app, declared via the workspace catalog so the version stays in one place.

Tsup apps (api, identity, loyalty, notifications, match-gateway, …)

// apps/<app>/tsup.config.ts
import { discoverPackages } from "@repo/build";
import { defineConfig } from "tsup";

export default defineConfig({
	entry: ["src/index.ts"],
	noExternal: discoverPackages(__dirname),
	external: ["pino", "pino-pretty"],
});

Next.js apps (web)

// apps/web/next.config.ts
const nextConfig: NextConfig = {
	serverExternalPackages: [
		"pino",
		"pino-pretty",
		"@opentelemetry/sdk-node",
		"@opentelemetry/exporter-trace-otlp-grpc",
		"@opentelemetry/instrumentation-http",
		"@opentelemetry/instrumentation-pino",
		"@opentelemetry/instrumentation-redis",
		"@opentelemetry/instrumentation-undici",
	],
	// …
};

Per-app package.json

{
	"dependencies": {
		"pino": "catalog:"
	}
}

The shared version is pinned in pnpm-workspace.yaml:

catalog:
  pino: 10.3.1
  pino-pretty: 11.3.0

New Node app checklist

When adding a new Node service:

1. Add "pino": "catalog:" to its dependencies (even if it never imports pino directly — @repo/logging does).
2. Add external: ["pino", "pino-pretty"] to its tsup.config.ts.
3. Verify with node -e "console.log(require.resolve('pino', { paths: ['./apps/<app>'] }))" — must resolve to node_modules/.pnpm/pino@*/....

Without these, PinoInstrumentation patches a phantom pino while the app keeps using the inlined copy, and pino logs disappear from App Insights.

Exception handling

import { recordException } from "@repo/observability/exception";

recordException(err, { "user.id": userId });

recordException does two things:

1. If there's an active recording span, it adds the exception as a span event with any extra attributes and marks the span status ERROR.
2. It always emits an OTel log record with the exception semantic conventions: exception.type, exception.message, exception.stacktrace, plus the extra attributes.

The log path is the important one — log records are not subject to trace sampling, so the exception lands in App Insights even when the trace it belongs to is dropped.

Where exceptions get captured

• BaseError constructor (@repo/error) calls recordException(this). Any class extending BaseError is automatically reported on instantiation.
• Global Node handlers (@repo/observability/node/errors.ts) capture unhandledRejection and uncaughtException, flush telemetry, and terminate the process.
• Next.js routes server-side render and route-handler errors through onRequestError (re-exported from @repo/observability/nextjs).
• Domain code can call recordException(err, attrs) directly when it catches and re-shapes an error.

Domain errors vs exceptions

This is the OTel exception pipeline — the mechanism that ships any thrown error into App Insights. DomainError subclasses call recordException automatically from BaseError, so no manual instrumentation is needed. How errors are modeled and mapped to transport responses is in API Patterns — Error Handling.

Sampling

Every Node service uses ParentBased(TraceIdRatio(OTEL_TRACES_SAMPLER_ARG)), hard-wired in node/sdk.ts. The Apollo Router (apps/graphql-router/router.yaml) and Traefik (apps/api-gateway/traefik/entrypoint.sh) are configured the same way.

If the incoming request carries a traceparent header, the upstream decision is honored. The native app samples at 5% and everything downstream inherits that decision, so a sampled-in trace always spans the whole chain. If there's no inbound parent (cron jobs, queue consumers, internal HTTP without context), the local OTEL_TRACES_SAMPLER_ARG decides. Per-environment ratios are documented in Azure Container Apps — Sampling per environment.