OpenTelemetry

Nomad emits OpenTelemetry metrics and spans for model serving and the code-mode gateway when telemetry export is enabled. Install the optional dependency extra to include the OTel SDK and OTLP/gRPC exporter:

pip install "nomad[otel]"

Enabling export

Nomad configures OTLP/gRPC trace and metric exporters when any of these are set:

• telemetry.enabled: true in nomad.config.ServerConfig or nomad.gateway.config.GatewayConfig.

• NOMAD_OTEL_ENABLED=true.

• Standard OTel environment variables such as OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_TRACES_ENDPOINT, OTEL_EXPORTER_OTLP_METRICS_ENDPOINT, OTEL_TRACES_EXPORTER, or OTEL_METRICS_EXPORTER.

Set telemetry.service_name or OTEL_SERVICE_NAME to control the exported service name. Set telemetry.otlp_endpoint or OTEL_EXPORTER_OTLP_ENDPOINT to control the collector endpoint. Set OTEL_SDK_DISABLED=true to disable SDK setup.

For OTel environment variable behavior and exporter configuration, see the OpenTelemetry Python documentation and Python exporter documentation.

Spans

nomad serve creates spans around managed Torch tool requests and batch execution. The code-mode gateway creates spans for MCP tool entrypoints, sandbox runs, and upstream MCP tool calls.

Model-serving metrics

The table is generated from src/nomad/metrics.py.

Metric	Type	Unit	Description
nomad.tool.requests	counter	{request}	Tool requests accepted by the Torch tool manager.
nomad.tool.request_errors	counter	{request}	Tool requests completed with an execution error.
nomad.tool.request_cancellations	counter	{request}	Tool requests cancelled while pending or executing.
nomad.tool.request_rejections	counter	{request}	Tool requests rejected before enqueueing.
nomad.tool.request.duration	histogram	s	End-to-end tool request duration, including queue wait.
nomad.tool.request.queue_wait	histogram	s	Time a tool request waits before batch execution starts.
nomad.tool.batch.duration	histogram	s	Wall time spent executing one model batch.
nomad.tool.batch.size	histogram	{request}	Number of requests included in an executed model batch.
nomad.tool.batch.oom_reductions	counter	{event}	Batch size reductions caused by out-of-memory errors.
nomad.tool.device.load_duration	histogram	s	Time spent moving a tool onto a device.
nomad.tool.device.offloads	counter	{event}	Tool offload operations to CPU.
nomad.tool.device.idle_evictions	counter	{event}	Device allocations removed after a tool idle timeout.
nomad.tool.disk.loads	counter	{event}	Tool load attempts from a registered source.
nomad.tool.disk.load_duration	histogram	s	Wall time spent loading a tool from a registered source.
nomad.tool.disk.unloads	counter	{event}	Resident CPU tool instances unloaded after a disk idle timeout.
nomad.device.cache_clears	counter	{event}	Accelerator cache clear attempts after server idle periods.
nomad.device.cache_clear.duration	histogram	s	Wall time spent clearing unused accelerator cache.
nomad.tool.queue.length	observable up/down counter	{request}	Pending requests per managed tool.
nomad.tool.inflight.requests	observable up/down counter	{request}	Requests currently assigned to executing batches.
nomad.tool.batch.configured_size	observable up/down counter	{request}	Current effective batch size per managed tool.
nomad.tool.pending_tools	observable up/down counter	{tool}	Number of tools waiting for scheduler dispatch.
nomad.device.slot.busy	observable gauge	1	Whether a managed device slot is busy.
nomad.device.tool.loaded	observable up/down counter	1	Whether a tool is resident on a managed device slot.
nomad.tool.resident	observable up/down counter	1	Whether a managed tool has a resident CPU instance.
nomad.device.memory.allocated	observable gauge	By	Best-effort accelerator memory currently allocated.
nomad.device.memory.reserved	observable gauge	By	Best-effort accelerator memory currently reserved by the backend.
nomad.device.utilization	observable gauge	1	Backend-reported device utilization ratio when available.

Gateway metrics

The gateway metrics cover MCP entrypoints, sandbox execution, and upstream MCP tool calls.

Metric	Type	Unit	Description
nomad.gateway.requests	counter	{request}	Code-mode gateway requests received.
nomad.gateway.request.duration	histogram	s	Code-mode gateway request duration.
nomad.gateway.sandbox.duration	histogram	s	Sandbox subprocess execution duration.
nomad.gateway.upstream_tool.calls	counter	{call}	Upstream MCP tool calls made through the code-mode gateway.
nomad.gateway.upstream_tool.duration	histogram	s	Duration of upstream MCP tool calls made through the gateway.