Loading

OTLP/HTTP endpoint

Stack Preview 9.2.0 Self-Managed

In addition to the ingestion of metrics data through the bulk API, Elasticsearch offers an alternative way to ingest data through the OpenTelemetry Protocol (OTLP).

The endpoint is available under /_otlp/v1/metrics.

Important

The recommended approach for sending OpenTelemetry Protocol (OTLP) data depends on your deployment:

For details on the recommended way to set up OpenTelemetry-based data ingestion, refer to the EDOT reference architecture.

Ingesting metrics data using the OTLP endpoint has the following advantages:

  • Improved ingestion performance, especially if the data contains many resource attributes.
  • Simplified index mapping: there's no need to manually create data streams, index templates, or define dimensions and metrics. Metrics are dynamically mapped using the metadata included in the OTLP requests.
Note

Elasticsearch only supports OTLP/HTTP, not OTLP/gRPC.

Don't send metrics from applications directly to the Elasticsearch OTLP endpoint, especially if there are many individual applications that periodically send a small amount of metrics. Instead, send data to an OpenTelemetry Collector first. This helps with handling many connections, and with creating bigger batches to improve ingestion performance.

To send data from an OpenTelemetry Collector to the Elasticsearch OTLP endpoint, use the OTLP/HTTP exporter. This is an example configuration:

extensions:
  basicauth/elasticsearch:
    client_auth:
      username: <user>
      password: <password>
exporters:
  otlphttp/elasticsearch-metrics:
    endpoint: <es_endpoint>/_otlp
    sending_queue:
      enabled: true
      sizer: bytes
      queue_size: 50_000_000
      block_on_overflow: true
      batch:
        flush_timeout: 1s
        min_size: 1_000_000
        max_size: 4_000_000
    auth:
      authenticator: basicauth/elasticsearch
service:
  extensions: [basicauth/elasticsearch]
  pipelines:
    metrics:
      exporters: [otlphttp/elasticsearch-metrics]
      receivers: ...
  1. 50MB uncompressed
  2. 1MB uncompressed
  3. 4MB uncompressed

The supported options for compression are gzip (default value of the OTLP/HTTP exporter) and none.

To track metrics in your custom application, use the OpenTelemetry language SDK of your choice.

Note

Only encoding: proto is supported, which the OTLP/HTTP exporter uses by default.

By default, metrics are ingested into the metrics-generic.otel-default data stream. You can influence the target data stream by setting specific attributes on your data:

  • data_stream.dataset or data_stream.namespace in attributes, with the following order of precedence: data point attribute -> scope attribute -> resource attribute
  • Otherwise, if the scope name contains /receiver/<somereceiver>, data_stream.dataset is set to the receiver name.
  • Otherwise, data_stream.dataset falls back to generic and data_stream.namespace falls back to default.

The target data stream name is constructed as metrics-${data_stream.dataset}.otel-${data_stream.namespace}.