Rejected requests

IMPORTANT: This documentation is no longer updated. Refer to Elastic's version policy and the latest documentation.

Rejected requests

When Elasticsearch rejects a request, it stops the operation and returns an error with a 429 response code. Rejected requests are commonly caused by:

A depleted thread pool. A depleted search or write thread pool returns a TOO_MANY_REQUESTS error message.
A circuit breaker error.
High indexing pressure that exceeds the indexing_pressure.memory.limit.

Check rejected tasks

edit

To check the number of rejected tasks for each thread pool, use the cat thread pool API. A high ratio of rejected to completed tasks, particularly in the search and write thread pools, means Elasticsearch regularly rejects requests.

resp = client.cat.thread_pool(
    v=True,
    h="id,name,queue,active,rejected,completed",
)
print(resp)

const response = await client.cat.threadPool({
  v: "true",
  h: "id,name,queue,active,rejected,completed",
});
console.log(response);

GET /_cat/thread_pool?v=true&h=id,name,queue,active,rejected,completed

write thread pool rejections frequently appear in the erring API and correlating log as EsRejectedExecutionException with either QueueResizingEsThreadPoolExecutor or queue capacity.

These errors are often related to backlogged tasks.

Check circuit breakers

edit

To check the number of tripped circuit breakers, use the node stats API.

resp = client.nodes.stats(
    metric="breaker",
)
print(resp)

const response = await client.nodes.stats({
  metric: "breaker",
});
console.log(response);

GET /_nodes/stats/breaker

These statistics are cumulative from node startup. For more information, see circuit breaker errors.

Check indexing pressure

edit

To check the number of indexing pressure rejections, use the node stats API.

resp = client.nodes.stats(
    human=True,
    filter_path="nodes.*.indexing_pressure",
)
print(resp)

const response = await client.nodes.stats({
  human: "true",
  filter_path: "nodes.*.indexing_pressure",
});
console.log(response);

GET _nodes/stats?human&filter_path=nodes.*.indexing_pressure

These stats are cumulative from node startup.

Indexing pressure rejections appear as an EsRejectedExecutionException, and indicate that they were rejected due to combined_coordinating_and_primary, coordinating, primary, or replica.

These errors are often related to backlogged tasks, bulk index sizing, or the ingest target’s refresh_interval setting.

Another cause of indexing pressure rejections might be the use of the semantic_text field type, which can cause rejections when indexing large batches of documents if the batch may otherwise incur an Out of Memory (OOM) error.

Prevent rejected requests

edit

Fix high CPU and memory usage

edit

If Elasticsearch regularly rejects requests and other tasks, your cluster likely has high CPU usage or high JVM memory pressure. For tips, see High CPU usage and High JVM memory pressure.

Fix for `semantic_text` ingestion issues

edit

When bulk indexing documents with the semantic_text field type, you may encounter rejections due to high memory usage during inference processing. These rejections will appear as an InferenceException in your cluster logs.

To resolve this issue:

Reduce the batch size of documents in your indexing requests.
If reducing batch size doesn’t resolve the issue, then consider scaling up your machine resources.
A last resort option is to adjust the indexing_pressure.memory.coordinating.limit cluster setting. The default value is 10% of the heap. Increasing this limit allows more memory to be used for coordinating operations before rejections occur.

This adjustment should only be considered after exhausting other options, as setting this value too high may risk Out of Memory (OOM) errors in your cluster. A cluster restart is required for this change to take effect.

« Red or yellow cluster health status Task queue backlog »

Rejected requests