Skip to content

Conversation

KANIOYH
Copy link

@KANIOYH KANIOYH commented Jul 21, 2025

This commit refactors the Elasticsearch batch_query
to utilize the msearch (multi-search) API instead of
sending individual queries in a loop.

The msearch API allows sending multiple search requests in a single
HTTP call, which significantly reduces network overhead and improves
query throughput, especially when performing batch operations.

Changes include:

  • Increased Elasticsearch thread_pool.search.queue_size in Dockerfile
  • Replaced batch_query calls with msearch API in module.py

Attention:

  • Set thread_pool.search.queue_size=1000 (with thread_pool.search.size=1),
    allowing more requests to queue but still executing them in a single thread

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant