The querier service handles queries using the PromQL query language. This document dives into the storage-specific details of the querier service. The general architecture documentation applies too. The querier is stateless. How it works The querier needs to have an almost up-to-date view over the entire storage bucket, in order to find the right blocks to lookup at query time. The querier can keep the bucket view updated in to two different ways:| Cortex
The store-gateway is the Cortex service responsible to query series from blocks. The store-gateway is required when running the Cortex blocks storage. The store-gateway is semi-stateful. How it works The store-gateway needs to have an almost up-to-date view over the storage bucket, in order to discover blocks belonging to their shard. The store-gateway can keep the bucket view updated in to two different ways: Periodically scanning the bucket (default) Periodically downloading the bucket inde...| Cortex
The compactor is an service which is responsible to: Compact multiple blocks of a given tenant into a single optimized larger block. This helps to reduce storage costs (deduplication, index size reduction), and increase query speed (querying fewer blocks is faster). Keep the per-tenant bucket index updated. The bucket index is used by queriers, store-gateways and rulers to discover new blocks in the storage. The compactor is stateless. How compaction works The blocks compaction has two main b...| Cortex
This page shares some tips and things to take in consideration when setting up a production Cortex cluster based on the blocks storage. Ingester Ensure a high number of max open file descriptors The ingester stores received series into per-tenant TSDB blocks. Both TSDB WAL, head and compacted blocks are composed by a relatively large number of files which gets loaded via mmap. This means that the ingester keeps file descriptors open for TSDB WAL segments, chunk files and compacted blocks whic...| Cortex
In order to query series inside blocks from object storage, the store-gateway has to know certain initial info from each block index. In order to achieve so, the store-gateway builds an index-header for each block and stores it on local disk; such index-header is built by downloading specific pieces of original block’s index and storing them on local disk. Index header is then used by store-gateway at query time. Store-gateways build the index-header with specific sections of the block’s ...| Cortex
The bucket index is a per-tenant file containing the list of blocks and block deletion marks in the storage. The bucket index itself is stored in the backend object storage, is periodically updated by the compactor, and used by queriers, store-gateways and rulers to discover blocks in the storage. The bucket index usage is optional and can be enabled via -blocks-storage.bucket-store.bucket-index.enabled=true (or its respective YAML config option). Benefits The querier, store-gateway and ruler...| Cortex