Elasticsearch Search skill for Claude Code

An index is done when its mapping is explicit (no surprise dynamic fields), every field is mapped for how it is queried, the application talks to an alias rather than a concrete index, relevance is measured against a judged query set, and shards land in the 10-50 GB range. The same guidance applies to OpenSearch unless a note says otherwise.

Version notes

• Elasticsearch 8.x/9.x and OpenSearch 2.x/3.x share the core query DSL, mappings, analyzers, aliases, _reindex, and aggregations covered here.
• Lifecycle: Elasticsearch uses ILM (_ilm/policy); OpenSearch uses ISM (_plugins/_ism/policies). Security, alerting, and vector features diverge - check the engine before writing those APIs.
• Use the official client for the engine and major version in use; mismatched clients (an ES 9 client against OpenSearch) fail on product checks.

Workflow

- [ ] 1. Inventory queries: search box, filters, sorts, aggregations, autocomplete
- [ ] 2. Write an explicit mapping and index template
- [ ] 3. Choose analyzers; test them with _analyze
- [ ] 4. Create a versioned index behind an alias; bulk load
- [ ] 5. Write queries: filter context for exact constraints, scoring only for text
- [ ] 6. Build a judged query set and measure relevance
- [ ] 7. Size shards, set lifecycle, check cluster health

2. Mapping

PUT _index_template/products
{
  "index_patterns": ["products-v*"],
  "template": {
    "settings": { "number_of_shards": 1, "number_of_replicas": 1, "refresh_interval": "1s" },
    "mappings": {
      "dynamic": "strict",
      "properties": {
        "sku":        { "type": "keyword" },
        "title":      { "type": "text", "analyzer": "english",
                        "fields": { "raw": { "type": "keyword", "ignore_above": 256 } } },
        "brand":      { "type": "keyword" },
        "price":      { "type": "scaled_float", "scaling_factor": 100 },
        "in_stock":   { "type": "boolean" },
        "created_at": { "type": "date" },
        "tags":       { "type": "keyword" },
        "attributes": { "type": "flattened" }
      }
    }
  }
}

Field is used for	Map as
Full-text search	text with a language or custom analyzer
Exact filter, terms aggregation, sort	keyword (add as .raw multi-field on text)
IDs, status codes, enums	keyword even when numeric - range queries on IDs are rare
Range filters and numeric aggregations	integer/long/scaled_float/date
Arbitrary user key/value bags	flattened (one field, keyword semantics) to avoid mapping explosion
Objects whose fields must match together in one array element	nested (costly; each element is a hidden doc)
Not queried at all	"enabled": false on the object, or "index": false on the field

Rules:

• Set "dynamic": "strict" (fail on unknown fields) or "dynamic": "false" (store but do not index) for application indices. Default dynamic mapping turns every string into text + keyword and grows the mapping without review.
• Field mappings cannot change type in place. Adding a new field is fine; changing an existing one means a new index and _reindex.
• Keep total fields well under the 1,000 default index.mapping.total_fields.limit; raising the limit treats a symptom.

3. Analyzers

"settings": {
  "analysis": {
    "filter": {
      "product_synonyms": { "type": "synonym_graph", "synonyms_set": "products-synonyms", "updateable": true }
    },
    "analyzer": {
      "title_index":  { "tokenizer": "standard", "filter": ["lowercase", "asciifolding", "english_stemmer"] },
      "title_search": { "tokenizer": "standard", "filter": ["lowercase", "asciifolding", "product_synonyms", "english_stemmer"] }
    }
  }
}

• Apply synonyms at search time (search_analyzer) with synonym_graph so updates do not require a reindex. Elasticsearch 8.10+ has a synonyms API (synonyms_set); OpenSearch and older ES use inline synonyms or synonyms_path files that must exist on every node. Define english_stemmer ("type": "stemmer", "language": "english") in the same filter block.
• Autocomplete: search_as_you_type field or an edge_ngram index analyzer with a plain search analyzer. Never apply edge n-grams at search time.
• Test before indexing: POST products-v3/_analyze {"analyzer": "title_index", "text": "Wi-Fi Routers"}.

4. Aliases and reindexing

The application reads and writes products (an alias), never products-v3.

PUT products-v4                                  // new mapping via template
POST _reindex?wait_for_completion=false
{ "source": { "index": "products-v3" }, "dest": { "index": "products-v4" } }
GET _tasks/<task_id>                             // poll until completed, check failures
POST _aliases
{ "actions": [
  { "remove": { "index": "products-v3", "alias": "products" } },
  { "add":    { "index": "products-v4", "alias": "products", "is_write_index": true } } ] }

• For big reindexes: set "refresh_interval": "-1" and "number_of_replicas": 0 on the destination, restore both afterwards, and use slices: auto.
• Writes that arrive during the reindex are lost unless the app dual-writes or you replay changes since the reindex start (from the source of truth by updated_at).
• Compare _count on old and new before switching; ask the user before deleting the old index.

5. Queries

GET products/_search
{
  "query": {
    "bool": {
      "must":   [ { "multi_match": { "query": "wireless router", "fields": ["title^3", "description"], "type": "best_fields" } } ],
      "filter": [ { "term": { "in_stock": true } }, { "range": { "price": { "lte": 200 } } }, { "terms": { "brand": ["netgear", "tp-link"] } } ],
      "should": [ { "term": { "tags": "bestseller" } } ]
    }
  },
  "aggs": { "brands": { "terms": { "field": "brand", "size": 20 } } },
  "size": 20,
  "_source": ["sku", "title", "price"]
}

• Put every yes/no constraint in filter: it skips scoring and is cached.
• term on a text field almost never matches (the indexed tokens are analyzed). Use match on text, term/terms on keyword.
• Deep pagination: from + size is capped at 10,000 by index.max_result_window. Use search_after with a point in time (PIT) and a tiebreaker sort, not a larger window.
• Boost freshness or popularity with function_score (field_value_factor with log1p, or a gauss decay on date) rather than sorting by it, which discards relevance.
• terms aggregations on high-cardinality fields are approximate across shards; raise shard_size or use composite aggregation to page through all buckets exactly.

6. Relevance measurement

Collect 30-100 real queries from logs with graded judgments (0-3) of the top results. Run _rank_eval with ndcg or mean_reciprocal_rank, change one thing (boosts, analyzer, synonyms), rerun, keep the change only if the metric improves without dropping important queries. Record the before/after table in the PR.

7. Shards, lifecycle, health

• Target 10-50 GB per primary shard (under ~200M docs). A 2 GB catalog needs one shard. Oversharding (many tiny shards) costs heap and slows cluster state more than undersized clusters do.
• Time-based data (logs, events): use data streams with a lifecycle policy that rolls over by max_primary_shard_size: 50gb or age, then deletes or moves to warm/cold.
• Bulk indexing: batches of 5-15 MB via _bulk, check errors: true in the response and retry only failed items, multiple concurrent workers, refresh off during initial loads.

For red/yellow clusters, unassigned shards, heap pressure, or slow queries, follow references/cluster-triage.md.

Gotchas

• refresh_interval controls visibility, not durability. Tests that index then search immediately need ?refresh=wait_for, not sleeps.
• Updates rewrite the whole document; frequently updated counters (views) belong elsewhere or in a separate, small index.
• wildcard/regexp queries with a leading * scan the term dictionary; use n-grams or the wildcard field type.
• Scripts in queries (script_score, painless) are slow per document; precompute fields at index time.
• Elasticsearch is not the source of truth. Keep the database authoritative and make the index rebuildable from it.
• Mapping the same field name with different types in different indices behind one alias breaks sorting and aggregations across them.

Verification

• GET products/_mapping matches the template; no unexpected fields.
• _analyze output for five tricky inputs (plurals, hyphens, accents, SKUs) is as intended.
• _rank_eval score recorded before and after relevance changes.
• GET _cat/shards/products*?v&h=index,shard,prirep,store,docs shows shard sizes in range.
• GET _cluster/health is green; a search with "profile": true shows no unexpected expensive clauses for the top three queries.