[Bug]: Archive trace returns 200 but does not write to Elasticsearch

[domolitom]

What happened?

Clicking "Archive" on a trace in the Jaeger UI shows "This trace has been archived", but nothing is written to Elasticsearch. No archive indices are created and no errors are logged.

Calling the API directly confirms the issue — it returns HTTP 200 with an empty response that looks like a search result, not a write acknowledgment:

POST /api/archive/{traceID}
→ 200 {"data":[],"total":0,"limit":0,"offset":0,"errors":[]}

Steps to reproduce

1. Deploy Jaeger v2.14.1 with Elasticsearch and traces_archive configured (see config below)
2. Open a trace in the Jaeger UI
3. Click the "Archive" button
4. UI shows "This trace has been archived"
5. Check Elasticsearch — no archive* indices exist

Expected behavior

The trace should be written to Elasticsearch under the archive index prefix, and archive-jaeger-span-* indices should appear.

Relevant log output

# No archive-related log output at all, even after clicking Archive.
# Startup logs show both stores initializing successfully:
Initializing storage 'archive_store_elasticsearch'
Elasticsearch detected  version=7
Initializing storage 'primary_store_elasticsearch'
Elasticsearch detected  version=7

Additional context

Primary storage works fine — jaeger-span-* and jaeger-service-* indices are created and populated. Only the archive write is silently failing.

Possibly related to #6225 (storage role assignment).

Jaeger setup

Jaeger backend version: v2.14.1

Storage backend: AWS OpenSearch 7.x (VPC endpoint)

Deployment configs:

extensions:
  jaeger_query:
    storage:
      traces: primary_store_elasticsearch
      traces_archive: archive_store_elasticsearch
  jaeger_storage:
    backends:
      primary_store_elasticsearch:
        elasticsearch:
          server_urls: ["https://my-opensearch.eu-west-1.es.amazonaws.com:443"]
      archive_store_elasticsearch:
        elasticsearch:
          server_urls: ["https://my-opensearch.eu-west-1.es.amazonaws.com:443"]
          indices:
            index_prefix: "archive"

[shuraih775]

I tried to reproduce this on Jaeger v2.14.1 with OpenSearch 2.11.1 (including security enabled). In my setup, clicking Archive successfully triggered bulk writes and resulted in creation of:

• archive-jaeger-span-*
• archive-jaeger-service-*

Primary indices were also working as expected.

One thing I noticed is that Jaeger itself does not log much during archive operations beyond the storage initialization message. The useful signal was actually in the OpenSearch logs. In my case I see entries like:

[archive-jaeger-span-YYYY-MM-DD] creating index, cause [auto(bulk api)]

and

adding template [archive-jaeger-span]

These confirm that the archive write path is being exercised.

Since in your environment primary indices are created but archive ones are not, it might help to check whether OpenSearch is rejecting or never receiving the archive bulk request.

Could you please check your OpenSearch logs around the time you click Archive and see if you observe any lines similar to the above (or any errors)?

Additionally, one configuration worth confirming:

• Is action.auto_create_index enabled in your OpenSearch domain?

If index auto-creation were disabled, archive index creation could fail (though typically primary ingestion would also be affected unless indices were pre-created).

Also, regarding the 200 response from:

POST /api/archive/{traceID}
→ {"data":[],"total":0,"limit":0,"offset":0,"errors":[]}

this appears when the trace is archived.
In case of errors it will be usually filled in the errors field.
But OpenSearch logs provides a more definitive signal.