The last MR of the project (!1290): artifact search stops reading Elasticsearch and starts reading OpenSearch, and the nightly Logstash pipeline is deleted. Everything below it in the stack was built for this moment. The merge is the cutover: nothing deploys piecemeal, so the new pipeline and the flip reach the main branch together.
The cutover (!1290).
The first half added per-request engine selection, so the two engines could be compared
on the same code: ?engine= picks the engine, a response header names which one
served, and the default stayed Elasticsearch while the comparison ran. A few small
query fixes went in with it, each proven to return identical results on Elasticsearch
before and after, so the comparison would measure the engines and not my edits.
The second half is the evidence. 226 request bodies, generated from the query class itself and replayed against both engines: transliteration in every mode, filters, pagination, access control, all 18 sort orders, all 27 aggregations the search page requests. Zero errors on either engine, all 18 sort orders identical in ids and totals, and every difference found traced to a cause and written into the MR as a table. Most trace to one already-documented mapping bound; a few are the new index being more correct than the old one, like 574 artifacts whose publications the old pipeline dropped.
Two finds this week were worth more than the parity run itself. The search results page reads an indexing timestamp that only Logstash ever wrote, and PHP’s date parser turns the missing value into “now”, so after the flip every result would have claimed it was indexed seconds ago, forever. A plausible lie, on exactly the feature this project is about, and no id comparison could ever catch it, since it lives in the template. The badge now renders only when the document carries the field.
The second find blocks the merge, on purpose. Five facets lose their “no value” option after the flip. Digging into why showed the real problem is live today: the facets exclude empty strings, and the current index is full of them, so one facet is silently hiding 129,294 documents from its counts right now, on the engine users are on. The correct fix is an aggregation redesign that changes visible numbers on the search page, which is a product change, not an engine change, so it needs its own MR, and it has to happen while the old index still exists to check against. I wrote both problems and the sized fix into a comment on the MR and asked the mentors to decide the order. The merge waits on their answer.
The MR ends by deleting the pipeline: the Logstash service, its config, and the nightly schedule, with a deployment note for the environments that don’t run an OpenSearch service yet, since for them this merge means an outage rather than a stale index until the service exists.
| # | Day | Date | A short description of the work done |
|---|---|---|---|
| 1 | Monday | 2026/08/10 | Planned the first half: the engine selection seam and the query fixes it carries |
| 2 | Tuesday | 2026/08/11 | Built the query fixes, each proven identical on Elasticsearch before and after |
| 3 | Wednesday | 2026/08/12 | Built the engine selection: the ?engine parameter, the response header, the admin viewer’s error states |
| 4 | Thursday | 2026/08/13 | Planned the second half; found the timestamp badge problem while sweeping what the templates read |
| 5 | Friday | 2026/08/14 | Ran the parity evidence: 226 bodies against both engines, every difference traced and tabled |
| 6 | Saturday | 2026/08/15 | Dug into the facet difference, measured the live defect behind it, sized the fix, and wrote the blocking question |
| 7 | Sunday | 2026/08/16 | Guarded the badge, flipped the default, deleted the pipeline, and opened the cutover MR (!1290) |