OpenSearch 3.9.0-CE-ubuntu24.04-r7 build trail
How ReadyStack built readystack/opensearch:3.9.0-CE-ubuntu24.04-r7 and its build kit, copied record by record: why the release exists, every chain, drive and stage it went through, the model work, the review rounds, the defects, the checks that verified it and when it was published. Each line names the record it comes from in brackets; a record that does not exist for this release says so.
This trail
As JSON- Image
readystack/opensearch:3.9.0-CE-ubuntu24.04-r7 - Index digestsha256:dcf889baf9fef8d89f7656aad6fb6df9a234299a0067e20b302395249ae82401
- linux/amd64 digestsha256:a094681b3c747c44cdfb0f02394892832573833d8a28a8bdb0166b181b701d20
- Archive
ReadyStack-opensearch-3.9.0-CE-ubuntu24.04-r7.tar.gz - Archive SHA-256 now421b684a6450c9bdddc51dd248e1db545725010fff756ca2f2182ffab98fcd20
- Pins record
docs/PINS.md,docs/provenance/PINS.jsonin the archive (since generated) - Generated2026-11-19T11:59:24Z by KitForge release/build_trail.py
- Sections9 sections, 8 fully recorded; 1 item not recorded
Signed with the release key The trail's signature is a Sigstore bundle beside it. Check it is ours before you rely on it:
cosign verify-blob --key https://readystack.dev/keys/cosign.pub \ --bundle ReadyStack-opensearch-3.9.0-CE-ubuntu24.04-r7.trail.json.sigstore ReadyStack-opensearch-3.9.0-CE-ubuntu24.04-r7.trail.json
IdentityWhy this release existsThe path through the harnessModel work per stageReviewDefectsVerificationPublicationPackages and CPU level
Identity
Fully recorded- ProductOpenSearch (opensearch) [MANIFEST.json]
- Release tag3.9.0-CE-ubuntu24.04-r7 [MANIFEST.json]
- Imagereadystack/opensearch:3.9.0-CE-ubuntu24.04-r7 [MANIFEST.json]
- Image index digestsha256:dcf889baf9fef8d89f7656aad6fb6df9a234299a0067e20b302395249ae82401 [docs/provenance/LOCKED_DIGEST.txt]
- linux/amd64 image digestsha256:a094681b3c747c44cdfb0f02394892832573833d8a28a8bdb0166b181b701d20 [ReadyStack-opensearch-3.9.0-CE-ubuntu24.04-r7.rebuild.json, subject.image_manifest_digest]
- ArchiveReadyStack-opensearch-3.9.0-CE-ubuntu24.04-r7.tar.gz (bytes 1155807606) [MANIFEST.json]
- Archive sha256 at publish421b684a6450c9bdddc51dd248e1db545725010fff756ca2f2182ffab98fcd20 [MANIFEST.json, tarball_sha256]
- Archive records agreeyes [MANIFEST.json, PUBLISH_MANIFEST.json and .cutover.json name the same sha256]
- Repack historynone: the archive is as first published [MANIFEST.json, repack_history]
- Upstream repositoryhttps://github.com/opensearch-project/OpenSearch [docs/provenance/SOURCE_TREE_HASH.json]
- Upstream ref3.9.0 [docs/provenance/SOURCE_TREE_HASH.json]
- Upstream commit4ee42a94e87f66fbf1e62a9871b1b87f91e02472 [docs/provenance/SOURCE_TREE_HASH.json]
- Source tree hash2ae217a7cfe15e67871dcb85116f6032694851802f619e60e9921326333b0902 (files 19243) [docs/provenance/SOURCE_TREE_HASH.json, tree_listing_sha256 (sha256 of the sorted file listing)]
- Kit tree hashsha256:9109f4598c85cba413bdb72ddb8cbb07531c36ef34e591ff0ca352b12df5af9c [ReadyStack-opensearch-3.9.0-CE-ubuntu24.04-r7.rebuild.json, inputs.tree_hash]
- SOURCE_DATE_EPOCH1795082360 (utc 2026-11-19T09:59:20Z) [ReadyStack-opensearch-3.9.0-CE-ubuntu24.04-r7.rebuild.json, inputs.source_date_epoch]
- Base images
- ref ubuntu:24.04@sha256:2198e4c65076e9cb0b3d695219df667c0e3b0072da64ca3c2648aca17dcecaf5 · platform digest sha256:77eb101c607d6e3d4f5ad0c8a02c2136c689b98aba1f359e6502ba517e5c57ec
- BuildKitmoby/buildkit:v0.31.2 (image digest sha256:e0e67ea9c4bb07bdef683e29f70f3b623d1eb9c82274a9ec04e6bb0511449fb0) [ReadyStack-opensearch-3.9.0-CE-ubuntu24.04-r7.rebuild.json, engine]
- Engine commit(s)
- commit 49732e5cd958e96fbcf301bfba0ea5c0b54e68ff · dirty yes · first drive 4916
- commit d4137780886c52e83fb42d4d1927198f26bf44c7 · dirty yes · first drive 4917
- commit 657323945e75094f81887e9b83e0cea60048a17a · dirty yes · first drive 4919
Why this release exists
Fully recorded- Chain1118 (created 2026-11-19T10:00:44Z · trigger customer_request · upstream version 3.9.0) [chains, chains #1118]
- Requestbuild request #800 (rebuild) (requested at 2026-11-19T09:59:21Z · requested by kitforge-agent · version 3.9.0) [build_requests, build_requests #800]
- Reasonrerelease agenda REVISION -> 3.9.0-CE-ubuntu24.04-r7 (decay loop, IMP-08) [build_requests, build_requests #800 request_reason]
- Supersedesreadystack/opensearch:3.9.0-CE-ubuntu24.04-r6 (chain 1062) [chains, chains #1062]
The path through the harness
Fully recorded1 chain · 3 drives · 18 stages · 2 rewinds
Chain 1118 release · ready_to_ship
Created 2026-11-19T10:00:44Z, upstream 3.9.0, build request 800 [chains, chains #1118]
| Drive | Started | Minutes | Build host | How it stopped |
|---|---|---|---|---|
| 4916 | 2026-11-19T10:01:23Z | 45.5 | build host 1 withhold-publish | ready_to_ship at package_archive; 19 ticks [drive_runs, drive_runs #4916] |
| 4917 | 2026-11-19T11:17:47Z | 9.7 | build host 1 withhold-publish | ready_to_ship at package_archive; 2 ticks [drive_runs, drive_runs #4917] |
| 4919 | 2026-11-19T11:45:41Z | — | build host 1 publish-held | not recorded [drive_runs, drive_runs #4919] |
Stages, in order
| Stage | First start | Passed | Attempts | Gate checks passed | Artifacts (SHA-256) |
|---|---|---|---|---|---|
intake |
2026-11-19T10:01:33Z | 2026-11-19T10:01:34Z | 1 | 4
|
order_json 9cecf30bc602… |
acquire |
2026-11-19T10:01:47Z | 2026-11-19T10:01:49Z | 1 | 12
|
source_tree 3c1cb97d3c4b…license_file c71d239df917…custody_inputs 03a7007d1349…source_tree_hash 979bf36d89aa…source_file_manifest 9fa22841de5a… |
plan |
2026-11-19T10:02:18Z | 2026-11-19T10:02:18Z | 1 | 10
|
recon_report 880a570cfc8f…input_pin_plan 5beeb39a3964…toolchain_lock 1ad18ffff1d7… |
assemble_kit |
2026-11-19T10:09:15Z | 2026-11-19T10:09:16Z | 1 | 27
|
dockerfile 4b620634609a…build_recipe 7b91a9f9c2ac…system_pinning f516a236c6e3…build_offline_sh 6ab4f08a6ec7…generate_sbom_sh ce0f1e6f1e70…first_boot_health 5cba9177ea4f…verify_offline_sh 3e8184ea230c…base_image_digests 79ac4273d7fe…dockerfile_offline 6c70cf6db5ea…first_build_digest cdef9dbb7e47… |
remediate |
2026-11-19T10:09:42Z | 2026-11-19T10:09:43Z | 1 | 8
|
vendored_apt e3b0c44298fc…remediation_log 2473e1b9b87a…security_report 552b7fbadf9d… |
build_kit |
2026-11-19T10:19:54Z | 2026-11-19T10:19:55Z | 1 | 9
|
cosign_stamp 8c66d5bf3501…locked_digest 2c4c0d256fee…freeze_manifest 2755c8c4797f…offline_rebuild_transcript 573e21d30499… |
verdict |
2026-11-19T10:24:29Z | 2026-11-19T10:24:30Z | 1 | 8
|
scout_verdict 19bfcd0ffce6… |
publish_image |
2026-11-19T10:26:26Z | 2026-11-19T10:26:27Z | 1 | 7
|
hub_readme f22cfdb27a76…promotion_record c11476ce89e2… |
boot_instance |
2026-11-19T10:26:56Z | 2026-11-19T10:26:59Z | 2 | 3
|
instance_handle 082d7aabbb0d… |
sbom |
2026-11-19T10:28:11Z | 2026-11-19T10:28:12Z | 1 | 5
|
sbom_md 013accd65e45…sbom_json f26424b6326f… |
schema_discovery |
2026-11-19T10:28:14Z | 2026-11-19T10:28:15Z | 1 | 7
|
schema_hash 31cadb103c72…dependency_tree b7845dcbda08…table_classification 7c88836984f9… |
admin_handbook |
2026-11-19T10:28:25Z | 2026-11-19T10:28:25Z | 1 | 7
|
api_map c0c143c6c12d…admin_handbook 9ee9ee06c170… |
db_handbook |
2026-11-19T10:28:36Z | 2026-11-19T10:28:37Z | 1 | 8
|
db_handbook 0997dabb9972…seed_isolation 65084d083961… |
seed_data |
2026-11-19T10:28:43Z | 2026-11-19T10:28:44Z | 1 | 6
|
seed_data_sql 9f648e8e5404…seed_cases_json 03f6119dd2f6… |
seed_validate |
2026-11-19T10:28:47Z | 2026-11-19T10:28:48Z | 1 | 3
|
seed_data_contract e103a6ab308c… |
usecase_validate |
2026-11-19T10:29:08Z | 2026-11-19T10:29:09Z | 1 | 10
|
failed_tests b2247b16f5f7…passed_tests 68a13a83e4a1…test_results 814031647d38…usecase_contract 92f853f48baa… |
finalize_docs |
2026-11-19T10:29:14Z | 2026-11-19T10:29:15Z | 1 | 9
|
defects_json 230d5736886b…release_notes 8523773da1c6…api_integration_guide 60a5c6907a02… |
package_archive invalidated 2026-11-19T11:45:17Z |
2026-11-19T10:46:48Z | 2026-11-19T11:27:29Z | 2 | 37 (final gate PASS)
|
sale_tar 76bfcdaabe72…checksums 9b0b1c53bf4e…verify_md da0e24c86bc2…rebuild_md 3c2b1770a8e3…manifest_txt f90f6566d237…actions_jsonl 272e77d56868…provenance_md 26e5679c1d52…assetchain_log_md e9eccaec0c0f…assetchain_log_json 74253fb389c7… |
[stage_runs]
Rewinds
- 2026-11-19T11:15:58Z:
package_archivetopackage_archiveon docs_redo (scripts/redo-docs.py --stage package_archive) [stage_events, stage_events #39737] - 2026-11-19T11:45:17Z:
package_archivetopackage_archiveon docs_redo (scripts/redo-docs.py --stage package_archive) [stage_events, stage_events #39751]
Model work per stage
Fully recordedRole: writer model. Calls up to 2026-11-19T11:27:29Z.
16 calls over 4 stages
| Stage | Calls | Refused | Errors | Max attempt | Forced commits | Real tokens | Lesson rules shown |
|---|---|---|---|---|---|---|---|
plan | 2 | 0 | 0 | 1 | 0 | 9232 | L-A183, L-A15 |
remediate | 2 | 0 | 0 | 1 | 0 | 9254 | L-A183, L-A15 |
usecase_validate | 1 | 0 | 0 | 1 | 0 | 2727 | L-A59 |
finalize_docs | 11 | 1 | 0 | 2 | 0 | 109049 | L-A180, L-A181, L-A182, L-A183, L-A184, L-A155, L-A114, L-A25, L-A15 |
[micro_step_runs] [lesson_recalls]
Review
Fully recordedRole: review model.
2 rounds, 2 counted, $1.27 recorded cost
| Round | Started | Critical | Major | Minor | Verdict | Cost | Cap in force | Archive reviewed |
|---|---|---|---|---|---|---|---|---|
| 1 | 2026-11-19T10:47:33Z | 0 | 6 | 0 | findings to repackage [held_rounds, held_rounds #2024] | $0.71 | 3 rounds, $40.00 | 735b13a13b1a… |
| 2 | 2026-11-19T11:27:57Z | 0 | 0 | 2 | clean to publish [held_rounds, held_rounds #2025] | $0.56 | 3 rounds, $40.00 | 76bfcdaabe72… |
Defects
Fully recorded- Ledger entries8 [defects, defects where product_slug=opensearch and this tag or chain]
- Disclosed in the archive8 [docs/defects/defects.json, entries]
8 ledger entries: by severity high 6, low 2; by status fixed-before-release 8.
| Defect | Severity | Status | Found by | Fixed in |
|---|---|---|---|---|
RSD-opensearch-7cebbae66eAPI guide: curl -X PUT :9200/_nodes -> 405 'Incorrect HTTP method for uri [/_nodes] and method [PUT], allowed: [GET]'; GET /_nodes -> 200 with node list (live r7 container) [defects, defects #340947] | high | fixed-before-release | held review (review model) | — |
RSD-opensearch-ab18d71231API guide: GET /_nodes -> 200; PUT /_nodes -> 405 allowed: [GET] on the running r7 container [defects, defects #340948] | high | fixed-before-release | held review (review model) | — |
RSD-opensearch-d4f338d75aAPI guide: Ran as written: HTTP 406 'Content-Type header [application/x-www-form-urlencoded] is not supported'; with the JSON header: 201 created. [defects, defects #340949] | high | fixed-before-release | held review (review model) | — |
RSD-opensearch-f9edf5b174API guide: Ran as written: HTTP 406 Content-Type application/x-www-form-urlencoded not supported [defects, defects #340950] | high | fixed-before-release | held review (review model) | — |
RSD-opensearch-4b7c88027aAPI guide: Sample Index step as written (no body): HTTP 400 parse_exception 'request body is required', so the following search finds nothing. [defects, defects #340951] | high | fixed-before-release | held review (review model) | — |
RSD-opensearch-d5730de50dAPI guide: POST /newidx/_doc/1 with JSON body on a fresh container -> 201 created (index auto-created) [defects, defects #340952] | high | fixed-before-release | held review (review model) | — |
RSD-opensearch-061e2beda1API guide: Cross-reference points to a section that does not show how to create rs-demo. [defects, defects #341063] | low | fixed-before-release | held review (review model) | — |
RSD-opensearch-779c9d56e7API guide: Same dangling cross-reference. [defects, defects #341064] | low | fixed-before-release | held review (review model) | — |
Verification
1 item not recorded- Reproduced before publishingyes (rebuilt digest sha256:a094681b3c747c44cdfb0f02394892832573833d8a28a8bdb0166b181b701d20 · published digest sha256:a094681b3c747c44cdfb0f02394892832573833d8a28a8bdb0166b181b701d20 · digest match yes · builder moby/buildkit:v0.31.2 · build seconds 228) [docs/provenance/REPRO_VERIFICATION.json, verified]
- Final contract gate (G8)PASS (checks artifact_coverage, assetchain_coverage, cosign, cpu_baseline, offline_rebuild, rebuild_manifest, scan_grade, scout_grade_a, script_copies, tag_base_os, thread_exit, trademark_shipped, unpack_files, upgrade_check, vex_shipped) [g8_result.json, result]
- Grade at releaseA (percent 94 · docker grade B · docker percent 83 · tool docker scout policy 1.26.0 · config docker-default · org vex no · at 2026-11-19T11:46:30Z · failing Docker Scout counts 104 packages with copyleft licences (GPL, LGPL, MPL), all of them operating-system packages (it counts only packages whose licence it can read; the SBOM lists every component). · corrections No outdated base images) [SCOUT_EVIDENCE.json, docker_scout_policy.corrected]
- Image signatureverified on the index and the linux/amd64 manifest (key fingerprint 04daa92344a52dbc · at 2026-11-19T11:45:51Z) [docs/provenance/IMAGE_EVIDENCE.json, signed]
- Image signature transparency-log entries
- log index 3993871947 · transparency-log entry
- log index 3993874434 · transparency-log entry
- rebuild.json signatureSigstore bundle, release key (key fingerprint 04daa92344a52dbc · log index 3995749632 · transparency-log entry · integrated 2026-11-19T11:56:30Z) [ReadyStack-opensearch-3.9.0-CE-ubuntu24.04-r7.rebuild.json.sigstore, verificationMaterial.tlogEntries[0]]
- Signed attestations
- kind provenance · digest sha256:dcf889baf9fef8d89f7656aad6fb6df9a234299a0067e20b302395249ae82401 · state attached · log index 3995755302 · transparency-log entry
- kind sbom · digest sha256:dcf889baf9fef8d89f7656aad6fb6df9a234299a0067e20b302395249ae82401 · state attached · log index 3995759142 · transparency-log entry
- kind vsa · digest sha256:dcf889baf9fef8d89f7656aad6fb6df9a234299a0067e20b302395249ae82401 · state attached · log index 3995762991 · transparency-log entry
- kind provenance · digest sha256:a094681b3c747c44cdfb0f02394892832573833d8a28a8bdb0166b181b701d20 · state attached · log index 3995767219 · transparency-log entry
- kind sbom · digest sha256:a094681b3c747c44cdfb0f02394892832573833d8a28a8bdb0166b181b701d20 · state attached · log index 3995771293 · transparency-log entry
- kind vsa · digest sha256:a094681b3c747c44cdfb0f02394892832573833d8a28a8bdb0166b181b701d20 · state attached · log index 3995775755 · transparency-log entry
- Pins recorddocs/PINS.md and docs/provenance/PINS.json in the archive (https://readystack.dev/products/opensearch/recipe) [docs/provenance/PINS.json]
Offline clean-room rebuilds
- 2026-11-19T10:47:40Z on build host 5 (Docker-in-Docker daemon with --network none): bit for bit yes, digest match yes, customer verify yes
rebuilt sha256:a094681b3c747c44cdfb0f02394892832573833d8a28a8bdb0166b181b701d20 [clean-room rebuild ledger, runs[906]]
- Not recorded: the policy tool's --result-file itself (its per-policy results are copied into SCOUT_EVIDENCE.json and the archive's IMAGE_EVIDENCE.json; the file is not kept)
Publication
Fully recorded 1 pending at generation- Image pushed (internal tag), publish_image passed2026-11-19T10:26:27Z [stage_runs, stage_runs chain 1118 stage publish_image]
- Archive manifests written2026-11-19T11:50:29Z [PUBLISH_MANIFEST.json, published_at]
- Pending at generation: the archive cut-over on prodasset, the release tag's promotion and the site event (this trail was generated after G8 and before them; they are appended as a 'publication' event in History)
Packages and CPU level
Fully recorded- x86-64 level the image needsv1 (used v4 · baseline v2 · elf files 815 · probed 2026-11-19T10:15:51Z · source build_kit) [release_images, isa_level_hard]
History
2 events- 2026-11-19T11:59:24Z generated: by KitForge release/build_trail.py · generator commit 657323945e75094f81887e9b83e0cea60048a17a
- 2026-11-19T12:03:17Z publication: promoted at 2026-11-19T12:00:51Z · promoted digest sha256:dcf889baf9fef8d89f7656aad6fb6df9a234299a0067e20b302395249ae82401 · promotion method imagetools_create_retag_by_digest_no_rebuild · site event 617 · checklist status complete · checklist at 2026-11-19T12:03:17Z · record workspace: deliverables/provenance/PROMOTION_RECORD.json, RELEASE_CHECKLIST.json
Records
25 records| Record | Where it is | Rows |
|---|---|---|
docs/defects/defects.json | inside the archive (ReadyStack-opensearch-3.9.0-CE-ubuntu24.04-r7.tar.gz) | — |
docs/provenance/IMAGE_EVIDENCE.json | inside the archive (ReadyStack-opensearch-3.9.0-CE-ubuntu24.04-r7.tar.gz) | — |
docs/provenance/LOCKED_DIGEST.txt | inside the archive (ReadyStack-opensearch-3.9.0-CE-ubuntu24.04-r7.tar.gz) | — |
docs/provenance/PINS.json | inside the archive (ReadyStack-opensearch-3.9.0-CE-ubuntu24.04-r7.tar.gz) | — |
docs/provenance/REPRO_VERIFICATION.json | inside the archive (ReadyStack-opensearch-3.9.0-CE-ubuntu24.04-r7.tar.gz) | — |
docs/provenance/SOURCE_TREE_HASH.json | inside the archive (ReadyStack-opensearch-3.9.0-CE-ubuntu24.04-r7.tar.gz) | — |
MANIFEST.json | beside the archive (prodasset /data/production/opensearch/3.9.0-CE-ubuntu24.04-r7/), as staged for publishing | — |
PUBLISH_MANIFEST.json | beside the archive (prodasset /data/production/opensearch/3.9.0-CE-ubuntu24.04-r7/), as staged for publishing | — |
ReadyStack-opensearch-3.9.0-CE-ubuntu24.04-r7.rebuild.json | beside the archive (prodasset /data/production/opensearch/3.9.0-CE-ubuntu24.04-r7/), as staged for publishing | — |
ReadyStack-opensearch-3.9.0-CE-ubuntu24.04-r7.rebuild.json.sigstore | beside the archive (prodasset /data/production/opensearch/3.9.0-CE-ubuntu24.04-r7/), as staged for publishing | — |
SCOUT_EVIDENCE.json | beside the archive (prodasset /data/production/opensearch/3.9.0-CE-ubuntu24.04-r7/), as staged for publishing | — |
clean-room rebuild ledger | the clean-room rebuild ledger (runstate/audit-2026-09/bit4bit.json, not published) | — |
build_requests | sadb, ReadyStack's engine database (not published) | 800 |
chains | sadb, ReadyStack's engine database (not published) | 1118 |
defects | sadb, ReadyStack's engine database (not published) | opensearch 3.9.0-CE-ubuntu24.04-r7 |
drive_runs | sadb, ReadyStack's engine database (not published) | chain 1118 |
held_rounds | sadb, ReadyStack's engine database (not published) | chain 1118 |
lesson_recalls | sadb, ReadyStack's engine database (not published) | chain 1118 |
micro_step_runs | sadb, ReadyStack's engine database (not published) | chain 1118 |
release_images | sadb, ReadyStack's engine database (not published) | opensearch 3.9.0-CE-ubuntu24.04-r7 |
stage_events | sadb, ReadyStack's engine database (not published) | chain 1118 |
stage_runs | sadb, ReadyStack's engine database (not published) | chain 1118 |
attestations.json | the chain's workspace on the controller (not published) | — |
deliverables/provenance/PROMOTION_RECORD.json | the chain's workspace on the controller (not published) | — |
g8_result.json | the chain's workspace on the controller (not published) | — |