pause 3.10.2-CE-scratch-r1 build trail
How ReadyStack built readystack/pause:3.10.2-CE-scratch-r1 and its build kit, copied record by record: why the release exists, every chain, drive and stage it went through, the model work, the review rounds, the defects, the checks that verified it and when it was published. Each line names the record it comes from in brackets; a record that does not exist for this release says so.
This trail
As JSON- Image
readystack/pause:3.10.2-CE-scratch-r1 - Index digestsha256:205b1d99aed2a00e741d04e7a9dede1ee8eca72e392088e3e2c0d1d0cf649253
- linux/amd64 digestsha256:6b74cfa71db2c51e1a21e3342402d3993573ea02a0c85063897eda87958dca26
- Archive
ReadyStack-pause-3.10.2-CE-scratch-r1.tar.gz - Archive SHA-256 now6d9d2b14aa0660407c002ab29867498315e1de8c6caca0643aed3bafb95e4d92
- Generated2026-11-02T23:33:35Z by KitForge release/build_trail.py
- Sections9 sections, 4 fully recorded; 11 items not recorded
Signed with the release key The trail's signature is a Sigstore bundle beside it. Check it is ours before you rely on it:
cosign verify-blob --key https://readystack.dev/keys/cosign.pub \ --bundle ReadyStack-pause-3.10.2-CE-scratch-r1.trail.json.sigstore ReadyStack-pause-3.10.2-CE-scratch-r1.trail.json
IdentityWhy this release existsThe path through the harnessModel work per stageReviewDefectsVerificationPublicationPackages and CPU level
Identity
1 item not recorded- Productpause (pause) [MANIFEST.json]
- Release tag3.10.2-CE-scratch-r1 [MANIFEST.json]
- Imagereadystack/pause:3.10.2-CE-scratch-r1 [MANIFEST.json]
- Image index digestsha256:205b1d99aed2a00e741d04e7a9dede1ee8eca72e392088e3e2c0d1d0cf649253 [docs/provenance/LOCKED_DIGEST.txt]
- linux/amd64 image digestsha256:6b74cfa71db2c51e1a21e3342402d3993573ea02a0c85063897eda87958dca26 [ReadyStack-pause-3.10.2-CE-scratch-r1.rebuild.json, subject.image_manifest_digest]
- ArchiveReadyStack-pause-3.10.2-CE-scratch-r1.tar.gz (bytes 446978915) [MANIFEST.json]
- Archive sha256 at publishf66275f4e724bda1c83b59671278e96853f465600009b867f348fd4180d98a4b [MANIFEST.json, repack_history[0].previous_tarball_sha256]
- Archive sha256 now6d9d2b14aa0660407c002ab29867498315e1de8c6caca0643aed3bafb95e4d92 [MANIFEST.json, tarball_sha256]
- Archive records agreeyes [MANIFEST.json, PUBLISH_MANIFEST.json and .cutover.json name the same sha256]
- Repack history
- at 2026-10-29 · label stackdocs · reason documents brought up to the stack's current runbook record (2026-10-29T02:34:55Z run, re-recorded with the runbook corrections); documents only, not image inputs; image and every build input unchanged · previous sha256 f66275f4e724bda1c83b59671278e96853f465600009b867f348fd4180d98a4b · sha256 6bd5069e9e8deb472f9e04f5855ff45c9c9d4640595ac99b8aff04943c3dacf2 · replaced docs/admin-guide/ADMIN_HANDBOOK.md, docs/api-guide/API_INTEGRATION_GUIDE.md, docs/db-handbook/DB_HANDBOOK.md, docs/stack/RUNBOOK.md, docs/stack/RUNBOOK_EVIDENCE.json, rebuild.json
- at 2026-10-29 · label stackdocs · reason documents brought up to the stack's current runbook record (2026-10-29T02:34:55Z run, re-recorded with the runbook corrections); documents only, not image inputs; image and every build input unchanged · previous sha256 6bd5069e9e8deb472f9e04f5855ff45c9c9d4640595ac99b8aff04943c3dacf2 · sha256 c764fdd354745fe90b3ed7e4b7711ca03a1e164c0a7b7d0c75ac57d20f747ff9 · replaced docs/admin-guide/ADMIN_HANDBOOK.md, docs/db-handbook/DB_HANDBOOK.md, docs/db-handbook/DB_HANDBOOK.md.command-transcripts.json, docs/stack/RUNBOOK.md, docs/stack/RUNBOOK_EVIDENCE.json, rebuild.json
- at 2026-10-29 · label notesfix · reason docs/RELEASE_NOTES.md Known issues and docs/defects/defects.json brought up to the defects ledger (1 defect(s) recorded after packaging: RSD-pause-2b72da4dde); documents only, not image inputs; image and every build input unchanged · previous sha256 c764fdd354745fe90b3ed7e4b7711ca03a1e164c0a7b7d0c75ac57d20f747ff9 · sha256 0f4136289261762f48aa7f53e81cca7e76a79faaaa6dd3c5bc978c9c6d56d798 · replaced docs/RELEASE_NOTES.md, docs/defects/defects.json, rebuild.json
- at 2026-11-01 · label kitfix · reason customer kit files brought up to the current engine: BUILD_OFFLINE.sh (kit root, docs/offline, docs/docker); the header of BUILD_OFFLINE.sh no longer says a --provenance rebuild meets a SLSA build level, comment lines only; not image inputs; image and every build input unchanged · previous sha256 0f4136289261762f48aa7f53e81cca7e76a79faaaa6dd3c5bc978c9c6d56d798 · sha256 6d9d2b14aa0660407c002ab29867498315e1de8c6caca0643aed3bafb95e4d92 · replaced BUILD_OFFLINE.sh, docs/docker/BUILD_OFFLINE.sh, docs/offline/BUILD_OFFLINE.sh, rebuild.json
- Upstream repositoryhttps://github.com/kubernetes/kubernetes [docs/provenance/SOURCE_TREE_HASH.json]
- Upstream refv1.36.6 [docs/provenance/SOURCE_TREE_HASH.json]
- Upstream commit1b2171b110bb2a2542e1322035c6583ec86eb976 [docs/provenance/SOURCE_TREE_HASH.json]
- Source tree hash803692e6a7661d2d72c4cdf0a2a2459bdf6646e2b6c955ed7362f279ee4fa004 (files 29206) [docs/provenance/SOURCE_TREE_HASH.json, tree_listing_sha256 (sha256 of the sorted file listing)]
- Kit tree hashsha256:d26c8946d1e290e2769b57d7941075807683cc6b6ad4924025b1ae38e7224161 [ReadyStack-pause-3.10.2-CE-scratch-r1.rebuild.json, inputs.tree_hash]
- SOURCE_DATE_EPOCH1793102572 (utc 2026-10-27T12:02:52Z) [ReadyStack-pause-3.10.2-CE-scratch-r1.rebuild.json, inputs.source_date_epoch]
- Base images
- ref golang:1.26.10-bookworm@sha256:632dff7351799a5a9954a48dc06670ade72ee44e8c351cdc0e5a57af2fea21af · platform digest sha256:88750ccdd0f2a77fcc00a14b1a48d6409bd8ddb5d7c2dd6ec0b4a2e916daf9d9
- BuildKitmoby/buildkit:v0.31.2 (image digest sha256:e0e67ea9c4bb07bdef683e29f70f3b623d1eb9c82274a9ec04e6bb0511449fb0) [ReadyStack-pause-3.10.2-CE-scratch-r1.rebuild.json, engine]
- Not recorded: the engine commit(s) that built it (drive_runs records them for drives from 2026-11-02; this release's drives were rebuilt from line logs, which do not name it)
Why this release exists
Fully recorded- Chain781 (created 2026-10-29T04:29:11Z · trigger customer_request · upstream version 3.10.2) [chains, chains #781]
- Requestbuild request #456 (new_product) (requested at 2026-10-29T03:13:38Z · requested by agent · version 3.10.2) [build_requests, build_requests #456]
- ReasonKubernetes VM Stack Kits (DevOps 2026-10-28); owner 2026-10-28 "pause all other efforts. This build list is your top priority"; owner 2026-10-29: publish every component through the normal pipeline, listed only at A (virthandler/virtlauncher may be unlisted, grade disclosed); both bundles ship as images [build_requests, build_requests #456 request_reason]
The path through the harness
2 items not recorded1 chain · 3 drives · 18 stages · 2 rewinds
Chain 781 release · shipped
Created 2026-10-29T04:29:11Z, upstream 3.10.2, build request 456 [chains, chains #781]
| Drive | Started | Minutes | Build host | How it stopped |
|---|---|---|---|---|
| 1061 | 2026-10-29T04:29:22Z | 3.8 | build host 2 withhold-publish | stage_failed at verdict [drive_runs, drive_runs #1061] rebuilt from a build line's log |
| 1063 | 2026-10-29T05:13:36Z | 8 | build host 2 withhold-publish | ready_to_ship [drive_runs, drive_runs #1063] rebuilt from a build line's log |
| 1066 | 2026-10-29T06:24:40Z | 7.5 | build host 2 publish-held | shipped [drive_runs, drive_runs #1066] rebuilt from a build line's log |
Stages, in order
| Stage | First start | Passed | Attempts | Gate checks passed | Artifacts (SHA-256) |
|---|---|---|---|---|---|
intake |
2026-10-29T04:29:28Z | 2026-10-29T04:29:29Z | 1 | 4
|
— |
acquire |
2026-10-29T04:29:42Z | 2026-10-29T04:29:43Z | 1 | 12
|
— |
plan |
2026-10-29T04:30:02Z | 2026-10-29T04:30:03Z | 1 | 10
|
— |
assemble_kit |
2026-10-29T04:30:54Z | 2026-10-29T05:14:11Z | 2 | 27
|
— |
remediate |
2026-10-29T04:31:21Z | 2026-10-29T05:14:13Z | 2 | 8
|
— |
build_kit |
2026-10-29T04:32:24Z | 2026-10-29T05:15:16Z | 2 | 9
|
— |
verdict |
2026-10-29T04:33:10Z | 2026-10-29T05:15:39Z | 2 | 8
|
— |
publish_image |
2026-10-29T05:16:09Z | 2026-10-29T05:16:10Z | 1 | 7
|
— |
boot_instance |
2026-10-29T05:16:11Z | 2026-10-29T05:16:12Z | 1 | 3
|
— |
sbom |
2026-10-29T05:16:18Z | 2026-10-29T05:16:19Z | 1 | 5
|
— |
schema_discovery |
2026-10-29T05:16:21Z | 2026-10-29T05:16:22Z | 1 | 7
|
— |
admin_handbook |
2026-10-29T05:16:23Z | 2026-10-29T05:16:24Z | 1 | 7
|
— |
db_handbook |
2026-10-29T05:16:26Z | 2026-10-29T05:16:27Z | 1 | 8
|
— |
seed_data |
2026-10-29T05:16:28Z | 2026-10-29T05:16:29Z | 1 | 6
|
— |
seed_validate |
2026-10-29T05:16:31Z | 2026-10-29T05:16:32Z | 1 | 3
|
— |
usecase_validate |
2026-10-29T05:16:34Z | 2026-10-29T05:16:35Z | 1 | 10
|
— |
finalize_docs |
2026-10-29T05:16:38Z | 2026-10-29T05:16:39Z | 1 | 9
|
— |
package_archive |
2026-10-29T05:21:31Z | 2026-10-29T06:32:05Z | 2 | 33 (final gate PASS)
|
— |
[stage_runs]
Rewinds
- 2026-10-29T05:13:41Z:
verdicttoassemble_kiton strategy_changed (chain 781: strategy changed — rewound from 'verdict' to assemble_kit and dropped the vendored tree so the change actually reaches the build.) [stage_events, stage_events #2343] - 2026-10-29T06:24:06Z:
package_archivetopackage_archiveon docs_redo (scripts/redo-docs.py) [stage_events, stage_events #1262]
- Not recorded: each drive's ticks and first failed check (drive_runs records them for drives from 2026-11-02; these drives were rebuilt from line logs)
- Not recorded: the sha256 of each stage's artifacts (stage_events records them for stages passed from 2026-11-02; this chain's events were rebuilt from stage_runs and logs) — the archive's CHECKSUMS.sha256 lists the sha256 of every file it ships
Model work per stage
3 items not recordedRole: writer model. Calls up to 2026-10-29T06:32:05Z.
4 calls over 2 stages
| Stage | Calls | Refused | Errors | Max attempt | Forced commits | Real tokens | Lesson rules shown |
|---|---|---|---|---|---|---|---|
plan | 2 | 0 | 0 | 1 | not recorded | not recorded | — |
remediate | 2 | 0 | 0 | 1 | not recorded | not recorded | — |
[micro_step_runs]
- Not recorded: forced commits (micro_step_runs records them for calls from 2026-11-02)
- Not recorded: real token counts (recorded per call from 2026-11-02; earlier rows hold only an estimate, which is not shown)
- Not recorded: the lesson rules shown to the writer model (lesson_recalls holds no rows for this chain: its recorder was silent from 2026-06-06 to 2026-11-02)
Review
3 items not recordedRole: review model.
1 round, 1 counted, $0.96 recorded cost
| Round | Started | Critical | Major | Minor | Verdict | Cost | Cap in force | Archive reviewed |
|---|---|---|---|---|---|---|---|---|
| 1 | 2026-10-29T05:22:29Z | 0 | 0 | 2 | clean to publish [held_rounds, held_rounds #145] rebuilt from a held-review state file | $0.96 | not recorded | not recorded |
Overrides
max_rounds: — to 6; by not recorded; why: not recorded predates override recording (2026-11-02): found in runstate/held-auto/pause.json, last written 2026-10-29T06:31Z; who set it, why, when and the value before are not known [review_overrides, review_overrides #15]
- Not recorded: the sha256 of the archive each round reviewed (held_rounds records it for rounds from 2026-11-02; these rounds were rebuilt from state files)
- Not recorded: the round cap in force at each round (the state files did not keep it)
- Not recorded: who raised max_rounds to 6 and why (review_overrides #15 was backfilled from a state file; overrides are recorded with who and why from 2026-11-02)
Defects
Fully recorded- Ledger entries3 [defects, defects where product_slug=pause and this tag or chain]
- Disclosed in the archive3 [docs/defects/defects.json, entries]
3 ledger entries: by severity low 2, medium 1; by status open 2, fixed-before-release 1.
| Defect | Severity | Status | Found by | Fixed in |
|---|---|---|---|---|
RSD-pause-f3910b3db4REBUILD.md: The sentence says pause runs only inside a cluster. [defects, defects #134417] | low | open | held review (review model) | — |
RSD-pause-b0865c387dRelease notes: Release notes and SECURITY_REPORT.md:19 say pause runs unprivileged as 65535, but the admin handbook records four sandboxes at uid 0 on the integration cluster because the pod specs set it. [defects, defects #134418] | low | fixed-before-release | held review (review model) | — |
RSD-pause-2b72da4ddeThe image's licences label omits LGPL-2.1-or-later: /pause links the GNU C Library statically [defects, defects #135523] | medium | open | held review (review model) | — |
Verification
2 items not recorded- Reproduced before publishingyes (rebuilt digest sha256:6b74cfa71db2c51e1a21e3342402d3993573ea02a0c85063897eda87958dca26 · published digest sha256:6b74cfa71db2c51e1a21e3342402d3993573ea02a0c85063897eda87958dca26 · digest match yes · builder moby/buildkit:v0.31.2 · build seconds 34) [docs/provenance/REPRO_VERIFICATION.json, verified]
- Final contract gate (G8)PASS (checks cosign, scan_grade, tag_base_os, vex_shipped, scout_grade_a, script_copies, three_way_sha, offline_rebuild, rebuild_manifest, artifact_coverage, assetchain_coverage) [stage_runs, stage_runs chain 781 package_archive gate_result.g8.result]
- Grade at releaseA (percent 100 · tool docker scout policy 1.26.0 · config docker-default · org vex no · at 2026-10-29T06:24:55Z) [SCOUT_EVIDENCE.json, docker_scout_policy]
- First own scan of the releaseA (at 2026-10-29T07:03:32Z · score 100 · rubric cerodeo-v1 · fixable critical 0 · fixable high 0 · syft 1.54.1 · grype 0.120.1 · grype db v6.1.10@2026-10-28T06:23:30Z) [image_scans, image_scans #1536]
- Image signatureverified on the index and the linux/amd64 manifest (key fingerprint 04daa92344a52dbc · at 2026-10-29T06:24:46Z) [docs/provenance/IMAGE_EVIDENCE.json, signed]
- Image signature transparency-log entries
- log index 3558919958 · transparency-log entry
- log index 3558921156 · transparency-log entry
- rebuild.json signatureSigstore bundle, release key (key fingerprint 04daa92344a52dbc · log index 3627860529 · transparency-log entry · integrated 2026-11-01T11:13:26Z) [ReadyStack-pause-3.10.2-CE-scratch-r1.rebuild.json.sigstore, verificationMaterial.tlogEntries[0]]
- Signed attestations
- kind provenance · digest sha256:205b1d99aed2a00e741d04e7a9dede1ee8eca72e392088e3e2c0d1d0cf649253 · state attached · log index 3560282981 · transparency-log entry
- kind sbom · digest sha256:205b1d99aed2a00e741d04e7a9dede1ee8eca72e392088e3e2c0d1d0cf649253 · state attached · log index 3560285513 · transparency-log entry
- kind vsa · digest sha256:205b1d99aed2a00e741d04e7a9dede1ee8eca72e392088e3e2c0d1d0cf649253 · state attached · log index 3560288655 · transparency-log entry
- kind provenance · digest sha256:6b74cfa71db2c51e1a21e3342402d3993573ea02a0c85063897eda87958dca26 · state attached · log index 3560291694 · transparency-log entry
- kind sbom · digest sha256:6b74cfa71db2c51e1a21e3342402d3993573ea02a0c85063897eda87958dca26 · state attached · log index 3560294529 · transparency-log entry
- kind vsa · digest sha256:6b74cfa71db2c51e1a21e3342402d3993573ea02a0c85063897eda87958dca26 · state attached · log index 3560297663 · transparency-log entry
Offline clean-room rebuilds
- 2026-10-29T05:22:32Z on build host 2 (Docker-in-Docker daemon with --network none): bit for bit yes, digest match yes, customer verify yes
rebuilt sha256:6b74cfa71db2c51e1a21e3342402d3993573ea02a0c85063897eda87958dca26 [clean-room rebuild ledger, runs[534]]
- Not recorded: the policy tool's --result-file itself (its per-policy results are copied into SCOUT_EVIDENCE.json and the archive's IMAGE_EVIDENCE.json; the file is not kept)
- Not recorded: the pins record (docs/PINS.md is not in this archive as generated; a repack that adds it is appended to this trail's history)
Publication
Fully recorded- Image pushed (internal tag), publish_image passed2026-10-29T05:16:10Z [stage_runs, stage_runs chain 781 stage publish_image]
- Release tag promoted2026-10-29T06:31:33Z (method imagetools_create_retag_by_digest_no_rebuild · digest sha256:205b1d99aed2a00e741d04e7a9dede1ee8eca72e392088e3e2c0d1d0cf649253 · tag readystack/pause:3.10.2-CE-scratch-r1) [deliverables/provenance/PROMOTION_RECORD.json, promoted_at]
- Archive manifests written2026-10-29T06:28:03Z [PUBLISH_MANIFEST.json, published_at]
- Archive cut over on prodasset, package_archive passed2026-10-29T06:32:05Z [stage_runs, stage_runs chain 781 stage package_archive]
- Cut-over markerpresent (chain 781) [.cutover.json]
- Release checklist and site event2026-10-29T06:32:02Z (status complete · site event 260 · steps grade a verified done · release tag promoted done · cosign signed done · release evidence recorded done · contract gate passed done · archive published done · hub readme live done · reproducibility recorded done · defects ledger published done · catalog available done · build request fulfilled done · seo regenerated skipped · site catalogue refreshed done · site grade recorded done · site queue synced done · customer requests charged done · owners notified done) [deliverables/provenance/RELEASE_CHECKLIST.json, run_at]
Packages and CPU level
Fully recorded- Packages in the image2 (scanned 2026-11-02T06:30:04Z · source backfill:daily-scan) [release_images, package_count]
- x86-64 level the image needsv1 (used v4 · baseline v2 · elf files 1 · probed 2026-11-02T17:14:34Z · source backfill) [release_images, isa_level_hard]
History
1 event- 2026-11-02T23:33:35Z generated: by KitForge release/build_trail.py · generator commit 916780cc218cf013bd623a8c085aaf73d3b053d0 · note backfill: built after the release from its records
Records
26 records| Record | Where it is | Rows |
|---|---|---|
docs/defects/defects.json | inside the archive (ReadyStack-pause-3.10.2-CE-scratch-r1.tar.gz) | — |
docs/provenance/IMAGE_EVIDENCE.json | inside the archive (ReadyStack-pause-3.10.2-CE-scratch-r1.tar.gz) | — |
docs/provenance/LOCKED_DIGEST.txt | inside the archive (ReadyStack-pause-3.10.2-CE-scratch-r1.tar.gz) | — |
docs/provenance/REPRO_VERIFICATION.json | inside the archive (ReadyStack-pause-3.10.2-CE-scratch-r1.tar.gz) | — |
docs/provenance/SOURCE_TREE_HASH.json | inside the archive (ReadyStack-pause-3.10.2-CE-scratch-r1.tar.gz) | — |
.cutover.json | beside the archive (prodasset /data/production/pause/3.10.2-CE-scratch-r1/) | — |
MANIFEST.json | beside the archive (prodasset /data/production/pause/3.10.2-CE-scratch-r1/) | — |
PUBLISH_MANIFEST.json | beside the archive (prodasset /data/production/pause/3.10.2-CE-scratch-r1/) | — |
ReadyStack-pause-3.10.2-CE-scratch-r1.rebuild.json | beside the archive (prodasset /data/production/pause/3.10.2-CE-scratch-r1/) | — |
ReadyStack-pause-3.10.2-CE-scratch-r1.rebuild.json.sigstore | beside the archive (prodasset /data/production/pause/3.10.2-CE-scratch-r1/) | — |
SCOUT_EVIDENCE.json | beside the archive (prodasset /data/production/pause/3.10.2-CE-scratch-r1/) | — |
clean-room rebuild ledger | the clean-room rebuild ledger (runstate/audit-2026-09/bit4bit.json, not published) | — |
build_requests | sadb, ReadyStack's engine database (not published) | 456 |
chains | sadb, ReadyStack's engine database (not published) | 781 |
defects | sadb, ReadyStack's engine database (not published) | pause 3.10.2-CE-scratch-r1 |
drive_runs | sadb, ReadyStack's engine database (not published) | chain 781 |
held_rounds | sadb, ReadyStack's engine database (not published) | chain 781 |
image_scans | sadb, ReadyStack's engine database (not published) | 1536 |
micro_step_runs | sadb, ReadyStack's engine database (not published) | chain 781 |
release_images | sadb, ReadyStack's engine database (not published) | pause 3.10.2-CE-scratch-r1 |
review_overrides | sadb, ReadyStack's engine database (not published) | 15 |
stage_events | sadb, ReadyStack's engine database (not published) | chain 781 |
stage_runs | sadb, ReadyStack's engine database (not published) | chain 781 |
attestations.json | the chain's workspace on the controller (not published) | — |
deliverables/provenance/PROMOTION_RECORD.json | the chain's workspace on the controller (not published) | — |
deliverables/provenance/RELEASE_CHECKLIST.json | the chain's workspace on the controller (not published) | — |