virt-operator v1.9.1-CE-debian12-r1 build trail
How ReadyStack built readystack/virtoperator:v1.9.1-CE-debian12-r1 and its build kit, copied record by record: why the release exists, every chain, drive and stage it went through, the model work, the review rounds, the defects, the checks that verified it and when it was published. Each line names the record it comes from in brackets; a record that does not exist for this release says so.
This trail
As JSON- Image
readystack/virtoperator:v1.9.1-CE-debian12-r1 - Index digestsha256:55aee39b1c7b4a718b56226fee362727e9dd124f5b83ed23bf0b85a3c0986ed7
- linux/amd64 digestsha256:78de21c348d6b5c03db14e9b14e4fe02f947114771f8897f53c2f52bf435a0a6
- Archive
ReadyStack-virtoperator-v1.9.1-CE-debian12-r1.tar.gz - Archive SHA-256 now98ed2577e92d0aaee8f8e8ceeb3d9d368657803f4e81d6ce9b538be1a025a8c0
- Generated2026-11-02T23:33:35Z by KitForge release/build_trail.py
- Sections9 sections, 4 fully recorded; 11 items not recorded
Signed with the release key The trail's signature is a Sigstore bundle beside it. Check it is ours before you rely on it:
cosign verify-blob --key https://readystack.dev/keys/cosign.pub \ --bundle ReadyStack-virtoperator-v1.9.1-CE-debian12-r1.trail.json.sigstore ReadyStack-virtoperator-v1.9.1-CE-debian12-r1.trail.json
IdentityWhy this release existsThe path through the harnessModel work per stageReviewDefectsVerificationPublicationPackages and CPU level
Identity
1 item not recorded- Productvirt-operator (virtoperator) [MANIFEST.json]
- Release tagv1.9.1-CE-debian12-r1 [MANIFEST.json]
- Imagereadystack/virtoperator:v1.9.1-CE-debian12-r1 [MANIFEST.json]
- Image index digestsha256:55aee39b1c7b4a718b56226fee362727e9dd124f5b83ed23bf0b85a3c0986ed7 [docs/provenance/LOCKED_DIGEST.txt]
- linux/amd64 image digestsha256:78de21c348d6b5c03db14e9b14e4fe02f947114771f8897f53c2f52bf435a0a6 [ReadyStack-virtoperator-v1.9.1-CE-debian12-r1.rebuild.json, subject.image_manifest_digest]
- ArchiveReadyStack-virtoperator-v1.9.1-CE-debian12-r1.tar.gz (bytes 1173305154) [MANIFEST.json]
- Archive sha256 at publishb703b090656950cdb33ae929e08cad959766067d9eb27b044026ed93ac4110c5 [MANIFEST.json, repack_history[0].previous_tarball_sha256]
- Archive sha256 now98ed2577e92d0aaee8f8e8ceeb3d9d368657803f4e81d6ce9b538be1a025a8c0 [MANIFEST.json, tarball_sha256]
- Archive records agreeyes [MANIFEST.json, PUBLISH_MANIFEST.json and .cutover.json name the same sha256]
- Repack history
- at 2026-10-29 · label stackdocs · reason documents brought up to the stack's current runbook record (2026-10-29T02:34:55Z run, re-recorded with the runbook corrections); documents only, not image inputs; image and every build input unchanged · previous sha256 b703b090656950cdb33ae929e08cad959766067d9eb27b044026ed93ac4110c5 · sha256 4e7a0e73726dde53389c67efc71197899a363f511845751a824727240c62960f · replaced docs/admin-guide/ADMIN_HANDBOOK.md, docs/db-handbook/DB_HANDBOOK.md, docs/db-handbook/DB_HANDBOOK.md.command-transcripts.json, docs/stack/RUNBOOK.md, docs/stack/RUNBOOK_EVIDENCE.json, rebuild.json
- at 2026-11-01 · label kitfix · reason customer kit files brought up to the current engine: BUILD_OFFLINE.sh (kit root, docs/offline, docs/docker); the header of BUILD_OFFLINE.sh no longer says a --provenance rebuild meets a SLSA build level, comment lines only; not image inputs; image and every build input unchanged · previous sha256 4e7a0e73726dde53389c67efc71197899a363f511845751a824727240c62960f · sha256 98ed2577e92d0aaee8f8e8ceeb3d9d368657803f4e81d6ce9b538be1a025a8c0 · replaced BUILD_OFFLINE.sh, docs/docker/BUILD_OFFLINE.sh, docs/offline/BUILD_OFFLINE.sh, rebuild.json
- Upstream repositoryhttps://github.com/kubevirt/kubevirt [docs/provenance/SOURCE_TREE_HASH.json]
- Upstream refv1.9.1 [docs/provenance/SOURCE_TREE_HASH.json]
- Upstream commitc3d19093f9c7b843b3879b18dc088911c081c39e [docs/provenance/SOURCE_TREE_HASH.json]
- Source tree hashde7abf04e51b6cf004d0365dfe6d8c5251edeeb69bfa0caa3d8b366b85fe6f36 (files 12717) [docs/provenance/SOURCE_TREE_HASH.json, tree_listing_sha256 (sha256 of the sorted file listing)]
- Kit tree hashsha256:75c163e68e8c70bb59cc117947d748b9f3bd5013401fa40565693c368b17eb30 [ReadyStack-virtoperator-v1.9.1-CE-debian12-r1.rebuild.json, inputs.tree_hash]
- SOURCE_DATE_EPOCH1791523273 (utc 2026-10-09T05:21:13Z) [ReadyStack-virtoperator-v1.9.1-CE-debian12-r1.rebuild.json, inputs.source_date_epoch]
- Base images
- ref quay.io/kubevirt/builder:2606231619-4fe2cd536f@sha256:a48a7dd5a1b80602269b8c714fdaa54f729a9a4a4d096f7ee0d15c94cef343e8 · platform digest sha256:a48a7dd5a1b80602269b8c714fdaa54f729a9a4a4d096f7ee0d15c94cef343e8
- BuildKitmoby/buildkit:v0.31.2 (image digest sha256:e0e67ea9c4bb07bdef683e29f70f3b623d1eb9c82274a9ec04e6bb0511449fb0) [ReadyStack-virtoperator-v1.9.1-CE-debian12-r1.rebuild.json, engine]
- Not recorded: the engine commit(s) that built it (drive_runs records them for drives from 2026-11-02; this release's drives were rebuilt from line logs, which do not name it)
Why this release exists
Fully recorded- Chain784 (created 2026-10-29T04:36:54Z · trigger customer_request · upstream version v1.9.1) [chains, chains #784]
- Requestbuild request #462 (new_product) (requested at 2026-10-29T03:13:57Z · requested by agent · version v1.9.1) [build_requests, build_requests #462]
- ReasonKubernetes VM Stack Kits (DevOps 2026-10-28); owner 2026-10-28 "pause all other efforts. This build list is your top priority"; owner 2026-10-29: publish every component through the normal pipeline, listed only at A (virthandler/virtlauncher may be unlisted, grade disclosed); both bundles ship as images [build_requests, build_requests #462 request_reason]
The path through the harness
2 items not recorded1 chain · 5 drives · 18 stages · 1 rewind
Chain 784 release · shipped
Created 2026-10-29T04:36:54Z, upstream v1.9.1, build request 462 [chains, chains #784]
| Drive | Started | Minutes | Build host | How it stopped |
|---|---|---|---|---|
| 1077 | 2026-10-29T04:37:06Z | 40 | build host 3 withhold-publish | ready_to_ship [drive_runs, drive_runs #1077] rebuilt from a build line's log |
| 1084 | 2026-10-29T07:24:25Z | 6.2 | build host 3 withhold-publish | ready_to_ship [drive_runs, drive_runs #1084] rebuilt from a build line's log |
| 1090 | 2026-10-29T08:35:23Z | 5.4 | build host 3 withhold-publish | infra_hub_rate_limit at package_archive [drive_runs, drive_runs #1090] rebuilt from a build line's log |
| 1091 | 2026-10-29T09:06:40Z | 5.7 | build host 3 withhold-publish | ready_to_ship [drive_runs, drive_runs #1091] rebuilt from a build line's log |
| 1093 | 2026-10-29T09:22:44Z | 8.5 | build host 3 publish-held | shipped [drive_runs, drive_runs #1093] rebuilt from a build line's log |
Stages, in order
| Stage | First start | Passed | Attempts | Gate checks passed | Artifacts (SHA-256) |
|---|---|---|---|---|---|
intake |
2026-10-29T04:37:12Z | 2026-10-29T04:37:12Z | 1 | 4
|
— |
acquire |
2026-10-29T04:37:21Z | 2026-10-29T04:37:21Z | 1 | 12
|
— |
plan |
2026-10-29T04:37:53Z | 2026-10-29T04:37:54Z | 1 | 10
|
— |
assemble_kit |
2026-10-29T04:47:52Z | 2026-10-29T04:47:53Z | 1 | 27
|
— |
remediate |
2026-10-29T04:48:35Z | 2026-10-29T04:48:36Z | 1 | 8
|
— |
build_kit |
2026-10-29T05:02:16Z | 2026-10-29T05:02:17Z | 1 | 9
|
— |
verdict |
2026-10-29T05:02:57Z | 2026-10-29T05:02:58Z | 1 | 8
|
— |
publish_image |
2026-10-29T05:03:40Z | 2026-10-29T05:03:40Z | 1 | 7
|
— |
boot_instance |
2026-10-29T05:03:42Z | 2026-10-29T05:03:43Z | 1 | 3
|
— |
sbom |
2026-10-29T05:03:52Z | 2026-10-29T05:03:53Z | 1 | 5
|
— |
schema_discovery |
2026-10-29T05:03:55Z | 2026-10-29T05:03:55Z | 1 | 7
|
— |
admin_handbook |
2026-10-29T05:03:58Z | 2026-10-29T07:24:31Z | 2 | 7
|
— |
db_handbook |
2026-10-29T05:04:00Z | 2026-10-29T07:24:35Z | 2 | 8
|
— |
seed_data |
2026-10-29T05:04:03Z | 2026-10-29T07:24:39Z | 2 | 6
|
— |
seed_validate |
2026-10-29T05:04:06Z | 2026-10-29T07:24:42Z | 2 | 3
|
— |
usecase_validate |
2026-10-29T05:04:08Z | 2026-10-29T07:24:45Z | 2 | 10
|
— |
finalize_docs |
2026-10-29T05:04:12Z | 2026-10-29T07:24:51Z | 2 | 9
|
— |
package_archive |
2026-10-29T05:17:01Z | 2026-10-29T09:31:12Z | 4 | 33 (final gate PASS)
|
— |
[stage_runs]
Rewinds
- 2026-10-29T09:22:13Z:
package_archivetopackage_archiveon docs_redo (scripts/redo-docs.py) [stage_events, stage_events #1293]
- Not recorded: each drive's ticks and first failed check (drive_runs records them for drives from 2026-11-02; these drives were rebuilt from line logs)
- Not recorded: the sha256 of each stage's artifacts (stage_events records them for stages passed from 2026-11-02; this chain's events were rebuilt from stage_runs and logs) — the archive's CHECKSUMS.sha256 lists the sha256 of every file it ships
Model work per stage
3 items not recordedRole: writer model. Calls up to 2026-10-29T09:31:12Z.
4 calls over 2 stages
| Stage | Calls | Refused | Errors | Max attempt | Forced commits | Real tokens | Lesson rules shown |
|---|---|---|---|---|---|---|---|
plan | 2 | 0 | 0 | 1 | not recorded | not recorded | — |
remediate | 2 | 0 | 0 | 1 | not recorded | not recorded | — |
[micro_step_runs]
- Not recorded: forced commits (micro_step_runs records them for calls from 2026-11-02)
- Not recorded: real token counts (recorded per call from 2026-11-02; earlier rows hold only an estimate, which is not shown)
- Not recorded: the lesson rules shown to the writer model (lesson_recalls holds no rows for this chain: its recorder was silent from 2026-06-06 to 2026-11-02)
Review
3 items not recordedRole: review model.
3 rounds, 3 counted, $2.31 recorded cost
| Round | Started | Critical | Major | Minor | Verdict | Cost | Cap in force | Archive reviewed |
|---|---|---|---|---|---|---|---|---|
| 1 | 2026-10-29T05:18:13Z | 0 | 2 | 2 | findings to tests_broken [held_rounds, held_rounds #210] rebuilt from a held-review state file | $0.90 | not recorded | not recorded |
| 2 | 2026-10-29T07:31:22Z | 0 | 1 | 1 | blocked to blocked [held_rounds, held_rounds #211] rebuilt from a held-review state file | $0.88 | not recorded | not recorded |
| 3 | 2026-10-29T09:12:44Z | 0 | 0 | 0 | clean to publish [held_rounds, held_rounds #212] rebuilt from a held-review state file | $0.53 | not recorded | not recorded |
Overrides
max_rounds: — to 6; by not recorded; why: not recorded predates override recording (2026-11-02): found in runstate/held-auto/virtoperator.json, last written 2026-10-29T09:15Z; who set it, why, when and the value before are not known [review_overrides, review_overrides #27]
- Not recorded: the sha256 of the archive each round reviewed (held_rounds records it for rounds from 2026-11-02; these rounds were rebuilt from state files)
- Not recorded: the round cap in force at each round (the state files did not keep it)
- Not recorded: who raised max_rounds to 6 and why (review_overrides #27 was backfilled from a state file; overrides are recorded with who and why from 2026-11-02)
Defects
Fully recorded- Ledger entries6 [defects, defects where product_slug=virtoperator and this tag or chain]
- Disclosed in the archive6 [docs/defects/defects.json, entries]
6 ledger entries: by severity high 3, low 3; by status fixed-before-release 6.
| Defect | Severity | Status | Found by | Fixed in |
|---|---|---|---|---|
RSD-virtoperator-729d61090eDocker Hub README: The only customer run command in the Docker Hub README starts a container that exits with status 1 straight away (no OPERATOR_IMAGE / in-cluster environment), so the 'Deployment' section hands out a… [defects, defects #134431] | high | fixed-before-release | held review (review model) | — |
RSD-virtoperator-184ab078eaRelease notes: The notes say the checks ran 'on the cluster that runs it', i.e. [defects, defects #134432] | high | fixed-before-release | held review (review model) | — |
RSD-virtoperator-8041cf0cafAdmin handbook: Contradicts the recorded state-dirs output (pki directory is mode 755, not root-only) and points to a check that is not above in this document. [defects, defects #134433] | low | fixed-before-release | held review (review model) | — |
RSD-virtoperator-0033728c35Database handbook: 'root-only' is contradicted by the directory mode printed 14 lines later. [defects, defects #134434] | low | fixed-before-release | held review (review model) | — |
RSD-virtoperator-71f5489c5bDocumentation: Round 1's major finding (validation claim stronger than the evidence) is fixed in RELEASE_NOTES.md but not in the sale record: the MANIFEST still presents '6 of 6 use cases passed' as the build's… [defects, defects #134867] | high | fixed-before-release | held review (review model) | — |
RSD-virtoperator-33812c06eeDocumentation: The sentence starts as a fragment ('the operator:') and omits virt-exportproxy, which the handbook says virt-operator v1.9.1 deploys unconditionally and without which KubeVirt is not Available. [defects, defects #134868] | low | fixed-before-release | held review (review model) | — |
Verification
2 items not recorded- Reproduced before publishingyes (rebuilt digest sha256:78de21c348d6b5c03db14e9b14e4fe02f947114771f8897f53c2f52bf435a0a6 · published digest sha256:78de21c348d6b5c03db14e9b14e4fe02f947114771f8897f53c2f52bf435a0a6 · digest match yes · builder moby/buildkit:v0.31.2 · build seconds 405) [docs/provenance/REPRO_VERIFICATION.json, verified]
- Final contract gate (G8)PASS (checks cosign, scan_grade, tag_base_os, vex_shipped, scout_grade_a, script_copies, three_way_sha, offline_rebuild, rebuild_manifest, artifact_coverage, assetchain_coverage) [stage_runs, stage_runs chain 784 package_archive gate_result.g8.result]
- Grade at releaseA (percent 94 · tool docker scout policy 1.26.0 · config docker-default · org vex no · at 2026-10-29T09:23:01Z · failing Docker Scout counts 3 packages with copyleft licences (GPL, LGPL), all of them operating-system packages (it counts only packages whose licence it can read; the SBOM lists every component).) [SCOUT_EVIDENCE.json, docker_scout_policy]
- First own scan of the releaseA (at 2026-10-29T09:31:27Z · score 100 · rubric cerodeo-v1 · fixable critical 0 · fixable high 0 · syft 1.54.1 · grype 0.120.1 · grype db v6.1.10@2026-10-29T06:23:34Z) [image_scans, image_scans #1549]
- Image signatureverified on the index and the linux/amd64 manifest (key fingerprint 04daa92344a52dbc · at 2026-10-29T09:22:49Z) [docs/provenance/IMAGE_EVIDENCE.json, signed]
- Image signature transparency-log entries
- log index 3558684024 · transparency-log entry
- log index 3558687201 · transparency-log entry
- rebuild.json signatureSigstore bundle, release key (key fingerprint 04daa92344a52dbc · log index 3632339228 · transparency-log entry · integrated 2026-11-01T14:19:57Z) [ReadyStack-virtoperator-v1.9.1-CE-debian12-r1.rebuild.json.sigstore, verificationMaterial.tlogEntries[0]]
- Signed attestations
- kind provenance · digest sha256:55aee39b1c7b4a718b56226fee362727e9dd124f5b83ed23bf0b85a3c0986ed7 · state attached · log index 3563791716 · transparency-log entry
- kind sbom · digest sha256:55aee39b1c7b4a718b56226fee362727e9dd124f5b83ed23bf0b85a3c0986ed7 · state attached · log index 3563793954 · transparency-log entry
- kind vsa · digest sha256:55aee39b1c7b4a718b56226fee362727e9dd124f5b83ed23bf0b85a3c0986ed7 · state attached · log index 3563796466 · transparency-log entry
- kind provenance · digest sha256:78de21c348d6b5c03db14e9b14e4fe02f947114771f8897f53c2f52bf435a0a6 · state attached · log index 3563798856 · transparency-log entry
- kind sbom · digest sha256:78de21c348d6b5c03db14e9b14e4fe02f947114771f8897f53c2f52bf435a0a6 · state attached · log index 3563801162 · transparency-log entry
- kind vsa · digest sha256:78de21c348d6b5c03db14e9b14e4fe02f947114771f8897f53c2f52bf435a0a6 · state attached · log index 3563803422 · transparency-log entry
Offline clean-room rebuilds
- 2026-10-29T05:18:17Z on build host 3 (Docker-in-Docker daemon with --network none): bit for bit yes, digest match yes, customer verify yes
rebuilt sha256:78de21c348d6b5c03db14e9b14e4fe02f947114771f8897f53c2f52bf435a0a6 [clean-room rebuild ledger, runs[535]]
- Not recorded: the policy tool's --result-file itself (its per-policy results are copied into SCOUT_EVIDENCE.json and the archive's IMAGE_EVIDENCE.json; the file is not kept)
- Not recorded: the pins record (docs/PINS.md is not in this archive as generated; a repack that adds it is appended to this trail's history)
Publication
Fully recorded- Image pushed (internal tag), publish_image passed2026-10-29T05:03:40Z [stage_runs, stage_runs chain 784 stage publish_image]
- Release tag promoted2026-10-29T09:30:38Z (method imagetools_create_retag_by_digest_no_rebuild · digest sha256:55aee39b1c7b4a718b56226fee362727e9dd124f5b83ed23bf0b85a3c0986ed7 · tag readystack/virtoperator:v1.9.1-CE-debian12-r1) [deliverables/provenance/PROMOTION_RECORD.json, promoted_at]
- Archive manifests written2026-10-29T09:26:41Z [PUBLISH_MANIFEST.json, published_at]
- Archive cut over on prodasset, package_archive passed2026-10-29T09:31:12Z [stage_runs, stage_runs chain 784 stage package_archive]
- Cut-over markerpresent (chain 784) [.cutover.json]
- Release checklist and site event2026-10-29T09:31:10Z (status complete · site event 273 · steps grade a verified done · release tag promoted done · cosign signed done · release evidence recorded done · contract gate passed done · archive published done · hub readme live done · reproducibility recorded done · defects ledger published done · catalog available done · build request fulfilled done · seo regenerated skipped · site catalogue refreshed done · site grade recorded done · site queue synced done · customer requests charged done · owners notified done) [deliverables/provenance/RELEASE_CHECKLIST.json, run_at]
Packages and CPU level
Fully recorded- Packages in the image113 (scanned 2026-11-02T06:30:04Z · source backfill:daily-scan) [release_images, package_count]
- x86-64 level the image needsv2 (used v4 · baseline v2 · elf files 281 · probed 2026-11-02T17:13:05Z · source backfill) [release_images, isa_level_hard]
History
1 event- 2026-11-02T23:33:35Z generated: by KitForge release/build_trail.py · generator commit 916780cc218cf013bd623a8c085aaf73d3b053d0 · note backfill: built after the release from its records
Records
26 records| Record | Where it is | Rows |
|---|---|---|
docs/defects/defects.json | inside the archive (ReadyStack-virtoperator-v1.9.1-CE-debian12-r1.tar.gz) | — |
docs/provenance/IMAGE_EVIDENCE.json | inside the archive (ReadyStack-virtoperator-v1.9.1-CE-debian12-r1.tar.gz) | — |
docs/provenance/LOCKED_DIGEST.txt | inside the archive (ReadyStack-virtoperator-v1.9.1-CE-debian12-r1.tar.gz) | — |
docs/provenance/REPRO_VERIFICATION.json | inside the archive (ReadyStack-virtoperator-v1.9.1-CE-debian12-r1.tar.gz) | — |
docs/provenance/SOURCE_TREE_HASH.json | inside the archive (ReadyStack-virtoperator-v1.9.1-CE-debian12-r1.tar.gz) | — |
.cutover.json | beside the archive (prodasset /data/production/virtoperator/v1.9.1-CE-debian12-r1/) | — |
MANIFEST.json | beside the archive (prodasset /data/production/virtoperator/v1.9.1-CE-debian12-r1/) | — |
PUBLISH_MANIFEST.json | beside the archive (prodasset /data/production/virtoperator/v1.9.1-CE-debian12-r1/) | — |
ReadyStack-virtoperator-v1.9.1-CE-debian12-r1.rebuild.json | beside the archive (prodasset /data/production/virtoperator/v1.9.1-CE-debian12-r1/) | — |
ReadyStack-virtoperator-v1.9.1-CE-debian12-r1.rebuild.json.sigstore | beside the archive (prodasset /data/production/virtoperator/v1.9.1-CE-debian12-r1/) | — |
SCOUT_EVIDENCE.json | beside the archive (prodasset /data/production/virtoperator/v1.9.1-CE-debian12-r1/) | — |
clean-room rebuild ledger | the clean-room rebuild ledger (runstate/audit-2026-09/bit4bit.json, not published) | — |
build_requests | sadb, ReadyStack's engine database (not published) | 462 |
chains | sadb, ReadyStack's engine database (not published) | 784 |
defects | sadb, ReadyStack's engine database (not published) | virtoperator v1.9.1-CE-debian12-r1 |
drive_runs | sadb, ReadyStack's engine database (not published) | chain 784 |
held_rounds | sadb, ReadyStack's engine database (not published) | chain 784 |
image_scans | sadb, ReadyStack's engine database (not published) | 1549 |
micro_step_runs | sadb, ReadyStack's engine database (not published) | chain 784 |
release_images | sadb, ReadyStack's engine database (not published) | virtoperator v1.9.1-CE-debian12-r1 |
review_overrides | sadb, ReadyStack's engine database (not published) | 27 |
stage_events | sadb, ReadyStack's engine database (not published) | chain 784 |
stage_runs | sadb, ReadyStack's engine database (not published) | chain 784 |
attestations.json | the chain's workspace on the controller (not published) | — |
deliverables/provenance/PROMOTION_RECORD.json | the chain's workspace on the controller (not published) | — |
deliverables/provenance/RELEASE_CHECKLIST.json | the chain's workspace on the controller (not published) | — |