diff --git a/CHANGELOG.md b/CHANGELOG.md index ace6ea72..11018bbd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -111,13 +111,13 @@ Applied, Rejected, Obstructed}` with receipt evidence and typed contract commit keeps a stable `QueryReadingIdentity` even if unrelated runtime-owned tick progress changes the observation freshness watermark. - `warp-core` now publishes observation artifacts under observation contract - version 3 and `echo:observation-artifact:v3` because contract evidence and - query reading identity are now part of the canonical reading envelope hashed - into observation artifacts. -- `echo-wasm-abi` now reports `ABI_VERSION` 11 for the expanded + version 4 and `echo:observation-artifact:v4` because contract evidence, query + reading identity, and retained-evidence posture are now part of the canonical + reading envelope hashed into observation artifacts. +- `echo-wasm-abi` now reports `ABI_VERSION` 12 for the expanded `ReadingEnvelope` response shape. Legacy retained reading envelopes that omit - the new optional contract/query identity fields decode those fields as - `None`. + the new optional contract, query identity, and retained-evidence fields + decode those fields as `None` or empty vectors. - `warp-core` now attaches contract package evidence to installed contract readings and receipt correlations. Installed QueryView readings carry package id, package name/version, artifact hash, schema hash, codec identity, diff --git a/crates/echo-wasm-abi/src/kernel_port.rs b/crates/echo-wasm-abi/src/kernel_port.rs index 3ee1c4bd..32d6955d 100644 --- a/crates/echo-wasm-abi/src/kernel_port.rs +++ b/crates/echo-wasm-abi/src/kernel_port.rs @@ -10,7 +10,7 @@ //! //! # ABI Version //! -//! The current ABI version is [`ABI_VERSION`] (11). All response types are +//! The current ABI version is [`ABI_VERSION`] (12). All response types are //! CBOR-encoded using the canonical rules defined in `docs/spec/js-cbor-mapping.md`. //! Breaking changes to response shapes or error codes require a bump to the //! ABI version. @@ -40,7 +40,7 @@ use serde::{ /// /// Increment when response types, error codes, or method signatures change /// in a backward-incompatible way. -pub const ABI_VERSION: u32 = 11; +pub const ABI_VERSION: u32 = 12; fn deserialize_opaque_id<'de, D>(deserializer: D) -> Result<[u8; 32], D::Error> where @@ -1666,6 +1666,71 @@ pub struct ContractEvidenceIdentity { pub op_kind: ContractOperationKind, } +/// Semantic role of retained contract evidence. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum RetainedEvidenceRole { + /// Generated contract artifact bytes. + ContractArtifact, + /// Scheduler receipt or receipt-adjacent material. + ContractReceipt, + /// Law witness or witness-adjacent material. + Witness, + /// Reading payload bytes. + ReadingPayload, + /// Encoded reading envelope bytes. + ReadingEnvelope, + /// Generated observer artifact bytes. + ObserverArtifact, +} + +/// Semantic coordinate for retained contract evidence. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct RetainedEvidenceCoordinate { + /// Installed contract evidence identity that owns the coordinate. + pub contract: ContractEvidenceIdentity, + /// Retained evidence role. + pub role: RetainedEvidenceRole, + /// Domain-separated semantic digest for the exact receipt, witness, reading, + /// or artifact coordinate. + pub semantic_digest: Vec, + /// Stable coordinate id used for missing-retention obstruction subjects. + pub coordinate_id: Vec, +} + +/// First-class reference to retained contract evidence bytes. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +pub struct RetainedEvidenceRef { + /// Semantic coordinate that explains what the retained bytes answer. + pub coordinate: RetainedEvidenceCoordinate, + /// Content-only hash for the retained bytes. + pub content_hash: Vec, + /// Retained byte length. + pub byte_len: u64, + /// Stable id that binds semantic coordinate, content hash, and byte length. + pub evidence_ref_id: Vec, +} + +/// Retained evidence availability posture. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(tag = "kind", rename_all = "snake_case")] +pub enum RetainedEvidencePosture { + /// The retained evidence is locally available. + Available { + /// Available retained evidence reference. + reference: Box, + }, + /// Required retained evidence is missing. + MissingRetention { + /// Coordinate that should eventually name retained evidence, if known. + coordinate: Option>, + /// Exact retained reference whose content is missing, if known. + reference: Option>, + /// Missing-retention id surfaced by the obstruction. + retention_id: Vec, + }, +} + /// Stable identity of the QueryView question answered by a reading. #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] pub struct QueryReadingIdentity { @@ -1698,6 +1763,9 @@ pub struct ReadingEnvelope { /// Stable query reading identity, when this envelope answered QueryView. #[serde(default)] pub query_identity: Option, + /// Retained evidence refs or missing-retention posture for this reading. + #[serde(default)] + pub retained_evidence: Vec, /// Witnesses or shell references that support the reading. pub witness_refs: Vec, /// Read-side parent/strand basis posture. diff --git a/crates/echo-wasm-abi/src/lib.rs b/crates/echo-wasm-abi/src/lib.rs index 4a590222..1ccb7bac 100644 --- a/crates/echo-wasm-abi/src/lib.rs +++ b/crates/echo-wasm-abi/src/lib.rs @@ -451,7 +451,7 @@ mod tests { #[test] fn kernel_port_abi_version_tracks_reading_envelope_contract_fields() { - assert_eq!(kernel_port::ABI_VERSION, 11); + assert_eq!(kernel_port::ABI_VERSION, 12); } #[test] @@ -1014,6 +1014,7 @@ mod tests { observer_basis: ReadingObserverBasis::CommitBoundary, contract: None, query_identity: None, + retained_evidence: Vec::new(), witness_refs: vec![ReadingWitnessRef::ResolvedCommit { reference }], parent_basis_posture: ObservationBasisPosture::Worldline, budget_posture: ReadingBudgetPosture::UnboundedOneShot, @@ -1033,7 +1034,10 @@ mod tests { entries.retain(|(key, _)| { !matches!( key, - Value::Text(text) if text == "contract" || text == "query_identity" + Value::Text(text) + if text == "contract" + || text == "query_identity" + || text == "retained_evidence" ) }); } @@ -1042,6 +1046,7 @@ mod tests { assert_eq!(legacy_reading, envelope.reading); assert_eq!(legacy_reading.contract, None); assert_eq!(legacy_reading.query_identity, None); + assert_eq!(legacy_reading.retained_evidence, Vec::new()); let optic_result = ObserveOpticResult::Reading(Box::new(OpticReading { envelope: envelope.reading.clone(), @@ -1120,6 +1125,57 @@ mod tests { )); } + #[test] + fn test_retained_evidence_posture_round_trip() { + use crate::kernel_port::{ + ContractEvidenceIdentity, ContractOperationKind, RetainedEvidenceCoordinate, + RetainedEvidencePosture, RetainedEvidenceRef, RetainedEvidenceRole, + }; + use alloc::boxed::Box; + + let contract = ContractEvidenceIdentity { + package_id: vec![20; 32], + echo_abi_version: 1, + package_name: "fixture-contract".into(), + package_version: "0.1.0".into(), + artifact_hash_hex: "aa".repeat(32), + codec_id: "fixture-codec".into(), + registry_version: 1, + wesley_generator_version: "echo-wesley-gen/0.1.0".into(), + helper_api_version: 1, + schema_sha256_hex: "bb".repeat(32), + op_id: 7, + op_kind: ContractOperationKind::Query, + }; + let coordinate = RetainedEvidenceCoordinate { + contract, + role: RetainedEvidenceRole::ReadingPayload, + semantic_digest: vec![21; 32], + coordinate_id: vec![22; 32], + }; + let reference = RetainedEvidenceRef { + coordinate: coordinate.clone(), + content_hash: vec![23; 32], + byte_len: 12, + evidence_ref_id: vec![24; 32], + }; + + let postures = vec![ + RetainedEvidencePosture::Available { + reference: Box::new(reference.clone()), + }, + RetainedEvidencePosture::MissingRetention { + coordinate: Some(Box::new(coordinate)), + reference: Some(Box::new(reference)), + retention_id: vec![25; 32], + }, + ]; + + let decoded: Vec = + decode_cbor(&encode_cbor(&postures).unwrap()).unwrap(); + assert_eq!(decoded, postures); + } + #[test] fn test_optic_intent_dispatch_result_variants_round_trip() { use crate::kernel_port::{ diff --git a/crates/echo-wesley-gen/tests/generation.rs b/crates/echo-wesley-gen/tests/generation.rs index e83a243d..8dd4aa63 100644 --- a/crates/echo-wesley-gen/tests/generation.rs +++ b/crates/echo-wesley-gen/tests/generation.rs @@ -331,6 +331,7 @@ mod tests { budget_posture: ReadingBudgetPosture::UnboundedOneShot, rights_posture: ReadingRightsPosture::KernelPublic, residual_posture: ReadingResidualPosture::Complete, + retained_evidence: Vec::new(), }, frame: request.frame, projection: request.projection, diff --git a/crates/warp-core/src/observation.rs b/crates/warp-core/src/observation.rs index 78db1d84..39f217a9 100644 --- a/crates/warp-core/src/observation.rs +++ b/crates/warp-core/src/observation.rs @@ -21,6 +21,7 @@ use thiserror::Error; use crate::attachment::{AttachmentOwner, AttachmentPlane}; use crate::clock::{GlobalTick, WorldlineTick}; +use crate::contract_obstruction::{ContractObstruction, ContractObstructionSubject}; use crate::contract_registry::{ContractEvidenceIdentity, ContractOperationKind}; use crate::coordinator::WorldlineRuntime; use crate::engine_impl::Engine; @@ -32,13 +33,16 @@ use crate::optic::{ OpticObstruction, OpticObstructionKind, OpticReading, ReadIdentity, WitnessBasis, }; use crate::provenance_store::{ProvenanceRef, ProvenanceService, ProvenanceStore}; +use crate::retained_evidence::{ + RetainedEvidenceCoordinate, RetainedEvidencePosture, RetainedEvidenceRef, RetainedEvidenceRole, +}; use crate::snapshot::Snapshot; use crate::strand::{StrandId, StrandRevalidationState}; use crate::tick_patch::SlotId; use crate::worldline::WorldlineId; -const OBSERVATION_VERSION: u32 = 3; -const OBSERVATION_ARTIFACT_DOMAIN: &[u8] = b"echo:observation-artifact:v3\0"; +const OBSERVATION_VERSION: u32 = 4; +const OBSERVATION_ARTIFACT_DOMAIN: &[u8] = b"echo:observation-artifact:v4\0"; const QUERY_READING_IDENTITY_DOMAIN: &[u8] = b"echo:query-reading-identity:v1\0"; const QUERY_READING_BASIS_DOMAIN: &[u8] = b"echo:query-reading-basis:v1\0"; const QUERY_READING_APERTURE_DOMAIN: &[u8] = b"echo:query-reading-aperture:v1\0"; @@ -891,6 +895,8 @@ pub struct ReadingEnvelope { pub contract: Option, /// Stable identity of the QueryView question answered by this reading. pub query_identity: Option, + /// Retained evidence refs or missing-retention posture for this reading. + pub retained_evidence: Vec, /// Witnesses or shell references that support the reading. pub witness_refs: Vec, /// Read-side parent/strand basis posture. @@ -917,6 +923,11 @@ impl ReadingEnvelope { .query_identity .as_ref() .map(QueryReadingIdentity::to_abi), + retained_evidence: self + .retained_evidence + .iter() + .map(retained_evidence_posture_to_abi) + .collect(), witness_refs: self .witness_refs .iter() @@ -950,6 +961,100 @@ fn contract_evidence_to_abi(evidence: &ContractEvidenceIdentity) -> abi::Contrac } } +fn retained_evidence_role_to_abi(role: RetainedEvidenceRole) -> abi::RetainedEvidenceRole { + match role { + RetainedEvidenceRole::ContractArtifact => abi::RetainedEvidenceRole::ContractArtifact, + RetainedEvidenceRole::ContractReceipt => abi::RetainedEvidenceRole::ContractReceipt, + RetainedEvidenceRole::Witness => abi::RetainedEvidenceRole::Witness, + RetainedEvidenceRole::ReadingPayload => abi::RetainedEvidenceRole::ReadingPayload, + RetainedEvidenceRole::ReadingEnvelope => abi::RetainedEvidenceRole::ReadingEnvelope, + RetainedEvidenceRole::ObserverArtifact => abi::RetainedEvidenceRole::ObserverArtifact, + } +} + +fn retained_evidence_coordinate_to_abi( + coordinate: &RetainedEvidenceCoordinate, +) -> abi::RetainedEvidenceCoordinate { + let coordinate_id = coordinate.coordinate_id(); + retained_evidence_coordinate_to_abi_with_id(coordinate, &coordinate_id) +} + +fn retained_evidence_coordinate_to_abi_with_id( + coordinate: &RetainedEvidenceCoordinate, + coordinate_id: &Hash, +) -> abi::RetainedEvidenceCoordinate { + abi::RetainedEvidenceCoordinate { + contract: contract_evidence_to_abi(&coordinate.contract), + role: retained_evidence_role_to_abi(coordinate.role), + semantic_digest: coordinate.semantic_digest.to_vec(), + coordinate_id: coordinate_id.to_vec(), + } +} + +fn retained_evidence_ref_to_abi(reference: &RetainedEvidenceRef) -> abi::RetainedEvidenceRef { + let coordinate_id = reference.coordinate.coordinate_id(); + retained_evidence_ref_to_abi_with_coordinate_id(reference, &coordinate_id) +} + +fn retained_evidence_ref_to_abi_with_coordinate_id( + reference: &RetainedEvidenceRef, + coordinate_id: &Hash, +) -> abi::RetainedEvidenceRef { + abi::RetainedEvidenceRef { + coordinate: retained_evidence_coordinate_to_abi_with_id( + &reference.coordinate, + coordinate_id, + ), + content_hash: reference.content_hash.to_vec(), + byte_len: reference.byte_len, + evidence_ref_id: reference + .evidence_ref_id_with_coordinate_id(coordinate_id) + .to_vec(), + } +} + +fn retained_evidence_posture_to_abi( + posture: &RetainedEvidencePosture, +) -> abi::RetainedEvidencePosture { + match posture { + RetainedEvidencePosture::Available(reference) => abi::RetainedEvidencePosture::Available { + reference: Box::new(retained_evidence_ref_to_abi(reference)), + }, + RetainedEvidencePosture::MissingCoordinate { + coordinate, + obstruction, + } => abi::RetainedEvidencePosture::MissingRetention { + coordinate: Some(Box::new(retained_evidence_coordinate_to_abi(coordinate))), + reference: None, + retention_id: retained_obstruction_id(obstruction), + }, + RetainedEvidencePosture::MissingContent { + reference, + obstruction, + } => { + let coordinate_id = reference.coordinate.coordinate_id(); + abi::RetainedEvidencePosture::MissingRetention { + coordinate: Some(Box::new(retained_evidence_coordinate_to_abi_with_id( + &reference.coordinate, + &coordinate_id, + ))), + reference: Some(Box::new(retained_evidence_ref_to_abi_with_coordinate_id( + reference, + &coordinate_id, + ))), + retention_id: retained_obstruction_id(obstruction), + } + } + } +} + +fn retained_obstruction_id(obstruction: &ContractObstruction) -> Vec { + match &obstruction.subject { + ContractObstructionSubject::Retention { retention_id } => retention_id.to_vec(), + _ => Vec::new(), + } +} + fn query_vars_digest(vars_bytes: &[u8]) -> Result { let digest = echo_wasm_abi::query_vars_digest_v1(vars_bytes); digest.as_slice().try_into().map_err(|_| { @@ -957,6 +1062,10 @@ fn query_vars_digest(vars_bytes: &[u8]) -> Result { }) } +fn retained_evidence_content_hash(bytes: &[u8]) -> Hash { + blake3::hash(bytes).into() +} + fn push_len_prefixed(hasher: &mut Hasher, bytes: &[u8]) { hasher.update(&(bytes.len() as u64).to_le_bytes()); hasher.update(bytes); @@ -2033,12 +2142,15 @@ impl ObservationService { &observer_plan, contract.as_ref(), )?; + let retained_evidence = + Self::retained_evidence_postures(contract.as_ref(), query_identity.as_ref(), payload); Ok(ReadingEnvelope { observer_plan, observer_instance: request.observer_instance.clone(), observer_basis: Self::observer_basis(request.frame), contract, query_identity, + retained_evidence, witness_refs, parent_basis_posture, budget_posture, @@ -2099,6 +2211,43 @@ impl ObservationService { })) } + fn retained_evidence_postures( + contract: Option<&ContractEvidenceIdentity>, + query_identity: Option<&QueryReadingIdentity>, + payload: &ObservationPayload, + ) -> Vec { + let (Some(contract), Some(query_identity)) = (contract, query_identity) else { + return Vec::new(); + }; + + let mut postures = Vec::new(); + let envelope_coordinate = RetainedEvidenceCoordinate::new( + contract.clone(), + RetainedEvidenceRole::ReadingEnvelope, + query_identity.reading_id, + ); + postures.push(RetainedEvidencePosture::missing_coordinate( + &envelope_coordinate, + )); + + if let ObservationPayload::QueryBytes(bytes) = payload { + let payload_coordinate = RetainedEvidenceCoordinate::new( + contract.clone(), + RetainedEvidenceRole::ReadingPayload, + query_identity.reading_id, + ); + postures.push(RetainedEvidencePosture::missing_content( + &RetainedEvidenceRef::new( + payload_coordinate, + retained_evidence_content_hash(bytes), + bytes.len() as u64, + ), + )); + } + + postures + } + fn query_basis_digest( resolved: &ResolvedObservationCoordinate, parent_basis_posture: &ObservationBasisPosture, @@ -3351,6 +3500,10 @@ mod tests { ObservationService::observe(&runtime, &provenance, &engine, request.clone()).unwrap(); assert_eq!(artifact.artifact_hash, same.artifact_hash); + assert!( + artifact.reading.retained_evidence.is_empty(), + "direct observer registration lacks installed package evidence" + ); assert_eq!( artifact.reading.observer_plan, ReadingObserverPlan::Authored { @@ -3366,20 +3519,40 @@ mod tests { ReadingResidualPosture::Complete ); assert_eq!(artifact.projection, request.projection); + let mut expected = 9_001_u32.to_le_bytes().to_vec(); + expected.extend_from_slice(b"counter=read"); + expected.extend_from_slice( + &artifact + .resolved + .resolved_worldline_tick + .as_u64() + .to_le_bytes(), + ); assert_eq!( artifact.payload, - ObservationPayload::QueryBytes({ - let mut expected = 9_001_u32.to_le_bytes().to_vec(); - expected.extend_from_slice(b"counter=read"); - expected.extend_from_slice( - &artifact - .resolved - .resolved_worldline_tick - .as_u64() - .to_le_bytes(), - ); - expected - }) + ObservationPayload::QueryBytes(expected.clone()) + ); + let abi = artifact.to_abi(); + assert!(abi.reading.retained_evidence.is_empty()); + assert!( + ObservationService::observe( + &runtime, + &provenance, + &engine, + builtin_one_shot( + ObservationCoordinate { + worldline_id, + at: ObservationAt::Frontier, + }, + ObservationFrame::CommitBoundary, + ObservationProjection::Head, + ), + ) + .unwrap() + .reading + .retained_evidence + .is_empty(), + "non-contract readings must not invent retained evidence refs" ); } @@ -3476,14 +3649,14 @@ mod tests { assert_ne!(baseline.artifact_hash, different_query.artifact_hash); assert_ne!(baseline.artifact_hash, different_basis.artifact_hash); assert_ne!(baseline.artifact_hash, different_schema.artifact_hash); - assert_eq!(baseline.resolved.observation_version, 3); + assert_eq!(baseline.resolved.observation_version, 4); assert_eq!( baseline.artifact_hash, - observation_artifact_hash_with_domain(&baseline, b"echo:observation-artifact:v3\0") + observation_artifact_hash_with_domain(&baseline, b"echo:observation-artifact:v4\0") ); assert_ne!( baseline.artifact_hash, - observation_artifact_hash_with_domain(&baseline, b"echo:observation-artifact:v2\0") + observation_artifact_hash_with_domain(&baseline, b"echo:observation-artifact:v3\0") ); let baseline_identity = baseline diff --git a/crates/warp-core/src/optic/tests.rs b/crates/warp-core/src/optic/tests.rs index cb5e0a81..bc29577c 100644 --- a/crates/warp-core/src/optic/tests.rs +++ b/crates/warp-core/src/optic/tests.rs @@ -122,6 +122,7 @@ fn reading_envelope() -> ReadingEnvelope { observer_basis: ReadingObserverBasis::CommitBoundary, contract: None, query_identity: None, + retained_evidence: Vec::new(), witness_refs: vec![ReadingWitnessRef::ResolvedCommit { reference: provenance(1, 2), }], diff --git a/crates/warp-core/src/retained_evidence.rs b/crates/warp-core/src/retained_evidence.rs index f2a86b60..5965a8ef 100644 --- a/crates/warp-core/src/retained_evidence.rs +++ b/crates/warp-core/src/retained_evidence.rs @@ -120,9 +120,16 @@ impl RetainedEvidenceRef { /// Stable id that binds semantic coordinate, content hash, and byte length. #[must_use] pub fn evidence_ref_id(&self) -> Hash { + let coordinate_id = self.coordinate.coordinate_id(); + self.evidence_ref_id_with_coordinate_id(&coordinate_id) + } + + /// Stable ref id when the caller already has the coordinate id. + #[must_use] + pub(crate) fn evidence_ref_id_with_coordinate_id(&self, coordinate_id: &Hash) -> Hash { let mut hasher = Hasher::new(); hasher.update(RETAINED_EVIDENCE_REF_ID_DOMAIN); - hasher.update(&self.coordinate.coordinate_id()); + hasher.update(coordinate_id); hasher.update(&self.content_hash); hasher.update(&self.byte_len.to_le_bytes()); hasher.finalize().into() @@ -141,8 +148,20 @@ impl RetainedEvidenceRef { pub enum RetainedEvidencePosture { /// The retained evidence is locally available. Available(RetainedEvidenceRef), - /// Required retained evidence is missing. - MissingRetention(ContractObstruction), + /// No retained descriptor exists for this semantic coordinate. + MissingCoordinate { + /// Coordinate that should eventually name retained evidence. + coordinate: RetainedEvidenceCoordinate, + /// Typed missing-retention obstruction for the coordinate. + obstruction: ContractObstruction, + }, + /// A descriptor is known, but its retained bytes are unavailable. + MissingContent { + /// Retained evidence reference whose bytes are unavailable. + reference: RetainedEvidenceRef, + /// Typed missing-retention obstruction for the exact reference. + obstruction: ContractObstruction, + }, } impl RetainedEvidencePosture { @@ -156,13 +175,19 @@ impl RetainedEvidencePosture { /// descriptor. #[must_use] pub fn missing_coordinate(coordinate: &RetainedEvidenceCoordinate) -> Self { - Self::MissingRetention(coordinate.missing_retention_obstruction()) + Self::MissingCoordinate { + coordinate: coordinate.clone(), + obstruction: coordinate.missing_retention_obstruction(), + } } /// Builds a missing posture for a descriptor whose content bytes are absent. #[must_use] pub fn missing_content(reference: &RetainedEvidenceRef) -> Self { - Self::MissingRetention(reference.missing_retention_obstruction()) + Self::MissingContent { + reference: reference.clone(), + obstruction: reference.missing_retention_obstruction(), + } } /// Returns the missing-retention obstruction when this posture obstructs. @@ -170,7 +195,8 @@ impl RetainedEvidencePosture { pub fn obstruction(&self) -> Option<&ContractObstruction> { match self { Self::Available(_) => None, - Self::MissingRetention(obstruction) => Some(obstruction), + Self::MissingCoordinate { obstruction, .. } + | Self::MissingContent { obstruction, .. } => Some(obstruction), } } } diff --git a/crates/warp-core/tests/installed_contract_intent_pipeline_tests.rs b/crates/warp-core/tests/installed_contract_intent_pipeline_tests.rs index 0f31672d..726fedd8 100644 --- a/crates/warp-core/tests/installed_contract_intent_pipeline_tests.rs +++ b/crates/warp-core/tests/installed_contract_intent_pipeline_tests.rs @@ -18,10 +18,10 @@ use warp_core::{ ObservationCoordinate, ObservationFrame, ObservationPayload, ObservationProjection, ObservationReadBudget, ObservationRequest, ObservationService, ObserverPlanId, OpticAdmissionTicket, OpticArtifactHandle, PatternGraph, PlaybackMode, ProvenanceService, - ReadingBudgetPosture, ReceiptCorrelationRecord, RuntimeError, SchedulerCoordinator, - SchedulerKind, TickDelta, TickReceiptRejection, TicketedRuntimeIngressAuthority, WorldlineId, - WorldlineRuntime, WorldlineState, WriterHead, WriterHeadKey, OPTIC_ADMISSION_TICKET_KIND, - OPTIC_ARTIFACT_HANDLE_KIND, + ReadingBudgetPosture, ReceiptCorrelationRecord, RetainedEvidencePosture, RetainedEvidenceRole, + RuntimeError, SchedulerCoordinator, SchedulerKind, TickDelta, TickReceiptRejection, + TicketedRuntimeIngressAuthority, WorldlineId, WorldlineRuntime, WorldlineState, WriterHead, + WriterHeadKey, OPTIC_ADMISSION_TICKET_KIND, OPTIC_ARTIFACT_HANDLE_KIND, }; const SCHEMA_SHA256_HEX: &str = "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"; @@ -973,6 +973,32 @@ fn external_contract_fixture_proves_mutation_query_retention_and_replay() { .contract .as_ref() .expect("external fixture reading must carry contract evidence"); + let expected_payload_hash: [u8; 32] = blake3::hash(&query_payload).into(); + assert_eq!(reading.reading.retained_evidence.len(), 2); + assert!(matches!( + &reading.reading.retained_evidence[0], + RetainedEvidencePosture::MissingCoordinate { + coordinate, + obstruction, + } if coordinate.role == RetainedEvidenceRole::ReadingEnvelope + && coordinate.contract == *query_contract + && coordinate.semantic_digest == query_identity.reading_id + && obstruction.kind == warp_core::ContractObstructionKind::MissingRetention + )); + assert!(matches!( + &reading.reading.retained_evidence[1], + RetainedEvidencePosture::MissingContent { + reference, + obstruction, + } if reference.coordinate.role == RetainedEvidenceRole::ReadingPayload + && reference.coordinate.contract == *query_contract + && reference.coordinate.semantic_digest == query_identity.reading_id + && reference.content_hash == expected_payload_hash + && reference.byte_len == query_payload.len() as u64 + && obstruction.kind == warp_core::ContractObstructionKind::MissingRetention + )); + let abi_reading = reading.to_abi(); + assert_eq!(abi_reading.reading.retained_evidence.len(), 2); let reading_coord = semantic_coordinate( query_contract, RetainedBlobRole::ReadingPayload, diff --git a/crates/warp-wasm/README.md b/crates/warp-wasm/README.md index a9302204..936505a4 100644 --- a/crates/warp-wasm/README.md +++ b/crates/warp-wasm/README.md @@ -13,14 +13,15 @@ See the repository root `README.md` for the full overview. Echo’s deterministic wire protocol can be used from JavaScript/TypeScript in web-based tools and playgrounds. - Exposes the current observation-first and intent-shaped control surface - (`ABI_VERSION` 11 in `echo-wasm-abi`): `observe(...)` is the only public + (`ABI_VERSION` 12 in `echo-wasm-abi`): `observe(...)` is the only public world-state read export, `scheduler_status()` is the read-only scheduler metadata export, and `dispatch_intent(...)` is application intent ingress. Accepted application ingress returns witnessed submission identity in addition to canonical intent identity. The current ABI also publishes strand settlement comparison, planning, execution entrypoints, settlement basis evidence, overlap revalidation - evidence, and read-side basis plus residual posture on observation artifacts. + evidence, read-side basis plus residual posture, and generic retained + evidence posture on observation artifacts. - The engine-backed boundary uses logical clocks only: `WorldlineTick` is per-worldline append identity and `GlobalTick` is runtime cycle correlation metadata. No wall-clock time enters Echo internals. diff --git a/crates/warp-wasm/src/lib.rs b/crates/warp-wasm/src/lib.rs index febfeb68..69d4c53b 100644 --- a/crates/warp-wasm/src/lib.rs +++ b/crates/warp-wasm/src/lib.rs @@ -918,6 +918,7 @@ mod init_tests { observer_basis: ReadingObserverBasis::CommitBoundary, contract: None, query_identity: None, + retained_evidence: Vec::new(), witness_refs: vec![ReadingWitnessRef::EmptyFrontier { worldline_id: WorldlineId::from_bytes([9; 32]), state_root: head.state_root.clone(), diff --git a/docs/BEARING.md b/docs/BEARING.md index a450f33d..6638adf3 100644 --- a/docs/BEARING.md +++ b/docs/BEARING.md @@ -24,9 +24,15 @@ Trusted runtime-control history is defined in The causal WAL doctrine and recovery design is defined in `docs/design/causal-wal-end-to-end.md`. +The WAL/WSC storage relationship is tracked in +`docs/method/backlog/v0.1.0/PLATFORM_wal-wsc-storage-relationship.md`. + The next ten jedit release-gate slices are planned in `docs/design/v0.1.0-jedit-next-ten-slices.md`. +The current sequencing filter for audited work items is maintained in +`docs/design/work-item-sequencing-and-prioritization.md`. + The filesystem lane for release-bar backlog cards is `docs/method/backlog/v0.1.0/`. @@ -81,6 +87,10 @@ crash-recoverable without giving applications tick or WAL authority. observers keyed by generated query op id. Observers receive canonical vars bytes and the resolved causal basis, emit `QueryBytes`, and stamp the `ReadingEnvelope` with authored observer plan identity. +- App-safe observation and WASM ABI surfaces carry generic retained-evidence + posture for installed contract QueryView readings. The envelope names + missing reading-envelope coordinates and missing reading-payload content + refs without exposing trusted runtime control or importing application nouns. - `echo-wesley-gen --contract-host` emits std-only query observer helpers for that seam: deterministic authored observer plan identity, typed context-vars decoders, and read-only observer constructors that install host closures into @@ -107,8 +117,9 @@ crash-recoverable without giving applications tick or WAL authority. per-intent applied/rejected semantics. - Contract-aware obstruction taxonomy and product-facing error surfaces still need release-grade stabilization. -- The semantic retention layer is local and in-memory; durable retained - artifact, witness, receipt, and reading recovery remains future work. +- The semantic retention layer is local and in-memory. App-safe readings can + now report generic missing-retention posture, but durable retained artifact, + witness, receipt, and reading recovery remains future work. - Generic external contract proof exists, but the release gate now requires real `jedit` follow-through from the sibling repository. jedit now has a local app/host split for its opt-in real-WASM witness; the remaining gap is diff --git a/docs/WorkItems.md b/docs/WorkItems.md new file mode 100644 index 00000000..7ba325d5 --- /dev/null +++ b/docs/WorkItems.md @@ -0,0 +1,307 @@ + + + +# Work Items + +Last audited: 2026-05-25. + +This is an inventory, not a replacement for the repo's planning system. When +there is disagreement, prefer the specific backlog card, design packet, issue, +or executable test over this summary. + +Sources checked during this audit: + +- [`docs/BEARING.md`](BEARING.md) +- [`docs/method/backlog/`](method/backlog/) +- [`backlog/`](../backlog/) +- open GitHub issues in `flyingrobots/echo` + +## Summary + +| Source | Open count | Notes | +| --------------------------------- | ---------: | ------------------------------------------ | +| `docs/method/backlog/asap/` | 13 | Immediate filesystem backlog. | +| `docs/method/backlog/v0.1.0/` | 21 | Release-bar lane. | +| `docs/method/backlog/up-next/` | 39 | Planned follow-on work. | +| `docs/method/backlog/inbox/` | 16 | Triage queue and older issue mirrors. | +| `docs/method/backlog/bad-code/` | 3 | Known local structural debt. | +| `docs/method/backlog/cool-ideas/` | 29 | Deliberately not current release work. | +| `backlog/bad-code/` | 5 | Older RE-series debt cards still present. | +| `backlog/cool-ideas/` | 2 | Older CI-series idea cards still present. | +| GitHub open issues | 46 | After closing completed `#281` and `#285`. | + +## Current Execution Gravity + +The active direction remains: + +```text +prove Echo with jedit as a real external app +without moving app nouns into Echo +and without giving application code tick, WAL, or trusted runtime authority +``` + +Current active signposts: + +- Echo's release feature bar: + [`docs/design/v0.1.0-release-plan.md`](design/v0.1.0-release-plan.md) +- sequencing and prioritization filter: + [`docs/design/work-item-sequencing-and-prioritization.md`](design/work-item-sequencing-and-prioritization.md) +- jedit external release gate: + [`docs/design/v0.1.0-jedit-release-gate.md`](design/v0.1.0-jedit-release-gate.md) +- next ten jedit/Echo release-gate slices: + [`docs/design/v0.1.0-jedit-next-ten-slices.md`](design/v0.1.0-jedit-next-ten-slices.md) +- causal WAL doctrine: + [`docs/design/causal-wal-end-to-end.md`](design/causal-wal-end-to-end.md) + +Progress bars from the current work stream: + +```text +[#####-----] Echo/jedit retained-evidence release-gate batch [5/10 slices] +[#####-----] PR checkpoint batch [5/5 slices before recommended PR] +[##########] Echo WAL truth boundary and runtime ACK plumbing [95/95 slices] +``` + +Immediate unfinished slice in the current batch: + +- Slice 6: Echo product-facing intent outcome API polish. +- Slice 7: jedit generated structural-history request path. +- Slice 8: generated package install path. +- Slice 9: jedit real mutation and query round trip. +- Slice 10: jedit non-happy path and release-gate report. + +## Known Cross-Repo And Storage Doctrine Gaps + +This inventory is Echo-local unless a row explicitly names another repository. +The following mission-critical gaps were not fully represented by the current +backlog lanes when this audit started: + +- `[Echo][jedit]` WSC causal-history persistence, export, and recovery. +- `[Echo]` WAL/WSC storage relationship and recovery authority. +- `[warp-ttd][Echo]` WAL-backed causal commit evidence read model. +- `[Echo]` JS/WASM/browser client release surface. +- `[Echo][Wesley]` package publish, generated package compatibility, and + versioning. +- `[Echo][Graft][Think]` post-jedit application portability checklist. +- `[Echo]` retained evidence posture versus durable recovery evidence. + +The Echo-owned follow-up cards are now in the `v0.1.0` lane: + +- [WAL/WSC Storage Relationship](method/backlog/v0.1.0/PLATFORM_wal-wsc-storage-relationship.md) +- [WSC Causal-History Storage](method/backlog/v0.1.0/PLATFORM_wsc-causal-history-storage.md) +- [Retained Evidence Durability Boundary](method/backlog/v0.1.0/PLATFORM_retained-evidence-durability-boundary.md) +- [JS/WASM/Browser Client Release Surface](method/backlog/v0.1.0/PLATFORM_js-wasm-browser-client-release-surface.md) +- [Package Publish And Versioning](method/backlog/v0.1.0/RELEASE_package-publish-and-versioning.md) + +## ASAP Backlog + +- [Docs cleanup](method/backlog/asap/DOCS_docs-cleanup.md) +- [Echo and git-warp compatibility sanity check](method/backlog/asap/KERNEL_echo-git-warp-compatibility-sanity-check.md) +- [Deterministic Trig Oracle](method/backlog/asap/MATH_deterministic-trig.md) +- [CI det-policy hardening](method/backlog/asap/PLATFORM_ci-det-policy-hardening.md) +- [CLI Scaffold (#47)](method/backlog/asap/PLATFORM_cli-scaffold.md) +- [Contract-Hosted File History Substrate](method/backlog/asap/PLATFORM_contract-hosted-file-history-substrate.md) +- [Contract QueryView Observer Bridge](method/backlog/asap/PLATFORM_contract-queryview-observer-bridge.md) +- [Explicit negative test mapping for decoder controls](method/backlog/asap/PLATFORM_decoder-negative-test-map.md) +- [Echo Contract Hosting Roadmap](method/backlog/asap/PLATFORM_echo-contract-hosting-roadmap.md) +- [Installed Wesley Contract Host Dispatch](method/backlog/asap/PLATFORM_installed-wesley-contract-host-dispatch.md) +- [Commit-ordered rollback playbooks for TTD integration](method/backlog/asap/PLATFORM_ttd-rollback-playbooks.md) +- [Reconcile TTD protocol schemas with warp-ttd](method/backlog/asap/PLATFORM_ttd-schema-reconciliation.md) +- [Wesley Compiled Contract Hosting Doctrine](method/backlog/asap/PLATFORM_wesley-compiled-contract-hosting-doctrine.md) + +## v0.1.0 Lane + +- [Release-Grade Quickstart](method/backlog/v0.1.0/DOCS_release-grade-quickstart.md) +- [Contract-Aware Receipts And Readings](method/backlog/v0.1.0/KERNEL_contract-aware-receipts-and-readings.md) +- [Contract Obstruction Taxonomy](method/backlog/v0.1.0/KERNEL_contract-obstruction-taxonomy.md) +- [Contract Reading Identity And Bounded Payloads](method/backlog/v0.1.0/KERNEL_contract-reading-identity-and-bounded-payloads.md) +- [Witnessed Intent Submission Persistence](method/backlog/v0.1.0/KERNEL_witnessed-intent-submission-persistence.md) +- [App-Safe Client Surface](method/backlog/v0.1.0/PLATFORM_app-safe-client-surface.md) +- [Contract Artifact Retention In echo-cas](method/backlog/v0.1.0/PLATFORM_contract-artifact-retention-in-echo-cas.md) +- [Contract Retention And Semantic Lookup Seams](method/backlog/v0.1.0/PLATFORM_contract-retention-and-semantic-lookup-seams.md) +- [External Contract Proof Fixture](method/backlog/v0.1.0/PLATFORM_external-contract-proof-fixture.md) +- [JS/WASM/Browser Client Release Surface](method/backlog/v0.1.0/PLATFORM_js-wasm-browser-client-release-surface.md) +- [jedit Real Echo Release Gate](method/backlog/v0.1.0/PLATFORM_jedit-real-echo-release-gate.md) +- [Package Publish And Versioning](method/backlog/v0.1.0/RELEASE_package-publish-and-versioning.md) +- [Product-Facing Intent Outcome API](method/backlog/v0.1.0/PLATFORM_product-facing-intent-outcome-api.md) +- [Reference Trusted Runtime Host Loop](method/backlog/v0.1.0/PLATFORM_reference-trusted-runtime-host-loop.md) +- [Retained Evidence Durability Boundary](method/backlog/v0.1.0/PLATFORM_retained-evidence-durability-boundary.md) +- [Versioned Contract And API Compatibility](method/backlog/v0.1.0/PLATFORM_versioned-contract-api-compatibility.md) +- [WAL/WSC Storage Relationship](method/backlog/v0.1.0/PLATFORM_wal-wsc-storage-relationship.md) +- [WSC Causal-History Storage](method/backlog/v0.1.0/PLATFORM_wsc-causal-history-storage.md) +- [v0.1.0 Release Candidate](method/backlog/v0.1.0/RELEASE_v0.1.0-release-candidate.md) +- [Authority Boundary Audit](method/backlog/v0.1.0/SECURITY_authority-boundary-audit.md) +- [v0.1.0 Replay And DIND Proof](method/backlog/v0.1.0/TEST_v0.1.0-replay-dind-proof.md) + +## Up Next + +- [KERNEL - Admission Outcome Family](method/backlog/up-next/KERNEL_admission-outcome-family.md) +- [KERNEL - Bounded Site and Admission Policy](method/backlog/up-next/KERNEL_bounded-site-and-admission-policy.md) +- [KERNEL - Braid and Settlement Admission Unification](method/backlog/up-next/KERNEL_braid-settlement-admission-unification.md) +- [Compliance reporting as a TTD protocol extension](method/backlog/up-next/KERNEL_compliance-protocol-envelope.md) +- [Contract Inverse Admission Hook](method/backlog/up-next/KERNEL_contract-inverse-admission-hook.md) +- [Contract Strands And Counterfactuals](method/backlog/up-next/KERNEL_contract-strands-and-counterfactuals.md) +- [KERNEL - Determinism escape hatches audit and closure](method/backlog/up-next/KERNEL_determinism-escape-hatches.md) +- [Dynamic Footprint Binding Runtime](method/backlog/up-next/KERNEL_dynamic-footprint-binding-runtime.md) +- [Generic Contract Braid Substrate](method/backlog/up-next/KERNEL_generic-contract-braid-substrate.md) +- [Intent-Only Contract Runtime Mutations](method/backlog/up-next/KERNEL_intent-only-contract-runtime-mutations.md) +- [SHA-256 to BLAKE3 Coordination](method/backlog/up-next/KERNEL_sha256-blake3.md) +- [Strand Runtime Graph Ontology](method/backlog/up-next/KERNEL_strand-runtime-graph-ontology.md) +- [Security/capabilities for fork/rewind/merge](method/backlog/up-next/KERNEL_time-travel-capabilities.md) +- [WARP optic boundary audit for topology and history operations](method/backlog/up-next/KERNEL_topology-mutation-intent-boundary-audit.md) +- [Authenticated Wesley Intent Admission Posture](method/backlog/up-next/PLATFORM_authenticated-wesley-intent-admission-posture.md) +- [Braid and settlement Intent paths](method/backlog/up-next/PLATFORM_braid-settlement-intent-paths.md) +- [In-Browser Visualization](method/backlog/up-next/PLATFORM_browser-visualization.md) +- [PLATFORM - Continuum admission family cutover](method/backlog/up-next/PLATFORM_continuum-admission-family-cutover.md) +- [Continuum Proof Family Runtime Cutover](method/backlog/up-next/PLATFORM_continuum-proof-family-runtime-cutover.md) +- [Add an explicit Echo CLI and MCP agent surface](method/backlog/up-next/PLATFORM_echo-agent-surface-cli-and-mcp.md) +- [echo-cas JS Bindings](method/backlog/up-next/PLATFORM_echo-cas-js-bindings.md) +- [Echo / git-warp witnessed suffix sync](method/backlog/up-next/PLATFORM_echo-git-warp-witnessed-suffix-sync.md) +- [Split echo-session-proto into retained bridge contracts vs legacy transport residue](method/backlog/up-next/PLATFORM_echo-session-proto-split.md) +- [Footprint Honesty Rewrite Proof Slice](method/backlog/up-next/PLATFORM_footprint-honesty-rewrite-proof-slice.md) +- [Graft Live Frontier Structural Readings](method/backlog/up-next/PLATFORM_graft-live-frontier-structural-readings.md) +- [Import outcome idempotence and loop law](method/backlog/up-next/PLATFORM_import-outcome-idempotence-and-loop-law.md) +- [Import outcome retention and novelty index](method/backlog/up-next/PLATFORM_import-outcome-retention-novelty-index.md) +- [Inverse operation Intent path](method/backlog/up-next/PLATFORM_inverse-operation-intent-path.md) +- [jedit Optic Intent / Observation Handoff](method/backlog/up-next/PLATFORM_jedit-hot-text-runtime-host-surface.md) +- [jedit Text Contract Hosting MVP](method/backlog/up-next/PLATFORM_jedit-text-contract-mvp.md) +- [Triage METHOD drift against ~/git/method](method/backlog/up-next/PLATFORM_method-sync-and-doctor-triage.md) +- [PLATFORM - Neighborhood publication stack documentation](method/backlog/up-next/PLATFORM_neighborhood-publication-stack.md) +- [Strand and support Intent paths](method/backlog/up-next/PLATFORM_strand-and-support-intent-paths.md) +- [WASM Runtime Integration](method/backlog/up-next/PLATFORM_wasm-runtime.md) +- [Wesley Footprint Honesty Artifact Attestation](method/backlog/up-next/PLATFORM_wesley-footprint-honesty-artifact-attestation.md) +- [Wesley Go Public](method/backlog/up-next/PLATFORM_wesley-go-public.md) +- [Wesley Migration Planning Phase B](method/backlog/up-next/PLATFORM_wesley-migration.md) +- [Wesley QIR Phase C](method/backlog/up-next/PLATFORM_wesley-qir-phase-c.md) +- [Wesley Type Pipeline in Browser](method/backlog/up-next/PLATFORM_wesley-type-pipeline-browser.md) + +## Inbox + +- [Wesley Docs](method/backlog/inbox/DOCS_wesley-docs.md) +- [Deterministic Rhai](method/backlog/inbox/KERNEL_deterministic-rhai.md) +- [First-class invariant documents](method/backlog/inbox/KERNEL_invariants-as-docs.md) +- [Security](method/backlog/inbox/KERNEL_security.md) +- [ABI nested evidence strictness](method/backlog/inbox/PLATFORM_abi-nested-evidence-strictness.md) +- [Editor Hot-Reload](method/backlog/inbox/PLATFORM_editor-hot-reload.md) +- [git-mind NEXUS](method/backlog/inbox/PLATFORM_git-mind-nexus.md) +- [Importer](method/backlog/inbox/PLATFORM_importer.md) +- [Legend progress in method status](method/backlog/inbox/PLATFORM_method-status-legend-progress.md) +- [Reconcile Relocated Wesley Echo Schemas](method/backlog/inbox/PLATFORM_reconcile-relocated-wesley-echo-schemas.md) +- [Runtime-Owned Footprint Directive Migration](method/backlog/inbox/PLATFORM_runtime-owned-footprint-directive-migration.md) +- [Signing Pipeline](method/backlog/inbox/PLATFORM_signing-pipeline.md) +- [Tooling & Misc](method/backlog/inbox/PLATFORM_tooling-misc.md) +- [TTD Hardening & Future](method/backlog/inbox/PLATFORM_ttd-hardening.md) +- [Wesley Boundary Grammar](method/backlog/inbox/PLATFORM_wesley-boundary-grammar.md) +- [Wesley Future](method/backlog/inbox/PLATFORM_wesley-future.md) + +## Known Bad Code + +- [RED/GREEN can't be separate commits](method/backlog/bad-code/red-green-lint-friction.md) +- [WASM control intent authority boundary is too implicit](method/backlog/bad-code/wasm-control-intent-authority-boundary.md) +- [xtask main.rs is a god file](method/backlog/bad-code/xtask-god-file.md) +- [RE-028 — Merkle-Tree Memoization in Snapshot Accumulator](../backlog/bad-code/RE-028-snapshot-accumulator-memoization.md) +- [RE-029 — Enforce det_fixed by Default](../backlog/bad-code/RE-029-concurrent-snapshot-fetching.md) +- [RE-030 — Converge QueryView Reads onto Optics](../backlog/bad-code/RE-030-queryview-optic-convergence.md) +- [RE-031 Capability Grant Validation Admission Integration](../backlog/bad-code/RE-031-capability-grant-validation-admission-integration.md) +- [RE-032: Publish Durable Scheduler Fault Evidence](../backlog/bad-code/RE-032-durable-scheduler-fault-evidence.md) + +## Cool Ideas + +- [Enforce Echo design vocabulary](method/backlog/cool-ideas/DOCS_glossary-enforcement.md) +- [Course Material](method/backlog/cool-ideas/DOCS_splash-guy-course-material.md) +- [Course Material](method/backlog/cool-ideas/DOCS_tumble-tower-course-material.md) +- [Expose parallel execution counterfactuals](method/backlog/cool-ideas/KERNEL_parallel-execution-counterfactuals.md) +- [TT3 — Rulial Diff](method/backlog/cool-ideas/KERNEL_rulial-diff.md) +- [Controlled Desync](method/backlog/cool-ideas/KERNEL_splash-guy-controlled-desync.md) +- [Lockstep Protocol](method/backlog/cool-ideas/KERNEL_splash-guy-lockstep-protocol.md) +- [Rules & State Model](method/backlog/cool-ideas/KERNEL_splash-guy-rules-and-state.md) +- [TT2 — Time Travel MVP](method/backlog/cool-ideas/KERNEL_time-travel-mvp.md) +- [Desync Breakers](method/backlog/cool-ideas/KERNEL_tumble-tower-desync-breakers.md) +- [Lockstep Harness](method/backlog/cool-ideas/KERNEL_tumble-tower-lockstep-harness.md) +- [Worldline Convergence Suite](method/backlog/cool-ideas/KERNEL_worldline-convergence.md) +- [Stage 0: AABB](method/backlog/cool-ideas/MATH_tumble-tower-stage-0-aabb.md) +- [Stage 1: Rotation](method/backlog/cool-ideas/MATH_tumble-tower-stage-1-rotation.md) +- [Stage 2: Friction](method/backlog/cool-ideas/MATH_tumble-tower-stage-2-friction.md) +- [Stage 3: Sleeping](method/backlog/cool-ideas/MATH_tumble-tower-stage-3-sleeping.md) +- [Continuum Contract Artifact Interchange](method/backlog/cool-ideas/PLATFORM_continuum-contract-artifact-interchange.md) +- [Cross-repo METHOD dashboard](method/backlog/cool-ideas/PLATFORM_cross-repo-method-dashboard.md) +- [API Evolution](method/backlog/cool-ideas/PLATFORM_deep-storage-api-evolution.md) +- [DiskTier](method/backlog/cool-ideas/PLATFORM_deep-storage-disk-tier.md) +- [GC Sweep & Eviction](method/backlog/cool-ideas/PLATFORM_deep-storage-gc-sweep-eviction.md) +- [Wire Protocol](method/backlog/cool-ideas/PLATFORM_deep-storage-wire-protocol.md) +- [Extract method crate to its own repo](method/backlog/cool-ideas/PLATFORM_method-crate-extract.md) +- [Method drift check as pre-push hook](method/backlog/cool-ideas/PLATFORM_method-drift-as-pre-push-hook.md) +- [Proof-Carrying Apertures](method/backlog/cool-ideas/PLATFORM_proof-carrying-apertures.md) +- [Reading envelope inspector](method/backlog/cool-ideas/PLATFORM_reading-envelope-inspector.md) +- [Visualization](method/backlog/cool-ideas/PLATFORM_splash-guy-visualization.md) +- [Visualization](method/backlog/cool-ideas/PLATFORM_tumble-tower-visualization.md) +- [WARPDrive POSIX Materialization Optic](method/backlog/cool-ideas/PLATFORM_warpdrive-posix-optic.md) +- [CI-001 — Causal "Multiverse" Puzzle Engine](../backlog/cool-ideas/CI-001-causal-puzzle-engine.md) +- [CI-002 — Deterministic Rule Profiling (Flamegraphs)](../backlog/cool-ideas/CI-002-deterministic-flamegraphs.md) + +## Open GitHub Issues + +| Issue | Title | Recommendation | +| ------------------------------------------------------- | ---------------------------------------------------------------- | ------------------------------------------------------------ | +| [#370](https://github.com/flyingrobots/echo/issues/370) | Track Echo v0.1.0 release bar | Keep open; top-level release tracker. | +| [#286](https://github.com/flyingrobots/echo/issues/286) | CI: Add unit tests for `classify_changes.cjs` and `matches()` | Keep open; current CI-hardening follow-up. | +| [#284](https://github.com/flyingrobots/echo/issues/284) | CI: Per-crate gate overrides in det-policy classification system | Keep open; current CI-hardening follow-up. | +| [#282](https://github.com/flyingrobots/echo/issues/282) | Commit-ordered rollback playbooks for TTD integration | Keep open; mirrored by ASAP rollback-playbook card. | +| [#279](https://github.com/flyingrobots/echo/issues/279) | Explicit negative test mapping for decoder controls | Keep open; partially implemented but exhaustiveness remains. | +| [#246](https://github.com/flyingrobots/echo/issues/246) | TT1: Security/capabilities for fork/rewind/merge in multiplayer | Keep open; capability policy remains future work. | +| [#239](https://github.com/flyingrobots/echo/issues/239) | Tooling: Reliving debugger UX | Keep open; mirrored in inbox/task DAG. | +| [#238](https://github.com/flyingrobots/echo/issues/238) | Demo 3: Tumble Tower docs course | Keep open as demo/course idea work. | +| [#237](https://github.com/flyingrobots/echo/issues/237) | Demo 3: Tumble Tower visualization | Keep open as demo/course idea work. | +| [#236](https://github.com/flyingrobots/echo/issues/236) | Demo 3: Tumble Tower controlled desync breakers | Keep open as demo/course idea work. | +| [#235](https://github.com/flyingrobots/echo/issues/235) | Demo 3: Tumble Tower lockstep harness | Keep open as demo/course idea work. | +| [#234](https://github.com/flyingrobots/echo/issues/234) | Demo 3: Tumble Tower Stage 3 physics | Keep open as demo/course idea work. | +| [#233](https://github.com/flyingrobots/echo/issues/233) | Demo 3: Tumble Tower Stage 2 physics | Keep open as demo/course idea work. | +| [#232](https://github.com/flyingrobots/echo/issues/232) | Demo 3: Tumble Tower Stage 1 physics | Keep open as demo/course idea work. | +| [#231](https://github.com/flyingrobots/echo/issues/231) | Demo 3: Tumble Tower Stage 0 physics | Keep open as demo/course idea work. | +| [#226](https://github.com/flyingrobots/echo/issues/226) | Demo 2: Splash Guy docs course | Keep open as demo/course idea work. | +| [#225](https://github.com/flyingrobots/echo/issues/225) | Demo 2: Splash Guy visualization | Keep open as demo/course idea work. | +| [#224](https://github.com/flyingrobots/echo/issues/224) | Demo 2: Splash Guy controlled desync lessons | Keep open as demo/course idea work. | +| [#223](https://github.com/flyingrobots/echo/issues/223) | Demo 2: Splash Guy lockstep harness | Keep open as demo/course idea work. | +| [#222](https://github.com/flyingrobots/echo/issues/222) | Demo 2: Splash Guy deterministic rules and state model | Keep open as demo/course idea work. | +| [#207](https://github.com/flyingrobots/echo/issues/207) | Backlog: Run noisy-line test for naming | Keep open; mirrored in tooling/misc inbox. | +| [#205](https://github.com/flyingrobots/echo/issues/205) | TT2: Reliving debugger MVP | Keep open; time-travel/debugger work remains future. | +| [#204](https://github.com/flyingrobots/echo/issues/204) | TT3: Provenance heatmap | Keep open; future debugger/provenance idea. | +| [#202](https://github.com/flyingrobots/echo/issues/202) | Spec: Provenance Payload v1 | Keep open; mirrored in security inbox. | +| [#199](https://github.com/flyingrobots/echo/issues/199) | TT3: Wesley worldline diff | Keep open; future diff tool. | +| [#198](https://github.com/flyingrobots/echo/issues/198) | W1: Provenance as query semantics | Keep open; mirrored in Wesley boundary grammar work. | +| [#195](https://github.com/flyingrobots/echo/issues/195) | Backlog: JS-ABI packet checksum v2 | Keep open; protocol-version follow-up. | +| [#194](https://github.com/flyingrobots/echo/issues/194) | W1: SchemaDelta vocabulary | Keep open; mirrored in Wesley boundary grammar work. | +| [#193](https://github.com/flyingrobots/echo/issues/193) | W1: Schema hash chain pinning | Keep open; mirrored in Wesley boundary grammar work. | +| [#190](https://github.com/flyingrobots/echo/issues/190) | M4: Determinism torture harness | Keep open; still useful as broader stress suite. | +| [#187](https://github.com/flyingrobots/echo/issues/187) | M4: Worldline convergence property suite | Keep open; generalized convergence suite remains open. | +| [#174](https://github.com/flyingrobots/echo/issues/174) | W1: Wesley as a boundary grammar | Keep open; mirrored in Wesley boundary grammar work. | +| [#173](https://github.com/flyingrobots/echo/issues/173) | S1: Deterministic Rhai surface | Keep open; mirrored in deterministic Rhai inbox. | +| [#172](https://github.com/flyingrobots/echo/issues/172) | TT3: Rulial diff / worldline compare MVP | Keep open; future compare tooling. | +| [#171](https://github.com/flyingrobots/echo/issues/171) | TT2: Time Travel MVP | Keep open; time-travel core remains future. | +| [#79](https://github.com/flyingrobots/echo/issues/79) | Docs/logging | Keep open; mirrored in tooling/misc inbox. | +| [#76](https://github.com/flyingrobots/echo/issues/76) | File watcher/debounce | Keep open; mirrored in hot-reload inbox. | +| [#75](https://github.com/flyingrobots/echo/issues/75) | Draft hot-reload spec | Keep open; mirrored in hot-reload inbox. | +| [#36](https://github.com/flyingrobots/echo/issues/36) | CI: verify signatures | Keep open; mirrored in signing pipeline. | +| [#35](https://github.com/flyingrobots/echo/issues/35) | Key management doc | Keep open; mirrored in signing pipeline. | +| [#34](https://github.com/flyingrobots/echo/issues/34) | CLI verify path | Keep open; mirrored in signing pipeline. | +| [#33](https://github.com/flyingrobots/echo/issues/33) | CI: sign release artifacts | Keep open; mirrored in signing pipeline. | +| [#25](https://github.com/flyingrobots/echo/issues/25) | Importer: TurtlGraph to Echo store | Keep open; mirrored in importer inbox. | +| [#24](https://github.com/flyingrobots/echo/issues/24) | Editor Hot-Reload | Keep open; mirrored in hot-reload inbox. | +| [#21](https://github.com/flyingrobots/echo/issues/21) | Spec: Security Contexts | Keep open; mirrored in security inbox. | +| [#20](https://github.com/flyingrobots/echo/issues/20) | Spec: Commit/Manifest Signing | Keep open; mirrored in signing pipeline. | + +## Issues Closed During This Audit + +| Issue | Disposition | Evidence | +| ------------------------------------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | +| [#285](https://github.com/flyingrobots/echo/issues/285) | Closed as completed. | `.github/workflows/det-gates.yml` computes `DETERMINISM_PATHS` from `det-policy.yaml`; `PLATFORM_ci-det-policy-hardening.md` marks the item completed. | +| [#281](https://github.com/flyingrobots/echo/issues/281) | Closed as completed/superseded. | `docs/determinism/RELEASE_POLICY.md` contains the staging/production blocker matrix and recommendation rules. | + +## Notes For Future Audits + +- Do not close open issues just because they are old. Several old issues are + mirrored in `docs/method/task-matrix.md`, backlog cards, and docs audits. +- The GitHub issue tracker contains both execution work and intentional idea + parking lots. Keep the issue open when the repo still carries a matching + backlog card or task-DAG node. +- Favor moving stale issue text into filesystem backlog cards before closing + the issue, unless the work is already clearly completed by merged code/docs. diff --git a/docs/design/causal-wal-end-to-end.md b/docs/design/causal-wal-end-to-end.md index e8c28c0c..f9d9d42a 100644 --- a/docs/design/causal-wal-end-to-end.md +++ b/docs/design/causal-wal-end-to-end.md @@ -694,6 +694,73 @@ Rebuildable indexes include: If an index cannot be rebuilt from committed WAL/checkpoint material, it is secretly state and should fail review. +## WAL Projection Into The WARP Graph And WSC + +Echo may project WAL-backed storage evidence into the WARP graph, but those +graph facts are not the WAL's recovery authority. + +```text +WAL bytes are the durable commit authority. +WARP graph facts track WAL segment evidence. +WSC serializes graph facts and may bundle or reference WAL bytes. +``` + +The graph may contain evidence facts such as: + +- `WalRoot`; +- `WalWriterEpoch`; +- `WalSegmentRef`; +- `WalCommitAnchor`; +- `RecoveryCertificateRef`. + +A `WalSegmentRef` should identify a segment by writer epoch, LSN range, digest +chain, segment digest, commit anchors, and sealed posture. Its storage locator +may point to local segment storage, CAS storage, or an object-store manifest, +but that locator is not causal identity. + +Good storage-locator shapes: + +```text +segments/0000000000000042.ecwal +wal://local/default/segments/0000000000000042.ecwal +cas://sha256/... +``` + +Bad canonical identity: + +```text +/Users/example/echo/wal/0000000000000042.ecwal +``` + +Recovery must not require pre-existing graph WAL nodes. The bootstrap path is: + +```text +configured WAL root or storage manifest +-> validate segment headers and commit chains +-> replay committed transactions +-> rebuild graph/read-model/index facts +-> expose WAL segment refs as projected evidence +``` + +WSC can serialize projected WAL facts in at least three modes: + +| Mode | Meaning | +| ------------------ | --------------------------------------------------------------------------------------- | +| Ref-only WSC | Contains graph facts plus WAL segment locators and digests. | +| Self-contained WSC | Contains graph facts plus embedded WAL segment bytes or bundled retained material. | +| CAS-addressed WSC | Contains graph facts plus content-addressed refs to WAL segments and retained material. | + +The bridge doctrine is: + +```text +Records are recorded. +Transactions are committed. +Segments are sealed. +Graph WAL facts are projected evidence. +WSC carries or references evidence. +The WAL commit boundary remains the authority. +``` + ## Retained Material Ordering If a committed WAL record references retained material, the material must diff --git a/docs/design/v0.1.0-jedit-next-ten-slices.md b/docs/design/v0.1.0-jedit-next-ten-slices.md index a45a88d8..0026671f 100644 --- a/docs/design/v0.1.0-jedit-next-ten-slices.md +++ b/docs/design/v0.1.0-jedit-next-ten-slices.md @@ -37,6 +37,24 @@ Open a PR after slice 5 and another after slice 10 unless a slice requires a small supporting PR in Wesley. Echo and jedit may have paired PRs. Echo must remain generic; jedit owns product nouns and product-facing adapter language. +## Progress + +```text +[#####-----] jedit release-gate next-ten plan [5/10 slices] +[#####-----] PR checkpoint batch [5/5 slices before recommended PR] +``` + +- [x] Slice 1: jedit release-gate doctrine cleanup. +- [x] Slice 2: jedit witness retained-evidence inventory. +- [x] Slice 3: Echo retained evidence ref surface check. +- [x] Slice 4: jedit adapter consumes Echo retained refs. +- [x] Slice 5: jedit replay witness shell. +- [ ] Slice 6: Echo product-facing intent outcome API polish. +- [ ] Slice 7: jedit generated structural-history request path. +- [ ] Slice 8: generated package install path. +- [ ] Slice 9: jedit real mutation and query round trip. +- [ ] Slice 10: jedit non-happy path and release-gate report. + ## Slice 1: jedit Release-Gate Doctrine Cleanup Repository: `jedit` @@ -122,10 +140,12 @@ Witness: - no jedit nouns are introduced into `warp-core` or `warp-wasm`; - app-safe retained evidence inspection does not expose trusted control. -Current audit note: the current WASM witness exposes inline observation -evidence, but not enough durable retained refs for jedit to satisfy the release -gate. The next Echo-side cut should add or expose generic retained evidence -references; it must not add editor/text nouns. +Current implementation note: Echo's app-safe observation and WASM ABI surfaces +now expose generic retained-evidence posture on installed contract QueryView +readings. The envelope reports missing reading-envelope coordinates and +missing reading-payload content refs until durable retention exists. This gives +jedit enough generic evidence shape to report honest missing-retention posture +without private runtime access or editor/text nouns in Echo. ## Slice 4: jedit Adapter Consumes Echo Retained Refs @@ -150,6 +170,12 @@ Witness: summary; - no direct Echo substrate access appears outside the adapter. +Current implementation note: jedit now projects text-window reading retained +evidence from the observer adapter into `TextBufferSessionPort`. The witness +consumes those adapter-projected refs instead of manufacturing reading refs +after observation. The current posture is still missing durable reading +envelope/payload material, which is correct until Echo/WSC retention closes. + ## Slice 5: jedit Replay Witness Shell Repository: `jedit` @@ -177,10 +203,14 @@ Witness: - same outcome/reading identity, or explicit replay obstruction; - no app-facing tick authority. -Current implementation note: jedit can request replay posture with `--replay` -and currently receives `durable_replay_unavailable` when the witness report -lacks durable replay proof. That obstruction is honest and must be replaced by -real replay evidence before `v0.1.0`. +Current implementation note: jedit can request durable replay posture with +`node scripts/jedit-echo-witness.mjs --json --replay` and currently receives +`durable_replay_unavailable` when the witness report lacks durable replay +proof. The session witness also has a local replay shell through +`node scripts/jedit-echo-powered-session.mjs --json --replay-local`, which +compares stable semantic identity and records that wall-clock cadence is not +semantic. Durable replay remains honestly unavailable until Echo/WSC recovery +closes. ## Slice 6: Echo Product-Facing Intent Outcome API Polish diff --git a/docs/design/work-item-sequencing-and-prioritization.md b/docs/design/work-item-sequencing-and-prioritization.md new file mode 100644 index 00000000..20ecd407 --- /dev/null +++ b/docs/design/work-item-sequencing-and-prioritization.md @@ -0,0 +1,174 @@ + + + +# Work Item Sequencing And Prioritization + +Status: active sequencing guide. + +Last updated: 2026-05-25. + +This guide adopts the work-item audit in [`docs/WorkItems.md`](../WorkItems.md) +as an inventory snapshot and turns it into an execution filter. It is not a +mandate to complete every historical backlog item before release. Work moves up +only when it shortens the path to a real `jedit`-on-Echo proof or hardens +evidence, replay, retention, authority, compatibility, or release +reproducibility around that proof. + +The current gravity remains: + +```text +prove Echo with jedit as a real external app +without moving app nouns into Echo +and without giving application code tick, WAL, or trusted runtime authority +``` + +The immediate release-proof plan is +[`docs/design/v0.1.0-jedit-next-ten-slices.md`](v0.1.0-jedit-next-ten-slices.md). +The broader release feature bar is +[`docs/design/v0.1.0-release-plan.md`](v0.1.0-release-plan.md). +The active inventory snapshot is [`docs/WorkItems.md`](../WorkItems.md). + +## Strategy + +Use five execution bands: + +| Band | Meaning | Planning intent | +| ---: | --------------------------------------------- | -------------------------------------------------------------------------------------------- | +| 0 | In-flight release-proof slices | Finish the narrow proof path before broadening scope. | +| 1 | Hard blockers for credible external-app proof | Pull in only what unlocks jedit, evidence, retention, app-safe API, or authority boundaries. | +| 2 | Release-candidate hardening and audit closure | Convert working paths into repeatable proof, docs, CI, and compatibility gates. | +| 3 | Near-follow-on platform work | Start after the release-proof path stabilizes. | +| 4 | Inbox, debt, demos, and ideas | Preserve visibility without stealing execution energy. | + +Priority rules, in order: + +1. Prefer work that shortens the path to a real `jedit`-on-Echo proof. +2. Prefer work that makes evidence, retention, replay, and contract boundaries + observable and testable. +3. Prefer work that tightens authority boundaries and app-safe APIs before + adding runtime surface area. +4. Prefer work that turns design claims into executable release gates, + fixtures, and proof artifacts. +5. Defer broad platform ambition, migration work, browser work, demo courses, + and idea-lot issues until the external-app proof is credible. + +Before executing any listed item, verify whether the backing card is still +current. If code has already landed, convert the item into closure, doc update, +or release-gate verification work instead of reimplementing it. + +## Storage Doctrine Gap + +The audit surfaced one release-bar storage gap that should stay visible while +the `jedit` proof proceeds: + +```text +WAL bytes are the durable commit authority. +WARP graph facts track WAL segment evidence. +WSC serializes graph facts and may bundle or reference WAL bytes. +``` + +The WARP graph may contain WAL evidence nodes, but those nodes are projected +facts, not recovery bootstrap authority. Echo recovery must be able to start +from a configured WAL root or storage manifest, validate committed segments, +and rebuild the graph/read-model/index facts. A graph-projected WAL filepath is +a storage locator, not causal identity. Causal identity comes from writer +epoch, LSN range, segment digest, commit digest chain, and validated commit +anchors. + +The doctrine is tracked by: + +- [`PLATFORM_wal-wsc-storage-relationship.md`](../method/backlog/v0.1.0/PLATFORM_wal-wsc-storage-relationship.md) +- [`PLATFORM_wsc-causal-history-storage.md`](../method/backlog/v0.1.0/PLATFORM_wsc-causal-history-storage.md) +- [`PLATFORM_retained-evidence-durability-boundary.md`](../method/backlog/v0.1.0/PLATFORM_retained-evidence-durability-boundary.md) + +## First Chunk + +The first chunk is the active release-proof package: + +```text +[#####-----] Echo/jedit retained-evidence release-gate batch [5/10 slices] +[#####-----] PR checkpoint batch [5/5 slices before recommended PR] +``` + +Finish the package in this order: + +| Seq | Band | Package | Source | Outcome | +| --: | ---- | ----------------------------------------------------- | ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | +| 1 | 0 | Slice 4: `jedit` adapter consumes Echo retained refs | Next-ten plan | `jedit` reads retained evidence from the adapter-projected Echo envelope instead of inventing reading refs in the witness layer. | +| 2 | 0 | Slice 5: `jedit` replay witness shell | Next-ten plan | Minimal replay witness shell proves the external-app evidence path can be rerun and inspected. | +| 3 | 0 | Slice 6 plus product-facing outcome API | Next-ten plan and v0.1.0 lane | Echo exposes a stable app-facing intent outcome shape without exposing scheduler or tick authority. | +| 4 | 0 | Slice 7: generated structural-history request path | Next-ten plan | `jedit` moves from retained-ref plumbing toward generated, product-owned request construction. | +| 5 | 0 | Retention, WAL/WSC, and semantic lookup proof package | v0.1.0 lane | Release-grade retained-evidence posture and lookup seams exist, with durable claims only where WAL/WSC-backed evidence exists. | + +Slice 5 is now closed. This is the recommended PR checkpoint before taking +Slice 6, unless a reviewer asks for a follow-up in the same branch. + +## Release-Proof Sequence + +This sequence is sorted by recommended execution, not by current folder or +issue number. + +| Seq | Band | Work package | Source | Why this spot | +| --: | ---- | -------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | +| 1 | 0 | Slice 4: `jedit` adapter consumes Echo retained refs | Next-ten plan | Unlocks the retained-evidence integration path already in progress. | +| 2 | 0 | Slice 5: `jedit` replay witness shell | Next-ten plan | Establishes replay/witness mechanics needed for proof, debugging, and release credibility. | +| 3 | 0 | Slice 6 and product-facing intent outcome API | Next-ten plan; [`PLATFORM_product-facing-intent-outcome-api.md`](../method/backlog/v0.1.0/PLATFORM_product-facing-intent-outcome-api.md) | Tightens the app-facing contract before more integration complexity lands. | +| 4 | 0 | Slice 7: `jedit` generated structural-history request path | Next-ten plan | Advances the external-app proof from retained refs toward product-owned generated request construction. | +| 5 | 0 | Contract retention, WAL/WSC, and semantic lookup proof package | [`PLATFORM_contract-artifact-retention-in-echo-cas.md`](../method/backlog/v0.1.0/PLATFORM_contract-artifact-retention-in-echo-cas.md); [`PLATFORM_contract-retention-and-semantic-lookup-seams.md`](../method/backlog/v0.1.0/PLATFORM_contract-retention-and-semantic-lookup-seams.md); [`PLATFORM_wal-wsc-storage-relationship.md`](../method/backlog/v0.1.0/PLATFORM_wal-wsc-storage-relationship.md) | Makes retained evidence discoverable without overclaiming durability or confusing WAL storage locators with causal identity. | +| 6 | 0 | Slice 8: generated package install path | Next-ten plan | Proves generated-path realism and reduces toy-demo risk. | +| 7 | 0 | External proof fixture and app-safe client surface | [`PLATFORM_external-contract-proof-fixture.md`](../method/backlog/v0.1.0/PLATFORM_external-contract-proof-fixture.md); [`PLATFORM_app-safe-client-surface.md`](../method/backlog/v0.1.0/PLATFORM_app-safe-client-surface.md) | Keeps the external app on bounded APIs instead of privileged internals. | +| 8 | 0 | Slice 9: `jedit` real mutation and query round trip | Next-ten plan | Decisive real-app interaction proof. | +| 9 | 1 | Contract-aware receipts/readings and bounded reading identity | [`KERNEL_contract-aware-receipts-and-readings.md`](../method/backlog/v0.1.0/KERNEL_contract-aware-receipts-and-readings.md); [`KERNEL_contract-reading-identity-and-bounded-payloads.md`](../method/backlog/v0.1.0/KERNEL_contract-reading-identity-and-bounded-payloads.md) | Strengthens what the app can prove it wrote and read. | +| 10 | 1 | Thin release-grade quickstart draft | [`DOCS_release-grade-quickstart.md`](../method/backlog/v0.1.0/DOCS_release-grade-quickstart.md) | Quickstarts expose integration drift early; final polish can wait for RC. | +| 11 | 1 | Witnessed submission persistence closure | [`KERNEL_witnessed-intent-submission-persistence.md`](../method/backlog/v0.1.0/KERNEL_witnessed-intent-submission-persistence.md) | Reconcile older persistence wording with landed WAL-backed ACK work before claiming release durability. | +| 12 | 1 | Reference trusted runtime host loop closure | [`PLATFORM_reference-trusted-runtime-host-loop.md`](../method/backlog/v0.1.0/PLATFORM_reference-trusted-runtime-host-loop.md) | Confirms trusted host behavior without handing scheduler authority to application code. | +| 13 | 1 | Contract obstruction taxonomy | [`KERNEL_contract-obstruction-taxonomy.md`](../method/backlog/v0.1.0/KERNEL_contract-obstruction-taxonomy.md) | Makes non-happy-path release evidence legible. | +| 14 | 1 | Slice 10: non-happy path and release-gate report | Next-ten plan | Closes the active ten-slice release-proof batch. | +| 15 | 1 | `jedit` real Echo release gate | [`PLATFORM_jedit-real-echo-release-gate.md`](../method/backlog/v0.1.0/PLATFORM_jedit-real-echo-release-gate.md) | Converts the proof from an ad hoc demo into a durable gate. | +| 16 | 1 | Versioned contract and API compatibility | [`PLATFORM_versioned-contract-api-compatibility.md`](../method/backlog/v0.1.0/PLATFORM_versioned-contract-api-compatibility.md) | Lowers generated-contract drift before release-candidate pressure. | +| 17 | 1 | CI det-policy hardening | [`PLATFORM_ci-det-policy-hardening.md`](../method/backlog/asap/PLATFORM_ci-det-policy-hardening.md) | Finishes current CI-hardening follow-ups after issue `#285` closure. | +| 18 | 1 | Decoder negative-test map | [`PLATFORM_decoder-negative-test-map.md`](../method/backlog/asap/PLATFORM_decoder-negative-test-map.md) | Completes security evidence around malformed input controls. | +| 19 | 2 | Authority boundary audit | [`SECURITY_authority-boundary-audit.md`](../method/backlog/v0.1.0/SECURITY_authority-boundary-audit.md) | Must run after concrete app-safe surfaces exist, before RC. | +| 20 | 2 | Replay/DIND proof and release-candidate readiness | [`TEST_v0.1.0-replay-dind-proof.md`](../method/backlog/v0.1.0/TEST_v0.1.0-replay-dind-proof.md); [`RELEASE_v0.1.0-release-candidate.md`](../method/backlog/v0.1.0/RELEASE_v0.1.0-release-candidate.md) | Consolidates determinism and replay claims into a repeatable release proof. | + +## Post-Proof Expansion + +After the release-proof sequence stabilizes, pull from these areas in order: + +1. Admission outcome family and bounded admission policy. +2. Installed/compiled contract hosting roadmap and doctrine cleanup. +3. TTD protocol schema reconciliation and rollback playbooks. +4. Continuum admission/proof-family cutovers. +5. Echo CLI/MCP agent surfaces. +6. Browser/WASM surface expansion. +7. Demo/course and visualization ideas. + +## Guardrails + +- Do not treat this sequence as a reason to reimplement already-landed code. +- Do not overclaim durable retention where only missing-retention posture + exists. +- Do not let `jedit` product nouns enter Echo core. +- Do not expose tick, WAL append, package install, scheduler, or trusted + recovery authority to application code. +- Do not close old GitHub issues merely because they are old; close them only + when the corresponding card, code path, or design claim is demonstrably + complete. +- Do not make WARP graph WAL nodes the recovery bootstrap. They are projected + evidence facts over WAL-backed history, not the authority that makes the WAL + recoverable. + +## Operating Cadence + +- Sprint A: finish slices 4-6, including product-facing outcome API and the + retained-evidence handoff. +- Sprint B: finish slices 7-9, including proof fixture and app-safe evidence + seams. +- Sprint C: finish slice 10, compatibility, authority audit, replay proof, and + quickstart hardening. +- Sprint D: cut release-candidate evidence, then selectively promote ASAP + hosting and schema work. + +Every sprint should end with executable proof: tests, release-gate output, +golden artifacts, or a documented audit. Design claims without witnesses do not +count as closure. diff --git a/docs/method/backlog/v0.1.0/PLATFORM_js-wasm-browser-client-release-surface.md b/docs/method/backlog/v0.1.0/PLATFORM_js-wasm-browser-client-release-surface.md new file mode 100644 index 00000000..229b34c9 --- /dev/null +++ b/docs/method/backlog/v0.1.0/PLATFORM_js-wasm-browser-client-release-surface.md @@ -0,0 +1,68 @@ + + + +# JS/WASM/Browser Client Release Surface + +Status: v0.1.0 release blocker if JS, WASM, Node, or browser packages ship. + +Depends on: + +- [App-safe client surface](./PLATFORM_app-safe-client-surface.md) +- [Product-facing intent outcome API](./PLATFORM_product-facing-intent-outcome-api.md) +- [Package publish and versioning](./RELEASE_package-publish-and-versioning.md) + +## Why now + +The release bar now includes serious application use. If Echo publishes +JavaScript, WASM, Node, or browser artifacts, those artifacts must carry only +the app-safe client surface. They must not accidentally expose trusted runtime +control, tick authority, WAL append authority, recovery authority, or package +install authority to application code. + +## Required behavior + +Published client artifacts may expose: + +- generated intent/query request helpers; +- canonical intent submission; +- intent outcome observation; +- bounded query reading requests; +- retained evidence posture; +- package/version compatibility checks. + +Published client artifacts may not expose: + +- scheduler step or tick; +- trusted runtime start/stop/drain control unless the artifact is explicitly a + trusted-host package; +- WAL append or recovery mutation; +- package installation authority; +- raw kernel object mutation. + +## Acceptance criteria + +- [ ] Release documentation names which JS/WASM/browser artifacts ship. +- [ ] Shipped application-facing packages expose only app-safe APIs. +- [ ] Trusted-host APIs, if shipped, are packaged and documented separately from + application client APIs. +- [ ] Browser and Node smoke tests prove generated requests can be submitted + and observed without tick authority. +- [ ] Package metadata binds Echo ABI version, WASM ABI version, generated + helper compatibility, and contract package version. +- [ ] Examples do not call trusted runtime control from application code. + +## Test plan + +- Add package export tests proving app-facing entry points do not re-export + trusted runtime controls. +- Add Node smoke test for submit/observe/query through app-safe API. +- Add browser or WASM-bindgen smoke test if browser artifacts ship. +- Add docs/quickstart check for the published artifact set. + +## Non-goals + +- Do not require browser artifacts if the release explicitly ships Rust/local + host APIs only. +- Do not build a UI framework. +- Do not implement streaming subscriptions. +- Do not expose raw privileged WASM exports to app code. diff --git a/docs/method/backlog/v0.1.0/PLATFORM_retained-evidence-durability-boundary.md b/docs/method/backlog/v0.1.0/PLATFORM_retained-evidence-durability-boundary.md new file mode 100644 index 00000000..4efaded6 --- /dev/null +++ b/docs/method/backlog/v0.1.0/PLATFORM_retained-evidence-durability-boundary.md @@ -0,0 +1,60 @@ + + + +# Retained Evidence Durability Boundary + +Status: v0.1.0 release blocker. + +Depends on: + +- [Contract retention and semantic lookup seams](./PLATFORM_contract-retention-and-semantic-lookup-seams.md) +- [WAL/WSC storage relationship](./PLATFORM_wal-wsc-storage-relationship.md) +- [Contract-aware receipts and readings](./KERNEL_contract-aware-receipts-and-readings.md) + +## Why now + +Echo can expose retained-evidence posture, but posture is not the same claim as +durable recovery. The release proof needs explicit language and tests that +distinguish "the runtime knows this reading was missing" from "the runtime can +recover this material after restart." + +## Required behavior + +Echo must distinguish: + +- retained evidence posture: present, missing, corrupt, redacted, or obstructed + from the current read model; +- durable recovery evidence: WAL/WSC-backed proof that the material reference, + semantic coordinate, and commit anchor can survive restart; +- semantic lookup: the product-level coordinate used to find a retained + reading, receipt, or artifact; +- byte identity: the content digest of retained bytes. + +## Acceptance criteria + +- [ ] Echo docs distinguish retained-evidence posture from durable recovery + evidence. +- [ ] App-safe readings do not imply durable payload recovery unless backed by + WAL/WSC evidence. +- [ ] Missing retained material returns typed obstruction or missing-retention + posture, not empty success. +- [ ] A retained reading ref is not treated as a query identity. +- [ ] A query identity does not imply payload retention. +- [ ] Cache hits are not evidence unless the semantic coordinate and material + digest match. + +## Test plan + +- Add a future retained-reading fixture proving missing payload posture is not + reported as a successful empty read. +- Add a future restart fixture proving only WAL/WSC-backed retained refs are + advertised as durable. +- Add a future semantic-coordinate fixture proving byte identity and semantic + lookup identity stay separate. + +## Non-goals + +- Do not require full object-store retention for every local development path. +- Do not implement streaming subscriptions. +- Do not collapse query identity, retained reading identity, and payload digest + into one noun. diff --git a/docs/method/backlog/v0.1.0/PLATFORM_wal-wsc-storage-relationship.md b/docs/method/backlog/v0.1.0/PLATFORM_wal-wsc-storage-relationship.md new file mode 100644 index 00000000..8769d4d6 --- /dev/null +++ b/docs/method/backlog/v0.1.0/PLATFORM_wal-wsc-storage-relationship.md @@ -0,0 +1,80 @@ + + + +# WAL/WSC Storage Relationship + +Status: v0.1.0 release doctrine blocker. + +Depends on: + +- [Causal WAL end-to-end design](../../../design/causal-wal-end-to-end.md) +- [WSC causal-history storage](./PLATFORM_wsc-causal-history-storage.md) +- [Retained evidence durability boundary](./PLATFORM_retained-evidence-durability-boundary.md) + +## Why now + +Echo now has a serious causal WAL doctrine and the jedit release gate needs +recoverable editing history. The WARP graph can represent graph-shaped evidence +about WAL segments, but recovery must not become circular. + +The unsafe model is: + +```text +graph facts tell Echo where the WAL is +WAL replay tells Echo how to rebuild graph facts +``` + +That makes recovery depend on the thing being recovered. + +The release-bar model is: + +```text +WAL bytes are the durable commit authority. +WARP graph facts track WAL segment evidence. +WSC serializes graph facts and may bundle or reference WAL bytes. +``` + +## Required doctrine + +Echo's WAL is the authority for committed causal history. Echo may project WAL +roots, writer epochs, segment references, commit anchors, recovery +certificates, and storage locators into the WARP graph as evidence-bearing +facts. These graph facts make WAL-backed history inspectable, portable, +queryable, and serializable to WSC, but they do not replace the WAL commit +boundary or bootstrap recovery. + +A WAL filepath is a storage locator, not causal identity. Causal identity comes +from writer epoch, LSN range, commit digest chain, segment digest, and +validated commit anchors. + +## Acceptance criteria + +- [ ] Echo documents that WAL segments are the primary durable commit authority. +- [ ] Echo documents that WARP graph WAL nodes are projected evidence facts, + not bootstrap recovery authority. +- [ ] Echo defines the minimum WAL evidence projected into the graph: + WAL root, writer epoch, segment reference, segment digest, LSN range, + commit anchors, and recovery certificate references where applicable. +- [ ] Echo defines WSC export modes for WAL evidence: + ref-only export, self-contained export, and CAS-addressed export. +- [ ] Echo documents that storage locators are not causal identity. +- [ ] Echo defines recovery bootstrap without requiring pre-existing graph WAL + nodes. +- [ ] Echo keeps record naming clear: records are recorded, transactions are + committed, segments are sealed. + +## Test plan + +- Add a design/doc check or lint fixture proving the WAL/WSC doctrine is linked + from BEARING, WorkItems, and the WAL design packet. +- Add a future recovery fixture proving recovery can start from a configured + WAL root or manifest without requiring pre-existing graph WAL nodes. +- Add a future export fixture proving graph-projected WAL segment refs can be + serialized without treating raw paths as causal identity. + +## Non-goals + +- Do not store the WAL inside the WARP graph as the primary recovery mechanism. +- Do not make absolute host paths part of causal identity. +- Do not require WSC to embed every WAL byte in every export. +- Do not let graph projection append WAL records or validate recovery. diff --git a/docs/method/backlog/v0.1.0/PLATFORM_wsc-causal-history-storage.md b/docs/method/backlog/v0.1.0/PLATFORM_wsc-causal-history-storage.md new file mode 100644 index 00000000..21b7c530 --- /dev/null +++ b/docs/method/backlog/v0.1.0/PLATFORM_wsc-causal-history-storage.md @@ -0,0 +1,71 @@ + + + +# WSC Causal-History Storage + +Status: v0.1.0 release doctrine blocker. + +Depends on: + +- [WAL/WSC storage relationship](./PLATFORM_wal-wsc-storage-relationship.md) +- [Contract retention and semantic lookup seams](./PLATFORM_contract-retention-and-semantic-lookup-seams.md) +- [v0.1.0 replay and DIND proof](./TEST_v0.1.0-replay-dind-proof.md) + +## Why now + +The jedit release gate is not only "can the app submit and observe once." +Editing history must eventually survive application lifecycles and support +materializing file artifacts from explicit causal points. Echo should define +how WSC participates in that recovery story before implementation choices split +WAL, graph, and export into separate durability systems. + +## Required behavior + +WSC is the portable container/export boundary for Echo causal-history material. +It may carry: + +- WARP graph facts and read-model facts; +- WAL segment evidence refs; +- segment digests and commit anchors; +- recovery certificate refs; +- retained material refs; +- embedded segment bytes or CAS refs when the export is self-contained. + +WSC must not become a second recovery authority that can contradict the WAL. + +## Export modes + +| Mode | Meaning | +| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Ref-only WSC | Serializes graph facts plus WAL segment locators and digests. Useful when the importer can access the same storage. | +| Self-contained WSC | Serializes graph facts plus embedded WAL segment bytes or bundled retained material. Useful for portable archive and recovery. | +| CAS-addressed WSC | Serializes graph facts plus content-addressed refs to WAL segments and retained material. Useful when CAS storage is shared or transported separately. | + +## Acceptance criteria + +- [ ] Echo defines WSC export posture for ref-only, self-contained, and + CAS-addressed causal-history material. +- [ ] Echo defines how exported WSC binds WAL segment digests, LSN ranges, and + commit anchors. +- [ ] Echo defines how jedit can request materialization from a causal point + without treating current editor memory as source of truth. +- [ ] Echo defines what happens when an export references unavailable WAL or + retained material. +- [ ] Echo documents that WSC import validates WAL-backed evidence rather than + trusting graph facts blindly. + +## Test plan + +- Add a future WSC export fixture with one committed WAL segment ref and one + retained material ref. +- Add a future self-contained export fixture proving segment bytes match the + segment digest and commit chain. +- Add a future obstruction fixture for ref-only WSC whose segment locator is + unavailable. + +## Non-goals + +- Do not implement full Continuum replica transport. +- Do not implement social lane or observer-rights governance. +- Do not make jedit file export mutate Echo history. +- Do not make WSC replace the WAL commit boundary. diff --git a/docs/method/backlog/v0.1.0/RELEASE_package-publish-and-versioning.md b/docs/method/backlog/v0.1.0/RELEASE_package-publish-and-versioning.md new file mode 100644 index 00000000..90d33081 --- /dev/null +++ b/docs/method/backlog/v0.1.0/RELEASE_package-publish-and-versioning.md @@ -0,0 +1,56 @@ + + + +# Package Publish And Versioning + +Status: v0.1.0 release blocker. + +Depends on: + +- [Versioned contract and API compatibility](./PLATFORM_versioned-contract-api-compatibility.md) +- [JS/WASM/Browser client release surface](./PLATFORM_js-wasm-browser-client-release-surface.md) +- [Release-grade quickstart](./DOCS_release-grade-quickstart.md) + +## Why now + +Real applications need to know whether a generated contract package, runtime +ABI, WASM ABI, and client helper set fit together. The jedit release gate should +not depend on local path accidents or unversioned generated artifacts. + +## Required behavior + +Echo release artifacts must state compatibility for: + +- Echo crate/package version; +- Echo ABI version; +- WASM ABI version, if shipped; +- Wesley generator version range; +- contract package version; +- schema hash; +- artifact hash; +- codec id; +- generated helper compatibility; +- installed package metadata. + +## Acceptance criteria + +- [ ] Echo documents release artifact names and publish targets. +- [ ] Echo defines compatibility checks for generated contract packages. +- [ ] Publish dry-run or equivalent release command is documented. +- [ ] jedit can consume Echo through documented versioned artifacts or an + explicitly documented local-development override. +- [ ] The release candidate checklist includes package metadata verification. +- [ ] CHANGELOG and release docs name package/API compatibility expectations. + +## Test plan + +- Add package metadata fixture with compatible and incompatible generated + package versions. +- Add release dry-run check where the repository supports one. +- Add quickstart validation that does not depend on an undocumented local path. + +## Non-goals + +- Do not publish until the release candidate gate is green. +- Do not force all neighboring repos to release in one commit. +- Do not treat local sibling path overrides as release-grade compatibility. diff --git a/docs/spec/SPEC-0009-wasm-abi.md b/docs/spec/SPEC-0009-wasm-abi.md index 3e079f2a..e3d87907 100644 --- a/docs/spec/SPEC-0009-wasm-abi.md +++ b/docs/spec/SPEC-0009-wasm-abi.md @@ -7,7 +7,7 @@ _Define the current deterministic browser boundary for intent ingress, scheduler Legend: PLATFORM -Current ABI version: 11 +Current ABI version: 12 Depends on: @@ -19,12 +19,14 @@ Depends on: The WASM boundary is where browser and host code meet the Echo runtime. It must be small, deterministic, and explicit about what kind of operation is crossing: intent admission, scheduler inspection, or observation. -ABI version 11 keeps the application-facing export shape from version 10, adds -an explicit trusted host-control export, and extends -`DispatchResponse` with witnessed submission identity for accepted application -ingress. Observation requests still name their observer plan, optional hosted -observer instance, read budget, and rights posture explicitly. Observation -artifacts continue to carry reading-envelope metadata for emitted readings. +ABI version 12 keeps the application-facing export shape from version 11, +retains the explicit trusted host-control export, and carries witnessed +submission identity for accepted application ingress. Observation requests +still name their observer plan, optional hosted observer instance, read budget, +and rights posture explicitly. Observation artifacts continue to carry +reading-envelope metadata for emitted readings, including contract/query +identity and generic retained-evidence posture when a contract QueryView +observer supplies a reading. ## Human users / jobs / hills @@ -69,9 +71,11 @@ application code. The observation request names the observer plan, optional hosted observer instance, read budget, and rights posture. The reading envelope names the observer plan, hosted observer instance when present, native observer basis, -witness refs, parent/basis posture, budget posture, rights posture, and -residual posture. Built-in observations currently emit `complete` residual -posture for clean derived readings. The ABI also names `residual`, +installed contract evidence when present, QueryView reading identity when +present, generic retained-evidence posture, witness refs, parent/basis posture, +budget posture, rights posture, and residual posture. Built-in observations +currently emit `complete` residual posture for clean derived readings and leave +contract/query/retained-evidence fields empty. The ABI also names `residual`, `plurality_preserved`, and `obstructed` so external consumers can recognize bounded non-clean readings without treating the payload as a generic state read.