diff --git a/docs/learning-mode/capabilities.md b/docs/learning-mode/capabilities.md index 83f766821..127d96a72 100644 --- a/docs/learning-mode/capabilities.md +++ b/docs/learning-mode/capabilities.md @@ -207,25 +207,60 @@ sandbox policy: - Analysis retains at most 10,000 unique denials and processes at most 1,000,000 ETW events. Reaching the unique-denial bound stops adding policy entries but continues bounded diagnostic accounting; reaching either bound - sets `summary.deniedResourcesTruncated` to `true`. + sets `summary.deniedResourcesTruncated` to `true`. Missing schemas for + supported denial events also set this flag. Incomplete results do not produce + policy previews or adjusted configurations. - `resource` is the user-visible identifier for the denied resource, interpreted by `resourceType`: an absolute `C:\…` path for `file`, the AppContainer **capability name** (e.g. `internetClient`) for `capability`, and the raw resource identifier otherwise. Well-known capability SIDs are resolved to their policy name; custom (hashed) capability SIDs that can't be reversed fall back to the `S-1-15-3-…` SID string. Named Section, - SymbolicLink, and Timer checks are verbose-only because the config has no - corresponding policy grants. Event 28 is - schema-discriminated: UI-shaped `Category`/`Detail` payloads emit `ui` - resources instead of treating the package SID as a capability. + SymbolicLink, Timer, and COM checks are verbose-only because the config has + no corresponding policy grants. Event 28 is schema-discriminated: UI-shaped + `Category`/`Detail` payloads emit `ui` resources instead of treating the + package SID as a capability. - `resourceType` is one of `file`, `ui`, `network`, `capability`, `other`; `accessType` is one of `read`, `write`, `execute`, `unknown`. Capability - denials are recorded under `block`; current `allow` traces expose capability - checks as empty-`ObjectType` access events that are omitted because they do - not carry a stable capability identifier. + denials are recorded under `block`; `allow` traces can expose capability + checks as empty-`ObjectType` access events. Known capabilities can be recovered + from their DACL payload; unresolved checks remain verbose diagnostics. - `filetime` is a decimal string containing the Windows `FILETIME` value, so JavaScript consumers retain all 64 bits without numeric precision loss. +### COM access checks + +On Windows builds with COM Learning Mode metadata, Kernel-General access-check +event 14 reports two additional object types: + +| `ObjectType` | Verbose reason | Identifier | +| ------------ | -------------- | ---------- | +| `ComActivationForClass` | `comActivation` | Activation CLSID | +| `ComCallOnInterface` | `comInterfaceCall` | Called interface IID | + +Both `captureDenials.mode: "block"` and `captureDenials.mode: "allow"` decode +these records identically. The mode still controls whether the denied operation +remains blocked or is permitted; it does not change the output classification. +MXC validates that the identifier is GUID-shaped. + +COM records do not enter the caller-facing `denials` array because MXC does not +expose an authorable COM grant. They are retained in the local verbose sibling +with `resourceType: "other"`, no `accessType`, and the original CLSID/IID in the +sanitized `ObjectName` property. The distinct reasons make COM activation and +interface calls recognizable without treating them as unknown object types or +policy-actionable denials. + +The `MXC.VerboseDenials` telemetry projection retains the COM reason and count +but removes all verbose properties, including the CLSID/IID. + +This coverage applies to classic COM activation checks instrumented in RPCSS and +COM interface-call checks instrumented in COMBASE. The separate WinRT +`CheckActivationPermissions` path does not currently attach this COM Learning +Mode metadata, so an MXC decoder-only change cannot identify those WinRT +activation denials as `ComActivationForClass`. `RPC Interface` events are +lower-layer LRPC diagnostics and remain `unsupportedObjectType`; they are not a +substitute for the COM permission event. + ### Verbose logging event signatures Every successful decode also writes a deterministic sibling file: @@ -235,7 +270,7 @@ policy denial occurrences plus diagnostic outcomes omitted from the policy file: ```json { - "version": 2, + "version": 3, "signatures": [ { "signature": { @@ -266,24 +301,33 @@ policy denial occurrences plus diagnostic outcomes omitted from the policy file: ``` Signatures are keyed by symbolic provider category, provider GUID, -provider-scoped event ID, closed outcome reason, PID, and sorted sanitized -properties. SIDs, capability names, GUIDs, PIDs/process identifiers, and +provider-scoped event ID, optional schema `eventName`, closed outcome reason, +PID, and sorted sanitized properties. The schema name distinguishes TraceLogging +events that share ID 0 and is separate from any payload field named `EventName`. +It is sanitized and bounded like other values, and is omitted when unavailable. +SIDs, capability names, GUIDs, PIDs/process identifiers, and non-file resource values are retained. Complete file paths are replaced with -``; standalone user/account names remain replaced with -``. +``; a path inside a command line or rendered array causes that entire +property value to be redacted. Standalone user/account names remain replaced +with ``. Properties whose names contain file paths are omitted. Exact header timestamps and timestamp-like properties are omitted so otherwise identical events deduplicate, and free-form decoder errors are never serialized. +This is an outcome summary, not an ordered event ledger: repeats become a count, +and one source event can produce several capability-denial outcomes. -Every valid actionable denial is classified as `actionable` in the verbose -file, including its first occurrence, later duplicates, and candidates observed -after the actionable file's unique-denial bound. Those occurrences deduplicate -under the same signature and increment its count. `accessType` and +Every valid policy denial is classified as `actionable` in the verbose file. +Its first occurrence, later duplicates, and candidates observed after the +actionable file's unique-denial bound are all retained. Those occurrences +deduplicate under the same signature and increment its count. `accessType` and `resourceType` are included when denial extraction determined them; diagnostic outcomes without those classifications omit the fields. Candidates excluded from the actionable output retain a closed diagnostic reason and their sanitized event properties: +- `comActivation` and `comInterfaceCall` identify recognized classic COM + permission decisions. They include `resourceType: "other"` and omit + `accessType`; their CLSID/IID remains only in the local verbose properties. - `notActionable` includes registry writes, registry checks whose access mask cannot be classified as a read, and recognized Section, SymbolicLink, and Timer checks. MXC has no corresponding policy grants, so reporting them in @@ -297,7 +341,8 @@ reason and their sanitized event properties: - `unsupportedObjectType` means the event names a resource outside the supported diagnostic model. Examples include `\BaseNamedObjects` as a Directory, ALPC Ports such as - `ubpmtaskhostchannel`, and RPC Interface GUIDs. + `ubpmtaskhostchannel`, and lower-layer RPC Interface GUIDs. The explicit COM + object types documented above are supported and do not use this outcome. Property values longer than 256 characters retain bounded prefix and suffix context plus a SHA-256 digest of the complete sanitized value. This keeps long @@ -305,26 +350,45 @@ named-object resources individually identifiable when they share a prefix without exceeding the per-property bound. Redaction occurs before the digest is computed, so neither retained context nor a digest is derived from a sensitive value. -Unknown event IDs from known Learning Mode providers are classified as -`unsupportedEventSchema`; the real ETL path retains their provider GUID and -PID without attempting an unsupported TDH payload decode. +Unknown providers and event IDs receive best-effort TDH decoding. Events with no +actionable extractor use `unsupportedEventSchema` and retain their sanitized +properties. Unknown providers use `provider: "other"` with their actual GUID +in the local file. Different provider GUIDs remain separate deduplication keys. +They do not create new actionable policy grants. Per-event TDH failures use closed diagnostic reasons: `eventPayloadMalformed` means the payload conflicts with its declared schema, `decoderLimitReached` means a nesting/element/work safety bound stopped decoding, and `unsupportedPropertyEncoding` means the decoder cannot consume that property shape. When TDH exposes it, the schema-declared name is retained -as the bounded `EventName` signature property. Free-form decoder errors are -never serialized. Failure to obtain the event schema remains a fatal analysis -error rather than being represented as a verbose logging signature. +in the optional `eventName` metadata field, with no partial payload properties. +Free-form decoder errors are +never serialized. Failure to obtain the event schema is retained as +`schemaUnavailable` rather than aborting the analysis, and marks actionable +results incomplete when the event belongs to a supported denial schema. +For brokered Event 28, scoped analysis marks the result incomplete even when +the missing schema prevents reading its workload PID; unattributed event +contents are not retained. To keep diagnostics bounded, verbose logging retains at most 4,096 distinct -signatures, 24 sorted properties per signature, and 256 characters per property -value. `overflowOccurrences` and `aggregateGroupsTruncated` indicate that +signatures and 16 MiB of compact signature data, with 24 sorted properties per +signature and 256 characters per property value. +`overflowOccurrences` and `aggregateGroupsTruncated` indicate that additional diagnostic groups were omitted. `actionableOverflowOccurrences` counts omitted actionable-denial occurrences, while `processedEventsTruncated` indicates that the 1,000,000-event limit prevented complete accounting. The actionable file itself is never reduced to make room for verbose logging. +The existing 64 MiB guarded analysis frame can also move verbose groups into +overflow accounting. These bounds are retained; the file does not claim complete +event-by-event coverage. + +Guarded WPR keeps only events from the exact job-attested process lifetimes, +including brokered capability attribution, regardless of provider. Its generated +relogging header is excluded from analysis so it does not add a diagnostic that +was absent from the source. A schema lookup failure while scoping brokered +Event 28 fails the capture, because its workload PID cannot be established. +Neither the unattributed event nor a potentially incomplete filtered trace is +returned. The host-wide source ETL is not transferred. The actionable and verbose logging files fail together: MXC stages both and reports capture failure unless both final artifacts are committed. The verbose logging path @@ -336,7 +400,9 @@ When stable telemetry is enabled and authorized, MXC may validate, compact, and send this redacted verbose document through `Microsoft.MXC/MXC.VerboseDenials`. Each event contains a valid JSON array of complete signatures and document reconstruction metadata. Before emission, MXC derives provider GUIDs from the -closed provider enum and drops every verbose property name and value. MXC does +closed provider enum and drops every verbose property name and value. For `other`, +the telemetry GUID is empty. Matching groups are combined again after these +values and the schema `eventName` are removed. MXC does not send the actionable denials file, workload-derived properties, or raw ETL through telemetry. See [MXC telemetry](../telemetry/telemetry.md). @@ -372,9 +438,10 @@ WPR's source ETL is host-wide, so the elevated guarded-WPR helper never transfers that file across the privilege boundary for `captureDenials` or `--audit`. After the sandbox process tree terminates, the helper uses the retained, job-attested process handles and their exact PID/creation/exit -`FILETIME` ranges to relog a second ETL. The -retained ETL contains only supported Learning Mode events whose event header -falls inside one of those attested process generations. Guarded analysis and +`FILETIME` ranges to relog a second ETL. The retained ETL contains events from +any provider attributed to those process generations, plus required ETL metadata. +Brokered capability events use their payload `ProcessId` rather than the +broker's header PID. Guarded analysis and retention both consume that same filtered ETL; filtering failure transfers no trace. The host-wide source remains in protected elevated scratch and is deleted with that scratch. The unelevated caller writes the filtered retained diff --git a/src/Cargo.lock b/src/Cargo.lock index 8356b6208..b0dc2b590 100644 --- a/src/Cargo.lock +++ b/src/Cargo.lock @@ -1374,7 +1374,9 @@ dependencies = [ "flatbuffers", "learning_mode_core", "process_security_environment_spec", + "tempfile", "thiserror", + "tracelogging", "windows", "windows-core", "wxc_common", diff --git a/src/backends/process_container/common/src/capture_output.rs b/src/backends/process_container/common/src/capture_output.rs index d575f5dbd..6c5bb9368 100644 --- a/src/backends/process_container/common/src/capture_output.rs +++ b/src/backends/process_container/common/src/capture_output.rs @@ -336,6 +336,7 @@ mod tests { let output_path = directory.path().join("denials.json"); let mut analysis = AnalysisResult::complete(Vec::new()); analysis.verbose_logging.record(VerboseLoggingSignature { + event_name: None, provider: VerboseLoggingProvider::KernelGeneral, provider_guid: "{A68CA8B7-004F-D7B6-A698-07E2DE0F1F5D}".to_string(), event_id: 14, diff --git a/src/core/learning_mode_core/src/analyze.rs b/src/core/learning_mode_core/src/analyze.rs index c924fc04b..9fcfdff5c 100644 --- a/src/core/learning_mode_core/src/analyze.rs +++ b/src/core/learning_mode_core/src/analyze.rs @@ -306,6 +306,7 @@ mod tests { let signature = |pid| VerboseLoggingAggregate { signature: VerboseLoggingSignature { provider: VerboseLoggingProvider::KernelGeneral, + event_name: None, provider_guid: "provider".to_string(), event_id: 14, reason: VerboseLoggingOutcomeReason::Actionable, @@ -351,6 +352,7 @@ mod tests { signatures: vec![VerboseLoggingAggregate { signature: VerboseLoggingSignature { provider: VerboseLoggingProvider::KernelGeneral, + event_name: None, provider_guid: "provider".to_string(), event_id: 14, reason: VerboseLoggingOutcomeReason::Actionable, @@ -395,6 +397,7 @@ mod tests { signature: VerboseLoggingSignature { provider: VerboseLoggingProvider::KernelGeneral, provider_guid: "provider".to_string(), + event_name: None, event_id: 14, reason: VerboseLoggingOutcomeReason::Actionable, pid: 1, @@ -428,6 +431,7 @@ mod tests { .map(|pid| VerboseLoggingAggregate { signature: VerboseLoggingSignature { provider: VerboseLoggingProvider::KernelGeneral, + event_name: None, provider_guid: "provider".to_string(), event_id: 14, reason: VerboseLoggingOutcomeReason::MissingObjectName, diff --git a/src/core/learning_mode_core/src/verbose_logging.rs b/src/core/learning_mode_core/src/verbose_logging.rs index a3870eac0..55c9be79c 100644 --- a/src/core/learning_mode_core/src/verbose_logging.rs +++ b/src/core/learning_mode_core/src/verbose_logging.rs @@ -17,10 +17,12 @@ pub const MAX_VERBOSE_LOGGING_GROUPS: usize = 4_096; /// 64 MiB frame limit for actionable denials and envelope overhead. pub const MAX_VERBOSE_LOGGING_SIGNATURE_BYTES: usize = 16 * 1024 * 1024; -/// Stable category for a known Learning Mode ETW provider. +/// Stable category for a Learning Mode ETW provider. #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)] #[serde(rename_all = "camelCase")] pub enum VerboseLoggingProvider { + /// Any other provider, identified by its GUID in the local output. + Other, /// Microsoft-Windows-Kernel-General. KernelGeneral, /// Microsoft-Windows-Privacy-Auditing-PermissiveLearningMode. @@ -33,8 +35,10 @@ pub enum VerboseLoggingProvider { pub enum VerboseLoggingOutcomeReason { /// The event produced a valid actionable denial. Actionable, - /// The provider is known, but the event ID is not a supported denial schema. + /// No actionable extractor supports this provider/event pair. UnsupportedEventSchema, + /// TDH could not resolve the event schema. + SchemaUnavailable, /// The event payload conflicted with its declared TDH schema. EventPayloadMalformed, /// A decoder safety bound prevented full payload processing. @@ -51,6 +55,10 @@ pub enum VerboseLoggingOutcomeReason { UnusableResourcePath, /// A capability event did not contain a usable capability denial. UnresolvedCapability, + /// A valid classic COM class-activation denial was retained for diagnostics. + ComActivation, + /// A valid classic COM interface-call denial was retained for diagnostics. + ComInterfaceCall, /// The event was valid but did not describe an actionable denial. NotActionable, } @@ -73,6 +81,9 @@ pub struct VerboseLoggingSignature { pub provider_guid: String, /// Provider-scoped ETW schema identifier. pub event_id: u16, + /// Sanitized schema name, separate from payload properties. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub event_name: Option, /// Closed exclusion category. pub reason: VerboseLoggingOutcomeReason, /// Process identifier from the event header. @@ -303,7 +314,7 @@ pub struct VerboseLoggingDocumentSummary { impl VerboseLoggingDocument { /// Current verbose logging document schema version. - pub const VERSION: u32 = 2; + pub const VERSION: u32 = 3; /// Builds an on-disk document from decoder aggregate state. #[must_use] @@ -371,6 +382,7 @@ mod tests { fn aggregates_and_sorts_sanitized_signatures() { let mut summary = VerboseLoggingSummary::default(); let signature = VerboseLoggingSignature { + event_name: None, provider: VerboseLoggingProvider::KernelGeneral, provider_guid: "{A68CA8B7-004F-D7B6-A698-07E2DE0F1F5D}".to_string(), event_id: 14, @@ -407,6 +419,7 @@ mod tests { for event_id in 0..MAX_VERBOSE_LOGGING_GROUPS as u16 { summary.record(VerboseLoggingSignature { provider: VerboseLoggingProvider::KernelGeneral, + event_name: None, provider_guid: "kernel".to_string(), event_id, reason: VerboseLoggingOutcomeReason::UnsupportedEventSchema, @@ -418,6 +431,7 @@ mod tests { } summary.record(VerboseLoggingSignature { provider: VerboseLoggingProvider::PrivacyAuditingPermissiveLearningMode, + event_name: None, provider_guid: "privacy".to_string(), event_id: u16::MAX, reason: VerboseLoggingOutcomeReason::UnsupportedEventSchema, @@ -439,6 +453,7 @@ mod tests { summary.record(VerboseLoggingSignature { provider: VerboseLoggingProvider::KernelGeneral, provider_guid: "kernel".to_string(), + event_name: None, event_id, reason: VerboseLoggingOutcomeReason::UnsupportedEventSchema, pid: 1, @@ -452,6 +467,7 @@ mod tests { provider_guid: "kernel".to_string(), event_id: u16::MAX, reason: VerboseLoggingOutcomeReason::Actionable, + event_name: None, pid: 1, access_type: Some(crate::AccessType::Read), resource_type: Some(crate::ResourceType::File), @@ -466,6 +482,43 @@ mod tests { assert_eq!(summary.actionable_overflow_occurrences, 0); } + #[test] + fn com_signature_overflow_is_not_counted_as_actionable() { + let mut summary = VerboseLoggingSummary::default(); + for event_id in 0..MAX_VERBOSE_LOGGING_GROUPS as u16 { + summary.record(VerboseLoggingSignature { + provider: VerboseLoggingProvider::KernelGeneral, + provider_guid: "kernel".to_string(), + event_id, + reason: VerboseLoggingOutcomeReason::UnsupportedEventSchema, + pid: 1, + event_name: None, + access_type: None, + resource_type: None, + properties: Vec::new(), + }); + } + + summary.record(VerboseLoggingSignature { + provider: VerboseLoggingProvider::KernelGeneral, + provider_guid: "kernel".to_string(), + event_id: u16::MAX, + reason: VerboseLoggingOutcomeReason::ComActivation, + event_name: None, + pid: 1, + access_type: None, + resource_type: Some(crate::ResourceType::Other), + properties: Vec::new(), + }); + + assert_eq!(summary.overflow_occurrences, 1); + assert_eq!(summary.actionable_overflow_occurrences, 0); + assert!(!summary + .signatures + .iter() + .any(|group| { group.signature.reason == VerboseLoggingOutcomeReason::ComActivation })); + } + #[test] fn byte_budget_leaves_guarded_analysis_protocol_headroom() { let mut summary = VerboseLoggingSummary::default(); @@ -476,6 +529,7 @@ mod tests { for pid in 0..MAX_VERBOSE_LOGGING_GROUPS as u32 { summary.record_with_byte_budget( VerboseLoggingSignature { + event_name: None, provider: VerboseLoggingProvider::KernelGeneral, provider_guid: "{A68CA8B7-004F-D7B6-A698-07E2DE0F1F5D}".to_string(), event_id: 14, @@ -514,6 +568,32 @@ mod tests { assert_eq!(parsed, document); } + #[test] + fn schema_name_is_optional_and_separate_from_payload() { + let old = serde_json::json!({ + "provider": "other", + "providerGuid": "provider", + "eventId": 0, + "reason": "unsupportedEventSchema", + "pid": 42, + "properties": [["EventName", "payload-name"]] + }); + let mut signature: VerboseLoggingSignature = serde_json::from_value(old).unwrap(); + assert!(signature.event_name.is_none()); + assert!(serde_json::to_value(&signature) + .unwrap() + .get("eventName") + .is_none()); + signature.event_name = Some("schema-name".into()); + let json = serde_json::to_value(&signature).unwrap(); + assert_eq!(json["eventName"], "schema-name"); + assert_eq!(json["properties"][0][1], "payload-name"); + assert_eq!( + serde_json::from_value::(json).unwrap(), + signature + ); + } + #[test] fn document_uses_actionable_vocabulary() { let mut summary = VerboseLoggingSummary::default(); @@ -521,6 +601,7 @@ mod tests { provider: VerboseLoggingProvider::KernelGeneral, provider_guid: "kernel".to_string(), event_id: 14, + event_name: None, reason: VerboseLoggingOutcomeReason::Actionable, pid: 1, access_type: Some(crate::AccessType::Read), @@ -531,12 +612,26 @@ mod tests { summary.mark_actionable_limit_reached(); let value = serde_json::to_value(VerboseLoggingDocument::new(&summary)).unwrap(); - assert_eq!(value["version"], 2); + assert_eq!(value["version"], 3); assert_eq!(value["signatures"][0]["signature"]["reason"], "actionable"); assert_eq!(value["summary"]["actionableOverflowOccurrences"], 2); assert_eq!(value["summary"]["actionableLimitReached"], true); } + #[test] + fn document_serializes_distinct_nonactionable_com_reasons() { + for (reason, expected) in [ + (VerboseLoggingOutcomeReason::ComActivation, "comActivation"), + ( + VerboseLoggingOutcomeReason::ComInterfaceCall, + "comInterfaceCall", + ), + ] { + assert!(!reason.is_actionable()); + assert_eq!(serde_json::to_value(reason).unwrap(), expected); + } + } + #[test] fn sibling_path_replaces_the_actionable_extension() { assert_eq!( diff --git a/src/core/learning_mode_platforms/windows/Cargo.toml b/src/core/learning_mode_platforms/windows/Cargo.toml index 457ef2975..3dc53179c 100644 --- a/src/core/learning_mode_platforms/windows/Cargo.toml +++ b/src/core/learning_mode_platforms/windows/Cargo.toml @@ -17,5 +17,7 @@ windows = { workspace = true } windows-core = { workspace = true } [target.'cfg(target_os = "windows")'.dev-dependencies] +tempfile = { workspace = true } +tracelogging = { workspace = true } flatbuffers = { workspace = true } process_security_environment_spec = { workspace = true } diff --git a/src/core/learning_mode_platforms/windows/src/capability_dacl.rs b/src/core/learning_mode_platforms/windows/src/capability_dacl.rs index bee880627..0a8a90676 100644 --- a/src/core/learning_mode_platforms/windows/src/capability_dacl.rs +++ b/src/core/learning_mode_platforms/windows/src/capability_dacl.rs @@ -674,6 +674,7 @@ mod tests { fn parts(name: &str, value: String) -> DecodedEventParts { DecodedEventParts { provider: PRIVACY_LEARNING_MODE_PROVIDER, + event_name: None, event_id: ACCESS_CHECK_EVENT_ID, props: vec![ ("ObjectType".to_string(), "\"\"".to_string()), @@ -852,6 +853,7 @@ mod tests { #[test] fn ignores_non_permissive_provider() { let event = DecodedEventParts { + event_name: None, provider: GUID::zeroed(), event_id: ACCESS_CHECK_EVENT_ID, props: Vec::new(), diff --git a/src/core/learning_mode_platforms/windows/src/etl_decode.rs b/src/core/learning_mode_platforms/windows/src/etl_decode.rs index b3eb07d4b..4250df8de 100644 --- a/src/core/learning_mode_platforms/windows/src/etl_decode.rs +++ b/src/core/learning_mode_platforms/windows/src/etl_decode.rs @@ -2,7 +2,7 @@ // Licensed under the MIT License. //! Sealed-ETL decoder: turns the `.etl` delivered by the Learning Mode trace API -//! produces into cross-platform [`DeniedResource`]s. +//! into cross-platform [`DeniedResource`]s. //! //! The trace is opened in **file mode** (`EVENT_TRACE_LOGFILEW.LogFileName`, //! without `PROCESS_TRACE_MODE_REAL_TIME`). `ProcessTrace` walks every @@ -18,11 +18,11 @@ //! The diagnostic console has a separate real-time, display-oriented ETW //! consumer in `tools/mxc_diagnostic_console`. It is a binary-private module //! that owns trace sessions and channels arbitrary provider events to a UI. -//! This backend instead reads sealed files synchronously, filters a fixed -//! provider/event vocabulary, bounds results, and skips malformed individual -//! events without invalidating the rest of the capture. Depending on the tool would invert the workspace dependency -//! direction; shared generic TDH primitives can be extracted later if another -//! runtime consumer needs them. +//! This backend reads sealed files synchronously and keeps bounded, deduplicated +//! diagnostics even for unfamiliar or malformed events. Only known schemas +//! produce actionable denials. Depending on the tool would invert the workspace +//! dependency direction; shared generic TDH primitives can be extracted later +//! if another runtime consumer needs them. use std::collections::{HashMap, HashSet}; use std::os::windows::ffi::OsStrExt; @@ -144,6 +144,7 @@ struct Accumulator<'visitor> { schema_cache: tdh_decode::EventSchemaCache, verbose_logging: VerboseLoggingSummary, verbose_logging_signature_bytes: usize, + skip_relog_header: bool, } impl<'visitor> Accumulator<'visitor> { @@ -167,6 +168,7 @@ impl<'visitor> Accumulator<'visitor> { schema_cache: tdh_decode::EventSchemaCache::default(), verbose_logging: VerboseLoggingSummary::default(), verbose_logging_signature_bytes: 0, + skip_relog_header: false, } } @@ -197,6 +199,7 @@ impl<'visitor> Accumulator<'visitor> { schema_cache: tdh_decode::EventSchemaCache::default(), verbose_logging: VerboseLoggingSummary::default(), verbose_logging_signature_bytes: 0, + skip_relog_header: false, } } @@ -220,10 +223,11 @@ impl<'visitor> Accumulator<'visitor> { schema_cache: tdh_decode::EventSchemaCache::default(), verbose_logging: VerboseLoggingSummary::default(), verbose_logging_signature_bytes: 0, + skip_relog_header: false, } } - fn add_raw_denial(&mut self, raw: RawDenial) { + fn add_raw_denial(&mut self, raw: RawDenial, event_name: Option<&str>) { if !self.event_in_scope(raw.pid, raw.filetime) { return; } @@ -235,6 +239,7 @@ impl<'visitor> Accumulator<'visitor> { &raw, &candidate, VerboseLoggingOutcomeReason::UnusableResourcePath, + event_name, ); return; } @@ -247,6 +252,7 @@ impl<'visitor> Accumulator<'visitor> { &raw, &candidate, VerboseLoggingOutcomeReason::UnusableResourcePath, + event_name, ); return; } @@ -254,7 +260,12 @@ impl<'visitor> Accumulator<'visitor> { } else { path_norm::to_user_visible(&raw.object_name).unwrap_or_else(|| raw.object_name.clone()) }; - self.record_raw_denial_outcome(&raw, &resource, VerboseLoggingOutcomeReason::Actionable); + self.record_raw_denial_outcome( + &raw, + &resource, + VerboseLoggingOutcomeReason::Actionable, + event_name, + ); let dedup_resource = match raw.resource_type { learning_mode_core::ResourceType::File | learning_mode_core::ResourceType::Other => { resource.to_ascii_lowercase() @@ -295,6 +306,7 @@ impl<'visitor> Accumulator<'visitor> { raw: &RawDenial, resource: &str, reason: VerboseLoggingOutcomeReason, + event_name: Option<&str>, ) { let mut properties = raw .verbose_logging_properties @@ -332,7 +344,7 @@ impl<'visitor> Accumulator<'visitor> { crate::extractors::bound_properties(properties.into_iter().collect::>()); self.record_outcome( raw.provider, - raw.event_id, + (raw.event_id, event_name), reason, raw.pid, (Some(raw.access_type), Some(raw.resource_type)), @@ -347,18 +359,18 @@ impl<'visitor> Accumulator<'visitor> { fn record_exclusion( &mut self, provider: VerboseLoggingProvider, - event_id: u16, + event: (u16, Option<&str>), reason: VerboseLoggingOutcomeReason, pid: u32, properties: Vec<(String, String)>, ) { - self.record_outcome(provider, event_id, reason, pid, (None, None), properties); + self.record_outcome(provider, event, reason, pid, (None, None), properties); } fn record_outcome( &mut self, provider: VerboseLoggingProvider, - event_id: u16, + event: (u16, Option<&str>), reason: VerboseLoggingOutcomeReason, pid: u32, classification: ( @@ -371,13 +383,39 @@ impl<'visitor> Accumulator<'visitor> { let signature = VerboseLoggingSignature { provider, provider_guid: crate::extractors::verbose_logging_provider_guid(provider), - event_id, + event_id: event.0, + event_name: crate::extractors::sanitize_event_name(event.1), reason, pid, access_type, resource_type, properties, }; + self.record_signature(signature); + } + + fn record_unknown_provider_outcome( + &mut self, + provider: windows::core::GUID, + event: (u16, Option<&str>), + reason: VerboseLoggingOutcomeReason, + pid: u32, + properties: Vec<(String, String)>, + ) { + self.record_signature(VerboseLoggingSignature { + provider: VerboseLoggingProvider::Other, + provider_guid: crate::extractors::format_guid_braced_uppercase(provider), + event_id: event.0, + event_name: crate::extractors::sanitize_event_name(event.1), + reason, + pid, + access_type: None, + resource_type: None, + properties, + }); + } + + fn record_signature(&mut self, signature: VerboseLoggingSignature) { self.verbose_logging.record_with_byte_budget( signature, &mut self.verbose_logging_signature_bytes, @@ -418,13 +456,8 @@ impl<'visitor> Accumulator<'visitor> { /// Handles a TDH decode failure for one event. /// - /// Schema-level failures (the event's manifest itself could not be - /// resolved) are fatal in every mode: they indicate the trace/schema - /// state is unreliable beyond this single event. Per-event decode - /// failures are fatal only for the raw diagnostic visitor (which needs - /// every event to succeed); in [`CollectionMode::Analyze`] they are - /// aggregated into the verbose logging summary for a known provider instead of - /// silently dropped. + /// Analysis records a closed diagnostic reason and continues. Raw diagnostic + /// visitors require every event to decode successfully and fail instead. fn record_event_decode_error( &mut self, provider: windows::core::GUID, @@ -432,7 +465,7 @@ impl<'visitor> Accumulator<'visitor> { pid: u32, error: tdh_decode::DecodeError, ) { - let fatal = matches!(self.mode, CollectionMode::Raw) || error.is_schema_error(); + let fatal = matches!(self.mode, CollectionMode::Raw); if fatal { if self.decode_error.is_none() { self.decode_error = @@ -440,26 +473,25 @@ impl<'visitor> Accumulator<'visitor> { } return; } + let reason = match error.event_kind() { + Some(tdh_decode::EventDecodeKind::PayloadMalformed) => { + VerboseLoggingOutcomeReason::EventPayloadMalformed + } + Some(tdh_decode::EventDecodeKind::DecoderLimitReached) => { + VerboseLoggingOutcomeReason::DecoderLimitReached + } + Some(tdh_decode::EventDecodeKind::UnsupportedPropertyEncoding) => { + VerboseLoggingOutcomeReason::UnsupportedPropertyEncoding + } + None => VerboseLoggingOutcomeReason::SchemaUnavailable, + }; + if reason == VerboseLoggingOutcomeReason::SchemaUnavailable + && is_learning_mode_event(provider, event_id) + { + self.truncated = true; + } + let event = (event_id, error.event_name()); if let Some(category) = crate::extractors::verbose_logging_provider_for_guid(provider) { - let reason = match error.event_kind() { - Some(tdh_decode::EventDecodeKind::PayloadMalformed) => { - VerboseLoggingOutcomeReason::EventPayloadMalformed - } - Some(tdh_decode::EventDecodeKind::DecoderLimitReached) => { - VerboseLoggingOutcomeReason::DecoderLimitReached - } - Some(tdh_decode::EventDecodeKind::UnsupportedPropertyEncoding) => { - VerboseLoggingOutcomeReason::UnsupportedPropertyEncoding - } - None => return, - }; - // Retain only the bounded schema-declared name. The free-form - // decoder message can include property values and is never emitted. - let properties = error - .event_name() - .map(|name| vec![("EventName".to_string(), name.to_string())]) - .map(crate::extractors::bound_properties) - .unwrap_or_default(); let classification = match event_id { crate::extractors::LEARNING_MODE_VIOLATION_EVENT_ID => ( Some(learning_mode_core::AccessType::Unknown), @@ -478,7 +510,9 @@ impl<'visitor> Accumulator<'visitor> { }, _ => (None, None), }; - self.record_outcome(category, event_id, reason, pid, classification, properties); + self.record_outcome(category, event, reason, pid, classification, Vec::new()); + } else { + self.record_unknown_provider_outcome(provider, event, reason, pid, Vec::new()); } } @@ -544,6 +578,39 @@ impl EtlDenialAnalyzer { let process_lifetimes = attested_process_lifetimes(membership)?; self.analyze_for_process_lifetimes(source_path, &process_lifetimes) } + + /// Analyzes a relogged trace for exact job-attested process generations. + /// + /// The relogger's generated transport header is not a source observation. + /// + /// # Errors + /// + /// Returns [`AnalyzeError`] when job evidence is invalid or the trace cannot + /// be decoded. + pub fn analyze_relogged_for_job_membership( + &self, + source_path: &Path, + membership: &JobMembershipSnapshot, + ) -> Result { + let lifetimes = attested_process_lifetimes(membership)?; + self.analyze_relogged_for_process_lifetimes(source_path, &lifetimes) + } + + pub(crate) fn analyze_relogged_for_process_lifetimes( + &self, + source_path: &Path, + lifetimes: &[ProcessLifetime], + ) -> Result { + let mut accumulator = Accumulator::analyze_for_process_lifetimes(lifetimes); + accumulator.skip_relog_header = true; + process_trace_file(source_path, &mut accumulator)?; + if accumulator.skip_relog_header { + return Err(AnalyzeError::Decode( + "relogged trace has no transport header".into(), + )); + } + accumulator.into_analysis() + } } impl DenialAnalyzer for EtlDenialAnalyzer { @@ -616,7 +683,7 @@ pub fn visit_raw_events( Ok(accumulator.raw_event_count) } -/// Builds a bounded decision vector for known-provider Learning Mode events in +/// Builds a bounded decision vector for all in-scope events in /// source order. `ProcessTrace` normalizes each timestamp to FILETIME before /// the exact process-generation test, so Trace Relogger can replay these /// decisions without comparing its raw trace-clock timestamps. @@ -643,7 +710,7 @@ pub(crate) fn select_learning_mode_events_for_relogging( fn dedup_to_resources>(raws: I) -> AnalysisResult { let mut accumulator = Accumulator::analyze(); for raw in raws { - accumulator.add_raw_denial(raw); + accumulator.add_raw_denial(raw, None); } accumulator .into_analysis() @@ -705,7 +772,9 @@ fn process_trace_file( // ERROR_SUCCESS (0) is end-of-file. ERROR_CANCELLED (1223) is expected // when our buffer callback stops after a processing bound or fatal error. - if status.0 != 0 && status.0 != 1223 { + if status.0 != 0 + && !(status.0 == 1223 && (accumulator.stop_requested || accumulator.decode_error.is_some())) + { return Err(AnalyzeError::Decode(format!( "ProcessTrace failed for '{}': Win32 error {}", source_path.display(), @@ -781,10 +850,18 @@ unsafe fn process_event_record(event_record: *mut EVENT_RECORD, acc: &mut Accumu let provider = header.ProviderId; let event_id = header.EventDescriptor.Id; - if matches!(acc.mode, CollectionMode::SelectForRelogging) { - if crate::extractors::verbose_logging_provider_for_guid(provider).is_none() { - return; + if acc.skip_relog_header { + acc.skip_relog_header = false; + if provider != windows::core::GUID::from_u128(0x68fdd900_4a3e_11d1_84f4_0000f80464e3) + || event_id != 0 + || header.EventDescriptor.Opcode != 0 + { + acc.decode_error = Some("relogged trace has an invalid transport header".into()); } + return; + } + + if matches!(acc.mode, CollectionMode::SelectForRelogging) { let event_index = acc.relog_event_count; let Some(next_event_count) = acc.relog_event_count.checked_add(1) else { acc.decode_error = @@ -822,29 +899,13 @@ unsafe fn process_event_record(event_record: *mut EVENT_RECORD, acc: &mut Accumu let mut analyze_filetime = None; if matches!(acc.mode, CollectionMode::Analyze) { - let Some(category) = crate::extractors::verbose_logging_provider_for_guid(provider) else { - // Unrelated provider: unrelated host traffic, ignored entirely - // (not aggregated as an excluded Learning Mode outcome). - return; - }; let Some(filetime) = normalized_filetime(header.TimeStamp, acc) else { return; }; analyze_filetime = Some(filetime); - if !is_learning_mode_event(provider, event_id) { - if !acc.event_in_scope(header.ProcessId, filetime) || !acc.begin_event() { - return; - } - // A known provider, but outside its supported event vocabulary: - // aggregate (as a signature with no decoded properties) without - // paying for a TDH decode. - acc.record_exclusion( - category, - event_id, - VerboseLoggingOutcomeReason::UnsupportedEventSchema, - header.ProcessId, - Vec::new(), - ); + let brokered = event_id == crate::extractors::CAPABILITY_DENIAL_EVENT_ID + && is_learning_mode_event(provider, event_id); + if !brokered && !acc.event_in_scope(header.ProcessId, filetime) { return; } } @@ -864,12 +925,16 @@ unsafe fn process_event_record(event_record: *mut EVENT_RECORD, acc: &mut Accumu }, Err(error) => { if matches!(acc.mode, CollectionMode::Analyze) { - if error.is_schema_error() { - acc.record_event_decode_error(provider, event_id, header.ProcessId, error); - return; - } let filetime = analyze_filetime.expect("analyze mode has normalized FILETIME"); - let pid = if event_id == crate::extractors::CAPABILITY_DENIAL_EVENT_ID { + if matches!(&error, tdh_decode::DecodeError::Schema(_)) + && event_id == crate::extractors::CAPABILITY_DENIAL_EVENT_ID + && is_learning_mode_event(provider, event_id) + { + acc.truncated = true; + } + let pid = if event_id == crate::extractors::CAPABILITY_DENIAL_EVENT_ID + && is_learning_mode_event(provider, event_id) + { let process_id = unsafe { tdh_decode::decode_event_property( event_record, @@ -927,10 +992,9 @@ fn select_capability_decode_result_for_relogging( header_pid, filetime, ), - Err(error) if error.is_schema_error() => { - acc.decode_error = Some(format!( - "failed to decode brokered capability event while scoping guarded trace: {error}" - )); + Err(tdh_decode::DecodeError::Schema(_)) => { + acc.decode_error = + Some("could not scope brokered capability event: schema unavailable".into()); acc.stop_requested = true; } Err(_) => { @@ -977,27 +1041,32 @@ fn select_event_for_relogging( acc.relog_selected_event_pids.push(pid); } -/// Extracts denials from one decoded, in-vocabulary event and feeds them -/// (or their closed outcome reason) into `acc`. -/// -/// Re-checks the provider/event vocabulary so it stays a single source of -/// truth for both the real ETW path (which gates before decoding, above) -/// and pure-composition tests that hand this already-"decoded" fixtures. +/// Extracts known denials and retains other scoped events as diagnostics. fn handle_decoded_event( parts: &DecodedEventParts, header_pid: u32, filetime: u64, acc: &mut Accumulator<'_>, ) { + let event = (parts.event_id, parts.event_name.as_deref()); let Some(category) = crate::extractors::verbose_logging_provider_for_guid(parts.provider) else { + if acc.event_in_scope(header_pid, filetime) && acc.begin_event() { + acc.record_unknown_provider_outcome( + parts.provider, + event, + VerboseLoggingOutcomeReason::UnsupportedEventSchema, + header_pid, + crate::extractors::sanitize_properties(&parts.props), + ); + } return; }; if !is_learning_mode_event(parts.provider, parts.event_id) { if acc.event_in_scope(header_pid, filetime) && acc.begin_event() { acc.record_exclusion( category, - parts.event_id, + event, VerboseLoggingOutcomeReason::UnsupportedEventSchema, header_pid, crate::extractors::sanitize_properties(&parts.props), @@ -1009,7 +1078,7 @@ fn handle_decoded_event( if acc.event_in_scope(header_pid, filetime) && acc.begin_event() { acc.record_outcome( category, - parts.event_id, + event, VerboseLoggingOutcomeReason::EventPayloadMalformed, header_pid, crate::extractors::verbose_logging_classification(parts), @@ -1030,18 +1099,18 @@ fn handle_decoded_event( // `UnresolvedCapability` outcome is counted only when none are recovered. let capability_candidates = crate::capability_dacl::extract_denials(parts, pid, filetime); for raw in capability_candidates.iter().cloned() { - acc.add_raw_denial(raw); + acc.add_raw_denial(raw, event.1); } match primary { - Ok(raw) => acc.add_raw_denial(raw), + Ok(raw) => acc.add_raw_denial(raw, event.1), Err(reason) => { let recovered_by_dacl = reason == VerboseLoggingOutcomeReason::UnresolvedCapability && !capability_candidates.is_empty(); if !recovered_by_dacl { acc.record_outcome( category, - parts.event_id, + event, reason, pid, crate::extractors::verbose_logging_classification(parts), @@ -1076,6 +1145,133 @@ mod tests { const SCOPED_END_FILETIME: u64 = 200; const SCOPED_EVENT_FILETIME: u64 = 150; + #[test] + fn callback_decodes_unknown_events_and_deduplicates_decode_failures() { + let mut accumulator = Accumulator::analyze(); + let payload = 42u32.to_le_bytes(); + for (provider, names) in [ + (windows::core::GUID::from_u128(1), Some(vec!["FutureValue"])), + ( + crate::extractors::KERNEL_GENERAL_PROVIDER, + Some(vec!["FutureValue"]), + ), + ( + windows::core::GUID::from_u128(2), + Some(vec!["First", "Missing"]), + ), + (windows::core::GUID::from_u128(3), None), + ] { + for time in [100, 200] { + // SAFETY: the initialized record and payload remain live + // throughout the synchronous callback. + let mut record: EVENT_RECORD = unsafe { core::mem::zeroed() }; + record.EventHeader.ProviderId = provider; + record.EventHeader.EventDescriptor.Id = 999; + record.EventHeader.ProcessId = 42; + record.EventHeader.TimeStamp = time; + record.UserData = payload.as_ptr().cast_mut().cast(); + record.UserDataLength = payload.len() as u16; + if let Some(names) = &names { + accumulator.schema_cache.insert_test_schema(&record, names); + } + unsafe { process_event_record(&mut record, &mut accumulator) }; + } + } + let analysis = accumulator.into_analysis().unwrap(); + let groups = &analysis.verbose_logging.signatures; + assert_eq!(analysis.verbose_logging.total_occurrences, 8); + assert_eq!(groups.len(), 4); + assert!(groups.iter().all(|group| group.count == 2)); + let decoded = groups + .iter() + .filter(|group| { + group.signature.reason == VerboseLoggingOutcomeReason::UnsupportedEventSchema + }) + .collect::>(); + assert_eq!(decoded.len(), 2); + assert!(decoded + .iter() + .all(|group| property(&group.signature, "FutureValue") == "42")); + for reason in [ + VerboseLoggingOutcomeReason::EventPayloadMalformed, + VerboseLoggingOutcomeReason::SchemaUnavailable, + ] { + let group = groups + .iter() + .find(|group| group.signature.reason == reason) + .unwrap(); + assert!(group.signature.properties.is_empty()); + assert_eq!(group.signature.provider, VerboseLoggingProvider::Other); + } + assert!(analysis.denials.is_empty()); + } + + #[test] + fn unknown_event_dedup_preserves_provider_identity_and_process_scope() { + let properties = [("ProcessId", "99"), ("FutureValue", "42")]; + let first = windows::core::GUID::from_u128(1); + let second = windows::core::GUID::from_u128(2); + let events = [ + event_with_provider(first, 28, 42, 150, &properties), + event_with_provider(first, 28, 42, 151, &properties), + event_with_provider(second, 28, 42, 152, &properties), + event_with_provider(first, 29, 42, 153, &properties), + event_with_provider(first, 28, 99, 150, &[("ProcessId", "42")]), + event_with_provider(first, 28, 42, 201, &properties), + kernel_event( + 14, + 42, + 160, + &[ + ("ObjectType", "File"), + ("ObjectName", r"C:\kept.txt"), + ("AccessMask", "1"), + ], + ), + ]; + let analysis = resources_from_events_for_process_lifetimes( + &events, + Some(&[ProcessLifetime { + pid: 42, + start_filetime: 100, + end_filetime: 200, + }]), + ); + let groups = &analysis.verbose_logging.signatures; + assert_eq!(groups.len(), 4); + assert_eq!(analysis.verbose_logging.total_occurrences, 5); + let repeated = groups + .iter() + .find(|group| { + group.signature.provider_guid + == crate::extractors::format_guid_braced_uppercase(first) + && group.signature.event_id == 28 + }) + .unwrap(); + assert_eq!(repeated.count, 2); + assert_eq!(repeated.signature.pid, 42); + let expected = vec![DeniedResource { + resource: r"C:\kept.txt".into(), + resource_type: ResourceType::File, + access_type: AccessType::Read, + pid: 42, + filetime: 160, + }]; + let serialize = |denials| { + let mut bytes = Vec::new(); + learning_mode_core::write_document( + &mut bytes, + &learning_mode_core::DenialsDocument::new( + denials, + learning_mode_core::DenialSummary::new(0, 1, false), + ), + ) + .unwrap(); + bytes + }; + assert_eq!(serialize(analysis.denials), serialize(expected)); + } + #[test] fn process_lifetime_index_matches_pid_and_merged_time_ranges() { let index = ProcessLifetimeIndex::new(&[ @@ -1118,7 +1314,7 @@ mod tests { } #[test] - fn relog_selection_tracks_known_provider_ordinals_and_exact_lifetimes() { + fn relog_selection_tracks_all_provider_ordinals_and_exact_lifetimes() { let mut accumulator = Accumulator::select_for_relogging(&[ProcessLifetime { pid: 42, start_filetime: 100, @@ -1143,8 +1339,8 @@ mod tests { visit(crate::extractors::KERNEL_GENERAL_PROVIDER, 999, 42, 200); visit(crate::extractors::KERNEL_GENERAL_PROVIDER, 14, 43, 150); - assert_eq!(accumulator.relog_event_count, 4); - assert_eq!(accumulator.relog_selected_event_indices, [0, 2]); + assert_eq!(accumulator.relog_event_count, 5); + assert_eq!(accumulator.relog_selected_event_indices, [0, 1, 3]); assert!(accumulator.decode_error.is_none()); } @@ -1195,6 +1391,7 @@ mod tests { }]); let parts = DecodedEventParts { provider: crate::extractors::KERNEL_GENERAL_PROVIDER, + event_name: None, event_id: crate::extractors::CAPABILITY_DENIAL_EVENT_ID, props: Vec::new(), }; @@ -1234,29 +1431,80 @@ mod tests { } #[test] - fn relog_selection_stops_on_capability_schema_failure() { + fn brokered_schema_failure_marks_incomplete_before_pid_resolution() { + for (provider, event_id, incomplete) in [ + (crate::extractors::KERNEL_GENERAL_PROVIDER, 28, true), + (crate::extractors::KERNEL_GENERAL_PROVIDER, 14, false), + (crate::extractors::PRIVACY_LEARNING_MODE_PROVIDER, 28, false), + (windows::core::GUID::from_u128(1), 28, false), + ] { + let mut accumulator = Accumulator::analyze_for_process_lifetimes(&[ProcessLifetime { + pid: 42, + start_filetime: 100, + end_filetime: 200, + }]); + let mut record = EVENT_RECORD::default(); + record.EventHeader.ProviderId = provider; + record.EventHeader.EventDescriptor.Id = event_id; + record.EventHeader.EventDescriptor.Version = u8::MAX; + record.EventHeader.ProcessId = 9000; + record.EventHeader.TimeStamp = 150; + if incomplete { + assert!(matches!( + unsafe { + tdh_decode::decode_event_parts(&mut record, &mut accumulator.schema_cache) + }, + Err(tdh_decode::DecodeError::Schema(_)) + )); + } + unsafe { process_event_record(&mut record, &mut accumulator) }; + assert_eq!(accumulator.truncated, incomplete); + assert!(accumulator.verbose_logging.is_empty()); + assert!(!accumulator.stop_requested); + assert!(accumulator.decode_error.is_none()); + + let valid = kernel_event( + 14, + 42, + 160, + &[ + ("ObjectType", "File"), + ("ObjectName", r"C:\kept.txt"), + ("AccessMask", "1"), + ], + ); + handle_decoded_event(&valid.parts, valid.pid, valid.filetime, &mut accumulator); + let analysis = accumulator.into_analysis().unwrap(); + assert_eq!(analysis.denied_resources_truncated, incomplete); + assert_eq!(analysis.denials.len(), 1); + assert_eq!(analysis.denials[0].resource, r"C:\kept.txt"); + assert_eq!(analysis.verbose_logging.total_occurrences, 1); + } + } + + #[test] + fn relog_selection_rejects_unattributable_capability_schema_failure() { let mut accumulator = Accumulator::select_for_relogging(&[ProcessLifetime { pid: 42, start_filetime: 100, end_filetime: 200, }]); - - select_capability_decode_result_for_relogging( - &mut accumulator, - 0, - Err(tdh_decode::DecodeError::Schema( - "manifest unavailable".to_string(), - )), - 9000, - 150, - ); + let mut record = EVENT_RECORD::default(); + record.EventHeader.ProviderId = crate::extractors::KERNEL_GENERAL_PROVIDER; + record.EventHeader.EventDescriptor.Id = crate::extractors::CAPABILITY_DENIAL_EVENT_ID; + record.EventHeader.EventDescriptor.Version = u8::MAX; + record.EventHeader.ProcessId = 9000; + record.EventHeader.TimeStamp = 150; + unsafe { process_event_record(&mut record, &mut accumulator) }; assert!(accumulator.relog_selected_event_indices.is_empty()); + assert!(accumulator.relog_selected_event_pids.is_empty()); + assert!(accumulator.verbose_logging.is_empty()); assert!(accumulator.stop_requested); - assert!(accumulator - .decode_error - .as_deref() - .is_some_and(|error| error.contains("manifest unavailable"))); + assert_eq!( + accumulator.decode_error.as_deref(), + Some("could not scope brokered capability event: schema unavailable") + ); } #[test] @@ -1284,6 +1532,7 @@ mod tests { let mut accumulator = Accumulator::raw(&mut visitor); let parts = DecodedEventParts { provider: windows::core::GUID::from_u128(0), + event_name: None, event_id: 1, props: Vec::new(), }; @@ -1341,7 +1590,7 @@ mod tests { } #[test] - fn analyze_mode_reports_schema_lookup_failure() { + fn analyze_mode_records_schema_lookup_failure_without_aborting() { let mut accumulator = Accumulator::analyze(); accumulator.record_event_decode_error( windows::core::GUID::from_u128(1), @@ -1350,8 +1599,84 @@ mod tests { tdh_decode::DecodeError::Schema("manifest unavailable".to_string()), ); - let error = accumulator.into_analysis().unwrap_err(); - assert!(error.to_string().contains("manifest unavailable")); + let analysis = accumulator.into_analysis().unwrap(); + let group = &analysis.verbose_logging.signatures[0]; + assert_eq!( + group.signature.reason, + VerboseLoggingOutcomeReason::SchemaUnavailable + ); + assert!(group.signature.properties.is_empty()); + assert_eq!(group.count, 1); + let mut bytes = Vec::new(); + learning_mode_core::write_verbose_logging_document( + &mut bytes, + &learning_mode_core::VerboseLoggingDocument::new(&analysis.verbose_logging), + ) + .unwrap(); + assert!(!String::from_utf8(bytes) + .unwrap() + .contains("manifest unavailable")); + } + + #[test] + fn schema_failure_completeness_follows_the_supported_event_vocabulary() { + let kernel = crate::extractors::KERNEL_GENERAL_PROVIDER; + let privacy = crate::extractors::PRIVACY_LEARNING_MODE_PROVIDER; + for (provider, event_id, incomplete) in [ + (kernel, 14, true), + (kernel, 27, true), + (kernel, 28, true), + (privacy, 14, true), + (privacy, 27, true), + (privacy, 4907, true), + (kernel, 999, false), + (privacy, 28, false), + (windows::core::GUID::from_u128(1), 14, false), + ] { + let mut accumulator = Accumulator::analyze(); + accumulator.record_event_decode_error( + provider, + event_id, + 42, + tdh_decode::DecodeError::Schema("manifest unavailable".into()), + ); + let event = kernel_event( + 14, + 42, + 150, + &[ + ("ObjectType", "File"), + ("ObjectName", r"C:\kept.txt"), + ("AccessMask", "1"), + ], + ); + handle_decoded_event(&event.parts, event.pid, event.filetime, &mut accumulator); + let analysis = accumulator.into_analysis().unwrap(); + assert_eq!( + analysis.denied_resources_truncated, incomplete, + "{provider:?} {event_id}" + ); + assert_eq!(analysis.denials.len(), 1); + assert_eq!(analysis.denials[0].resource, r"C:\kept.txt"); + assert_eq!(analysis.verbose_logging.total_occurrences, 2); + assert!(analysis.verbose_logging.signatures.iter().any(|group| { + group.signature.reason == VerboseLoggingOutcomeReason::SchemaUnavailable + })); + let mut bytes = Vec::new(); + let summary = learning_mode_core::DenialSummary::new( + 0, + analysis.denials.len(), + analysis.denied_resources_truncated, + ); + learning_mode_core::write_document( + &mut bytes, + &learning_mode_core::DenialsDocument::new(analysis.denials, summary), + ) + .unwrap(); + assert!(String::from_utf8(bytes) + .unwrap() + .contains(&format!("\"deniedResourcesTruncated\": {incomplete}"))); + } } fn raw(path: &str, access: AccessType, rt: ResourceType) -> RawDenial { @@ -1502,11 +1827,14 @@ mod tests { .map(|index| (format!(r"c:\data\{index}.txt"), AccessType::Read)) .collect(); - accumulator.add_raw_denial(raw( - r"C:\data\overflow.txt", - AccessType::Read, - ResourceType::File, - )); + accumulator.add_raw_denial( + raw( + r"C:\data\overflow.txt", + AccessType::Read, + ResourceType::File, + ), + None, + ); assert!( !accumulator.stop_requested, @@ -1527,12 +1855,18 @@ mod tests { // Processing continues past the cap: a further overflow candidate is // still aggregated (not silently discarded), and a duplicate of an // already-actionable denial retains the same actionable classification. - accumulator.add_raw_denial(raw( - r"C:\data\overflow-2.txt", - AccessType::Read, - ResourceType::File, - )); - accumulator.add_raw_denial(raw(r"C:\data\0.txt", AccessType::Read, ResourceType::File)); + accumulator.add_raw_denial( + raw( + r"C:\data\overflow-2.txt", + AccessType::Read, + ResourceType::File, + ), + None, + ); + accumulator.add_raw_denial( + raw(r"C:\data\0.txt", AccessType::Read, ResourceType::File), + None, + ); assert!(!accumulator.stop_requested); assert_eq!( @@ -1645,6 +1979,7 @@ mod tests { parts: DecodedEventParts { provider, event_id, + event_name: None, props: kv .iter() .map(|(k, v)| ((*k).to_string(), (*v).to_string())) @@ -1804,6 +2139,110 @@ mod tests { assert_eq!(out[2].resource_type, ResourceType::Ui); } + #[test] + fn com_access_checks_are_distinct_verbose_only_outcomes_in_both_modes() { + let activation_clsid = "{A47979D2-C419-11D9-A5B4-001185AD2B89}"; + let call_iid = "{00000132-0000-0000-C000-000000000046}"; + let events = vec![ + kernel_event( + 14, + 100, + 1, + &[ + ("Mode", "\"Normal\""), + ("ObjectType", "\"ComActivationForClass\""), + ("ObjectName", activation_clsid), + ("AccessMask", "0x1"), + ], + ), + kernel_event( + 14, + 101, + 2, + &[ + ("Mode", "\"Permissive\""), + ("ObjectType", "\"ComActivationForClass\""), + ("ObjectName", "\"{a47979d2-c419-11d9-a5b4-001185ad2b89}\""), + ("AccessMask", "0xffffffff"), + ], + ), + kernel_event( + 14, + 102, + 3, + &[ + ("Mode", "\"Permissive\""), + ("ObjectType", "\"ComCallOnInterface\""), + ("ObjectName", call_iid), + ("AccessMask", "0x2"), + ], + ), + ]; + + let analysis = resources_from_events(&events); + + assert!(analysis.denials.is_empty()); + let activation_signatures = analysis + .verbose_logging + .signatures + .iter() + .filter(|group| property(&group.signature, "ObjectType") == "ComActivationForClass") + .collect::>(); + assert_eq!(activation_signatures.len(), 2); + assert!(activation_signatures.iter().all(|group| { + group.signature.reason == VerboseLoggingOutcomeReason::ComActivation + && group.signature.resource_type == Some(ResourceType::Other) + && group.signature.access_type.is_none() + && group.count == 1 + })); + assert!(activation_signatures + .iter() + .any(|group| property(&group.signature, "ObjectName") == activation_clsid)); + + let call = analysis + .verbose_logging + .signatures + .iter() + .find(|group| property(&group.signature, "ObjectType") == "ComCallOnInterface") + .expect("COM interface call should remain in verbose logging"); + assert_eq!( + call.signature.reason, + VerboseLoggingOutcomeReason::ComInterfaceCall + ); + assert_eq!(call.signature.resource_type, Some(ResourceType::Other)); + assert!(call.signature.access_type.is_none()); + assert_eq!(property(&call.signature, "ObjectName"), call_iid); + } + + #[test] + fn malformed_com_identifier_remains_classified_verbose_diagnostic() { + let events = vec![kernel_event( + 14, + 42, + 1, + &[ + ("Mode", "\"Normal\""), + ("ObjectType", "\"ComActivationForClass\""), + ("ObjectName", "\"not-a-clsid\""), + ("AccessMask", "0x1"), + ], + )]; + + let analysis = resources_from_events(&events); + + assert!(analysis.denials.is_empty()); + assert_eq!(analysis.verbose_logging.signatures.len(), 1); + let signature = &analysis.verbose_logging.signatures[0].signature; + assert_eq!( + signature.reason, + VerboseLoggingOutcomeReason::EventPayloadMalformed + ); + assert_eq!(signature.resource_type, Some(ResourceType::Other)); + assert!(signature.access_type.is_none()); + assert_eq!(property(signature, "ObjectType"), "ComActivationForClass"); + assert_eq!(property(signature, "ObjectName"), "not-a-clsid"); + } + #[test] fn unidentified_capability_events_are_omitted() { let events = vec![ @@ -2076,6 +2515,7 @@ mod tests { parts: DecodedEventParts { provider: crate::extractors::KERNEL_GENERAL_PROVIDER, event_id: 14, + event_name: None, props: vec![ ("Mode".to_string(), "\"Permissive\"".to_string()), ("ObjectType".to_string(), format!("\"{object_type}\"")), @@ -2126,6 +2566,7 @@ mod tests { parts: DecodedEventParts { provider: crate::extractors::KERNEL_GENERAL_PROVIDER, event_id: 14, + event_name: None, props: vec![ ("ObjectType".to_string(), "\"Section\"".to_string()), ("ObjectName".to_string(), format!("\"{object_name}\"")), @@ -2263,10 +2704,7 @@ mod tests { } #[test] - fn unrelated_provider_is_ignored_without_accounting() { - // A provider outside the known Learning Mode vocabulary must not - // contribute any verbose logging accounting at all, even though its event - // ID happens to collide with a known access-check ID. + fn unknown_provider_is_retained_without_creating_actionable_denials() { let events = vec![event_with_provider( windows::core::GUID::from_u128(0xdead_beef), 14, @@ -2281,16 +2719,21 @@ mod tests { let out = resources_from_events(&events); assert!(out.denials.is_empty()); - assert!( - out.verbose_logging.is_empty(), - "unrelated providers are ignored, not aggregated" + assert_eq!(out.verbose_logging.signatures.len(), 1); + let group = &out.verbose_logging.signatures[0]; + assert_eq!(group.signature.provider, VerboseLoggingProvider::Other); + assert_eq!( + group.signature.provider_guid, + "{00000000-0000-0000-0000-0000DEADBEEF}" ); + assert_eq!(property(&group.signature, "ObjectName"), ""); + assert_eq!(group.count, 1); } #[test] fn actionable_candidates_and_duplicates_share_one_verbose_logging_signature() { let make_event = |sequence_no: u64| { - kernel_event( + let mut event = kernel_event( 14, 7, sequence_no, @@ -2300,7 +2743,9 @@ mod tests { ("AccessMask", "0x1"), ("UserName", "\"jsmith\""), ], - ) + ); + event.parts.event_name = Some("AccessCheck".into()); + event }; let events = vec![make_event(1), make_event(2), make_event(3)]; @@ -2322,6 +2767,10 @@ mod tests { VerboseLoggingProvider::KernelGeneral ); assert_eq!(actionable_group.signature.event_id, 14); + assert_eq!( + actionable_group.signature.event_name.as_deref(), + Some("AccessCheck") + ); assert_eq!( actionable_group.count, 3, "all three occurrences are retained" @@ -2385,7 +2834,7 @@ mod tests { // capability candidate must not also surface an `UnresolvedCapability` // exclusion for the same event. let dacl = "hex:000000000000000001000000010200000000000F0300000001000000"; - let events = vec![permissive_event( + let mut events = vec![permissive_event( 14, 5900, 21, @@ -2397,6 +2846,7 @@ mod tests { ("Dacl", dacl), ], )]; + events[0].parts.event_name = Some("CapabilityCheck".into()); let out = resources_from_events(&events); assert_eq!(out.denials.len(), 1); @@ -2406,6 +2856,7 @@ mod tests { assert_eq!(signature.reason, VerboseLoggingOutcomeReason::Actionable); assert_eq!(signature.access_type, Some(AccessType::Unknown)); assert_eq!(signature.resource_type, Some(ResourceType::Capability)); + assert_eq!(signature.event_name.as_deref(), Some("CapabilityCheck")); } fn find_signature( @@ -2446,7 +2897,7 @@ mod tests { for pid in 0..learning_mode_core::MAX_VERBOSE_LOGGING_GROUPS as u32 { accumulator.record_exclusion( VerboseLoggingProvider::KernelGeneral, - 14, + (14, None), VerboseLoggingOutcomeReason::Actionable, pid, properties.clone(), @@ -2606,7 +3057,10 @@ mod tests { VerboseLoggingOutcomeReason::EventPayloadMalformed ); assert_eq!(malformed.signature.pid, 9); - assert_eq!(property(&malformed.signature, "EventName"), "AccessCheck"); + assert_eq!( + malformed.signature.event_name.as_deref(), + Some("AccessCheck") + ); assert_eq!( find_signature(&out.verbose_logging, 27).signature.reason, VerboseLoggingOutcomeReason::DecoderLimitReached @@ -2622,11 +3076,7 @@ mod tests { Some(ResourceType::Capability) ); assert!(out.verbose_logging.signatures.iter().all(|aggregate| { - aggregate - .signature - .properties - .iter() - .all(|(name, _)| name == "EventName") + aggregate.signature.properties.is_empty() && aggregate.signature.event_name.is_some() })); } @@ -2672,4 +3122,29 @@ mod tests { None ); } + + #[test] + fn decode_failure_schema_names_are_sanitized_for_all_providers() { + for provider in [ + crate::extractors::KERNEL_GENERAL_PROVIDER, + windows::core::GUID::from_u128(1), + ] { + let mut accumulator = Accumulator::analyze(); + accumulator.record_event_decode_error( + provider, + 999, + 42, + tdh_decode::DecodeError::event( + tdh_decode::EventDecodeKind::PayloadMalformed, + "private decoder message".into(), + Some(r"C:\Users\private\schema".into()), + ), + ); + let analysis = accumulator.into_analysis().unwrap(); + let group = &analysis.verbose_logging.signatures[0]; + assert_eq!(group.signature.event_name.as_deref(), Some("")); + assert!(group.signature.properties.is_empty()); + assert_eq!(group.count, 1); + } + } } diff --git a/src/core/learning_mode_platforms/windows/src/etl_filter.rs b/src/core/learning_mode_platforms/windows/src/etl_filter.rs index 1c61e2b90..f9ae1ae46 100644 --- a/src/core/learning_mode_platforms/windows/src/etl_filter.rs +++ b/src/core/learning_mode_platforms/windows/src/etl_filter.rs @@ -1,8 +1,6 @@ // Copyright (c) Microsoft Corporation. // Licensed under the MIT License. -//! Process-scoped ETL rewriting for guarded WPR captures. - use std::collections::HashSet; use std::os::windows::ffi::OsStrExt; use std::path::Path; @@ -19,7 +17,7 @@ use windows::Win32::System::Diagnostics::Etw::{ }; use crate::etl_decode::select_learning_mode_events_for_relogging; -use crate::extractors::{is_learning_mode_event, verbose_logging_provider_for_guid}; +use crate::extractors::is_learning_mode_event; use crate::process_lifetime::{attested_process_lifetimes, JobMembershipSnapshot}; use crate::tdh_decode; @@ -90,19 +88,13 @@ impl RelogSelectionState { } } - fn observe_known_provider_event(&self, pid: u32) -> bool { + fn observe_event(&self, pid: u32) -> bool { let event_index = self.event_cursor.fetch_add(1, Ordering::Relaxed); let selected_index = self.selected_event_cursor.load(Ordering::Relaxed); if self.selected_event_indices.get(selected_index) != Some(&event_index) { return false; } - // The first ProcessTrace pass selected this ordinal using exact PID and - // lifetime bounds. Revalidate the second pass's current PID before - // injection so a different equal-timestamp ordering cannot substitute - // a foreign process's event at the same ordinal. Do not advance the - // selected-event index on mismatch: the final count reconciliation then - // fails closed and the partial destination is deleted. if self.selected_event_pids.get(selected_index) != Some(&pid) || !self.attested_pids.contains(&pid) { @@ -130,8 +122,13 @@ impl ITraceEventCallback_Impl for ProcessScopedTraceFilter_Impl { ) -> windows::core::Result<()> { let header_event = header_event.ok()?; let relogger = relogger.ok()?; - // SAFETY: both interfaces are valid for this synchronous callback. - // The trace header is required for a standalone output ETL. + // SAFETY: Trace Relogger keeps the header and its interfaces valid + // throughout this synchronous callback. + let record = unsafe { header_event.GetEventRecord()? }; + let record = unsafe { record.as_ref() }.ok_or_else(|| { + windows::core::Error::from_hresult(windows::Win32::Foundation::E_POINTER) + })?; + self.selection.observe_event(record.EventHeader.ProcessId); unsafe { relogger.Inject(header_event)? }; Ok(()) } @@ -156,9 +153,6 @@ impl ITraceEventCallback_Impl for ProcessScopedTraceFilter_Impl { )); }; let header = &record.EventHeader; - if verbose_logging_provider_for_guid(header.ProviderId).is_none() { - return Ok(()); - } let is_supported_capability_event = header.EventDescriptor.Id == crate::extractors::CAPABILITY_DENIAL_EVENT_ID && is_learning_mode_event(header.ProviderId, header.EventDescriptor.Id); @@ -183,7 +177,7 @@ impl ITraceEventCallback_Impl for ProcessScopedTraceFilter_Impl { } else { header.ProcessId }; - if self.selection.observe_known_provider_event(effective_pid) { + if self.selection.observe_event(effective_pid) { // SAFETY: both interfaces are valid for this synchronous callback, // and Inject clones the event into the output trace. unsafe { relogger.Inject(event)? }; @@ -321,7 +315,7 @@ fn relog_trace_with( let consumed_event_count = selection.consumed_event_count(); if consumed_event_count != expected_event_count { return Err(AnalyzeError::Decode(format!( - "Trace Relogger observed {consumed_event_count} known-provider Learning Mode events, but ProcessTrace selected {expected_event_count}" + "Trace Relogger observed {consumed_event_count} events, but ProcessTrace selected {expected_event_count}" ))); } let consumed_selected_event_count = selection.consumed_selected_event_count(); @@ -365,6 +359,174 @@ fn windows_error(operation: &str, error: windows::core::Error) -> AnalyzeError { mod tests { use super::*; + #[test] + fn private_trace_relogging_preserves_unknown_events() { + use windows::core::PCWSTR; + use windows::Win32::System::Diagnostics::Etw::{ + ControlTraceW, EnableTraceEx2, StartTraceW, CONTROLTRACE_HANDLE, + EVENT_TRACE_CONTROL_STOP, EVENT_TRACE_PRIVATE_IN_PROC, EVENT_TRACE_PRIVATE_LOGGER_MODE, + EVENT_TRACE_PROPERTIES, WNODE_FLAG_TRACED_GUID, + }; + tracelogging::define_provider!( + TEST_PROVIDER, + "MxcTest.Redaction", + id("a93bc25e-f2de-485a-90e7-a2d8970b22a9") + ); + + let directory = tempfile::tempdir().unwrap(); + let source = directory.path().join("source.etl"); + let destination = directory.path().join("scoped.etl"); + let provider = windows::core::GUID::from_u128(0xa93bc25e_f2de_485a_90e7_a2d8970b22a9); + let name = format!("mxc-verbose-test-{}", std::process::id()) + .encode_utf16() + .chain([0]) + .collect::>(); + let path = source + .as_os_str() + .encode_wide() + .chain([0]) + .collect::>(); + let mut locations = 2u16.to_le_bytes().to_vec(); + for value in [r"C:\Users\alice\secret.txt", r"D:\private.txt"] { + locations.extend(value.encode_utf16().chain([0]).flat_map(u16::to_le_bytes)); + } + let mut storage = vec![0u64; 512]; + let properties = storage.as_mut_ptr().cast::(); + let mut session = CONTROLTRACE_HANDLE::default(); + // SAFETY: the aligned buffer holds both null-terminated strings and + // outlives the synchronous ETW calls. + unsafe { + (*properties).Wnode.BufferSize = (storage.len() * 8) as u32; + (*properties).Wnode.Guid = provider; + (*properties).Wnode.Flags = WNODE_FLAG_TRACED_GUID; + (*properties).Wnode.ClientContext = 1; + (*properties).BufferSize = 64; + (*properties).LogFileMode = + EVENT_TRACE_PRIVATE_LOGGER_MODE | EVENT_TRACE_PRIVATE_IN_PROC; + (*properties).LoggerNameOffset = std::mem::size_of::() as u32; + (*properties).LogFileNameOffset = + (*properties).LoggerNameOffset + (name.len() * 2) as u32; + assert!((*properties).LogFileNameOffset as usize + path.len() * 2 <= storage.len() * 8); + std::ptr::copy_nonoverlapping( + path.as_ptr(), + storage + .as_mut_ptr() + .cast::() + .add((*properties).LogFileNameOffset as usize) + .cast::(), + path.len(), + ); + assert_eq!(TEST_PROVIDER.register(), 0); + let started = StartTraceW(&mut session, PCWSTR(name.as_ptr()), properties); + if started.0 != 0 { + TEST_PROVIDER.unregister(); + panic!("private StartTraceW failed: {}", started.0); + } + let enabled = EnableTraceEx2(session, &provider, 1, 5, u64::MAX, 0, 0, None); + let emit = || { + [ + tracelogging::write_event!( + TEST_PROVIDER, + "CompositeProperties", + cstr8("CommandLine", r"cmd.exe /c type C:\Users\alice\secret.txt"), + raw_field_slice("FutureLocations", CStr16, &locations), + u32("SafeIdentifier", &42), + u32("C:\\Users\\alice\\secret.txt", &42), + cstr8("EventName", "payload-name"), + ), + tracelogging::write_event!( + TEST_PROVIDER, + "OtherCompositeProperties", + cstr8("CommandLine", r"cmd.exe /c type C:\Users\alice\secret.txt"), + raw_field_slice("FutureLocations", CStr16, &locations), + u32("SafeIdentifier", &42), + u32("C:\\Users\\alice\\secret.txt", &42), + cstr8("EventName", "payload-name"), + ), + ] + }; + let first = emit(); + let second = emit(); + let stopped = ControlTraceW( + session, + PCWSTR(name.as_ptr()), + properties, + EVENT_TRACE_CONTROL_STOP, + ); + let unregistered = TEST_PROVIDER.unregister(); + for status in [enabled.0, stopped.0, unregistered] + .into_iter() + .chain(first) + .chain(second) + { + assert_eq!(status, 0); + } + } + if let Some(path) = std::env::var_os("MXC_TEST_ETL_OUTPUT") { + std::fs::copy(&source, path).expect("failed to retain the test ETL for CLI replay"); + } + let lifetimes = [ProcessLifetime { + pid: std::process::id(), + start_filetime: 0, + end_filetime: u64::MAX, + }]; + let native = crate::EtlDenialAnalyzer + .analyze_for_process_lifetimes(&source, &lifetimes) + .unwrap(); + filter_trace_for_process_lifetimes(&source, &destination, &lifetimes).unwrap(); + let guarded = crate::EtlDenialAnalyzer + .analyze_relogged_for_process_lifetimes(&destination, &lifetimes) + .unwrap(); + let provider_guid = crate::extractors::format_guid_braced_uppercase(provider); + let groups = native + .verbose_logging + .signatures + .iter() + .filter(|group| group.signature.provider_guid == provider_guid) + .collect::>(); + assert_eq!(groups.len(), 2, "{groups:?}"); + assert_eq!( + groups + .iter() + .map(|group| group.signature.event_name.as_deref()) + .collect::>(), + [ + Some("CompositeProperties"), + Some("OtherCompositeProperties") + ] + ); + for group in groups { + assert_eq!(group.count, 2); + for name in ["CommandLine", "FutureLocations"] { + assert_eq!( + group + .signature + .properties + .iter() + .find(|(key, _)| key == name), + Some(&( + name.to_string(), + crate::extractors::REDACTED_PATH.to_string() + )), + ); + } + assert!(group + .signature + .properties + .contains(&("SafeIdentifier".into(), "42".into()))); + assert!(group + .signature + .properties + .contains(&("EventName".into(), "payload-name".into()))); + assert!(!group + .signature + .properties + .iter() + .any(|(name, _)| name.contains(r"C:\Users"))); + } + assert_eq!(native.verbose_logging, guarded.verbose_logging); + } + const START: u64 = 100; const END: u64 = 200; @@ -380,7 +542,7 @@ mod tests { fn selected_ordinal_with_foreign_pid_is_not_injected() { let selection = RelogSelectionState::new(vec![0], vec![42], &[lifetime(42)]); - assert!(!selection.observe_known_provider_event(99)); + assert!(!selection.observe_event(99)); assert_eq!(selection.consumed_event_count(), 1); assert_eq!( selection.consumed_selected_event_count(), @@ -393,7 +555,7 @@ mod tests { fn selected_ordinal_with_attested_pid_is_injected() { let selection = RelogSelectionState::new(vec![0], vec![42], &[lifetime(42)]); - assert!(selection.observe_known_provider_event(42)); + assert!(selection.observe_event(42)); assert_eq!(selection.consumed_event_count(), 1); assert_eq!(selection.consumed_selected_event_count(), 1); } diff --git a/src/core/learning_mode_platforms/windows/src/extractors.rs b/src/core/learning_mode_platforms/windows/src/extractors.rs index 8c7dbb5ef..697bb92e7 100644 --- a/src/core/learning_mode_platforms/windows/src/extractors.rs +++ b/src/core/learning_mode_platforms/windows/src/extractors.rs @@ -33,9 +33,10 @@ //! classified registry reads are actionable; writes and unknown registry //! access are retained only as verbose diagnostics. Named Section, //! SymbolicLink, and Timer objects are likewise verbose-only because MXC has -//! no corresponding policy grants. -//! Other object types are dropped until their access-mask vocabulary is -//! understood. The [`AccessType`] is derived from the +//! no corresponding policy grants. COM activation and interface-call checks +//! remain diagnostic, with their CLSID/IID retained in sanitized properties. +//! Other object types remain diagnostic until their access-mask vocabulary +//! is understood. The [`AccessType`] is derived from the //! `AccessMask` field (see [`access_type_from_mask`]). Emitted under both //! learning modes (`block` → `Mode="Normal"`, `allow` → //! `Mode="Permissive"`). @@ -76,6 +77,8 @@ pub(crate) const ACCESS_CHECK_EVENT_ID: u16 = 14; pub(crate) const LEARNING_MODE_VIOLATION_EVENT_ID: u16 = 27; pub(crate) const CAPABILITY_DENIAL_EVENT_ID: u16 = 28; pub(crate) const PRIVACY_ACCESS_CHECK_EVENT_ID: u16 = 4907; +const COM_ACTIVATION_OBJECT_TYPE: &str = "ComActivationForClass"; +const COM_CALL_OBJECT_TYPE: &str = "ComCallOnInterface"; /// Pre-decoded event payload handed to the extractors. /// @@ -88,6 +91,8 @@ pub struct DecodedEventParts { pub provider: GUID, /// Originating ETW event ID. pub event_id: u16, + /// Schema-declared event name. + pub event_name: Option, /// `(name, value)` pairs from the decoded payload. String values are /// often TDH-quoted; extractors trim the surrounding quotes. pub props: Vec<(String, String)>, @@ -197,6 +202,9 @@ pub(crate) fn verbose_logging_classification( ), Some(ResourceType::Other), ), + COM_ACTIVATION_OBJECT_TYPE | COM_CALL_OBJECT_TYPE => { + (None, Some(ResourceType::Other)) + } "" => (Some(AccessType::Unknown), Some(ResourceType::Capability)), _ => (None, None), } @@ -251,9 +259,8 @@ pub(crate) fn effective_capability_event_pid(process_id: Option<&str>) -> Option /// Maps a raw ETW provider GUID to its symbolic verbose logging category. /// -/// Returns `None` for providers outside the Learning Mode vocabulary; those -/// events are ignored entirely (not aggregated), since they are unrelated -/// host traffic rather than an excluded Learning Mode outcome. +/// Returns `None` outside the actionable vocabulary. Verbose logging retains +/// those events as `Other` after process scoping. pub(crate) fn verbose_logging_provider_for_guid(provider: GUID) -> Option { if provider == KERNEL_GENERAL_PROVIDER { Some(VerboseLoggingProvider::KernelGeneral) @@ -270,6 +277,7 @@ pub(crate) fn verbose_logging_provider_for_guid(provider: GUID) -> Option String { match provider { + VerboseLoggingProvider::Other => String::new(), VerboseLoggingProvider::KernelGeneral => { format_guid_braced_uppercase(KERNEL_GENERAL_PROVIDER) } @@ -281,7 +289,7 @@ pub(crate) fn verbose_logging_provider_guid(provider: VerboseLoggingProvider) -> // Keep the serialized spelling independent of formatting changes in the // `windows` crate: verbose signatures require braces and uppercase hex. -fn format_guid_braced_uppercase(guid: GUID) -> String { +pub(crate) fn format_guid_braced_uppercase(guid: GUID) -> String { format!( "{{{:08X}-{:04X}-{:04X}-{:02X}{:02X}-{:02X}{:02X}{:02X}{:02X}{:02X}{:02X}}}", guid.data1, @@ -413,20 +421,32 @@ fn is_identity_property(name: &str) -> bool { fn looks_like_file_path_property(name: &str, value: &str, object_type: Option<&str>) -> bool { let normalized = NormalizedPropertyName(name); - if normalized.ends_with("path") || normalized.ends_with("filename") { + if normalized.ends_with("path") + || normalized.ends_with("filename") + || normalized.ends_with("filenamestring") + { return true; } - if !normalized.equals("objectname") && !normalized.equals("resource") { - return false; - } - if object_type.is_some_and(|object_type| object_type.eq_ignore_ascii_case("File")) { + if (normalized.equals("objectname") || normalized.equals("resource")) + && object_type.is_some_and(|object_type| object_type.eq_ignore_ascii_case("File")) + { return true; } - crate::path_norm::is_user_visible_absolute(value) - || looks_like_dos_device_filesystem_path(value) - || looks_like_nt_filesystem_path(value) + value.char_indices().any(|(offset, _)| { + let candidate = &value[offset..]; + if offset > 0 + && candidate.get(1..4) == Some("://") + && (value.as_bytes()[offset - 1].is_ascii_alphanumeric() + || matches!(value.as_bytes()[offset - 1], b'+' | b'-' | b'.')) + { + return false; + } + crate::path_norm::is_user_visible_absolute(candidate) + || looks_like_dos_device_filesystem_path(candidate) + || looks_like_nt_filesystem_path(candidate) + }) } fn looks_like_dos_device_filesystem_path(value: &str) -> bool { @@ -547,7 +567,7 @@ pub(crate) fn sanitize_properties(props: &[(String, String)]) -> Vec<(String, St .map(|(_, value)| value.trim_matches('"')); let mut sanitized = std::collections::BTreeMap::new(); for (name, raw_value) in props { - if is_timestamp_like_property(name) { + if is_timestamp_like_property(name) || looks_like_file_path_property("", name, None) { continue; } let value = raw_value.trim_matches('"'); @@ -606,15 +626,24 @@ pub(crate) fn sanitize_properties(props: &[(String, String)]) -> Vec<(String, St bound_properties(sanitized.into_iter().collect()) } +pub(crate) fn sanitize_event_name(name: Option<&str>) -> Option { + let name = name?; + sanitize_properties(&[("EventName".into(), name.into())]) + .into_iter() + .next() + .map(|(_, value)| value) +} + +/// Builds a denial from an access-check event. /// /// The `ObjectType` field selects the resource type: `File` and `Key` /// (registry) map to concrete resources, an **empty** `ObjectType` is a /// brokered-capability check, and the observed named-object types `Section`, /// `SymbolicLink`, and `Timer` map to [`ResourceType::Other`]. Only registry -/// reads are actionable; other registry access and those named-object types are -/// excluded because MXC has no corresponding policy grants. Other object types -/// are dropped until their access-mask vocabulary is understood. An absent -/// `ObjectType` field drops the event. +/// reads are actionable; other registry access, named-object and COM checks +/// are excluded because MXC has no corresponding policy grants. Other object +/// types remain diagnostic until their access-mask vocabulary is understood. +/// An absent `ObjectType` field excludes the event from actionable output. /// /// For file/registry resources the [`AccessType`] is derived from the /// event's `AccessMask` field (the desired access the caller was denied; @@ -633,9 +662,11 @@ pub(crate) fn sanitize_properties(props: &[(String, String)]) -> Vec<(String, St /// [`VerboseLoggingOutcomeReason::MissingObjectName`]; capability: an /// unidentified brokered check, /// [`VerboseLoggingOutcomeReason::UnresolvedCapability`] — [`crate::capability_dacl`] -/// may still recover it from the event's DACL payload), or a self-access, -/// non-read registry, or recognized named-object check that isn't actionable -/// ([`VerboseLoggingOutcomeReason::NotActionable`]). +/// may still recover it from the event's DACL payload), a malformed COM +/// identifier ([`VerboseLoggingOutcomeReason::EventPayloadMalformed`]), or a +/// non-actionable check. Valid COM checks use their dedicated verbose reasons; +/// self-access, non-read registry and named-object checks use +/// [`VerboseLoggingOutcomeReason::NotActionable`]. pub fn build_denial_from_access_check( parts: &DecodedEventParts, pid: u32, @@ -652,6 +683,7 @@ pub fn build_denial_from_access_check( "Section" | "SymbolicLink" | "Timer" => ResourceType::Other, // A present-but-empty object type is a brokered-capability check. "" => ResourceType::Capability, + COM_ACTIVATION_OBJECT_TYPE | COM_CALL_OBJECT_TYPE => ResourceType::Other, _ => return Err(VerboseLoggingOutcomeReason::UnsupportedObjectType), }; @@ -668,6 +700,19 @@ pub fn build_denial_from_access_check( (_, Some(name)) => name, }; + if is_com_object_type(object_type_str) && !is_guid_identifier(&object_name) { + return Err(VerboseLoggingOutcomeReason::EventPayloadMalformed); + } + match object_type_str { + COM_ACTIVATION_OBJECT_TYPE => { + return Err(VerboseLoggingOutcomeReason::ComActivation); + } + COM_CALL_OBJECT_TYPE => { + return Err(VerboseLoggingOutcomeReason::ComInterfaceCall); + } + _ => {} + } + if resource_type == ResourceType::File { let app_path = find_prop(&parts.props, "AppPath") .or_else(|| find_prop(&parts.props, "ApplicationPath")) @@ -1002,6 +1047,30 @@ fn find_prop<'a>(props: &'a [(String, String)], name: &str) -> Option<&'a String props.iter().find(|(k, _)| k == name).map(|(_, v)| v) } +fn is_com_object_type(object_type: &str) -> bool { + matches!( + object_type, + COM_ACTIVATION_OBJECT_TYPE | COM_CALL_OBJECT_TYPE + ) +} + +fn is_guid_identifier(value: &str) -> bool { + let value = match (value.strip_prefix('{'), value.strip_suffix('}')) { + (Some(value), Some(_)) => &value[..value.len() - 1], + (None, None) => value, + _ => return false, + }; + + value.len() == 36 + && value.bytes().enumerate().all(|(index, byte)| { + if matches!(index, 8 | 13 | 18 | 23) { + byte == b'-' + } else { + byte.is_ascii_hexdigit() + } + }) +} + #[cfg(test)] mod tests { use super::*; @@ -1025,6 +1094,7 @@ mod tests { DecodedEventParts { provider, event_id, + event_name: None, props: kv .iter() .map(|(k, v)| ((*k).to_string(), (*v).to_string())) @@ -1350,6 +1420,84 @@ mod tests { } } + #[test] + fn access_check_com_denials_use_distinct_verbose_reasons() { + for (object_type, object_name, mode, expected_reason) in [ + ( + COM_ACTIVATION_OBJECT_TYPE, + "{A47979D2-C419-11D9-A5B4-001185AD2B89}", + "\"Normal\"", + VerboseLoggingOutcomeReason::ComActivation, + ), + ( + COM_CALL_OBJECT_TYPE, + "{00000132-0000-0000-C000-000000000046}", + "\"Permissive\"", + VerboseLoggingOutcomeReason::ComInterfaceCall, + ), + ] { + let p = parts( + 14, + &[ + ("Mode", mode), + ("ObjectType", object_type), + ("ObjectName", object_name), + ("AccessMask", "0xffffffff"), + ], + ); + + assert_eq!(extract_denial(&p, 42, FIXED_FILETIME), Err(expected_reason)); + assert_eq!( + verbose_logging_classification(&p), + (None, Some(ResourceType::Other)) + ); + } + } + + #[test] + fn access_check_com_denials_require_guid_identifier() { + for (object_name, expected) in [ + (None, VerboseLoggingOutcomeReason::MissingObjectName), + (Some("\"\""), VerboseLoggingOutcomeReason::MissingObjectName), + ( + Some("\"not-a-guid\""), + VerboseLoggingOutcomeReason::EventPayloadMalformed, + ), + ( + Some("\"{00000132-0000-0000-C000-000000000046\""), + VerboseLoggingOutcomeReason::EventPayloadMalformed, + ), + ] { + let mut properties = vec![("ObjectType", COM_CALL_OBJECT_TYPE)]; + if let Some(object_name) = object_name { + properties.push(("ObjectName", object_name)); + } + let p = parts(14, &properties); + + assert_eq!(extract_denial(&p, 42, FIXED_FILETIME), Err(expected)); + assert_eq!( + verbose_logging_classification(&p), + (None, Some(ResourceType::Other)) + ); + } + } + + #[test] + fn access_check_rpc_interface_remains_unsupported() { + let p = parts( + 14, + &[ + ("ObjectType", "\"RPC Interface\""), + ("ObjectName", "\"f6beaff7-1e19-4fbb-9f8f-b89e2018337c\""), + ], + ); + assert_eq!( + extract_denial(&p, 1, FIXED_FILETIME), + Err(VerboseLoggingOutcomeReason::UnsupportedObjectType) + ); + assert_eq!(verbose_logging_classification(&p), (None, None)); + } + #[test] fn access_check_unrecognized_object_type_is_dropped() { let p = parts( @@ -1708,7 +1856,13 @@ mod tests { r"\Device\MountPointManager", Some("Section") )); - for identifier in [r"\??\FDC#GENERIC_FLOPPY_DRIVE", r"\\.\PhysicalDrive0"] { + for identifier in [ + r"\??\FDC#GENERIC_FLOPPY_DRIVE", + r"\\.\PhysicalDrive0", + "https://example.com/resource", + "custom+a://example.com/resource", + r"\Device\NamedPipe\mxc", + ] { assert!(!looks_like_file_path_property( "ObjectName", identifier, @@ -1802,6 +1956,91 @@ mod tests { assert_eq!(value_for("ObjectName"), Some(REDACTED_PATH)); } + #[test] + fn future_property_names_do_not_bypass_path_redaction() { + let properties = sanitize_properties(&[ + ("FutureLocation".into(), r"C:\Users\private\file.txt".into()), + ("LogFileNameString".into(), "ReloggedFile.ETL".into()), + ( + "UnrecognizedField".into(), + r"\\server\share\private.txt".into(), + ), + ]); + assert!(properties.iter().all(|(_, value)| value == REDACTED_PATH)); + } + + #[test] + fn sanitize_properties_omits_sensitive_property_names() { + let props = [ + (r"C:\Users\alice\secret.txt".into(), "42".into()), + (r"lookup \\server\share\private.txt".into(), "43".into()), + (r"\Device\HarddiskVolume3\private.txt".into(), "44".into()), + ("SafeIdentifier".into(), "42".into()), + ]; + assert_eq!( + sanitize_properties(&props), + [("SafeIdentifier".into(), "42".into())] + ); + } + + #[test] + fn event_names_are_sanitized_and_bounded() { + assert_eq!(sanitize_event_name(None), None); + assert_eq!( + sanitize_event_name(Some("AccessCheck")).as_deref(), + Some("AccessCheck") + ); + assert_eq!( + sanitize_event_name(Some(r"C:\Users\alice\secret.txt")).as_deref(), + Some(REDACTED_PATH) + ); + let first = sanitize_event_name(Some(&format!("{}First", "a".repeat(400)))).unwrap(); + let second = sanitize_event_name(Some(&format!("{}Second", "a".repeat(400)))).unwrap(); + assert_ne!(first, second); + assert!(first.chars().count() <= MAX_SIGNATURE_VALUE_LEN); + assert!(second.chars().count() <= MAX_SIGNATURE_VALUE_LEN); + } + + #[test] + fn sanitize_properties_redacts_embedded_file_paths() { + let long_value = format!("{}C:\\Users\\alice\\secret.txt", "prefix ".repeat(100)); + for value in [ + r"cmd.exe /c type C:\Users\alice\secret.txt", + r#"--input="C:\Program Files\private\data.txt""#, + r"open \\server\share\private.txt", + r"open \??\C:\Users\alice\secret.txt", + r"open \\?\Volume{1234}\private.txt", + r"open \Device\HarddiskVolume3\Users\alice\secret.txt", + "message: C:/Users/alice/secret.txt", + "\u{03bb}: C:\\Users\\alice\\secret.txt", + ] + .into_iter() + .chain([long_value.as_str()]) + { + assert_eq!( + sanitize_properties(&[("CommandLine".into(), value.into())]), + [("CommandLine".into(), REDACTED_PATH.into())], + "{value}", + ); + } + } + + #[test] + fn sanitize_properties_redacts_paths_in_rendered_arrays() { + for value in [ + r#"["C:\Users\alice\secret.txt", "D:\private.txt"]"#, + r#"["safe", "\\server\share\private.txt"]"#, + r#"["safe", "\Device\HarddiskVolume3\private.txt"]"#, + r#"["C:\\Users\\alice\\secret.txt"]"#, + ] { + assert_eq!( + sanitize_properties(&[("FutureLocations".into(), value.into())]), + [("FutureLocations".into(), REDACTED_PATH.into())], + "{value}", + ); + } + } + #[test] fn sanitize_properties_redacts_absolute_path_despite_non_file_object_type() { for path in [ diff --git a/src/core/learning_mode_platforms/windows/src/tdh_decode.rs b/src/core/learning_mode_platforms/windows/src/tdh_decode.rs index 63ce8ebaa..77cf17628 100644 --- a/src/core/learning_mode_platforms/windows/src/tdh_decode.rs +++ b/src/core/learning_mode_platforms/windows/src/tdh_decode.rs @@ -88,6 +88,16 @@ pub(crate) struct EventSchemaCache { schemas: HashMap, } +#[cfg(test)] +impl EventSchemaCache { + pub(crate) fn insert_test_schema(&mut self, record: &EVENT_RECORD, names: &[&str]) { + self.schemas.insert( + EventSchemaKey::from_record(record), + tests::uint32_property_buffer(names), + ); + } +} + #[derive(Debug)] pub(crate) enum DecodeError { Schema(String), @@ -106,10 +116,6 @@ pub(crate) enum EventDecodeKind { } impl DecodeError { - pub(crate) fn is_schema_error(&self) -> bool { - matches!(self, Self::Schema(_)) - } - pub(crate) fn event_kind(&self) -> Option { match self { Self::Event { kind, .. } => Some(*kind), @@ -210,11 +216,12 @@ pub unsafe fn decode_event_parts( let pointer_size = pointer_size_from_header_flags(header.Flags); let event_name = schema_event_name(buffer.as_bytes(), info); let props = decode_properties(buffer.as_bytes(), info, event_record, pointer_size) - .map_err(|error| map_property_decode_error(error, event_name))?; + .map_err(|error| map_property_decode_error(error, event_name.clone()))?; Ok(DecodedEventParts { provider: header.ProviderId, event_id, + event_name, props, }) } @@ -1005,7 +1012,7 @@ mod tests { .collect() } - fn uint32_property_buffer(names: &[&str]) -> TdhInfoBuffer { + pub(super) fn uint32_property_buffer(names: &[&str]) -> TdhInfoBuffer { assert!(!names.is_empty()); let encoded_names = names .iter() diff --git a/src/core/mxc_engine/src/verbose_telemetry.rs b/src/core/mxc_engine/src/verbose_telemetry.rs index 9f15efac5..b34b53eea 100644 --- a/src/core/mxc_engine/src/verbose_telemetry.rs +++ b/src/core/mxc_engine/src/verbose_telemetry.rs @@ -7,7 +7,8 @@ use std::io::Read; use std::path::Path; use learning_mode_core::{ - verbose_logging_sibling_path, VerboseLoggingDocument, VerboseLoggingProvider, + verbose_logging_sibling_path, VerboseLoggingAggregate, VerboseLoggingDocument, + VerboseLoggingProvider, }; use wxc_common::hashing::sha256_hex; use wxc_common::models::{ContainmentBackend, ScriptResponse}; @@ -145,16 +146,25 @@ fn prepare_document(path: &Path) -> Result { } fn project_for_telemetry(mut document: VerboseLoggingDocument) -> VerboseLoggingDocument { - for aggregate in &mut document.signatures { + let mut groups = std::collections::BTreeMap::new(); + for mut aggregate in document.signatures { aggregate.signature.provider_guid = canonical_provider_guid(aggregate.signature.provider).to_string(); + aggregate.signature.event_name = None; aggregate.signature.properties.clear(); + let count = groups.entry(aggregate.signature).or_insert(0u64); + *count = count.saturating_add(aggregate.count); } + document.signatures = groups + .into_iter() + .map(|(signature, count)| VerboseLoggingAggregate { signature, count }) + .collect(); document } fn canonical_provider_guid(provider: VerboseLoggingProvider) -> &'static str { match provider { + VerboseLoggingProvider::Other => "", VerboseLoggingProvider::KernelGeneral => "{A68CA8B7-004F-D7B6-A698-07E2DE0F1F5D}", VerboseLoggingProvider::PrivacyAuditingPermissiveLearningMode => { "{811A1DDB-2E69-5F25-ADC0-4B186170E760}" @@ -215,13 +225,18 @@ mod tests { }; use wxc_common::models::{CaptureDenialsOutput, SandboxOutputMetadata}; - fn aggregate(event_id: u16, value: &str) -> VerboseLoggingAggregate { + fn aggregate_with_reason( + event_id: u16, + value: &str, + reason: VerboseLoggingOutcomeReason, + ) -> VerboseLoggingAggregate { VerboseLoggingAggregate { signature: VerboseLoggingSignature { + event_name: None, provider: VerboseLoggingProvider::KernelGeneral, provider_guid: "{a68ca8b7-004f-d7b6-a698-07e2de0f1f5d}".to_string(), event_id, - reason: VerboseLoggingOutcomeReason::UnsupportedEventSchema, + reason, pid: 42, access_type: None, resource_type: None, @@ -231,6 +246,14 @@ mod tests { } } + fn aggregate(event_id: u16, value: &str) -> VerboseLoggingAggregate { + aggregate_with_reason( + event_id, + value, + VerboseLoggingOutcomeReason::UnsupportedEventSchema, + ) + } + fn document(signatures: Vec) -> VerboseLoggingDocument { VerboseLoggingDocument { version: VerboseLoggingDocument::VERSION, @@ -340,6 +363,28 @@ mod tests { })); } + #[test] + fn telemetry_preserves_com_reason_but_strips_clsid() { + let mut document = document(vec![aggregate_with_reason( + 14, + "{A47979D2-C419-11D9-A5B4-001185AD2B89}", + VerboseLoggingOutcomeReason::ComActivation, + )]); + document.signatures[0].signature.resource_type = + Some(learning_mode_core::ResourceType::Other); + + document = project_for_telemetry(document); + + let signature = &document.signatures[0].signature; + assert_eq!(signature.reason, VerboseLoggingOutcomeReason::ComActivation); + assert_eq!( + signature.resource_type, + Some(learning_mode_core::ResourceType::Other) + ); + assert!(signature.access_type.is_none()); + assert!(signature.properties.is_empty()); + } + #[test] fn malformed_artifact_never_falls_back_to_raw_bytes() { let directory = tempfile::tempdir().unwrap(); @@ -355,6 +400,7 @@ mod tests { let injected = "customer-secret"; let mut doc = document(vec![aggregate(1, injected)]); doc.signatures[0].signature.provider_guid = injected.to_string(); + doc.signatures[0].signature.event_name = Some(injected.to_string()); doc.signatures[0].signature.properties = vec![(injected.to_string(), injected.to_string())]; let projected = project_for_telemetry(doc); @@ -367,4 +413,29 @@ mod tests { ); assert!(projected.signatures[0].signature.properties.is_empty()); } + + #[test] + fn telemetry_deduplicates_after_removing_unknown_provider_guids_and_properties() { + let mut first = aggregate(999, "first-secret"); + first.signature.provider = VerboseLoggingProvider::Other; + first.signature.provider_guid = "first-provider".into(); + first.signature.event_name = Some("first-event".into()); + first.count = 3; + let mut second = first.clone(); + second.signature.provider_guid = "second-provider".into(); + second.signature.event_name = Some("second-event".into()); + second.signature.properties[0].1 = "second-secret".into(); + second.count = 4; + let mut input = document(vec![first, second]); + input.summary.total_occurrences = 7; + + let projected = project_for_telemetry(input); + + assert_eq!(projected.signatures.len(), 1); + assert_eq!(projected.signatures[0].count, 7); + assert!(projected.signatures[0].signature.provider_guid.is_empty()); + assert!(projected.signatures[0].signature.event_name.is_none()); + assert!(projected.signatures[0].signature.properties.is_empty()); + assert_eq!(projected.summary.total_occurrences, 7); + } } diff --git a/src/host/plm/src/elevated.rs b/src/host/plm/src/elevated.rs index e9ac125b7..2ef4a4e60 100644 --- a/src/host/plm/src/elevated.rs +++ b/src/host/plm/src/elevated.rs @@ -2138,7 +2138,7 @@ fn write_analysis_response( membership: &JobMembershipSnapshot, ) -> Result<()> { let analysis = EtlDenialAnalyzer - .analyze_for_job_membership(trace_path, membership) + .analyze_relogged_for_job_membership(trace_path, membership) .context("failed to decode guarded WPR trace for the sandbox process tree")?; write_serialized_analysis_response(pipe, &analysis) } diff --git a/src/host/plm/src/stop.rs b/src/host/plm/src/stop.rs index e88890508..13b91d3ff 100644 --- a/src/host/plm/src/stop.rs +++ b/src/host/plm/src/stop.rs @@ -527,7 +527,16 @@ mod tests { #[test] fn truncated_actionable_denials_snapshot_but_skip_adjusted_config() { - let document = DenialsDocument::new(Vec::new(), DenialSummary::new(0, 0, true)); + let document = DenialsDocument::new( + vec![DeniedResource { + resource: "internetClient".to_string(), + resource_type: ResourceType::Capability, + access_type: AccessType::Unknown, + pid: 42, + filetime: 1, + }], + DenialSummary::new(0, 1, true), + ); let (_directory, log_dir, adjusted) = postprocess_fixture(&document); diff --git a/src/testing/wxc_e2e_tests/tests/e2e_telemetry_etw.rs b/src/testing/wxc_e2e_tests/tests/e2e_telemetry_etw.rs index c9d0f0f57..b4524a948 100644 --- a/src/testing/wxc_e2e_tests/tests/e2e_telemetry_etw.rs +++ b/src/testing/wxc_e2e_tests/tests/e2e_telemetry_etw.rs @@ -135,6 +135,8 @@ fn build_instrumented_wxc_exec() -> PathBuf { "build", "-p", "wxc", + "-p", + "plm", // `test-support` is what makes `MXC_TEST_LOCALAPPDATA_OVERRIDE` // observable to the child; without it the consent store is the real // per-user one and this test would mutate developer state. @@ -236,12 +238,14 @@ fn run_traced_execution(exe: &Path, local_app_data: &Path) -> std::process::Outp fn write_capture_config(workdir: &Path) -> PathBuf { let config_path = workdir.join("capture-config.json"); let output_path = workdir.join("denials.json"); + let denied_file = workdir.join("denied.txt"); + std::fs::write(&denied_file, "telemetry E2E fixture").expect("failed to write denied fixture"); let config = serde_json::json!({ "version": "0.9.0-alpha", "containerId": "TelemetryVerboseDenialsE2E", "containment": "processcontainer", "process": { - "commandLine": "cmd.exe /c type C:\\Windows\\System32\\config\\SAM >nul 2>&1 & exit /b 0" + "commandLine": format!("cmd.exe /d /c type \"{}\" & exit /b 0", denied_file.display()) }, "processContainer": { "captureDenials": { @@ -297,7 +301,7 @@ fn find_verbose_artifact(workdir: &Path) -> Option { /// Decodes an `.etl` to XML and returns the text, or `None` if it holds no /// decodable events (`tracerpt` reports failure on an empty trace). fn decode_trace(etl: &Path, workdir: &Path) -> Option { - let dump = workdir.join("dump.xml"); + let dump = workdir.join(etl.file_name()?).with_extension("xml"); let _ = std::fs::remove_file(&dump); Command::new("tracerpt") .arg(etl) @@ -474,7 +478,7 @@ fn test_verbose_denials_etw_payload_honors_consent() { &std::fs::read(&verbose_path).expect("failed to read verbose artifact"), ) .expect("verbose artifact was not valid JSON"); - assert_eq!(verbose["version"], 2); + assert_eq!(verbose["version"], 3); let granted_dump = decode_trace(&granted_etl, &workdir) .expect("tracerpt produced no output for the verbose telemetry run");