AI Diagnostics
How AI Analysis Works
When you open a guided troubleshooting session and describe what's wrong, ET Ducky runs a multi-step investigation rather than a single AI call. The agent gathers evidence from the host in rounds, the AI decides what to look at next based on what each round surfaces, and the conclusion (along with any proposed remediation) lands back in the session for your review.
- You describe the issue in natural language (or attach a ticket).
- The cloud orchestrator starts an AI-driven dynamic ETW investigation against the target agent (see below).
- The AI picks ETW providers from the host's actual catalog, enables them on a separate diagnostic session, captures briefly, and reads the results.
- If the first round isn't conclusive, the AI adjusts — enables more providers, drops noisy ones, narrows keyword masks — and runs another round.
- Once the AI has enough evidence, it produces a root cause + recommendations writeup. In live sessions where remediation is well-defined, it also proposes a remediation script that goes through the org's existing automations approval gate before any execution.
Example Queries
- “Why is the SQL Server service failing to start?”
- “What processes are accessing C:\ProgramData?”
- “Show me all failed authentication attempts in the last hour”
- “Why is the system slow right now?”
- “What changed before the application started crashing?”
- “DNS resolution is intermittent on this box, find out why”
- “Logon stalls at ‘preparing windows’ for 90 seconds, what's responsible?”
Dynamic ETW Diagnostic Sessions
Every Windows host registers between roughly 1,500 and 2,500 ETW providers. The traditional approach (collect everything, analyse after) is expensive; the manual approach (operator picks providers) is impractical for non-experts. ET Ducky's AI orchestrates the session layer instead: it picks providers for each round based on what the captured evidence suggests is worth looking at next.
How it works on the host
- The agent enumerates its full ETW provider catalog at startup and uploads a tier-filtered inventory to the cloud: ~50 well-known "Tier 1" providers (kernel, networking, .NET, security baseline) plus any providers marked recently-active become the AI's working set. The long tail is reachable but not shown by default.
- When an investigation begins, the agent spins up a separate diagnostic ETW session with a unique name — never the always-on baseline session. The two sessions share no state.
- The AI's per-round plan (which providers to enable, which to disable, how long to capture) is applied to the live diagnostic session. The agent counts events into a per-(provider, event id, task, opcode) tally; only the counts cross back to the cloud.
- Before tear-down, a local deterministic rule engine runs over a bounded sample of the round's events. When it recognises a known failure pattern (Azure AD Conditional Access denials, TLS untrusted-root chains, Kerberos SPN-unknown errors, etc.) it emits a verdict — severity plus pre-authored explanation and suggestion text, no raw event payloads — that ships alongside the counts summary. The AI planner sees these verdicts as authoritative findings; when one explains the symptom the investigation can conclude immediately instead of running additional rounds.
- When the investigation finishes, the diagnostic session is torn down. Raw event buffers are purged. The conclusion travels; the evidence stays on the endpoint.
What the AI sees per round
- The user's reported issue
- What's currently enabled on the diagnostic session
- The previous round's counts-only snapshot (top buckets by event count)
- Any verdicts the agent's local rule engine emitted over the previous round's events
- Remaining budget headroom (how many more providers the session can take)
- The host's tier-filtered provider catalog
The AI returns a structured plan: enables, disables, capture seconds, and a "should we conclude?" flag with confidence. The loop applies, captures, snapshots, re-plans — up to a default 5 rounds (hard ceiling 10) with each round capturing for 10–120 seconds.
Deterministic Verdicts & Connection Stories
ET Ducky's AI doesn't read a wall of raw events from scratch for every question. Two pre-processing layers run between the captured events and the model so the AI sees structured, semantically dense input — and so the answer is deterministic for the patterns we can already recognise.
Pre-classified verdicts
A local rule engine scans the captured events against a curated set of known failure patterns: Azure AD Conditional Access denials (AADSTS error codes), TLS handshake failures (untrusted root CAs, unknown_ca alerts, protocol-version mismatches), Kerberos errors (SPN unknown, pre-authentication required), DNS failures (NXDOMAIN, SERVFAIL), HTTP-layer failures (401, 403 with service-specific diagnostics), and more.
When a rule matches, the engine emits a verdict: a severity, a short title, a 1–3 sentence explanation, and an actionable next step. Verdicts appear at the top of the AI's prompt as "pre-classified findings"; the AI treats them as authoritative ground truth and writes a confident, specific answer instead of inferring from raw events and hedging.
The rule engine is deterministic and cheap. For the common patterns it covers, an answer that previously required several AI inference passes now lands with a single tight synthesis call.
Connection stories
For network-shaped questions, captured events are also grouped into per-destination connection stories: one row per (process, hostname) pair, with the chronological chain of DNS → TCP → TLS → HTTP → auth events for that one logical operation. A story collapses what used to be six to eight raw events spread across the sample into a single dense narrative the AI can read in one pass. Network destinations with mixed outcomes (some steps OK, some failed) sort first so the AI's attention lands on what's interesting.
Where this applies
Both layers run inside the unified query engine that powers Ducky Chat (the desktop "ask the AI about my system" path) and the cloud orchestrator's hypothesis loop (the dashboard's guided-troubleshooting investigation). The same rule registry serves both surfaces; a verdict that fires for a Ducky Chat question fires the same way for a live investigation, so the diagnoses are consistent across single-machine and fleet contexts.
Budget & Safety Caps
The AI is constrained by the engine, not trusted to behave. Caps live inside the dynamic capture engine; the AI can't bypass them. A request that would exceed a cap is refused with a structured error and the AI re-plans.
| Cap | Default |
|---|---|
| Concurrent providers per investigation | 16 |
| Aggregate events/sec per session | 25,000/s |
| Aggregate events/sec across all diagnostic sessions on the host | 50,000/s |
| In-flight buffer memory per session | 64 MiB |
| Hypothesis-loop rounds per investigation | 5 default / 10 hard ceiling |
| Per-round capture window | 10–120 seconds |
| Concurrent investigations per host | 4 |
If observed throughput exceeds a cap mid-capture (the per-provider estimate was wrong, the host is busier than expected), the engine triggers emergency scale-back: the most-recently-enabled provider is automatically disabled and the AI is told so it can re-plan rather than retry. Investigations that go 15 minutes without a command from the cloud are reaped by an on-agent sweeper.
Remediation Script Handoff
When an investigation concludes with a well-defined fix, the AI generates a remediation script (PowerShell on Windows, Bash on Linux) and submits it to the org's Process Automations approval flow. Mode-dependent behavior:
- Live guided-troubleshooting session, operator attending: the script is created with
auto_run_eligible = true. After org-admin approval, the operator can run it immediately in-session through the existing automations execution path (which itself enforces command security policy). - Background / scheduled / dashboard-initiated investigation, no operator: the script is created with
auto_run_eligible = falseand stays as analysis-only until a human picks it up.
The product invariant — ET Ducky does not take unauthorised actions on customer hosts — holds for AI-generated scripts the same way it holds for human-written ones. The approval gate, per-command security policy, and org-admin role check apply equally. AI-generated remediations are tagged with Origin = "process_automation" and a CreatedBy = "dynamic-etw:<jobId>" marker so the audit trail makes the source unambiguous.
Query Pool System
| Tier | Monthly Queries | Cost |
|---|---|---|
| BYOK | Unlimited | Free (your provider key: Anthropic, OpenAI, or Azure OpenAI) |
| Professional | 1,000 | $39/mo |
| Business | 5,000 | $99/mo |
| Enterprise | 50,000 | $249/mo |
What Consumes Queries
- Live Session Question: 1 query, plus one additional query per round of the dynamic ETW investigation that fires for that question (default cap 5 rounds, hard ceiling 10).
- Multi-Agent Question: 1 per agent per question
- Cross-Correlation: 1 additional per device
- Alert AI Analysis: 1 per alert (cached 24 hours)
- Remediation Script Generation: 1 query when the investigation concludes with a proposed fix
Shared across all org members. Unused queries do not roll over.
Inflection Root-Cause Analysis (RCA)
Every agent’s heartbeat is scanned for inflections — sudden metric movements like CPU spikes, memory drops, or disk-time surges. The dashboard surfaces unacknowledged inflections as Anomalies on each agent row. Clicking View anomalies opens a per-agent drawer where you can select one or more anomalies and click Analyze (RCA) to launch a focused AI investigation.
What RCA does
- Brackets the timeline. The cloud takes the min/max timestamps of the selected inflections and pads ±60 seconds, then queries every captured event in that window from the agent’s event store.
- Builds a focused prompt. The model receives the inflection list (metric, direction, before/after values, percentage change) plus events grouped by ProviderName, severity-tagged, and message-truncated. Up to 500 events are included to stay within token budgets.
- Returns structured output. The response is split into three sections — Root Cause, Evidence, Recommendations — rendered as collapsible panels in a modal overlay. OS-tagged commands are suggested where evidence supports them (PowerShell on Windows, bash/systemctl/journalctl on Linux).
When evidence is thin
If the bracket contains zero captured events — for example, the inflection happened before the agent started shipping events, or capture was disabled during the spike — the model is told to reason from the inflection metric alone and to be explicit about the uncertainty. You’ll see a “Zero events in bracket” note in the Evidence section.
Cost & provider
Each RCA consumes 1 AI query from your workspace pool, just like a live-session question. The call routes through the same BYOK / platform-key resolver as the rest of the AI surface — if you’ve set a custom provider key in Team → AI, RCA uses it; otherwise the platform default is used. Token cost and elapsed time are surfaced in the modal footer.
Bring Your Own AI Key (BYOK)
Team → AI. By default, AI features (live-session questions, RCA, smart reports, alert enrichment, script generation) use ET Ducky’s platform key and draw from your workspace query pool. With BYOK you point ET Ducky at your own provider account instead — useful for data residency, cost control, or higher limits.
Choosing a provider
Open Team → AI and pick a provider:
- Anthropic Claude — key, plus an optional model override.
- OpenAI — key, plus an optional model override.
- Microsoft Copilot (Azure OpenAI) — key, plus the Azure Endpoint URL and the deployment name in the Model field (both required for Azure).
Paste your provider key and save. The key is never shown again (only a hash is stored); once saved, the tab shows a green “A key is saved” badge and the field placeholder reads “leave blank to keep it.” To rotate the key, type a new one; to update only the model or endpoint, leave the key field blank.
What BYOK changes
- AI calls from your workspace route through your provider key — their cost, your bill.
- BYOK calls do not deduct from your platform query quota.
- If a BYOK call fails (bad key, provider outage), ET Ducky falls back to the platform key so troubleshooting isn’t blocked, and records the failure on the AI tab.
Usage
The AI usage card on the same tab breaks down token use by feature (live queries, RCA, smart reports, and so on) so you can see where your AI spend is going — whether a call ran on BYOK or the platform key.