What Agents Are Supposed to Do Most of a SIEM is reactive. A detection fires, someone gets paged, someone goes and looks. RunReveal's Agents feature tries to move part of that work onto a schedule. You write a prompt, pick a model, set a cron expression, and an LLM goes off, queries your logs with the same tools you'd use, and sends you what it found. There are six starting templates in the console: Weekly Threat Hunt: a recurring hunt that ends in a written report. It doesn't create detections. Detection Coverage Gap Analysis: compares your sources against your detections and suggests what's missing. Detection Tuning: looks for noisy detections and proposes changes. Data Volume Anomaly Detection: watches source volumes for spikes and drops. A source going quiet is often the first sign something broke, or that someone turned logging off. Data Saving Analysis: points at high-volume, low-value logs you could filter or drop. Open Investigation Analysis: summarises open cases and flags the ones nobody has touched. Why I Didn't Run One Every other part of this series is built on things I actually ran. This one has a gap in the middle, and I'd rather say so up front than dress it up. An agent needs an AI provider connected to the workspace: Anthropic, OpenAI, Google AI, Bedrock or Vertex. RunReveal doesn't ship its own model. You bring your own key, or for Bedrock, an IAM role that RunReveal assumes in your AWS account. The workspace I've been testing in is a shared lab, and connecting my personal API key to it would mean my own billing account paying to process company data. I didn't want that, and setting up a proper Bedrock role is somebody else's call. So no provider is connected, and no agent has ever completed a run here. That turned out to be less of a dead end than I expected. You can learn a lot about an AI feature from everything around the model: what the API insists on, what the defaults are, what instructions the agent gets before it starts, and what gets written to the audit log. That's what the rest of this article cover What the Create API Actually Insists On Back in September I created a test agent through the API to see the real request and response shapes. Two of my first attempts failed, and the errors tell you more than the docs do: POST /agents/create (no provider) -> "AI provider is required. Please select a provider (e.g., anthropic, openai, googleai, bedrock, vertex)" POST /agents/create (provider, no model) -> "AI model is required. Please select a model for the chosen provider" So provider and model are both mandatory, and the provider list is a fixed set of five. What isn't checked is whether the model exists. I passed a model name I made up, not-a-real-model-xyz, and the agent was created without complaint. You'd only find out at the first scheduled run, which could be days later. The full set of fields create accepts is wider than the console shows: name, displayName, description, prompt, provider, model, schedule (cron), notificationNames, tools, parameters, riskScore, notes, plus two switches worth knowing about: disableRunRevealSystemPrompt and skipPermissions. I didn't test either. The names alone tell you there's a RunReveal system prompt sitting in front of yours, and a permission check that can be turned off. The Defaults Point at Email, Twice This is the part I'd want anyone to read before building an agent through the API. What the API did: I sent "enabled": false and no notification channel. The agent came back with enabled: true, notificationNames: ["email"], and a next run scheduled about two days out. The false I sent was silently ignored, and the email channel was filled in for me. I caught it because I read the object that came back instead of assuming the request did what I asked. Fixing it took a second call, and that call is a full-replace PUT: my first attempt sent only the fields I wanted to change, and failed: PUT /agents/update (just enabled + notificationNames) -> Key: 'UpdateAgentRequest.DisplayName' Error:Field validation for 'DisplayName' failed on the 'required' tag Key: 'UpdateAgentRequest.Prompt' Error:Field validation for 'Prompt' failed on the 'required' tag You have to fetch the agent, change the two fields, and send the whole thing back. One more inconsistency while you're there: update wants enabled as a real JSON boolean, while the detections enable endpoint wants it as a string. Same word, opposite types, same API. The second time email shows up is new since I first wrote about this. The workspace now ships built-in agent skills (more on those below), and one of them is called notifications. This is the instruction it gives the agent, word for word: "If you are working in a background context and cannot confirm, look for the channel named email. Most workspaces should have a channel named email, if not just use the oldest channel with the type email." A scheduled agent is a background context by definition. So if you leave the channel empty, the API fills in email, and if the agent decides to send something on its own, its own instructions also send it to email. In most workspaces that channel goes to real people. In ours it goes to a colleague's inbox. That's the reason I neutralised the test agent within a minute of creating it, and then deleted it. My rule for anyone trying this: create a dedicated test channel first (a Slack channel only you are in, or a webhook you control), set it explicitly on the agent, and read the object back after every create and update. Don't rely on enabled: false to keep anything quiet. One thing that did work properly: DELETE /agents/delete?id=… really deletes the agent. My test agent was gone from /agents/list straight afterwards. That stood out, because the delete endpoints for detections and pipeline steps I'd hit earlier in this series don't actually remove anything. Built-in Skills: The Instructions Behind the Agent Skills are markdown playbooks an agent loads when it needs them. When I checked on 28 September, the skills list in this workspace was empty. Checking again today, there are three, and none of them were written by anyone on our side: GET /agents/skills/list data-analysis sources -> tables -> schema -> query workflow, ClickHouse syntax, dashboards detection-authoring Sigma vs SQL, lookup order, pre-validation, naming, categories, MITRE notifications choosing a channel, Handlebars templates, NotificationSend These are worth reading even if you never run an agent, because they're RunReveal telling its own AI how to use the platform properly. A lot of it is good advice for humans too. Some of the specifics I hadn't seen written down anywhere else: Default to Sigma for single-event matching. Use SQL only for aggregation, correlation, time-window counting or joins. Sigma runs in real time per event, SQL runs on a schedule. Mandatory pre-validation before creating a detection: read the schema, sample 5 rows from the last 7 days, get the event-name distribution, then test the WHERE clause with LIMIT 5 and check it runs in under 1,500 ms. Performance targets: a detection on a 15-minute schedule should run in under 1,000 ms, an hourly one under 2,000 ms. Use count(), not count(*). Naming: displayName is "{Service} {Description In Title Case}" ("Okta Push Fatigue"), the slug is lowercase with hyphens, 3 to 128 characters, no underscores. Categories go in a fixed order: service, then an OCSF category, then signal or non-signal, then optional tags. Example: [okta, identity-access-management, signal, privilege-escalation]. MITRE fields are validated by regex. TA#### for tactics, T#### or T####.### for techniques. t1078, T1136.1 and TA001 are all rejected. "Zero results != broken detection." That line made me laugh, given that Part 6 covered a detection that returned zero results because its filter could never match. Both things are true; you have to check which one you're looking at. The data-analysis skill also has a sensible habit I'd copy: when no timeframe is given, default to the last 24 hours but tell the user you assumed it. And it warns the agent to check how big rawLog is before querying it, because full-table scans time out with a 5xx. Built-in Resources, and What Sigmalite Changes Alongside skills there are three reference documents agents can load: a detection engineering guide, a list of console URL routes (so the agent can link you to the right page), and a reference for sigmalite, the Sigma dialect RunReveal actually runs. The sigmalite one is the most useful, because it lists where RunReveal differs from upstream Sigma: Added: a cidr modifier, for matching IP fields against ranges. Upstream Sigma has no native equivalent. Not supported: gt, gte, lt, lte, utf16le, utf16be. The rule fails to parse if you use them. Numeric comparisons belong in a SQL detection. Extra fields: sourcenames and sourcetypes to scope a rule, score for signal chaining, mitre, disabled, and notes. detection: local: DestinationIp|cidr: - "10.0.0.0/8" - "172.16.0.0/12" - "192.168.0.0/16" condition: not local If you're porting Sigma rules from a public repo, the unsupported modifiers are the thing to grep for first. What the Audit Log Shows RunReveal writes its own audit trail into the same logs table as everything else, so I could query my September test back out of it. Every agent API call is there, failures included, with the error message attached: SELECT receivedAt, eventName, rawLog FROM logs WHERE receivedAt >= now() - INTERVAL 60 DAY AND sourceType ILIKE '%runreveal%' AND eventName IN ('agents_create','agents_update','agents_delete') ORDER BY receivedAt A failed create looks like this (workspace and IP details removed): { "event_name": "agents_create", "actor_email": "Testing", "readonly": false, "supplemental_data": { "error": "true", "error_message": "AI model is required. Please select a model for the chosen provider", "user_agent": "curl/8.5.0" } } Three things in there are worth knowing if you ever investigate agent activity: actor_email isn't an email. For API calls it's the name of the API token, here "Testing". If your tokens have vague names, your audit trail will too. Name tokens after the person or the automation that owns them. There's no request body. The log tells you an agent was created and that it succeeded. It doesn't tell you it was created with enabled: true and an email channel when the caller asked for neither. The silent default from the section above is invisible in the audit trail. Reads are logged too. agents_list, agent_runs_list, agent_tasks_list, chat_usage_summary and workspace_ai_models_info all show up, so you can see who has been looking at the AI features, not just who changed them. If you want a detection on this, agents_create and agents_update events where readonly is false and the error field is empty is a reasonable start. An agent is code that runs on a schedule with access to your logs and, by default, the ability to write Sigma rules. New ones deserve the same attention as a new scheduled task on a server. The Trust Boundary The tool permissions on a new agent are on by default. The list includes reading schemas, query history, sources and detections, and also creating and updating Sigma rules and sending notifications. Put that next to everything above: it can be created with enabled: true when you asked for false, it defaults to emailing a real channel, its own instructions tell it to fall back to email when it can't ask, it's allowed to write detection rules, and there's a skipPermissions switch in the API. None of that is a vulnerability. It's a product built to be useful out of the box, which is what most people want. But it means the safe configuration is something you build on purpose, not something you get by default What I would check before turning one on A dedicated test notification channel exists and is set explicitly on the agent. Tool permissions are trimmed to read-only for the first few runs. Turn rule creation on only once you trust the output. The model name is real, so the first run doesn't fail silently days from now. The object you get back from create and update matches what you sent. The API token used has a name that identifies its owner, so the audit trail means something. Someone has read the built-in skills, since they shape what the agent will do. Where This Leaves Agents I can't tell you how good a RunReveal agent's threat hunt report is, because I haven't seen one. What I can tell you is that the scaffolding is more thought through than I expected. The skills carry real detection-engineering rules, the sigmalite reference is honest about what doesn't work, and every call lands in an audit log you can query with SQL. The rough edges are in the defaults, and they all lean the same way: towards running, and towards email. If a proper provider gets connected to this workspace later, I'll run the Weekly Threat Hunt template against it and write up what comes back, good or bad. ----------------------------------------------------Dean------------------------------------------------

Seven Tools for Shaping Data In Flight Earlier introduced pipelines as an ordered chain of steps, using one real example that happened to include transform, enrich, filter, detect, and destination steps. What I didn't appreciate until I read through RunReveal's log-processing documentation properly is how much depth sits behind each of those step types individually. This article covers the ones worth understanding on their own: Transforms, Filtering, Dropping, Masking, Sampling, Enrichments, and Set TTL. Enrichments — Tagging Events Without Touching Storage Rules An enrichment matches incoming events against a pattern and attaches supplemental key-value data to them. Rules are comma-delimited rows following match_type, pattern,data, matched against sources you select. The real value isn't the tagging itself — it's what you do with the tag afterward. The documented patterns for this are genuinely practical, not abstract examples: Exclude known-good activity from detections — tag events from service accounts or automation systems, then have your detection query check for that tag and skip it, rather than hand-listing every service account inside every detection's WHERE clause. Tag trusted IP ranges — flag corporate VPN or known cloud infrastructure ranges via CIDR match, so network-based detections can exclude them centrally instead of repeating the same exclusion list everywhere. Tag events for dropping — rather than hard-coding drop patterns into a Drop step directly, tag candidates with an enrichment first. That keeps drop rules centralized, auditable, and visible before anything actually disappears. Classify by risk level — tag actors or resources by sensitivity (a production database ARN, an admin IAM role pattern) using regex rules, feeding tiered alerting downstream. Something worth remembering about how rules get edited: Clicking a rule in the rules table does not open an inline editor. To change one, you delete the existing rule and add a new one with the updated values. If you add a rule with the same pattern as an existing one instead of deleting first, the UI adds a duplicate entry rather than replacing it. Enrichments are also fully API-manageable — create, update, append rules, get, list, and delete all have real endpoints, which means this is scriptable rather than console-only if you're managing rule sets at any real scale. Filtering vs. Dropping vs. Sampling — Three Ways to Reduce Volume, and When Each Applies These three get confused with each other constantly, and RunReveal's documentation actually spells out the distinction cleanly enough that I'll just relay it directly, because it's the most useful framing I've seen for this problem on any SIEM. Filtering — reusable, regex on the raw log A Filter matches a regex pattern against the entire raw log JSON string, not a specific field — so it can catch content anywhere in the event. Filters are managed centrally and can apply globally across all sources or be scoped to specific ones. Dropped events never reach ClickHouse and never show up in Explore. The tradeoff: because it's a shared, reusable rule set, it's the right tool when you want the same drop logic — health-check traffic, known bot user agents — applied consistently across multiple pipelines. Dropping — pipeline-specific, precondition-based A Drop step lives inside one specific pipeline and uses preconditions against normalized fields (eventName regex, an IP CIDR match, a missing-field check) rather than a raw-log regex. Multiple preconditions on the same step AND together. This is the right tool for a one-off rule scoped to a single pipeline, versus something you want shared everywhere. Sampling — statistical, not absolute A Sample step keeps a configured percentage of matching events (1-99%) and discards the rest, while non-matching events pass through untouched. The real-world examples given are telling: sample S3 data events at 10%, but keep VPC Flow Logs' rejected connections and CloudFlare's error responses at full fidelity even while sampling the successful/accepted traffic. The decision in one line: Filter for shared rules across pipelines. Drop for a one-off, pipeline-specific rule. Sample when you want to keep statistical visibility into high-volume noise rather than delete it outright. Masking — Protecting Fields Without Losing the Event Masking mutates a field's value in place — every step and destination after it in the pipeline sees the masked value, not the original. Three modes: Redact — replace the entire value with a static token. Hash — replace with a deterministic, non-reversible pseudonym. Equal input values produce equal hashes within a workspace, so a masked field stays joinable and countable in detections even though the real value is gone. Partial — replace only the regex-matched portion, leaving the rest visible. The documented example: strip the local part of an email address but keep the domain, so you can still group and filter by domain without storing who exactly it was. One detail worth knowing: a Mask rule targeting a field that doesn't exist on a given event is silently skipped rather than erroring, which means it's safe to write one mask config that references fields only present on some of your sources. Mask vs. Drop, in one sentence: mask when you still need the event for analysis but must protect or pseudonymize a specific field; drop when the entire event has zero value and should never be stored at all. Transforms — Reshaping a Value Before It Lands A transform rule is a chain of processors run in order against the raw log, always ending in an Output to Field step that writes the final result to a normalized field. Available processors include GJSON-style path extraction, regex find/replace, string prepend/append, trim, substring by index (negative indices allowed, counting from the end), split-by-delimiter, and date format conversion. Rules are independent of each other, and a processor's output becomes the next processor's input — so "Extract JSON Field" partway through a chain will fail if the prior processor handed it a plain string instead of JSON. That's listed as one of the documented common mistakes, alongside forgetting the mandatory Output to Field step, which just means the rule silently doesn't reach a normalized field at all rather than erroring loudly. Set TTL — Retention, Per Event A Set TTL step lets you override retention on a per-event basis. If you never touch it, ingestion applies the platform default of 550 days automatically — this is the same number referenced back in Part 2's retention discussion, now with the mechanism that actually controls it at the event level rather than just the workspace default. Hands-On: What the API Doesn't Document Everything above is real, but it's still secondhand — pulled from RunReveal's documentation rather than from actually operating the platform. Since I have live API access to a workspace, it seemed worth actually exercising two of these tools directly rather than just relaying what the docs say about them, and writing down what happened, including the parts that didn't work the way the docs implied. A transform that's already running, verified against real ingested data Rather than build a synthetic example, I went looking for a transform rule already active in the workspace and checked its actual output against actual ingested events. One was already running on a lab source: a processor chain that extracts an "environment" field, uppercases it, and writes the result to a normalized tag field. A real raw event ingested through that source carries this in its JSON payload: {"environment":"dev","application":"", ...} And the same event's tags, queried directly from ClickHouse: tags = {"RR_Pipeline":"Default Pipeline","RR_Topic":"Default Topic", "environment_CAPITAL":"DEV"} That's the whole point of a transform in one real pair of before/after values: "dev" goes in, "DEV" comes out, on an event nobody hand-crafted for this article. Building an enrichment rule from scratch — and hitting five undocumented behaviors The docs describe enrichment rules conceptually ("match_type,pattern,data" rows) but the actual API schema for creating one isn't published anywhere I could find. So I built one by trial and error against the live create endpoint, reading the validation errors back from the server to figure out the real field names. Five things came out of that exercise that aren't in the documentation: The source list field is called sources, and it takes source names, not source IDs — passing a valid source ID here fails with "invalid source name," even though every other reference to sources in this API is ID-based. The match-type field is called type, not matchType or match_type (both of those are silently accepted and silently ignored — you get back the actual error "invalid match type ''" with no hint that the key name itself was wrong). Valid values are exact, regex, or cidr — the docs' own contains-style examples describe the behavior, not the literal parameter name. A newly created rule comes back with isActive: false by default. It exists, it's saved, and it does nothing until you explicitly flip it on — there's no warning in the create response that the rule is inert. Updating a rule doesn't accept a partial patch. Sending just {id, isActive: true} returns a bare "unknown error" with no explanation; the update endpoint wants the full object — name, sources, and rules — resent every time, or it fails silently on the missing fields. The update also regenerates the rule's internal ID rather than editing it in place. That's a nice confirmation, at the API level, of the console-side detail mentioned earlier in this article — there really is no in-place edit happening anywhere in this system, UI or API; every "edit" is a delete-and-recreate under the hood. The one that actually matters operationally: Activating an enrichment rule does not retroactively tag events that were already ingested before activation, even when their content matches the rule's pattern. I confirmed this directly: queried a set of already-ingested events matching the rule's regex, activated the rule, then re-ran the identical query — the tag was absent both times. Enrichment only applies going forward, at ingest time. If you're rolling out a new tagging rule, don't expect it to backfill your existing data; budget for a re-ingestion or accept the gap. One more thing worth knowing if you're scripting any of this: the delete endpoint for enrichments returns a 405 on a plain POST and needs the literal DELETE HTTP method to be accepted at all — and even then, it only clears the rule's contents rather than removing the enrichment record itself, which then lingers, empty and inert, in the list output. That matches a pattern already seen elsewhere in this API: detections and pipelines have the same non-functional delete behavior. Plan for disable-in-place as the real cleanup mechanism here, not delete. ------------------------------------------------------------Dean--------------------------------------

What I Was Trying to Do Every SIEM bill I've ever looked at has the same problem: a big chunk of what you pay to store is noise nobody will ever query. RunReveal's answer is the pipeline. Events are matched, transformed, enriched and, if you want, thrown away before they ever reach storage. One of the part in this series covered this from the console side. For the last part of the series I wanted to do it properly, through the API, and prove it works with real events instead of trusting a green tick in the UI. The plan was small on purpose: Build a pipeline with one drop rule: throw away any event whose marker field equals article-drop-me. Route only webhook traffic into that pipeline, so nothing else in the workspace is touched. Send two events through a test webhook: one marked article-drop-me, one marked article-keep-me. Query the logs table. If only the keep-me event is there, the drop rule works. That isn't what happened. I'm writing it up anyway, because what I hit on the way is more useful than a clean result would have been. There are four API behaviours here I couldn't find in the docs, and I'd want to know about every one of them before automating anything against this API. How Routing Actually Works The docs describe this in prose, but the clearest picture comes from reading a working pipeline back out of the API. The model is: Source -> Topic (precondition + rank) -> Pipeline (ordered steps) -> destination step -> ClickHouse Topics are checked top to bottom by rank. The first topic whose precondition matches wins. Each topic points at one pipeline. Anything no topic catches goes to the default pipeline. A pipeline is an ordered list of steps: transform, enrich, filter, detect, drop, destination. The destination step is what actually writes the event to storage. A pipeline without one stores nothing. Our workspace has one catch-all topic (precondition type all) wired to a pipeline that's been running since the workspace was set up. Its steps, as the API returns them: transform precondition: all settings: {"transform_id": "*"} enrich precondition: all settings: {"enrichment_id": "*"} filter precondition: normalized.src.asNumber not_equal … settings: {"filter_id": "*"} detect precondition: normalized.src.asNumber is_empty destination precondition: all settings: {"destination_id": "default-clickhouse"} Keep that transform step in mind. My test pipeline didn't have one, and it's one of the two things I still suspect. Building the Steps: Two Broken Endpoints, One That Works The API offers three ways to put steps into a pipeline. I tried all three, with payloads copied from the working pipeline above so the shape couldn't be the problem. POST /pipelines/add-step returned HTTP 500 "unknown error". I tried settings as a JSON object and as base64. Same result both times, again on 13 October. POST /pipelines/execute-step returned 502 Bad Gateway, even against the working pipeline. That's a gateway error, not a validation error. POST /pipelines/save-steps worked, first time. It replaces the whole step list in one call. This is the save-steps call that built my test pipeline: POST /pipelines/save-steps { "id": "", "steps": [ {"type": "drop", "rank": 1, "settings": {}, "precondition": [{"field": "marker", "value": "article-drop-me", "type": "equal"}]}, {"type": "destination", "rank": 2, "settings": {"destination_id": "default-clickhouse"}, "precondition": [{"field": "", "value": "", "type": "all"}]} ] } Gotcha 1: If you're scripting pipelines, skip add-step and go straight to save-steps. Read the current steps, change the list, and write the whole list back. That's also how the console behaves, as far as I can tell. Wiring It Up Without Touching Production Traffic The pipeline does nothing until a topic points at it. I created a topic called webhook-drop-test with a precondition of sourceType equal webhook, then attached the pipeline and moved the topic above the catch-all: POST /topics/set-pipeline {"id": "", "pipeline_id": ""} POST /topics/move {"id": "", "rank": 1} Both calls returned success. The important check came straight afterwards: our Google Workspace events kept arriving at their normal rate through the catch-all. So the topic precondition did exactly what it should and only caught webhook traffic. The only webhook source in the workspace is an old test source from an internal tool, idle since early July, so nothing real was at risk. Then the two test events, sent to that source's webhook URL with its bearer token: {"marker": "article-drop-me", "run": "20261014T191844Z", "message": "pipeline drop test"} -> HTTP 200 {"marker": "article-keep-me", "run": "20261014T191844Z", "message": "pipeline drop test"} -> HTTP 200 The Result: Nothing Arrived I searched the logs table for the run ID, across every source type, for several minutes. Neither event was there. Not the dropped one, which was expected, and not the kept one either, which wasn't. SELECT receivedAt, sourceType, sourceID, rawLog FROM logs WHERE sourceType != 'runreveal-audit' AND rawLog LIKE '%pipeline drop test%' -> 0 rows A small trap while I was doing this: my first search matched rows, and they were all my own searches. Every query you run is logged to the runreveal-audit source, with the full SQL text in it. Searching rawLog for a string will find your own query unless you exclude that source. My first theory was the missing transform step. The working pipeline starts with one, mine didn't, and a pipeline that skips transform might be handing the destination an event it can't write. To test that properly I needed a control: send the same kind of event down the normal path, with my topic out of the way, and confirm it lands. Getting that control set up turned up two more gotchas. Gotcha 2: topics/move Renumbers, It Does Not Reorder To get my topic out of the way I moved it to rank 2, the catch-all's slot, expecting the two to swap places. This is what came back: before: 1 webhook-drop-test 2 test-topic (catch-all) move webhook-drop-test -> rank 2 after: 2 webhook-drop-test 3 test-topic (catch-all) The catch-all got pushed down to make room. My topic was still checked first, so my control event went through the test pipeline anyway and proved nothing. Rank is a sort value, not a slot. If you want a topic below another one, move it to a rank higher than that topic's current rank, and always read the list back afterwards. Gotcha 3: topics/update Is a Full Replace Plan B was to disable my topic instead of moving it. I sent only the field I wanted to change: PUT /topics/update {"id": "", "enabled": false} -> {"id": "…", "name": "", "enabled": false, "precondition": [{"field": "", "value": "", "type": ""}], "rank": 2} It disabled the topic, and it also wiped the name and the precondition, because every field I didn't send was overwritten with an empty value. On a disabled test topic that doesn't matter. On a live topic it could be serious: I don't know how the router treats a precondition with an empty type, and the worst case is that it matches everything. Always send the full object to topics/update, never just the field you're changing. Gotcha 4, and the reason I have not finished this test: With my topic disabled, the control event went through the normal catch-all path, the same one our Google Workspace data uses every minute. The webhook returned HTTP 200. The event never appeared. So the problem isn't my pipeline, or not only my pipeline. This webhook source is accepting events and storing none of them, and nothing in the API response tells you. The last event it ever stored is from 5 July. A 200 from a webhook only means the request was received. It doesn't mean the event was stored. If you're running anything that pushes into a RunReveal webhook, it's worth a scheduled query that alerts when a source goes quiet, because the webhook itself will never tell you. What I Can and Cannot Say Yet Confirmed, with the calls above: save-steps builds pipelines reliably. add-step (500) and execute-step (502) are still broken. A topic with a sourceType precondition isolates traffic cleanly. Other sources were unaffected. topics/move renumbers rather than reorders, and topics/update overwrites fields you don't send. This webhook source returns 200 and stores nothing, even on the default path. Not yet proven: Whether the drop rule itself works. Nothing reached the pipeline in a way I could measure. Whether a pipeline without a transform step loses events. That's still my best guess, but the broken source means I can't test it with this setup. Next is a fresh webhook source built only for this test, a control event to confirm it stores, and then the same two-event run again, with and without a transform step. I'll add the result here when I have it, whichever way it goes. Closing the Series That's eleven parts: what RunReveal is, how it's built, searching it, processing logs in flight, getting data in, detections, investigations, scheduled agents, notifications and the MCP server I built, and how it compares with Splunk and Sentinel. If there's one thing that runs through all of it, it's this. The core idea is good: your logs in ClickHouse, queried with plain SQL, with detections that are just scheduled queries. The parts around the edges are younger than the core. Most of what I hit in this series, from deletes that don't delete to webhooks that return 200 and store nothing, came from actually using the API rather than reading about it. Test it the same way before you trust it with anything important. ---------------------------------------------------Dean-------------------------------------------------

Two things also changed under my feet while I was writing. ClickHouse bought RunReveal (announced 1 September 2026), and Microsoft is in the middle of moving Sentinel out of the Azure portal and into the Defender portal. A comparison that ignores either of those would be out of date the day it went up. Where my knowledge comes from: RunReveal: hands-on, everything in this article was run against a real workspace this week. Splunk and Sentinel: their current official documentation and pricing pages, checked in October 2026. What Changed in 2026 RunReveal is now part of ClickHouse ClickHouse announced it had acquired RunReveal on 1 September 2026. That isn't as odd, RunReveal was always built on top of ClickHouse, with every log landing in one big columnar logs table. ClickHouse's own announcement says existing contracts and support stay the same, and that RunReveal remains available through its bring-your-own-database model, where your security data sits in a ClickHouse cluster you control The practical effect for anyone evaluating it today: the pricing page no longer lists the tiers I recorded back in September. It now says RunReveal has joined ClickHouse, that a joint offering is being worked on, and to contact sales. I'll come back to what that means for cost later on. Sentinel is moving house Microsoft originally planned to retire the Sentinel experience in the Azure portal on 1 July 2026. That date has since moved to 31 March 2027. After that, you manage Sentinel in the Microsoft Defender portal. New customers have been sent to the Defender portal by default since July 2025, and Microsoft's docs say Sentinel works there even without Defender XDR or an E5 licence. The bigger change is underneath. Sentinel now has two storage tiers: the analytics tier (the classic Log Analytics data that analytics rules, hunting and incidents run on) and a data lake tier in open Parquet format that keeps data for up to 12 years. Microsoft's docs say analytics-tier data is mirrored into the lake, so you keep one copy. You query the lake with KQL jobs or Jupyter notebooks, and promote data back into the analytics tier when you need detections to see it. Splunk is a Cisco product with three ways to pay Cisco completed its Splunk acquisition in March 2024, and the pricing story has spread out since then. Splunk's own pricing FAQ now lists three models for Enterprise Security: ingest-based, workload-based and activity-based. Splunk Cloud Platform offers all three; self-managed Splunk Enterprise offers ingest or workload. On the query side, Splunk Cloud's Federated Search for Amazon S3 lets you search S3 data without indexing it first. It uses SPL2, not classic SPL, and Splunk positions it for archival or low-value data rather than real-time detection. The Short Version Query language: RunReveal: ClickHouse SQL, plus Sigma rules Splunk: SPL (and SPL2 for federated search) Microsoft Sentinel: KQL Storage engine: RunReveal: ClickHouse, columnar, one wide logs table Splunk: Splunk indexers and buckets (SmartStore in the cloud) Microsoft Sentinel: Log Analytics (analytics tier) plus a Parquet data lake tier Detections: RunReveal: Scheduled SQL or Sigma, defined as text from the start Splunk: Correlation searches in Enterprise Security Microsoft Sentinel: Analytics rules (scheduled and near-real-time KQL) Allowlisting: RunReveal: SQL WHERE clause or an enrichment tag Splunk: Lookup tables Microsoft Sentinel: Watchlists How you pay: RunReveal: Was storage-based ("no ingest fees"); now "contact us" after the ClickHouse deal Splunk: Ingest, workload or activity-based Microsoft Sentinel: Analytics tier per GB (pay-as-you-go or commitment tiers) plus data lake charges Where it fits best: RunReveal: SQL-fluent teams, cloud and SaaS-heavy log sources Splunk: Large enterprises, deep app ecosystem, existing SPL skills Microsoft Sentinel: Microsoft 365 and Azure-heavy shops Architecture: Where Your Logs Actually Live RunReveal One table does most of the work. Every source lands in logs, with a handful of normalized columns (sourceType, eventName, actor, srcIP, eventTime, receivedAt) and the original event kept in rawLog. You query the normalized columns when they're enough and dig into rawLog with ClickHouse JSON functions when they aren't. Written article showed the one rule that matters most for speed: filter on receivedAt, because that's what the table is ordered by. Every query in this article does that, and none of them took longer than about 0.8 seconds over a week of data. Splunk Splunk parses events into indexes made of time-bucketed files, and extracts most fields at search time. That schema-on-read approach is why Splunk copes with almost any messy log you throw at it, and why the add-on (TA) for each source matters so much: the TA is what turns raw text into the field names your searches use. Splunk Cloud's SmartStore keeps the bulk of indexed data in object storage with a local cache, which changes the cost profile but not how you write searches. Microsoft Sentinel Sentinel sits on Log Analytics workspaces. Each connector writes to its own table, so instead of one big logs table you get dozens of purpose-built ones. That's great for well-supported Microsoft sources and more awkward for third-party ones, where the table you end up with depends on which version of the connector you installed. Google Workspace is a good example. The older Azure Functions connector writes to custom tables such as GWorkspace_ReportsAPI_token_CL. The newer schema uses a GoogleWorkspaceReports table, which Microsoft's table reference marks as a Basic log table that supports lake-only ingestion. Same data, different tables, different query. One Real Detection, Three Languages Here's the problem I picked, because it's a real one: a third-party OAuth app gets granted the full Gmail scope, https://mail.google.com/. That scope means read, send and delete for the whole mailbox. Consent phishing campaigns go after exactly this, so most Google Workspace detection packs include some version of the rule. First, what the data actually looks like Google records each grant as an authorize event under the token application. The interesting parts (app name, client ID, scopes) aren't top-level fields. They sit inside an array of name/value pairs in events[0].parameters, which every platform has to unpack somehow. "events": [{ "name": "authorize", "parameters": [ {"name": "client_id", "value": "1173955..."}, {"name": "app_name", "value": "..."}, {"name": "client_type", "value": "WEB"}, {"name": "scope_data", "multiMessageValue": [...]}, {"name": "scope", "multiValue": ["https://www.googleapis.com/auth/..."]} ] }] RunReveal (ClickHouse SQL), My first attempt picked parameters out by position (the first one is client_id, the second app_name, the fifth scope). It worked, and it's a trap: nothing in Google's API promises that order, and if a parameter is ever added or dropped the rule silently starts reading the wrong field. This version finds each parameter by its name instead: WITH JSONExtractArrayRaw(JSONExtractRaw(rawLog,'events'),1,'parameters') AS params, arrayFirst(p -> JSONExtractString(p,'name') = 'app_name', params) AS p_app, arrayFirst(p -> JSONExtractString(p,'name') = 'scope', params) AS p_scope SELECT JSONExtractString(p_app,'value') AS app_name, count() AS grants, uniqExact(actor['email']) AS users, uniqExact(srcIP) AS ips FROM logs WHERE receivedAt > now() - INTERVAL 7 DAY AND sourceType = 'gsuite' AND eventName = 'authorize' AND has(JSONExtract(p_scope,'multiValue','Array(String)'), 'https://mail.google.com/') GROUP BY app_name ORDER BY grants DESC Run against workspace, it came back in under half a second with exactly one app: our email-security vendor's integration, holding the full Gmail scope for 8 users, with 432 grants from 274 different IP addresses over the week. Both versions, by position and by name, returned the same 432, which is how I checked that the name-based one wasn't missing anything. Read that result cold and it looks alarming: a mailbox-wide scope, used from hundreds of IPs. It's actually expected. Cloud email-security products scan mail through the Gmail API from a pool of cloud addresses. That's exactly why this rule needs an allowlist before it goes anywhere near a notification channel, which is the next section. Splunk (SPL), untested translation In Splunk, the work happens in field extraction. The Splunk Add-on for Google Workspace collects the Reports API activity, including the token endpoint. How the parameters array comes out as fields depends on the add-on version, so treat the field names below as placeholders and check them against your own events first: index=gws "events{}.name"="authorize" "https://mail.google.com/" | spath path="events{}.parameters{}" output=params | mvexpand params | eval pname=spath(params,"name"), pval=spath(params,"value") | eval app_name=if(pname="app_name", pval, null()) | stats values(app_name) AS app_name, values(actor.email) AS user, values(ipAddress) AS ip BY id.uniqueQualifier | stats count AS grants, dc(user) AS users, dc(ip) AS ips BY app_name | lookup oauth_app_allowlist app_name OUTPUT approved | where isnull(approved) It's noticeably more work than the SQL, and most of that is unpacking the array (spath, then mvexpand, then pulling name and value back out). In fairness, a well-built add-on usually flattens this for you at search time, and then the rule shrinks to a couple of lines. That's the Splunk trade in a nutshell: a lot depends on the add-on, and when the add-on is good, writing searches is pleasant. Microsoft Sentinel (KQL), untested translation KQL handles the nested JSON cleanly with mv-apply, which loops over the parameters array per event. Again, the column names depend on which Google Workspace connector populated your table, so adjust them before using this. I've written it against the raw JSON so it doesn't depend on any one connector's flattening: let allowlist = _GetWatchlist("OAuthAppAllowlist") | project SearchKey; GoogleWorkspaceReports // or GWorkspace_ReportsAPI_token_CL on the older connector | where TimeGenerated > ago(7d) | extend raw = todynamic(RawEvent) // RawEvent: whichever column holds the original JSON | extend ev = raw.events[0], Actor = tostring(raw.actor.email), Ip = tostring(raw.ipAddress) | where tostring(ev.name) == "authorize" | mv-apply p = ev.parameters on ( summarize AppName = take_anyif(tostring(p.value), tostring(p.name) == "app_name"), Scopes = take_anyif(p.multiValue, tostring(p.name) == "scope") ) | where set_has_element(Scopes, "https://mail.google.com/") | where AppName !in (allowlist) | summarize Grants = count(), Users = dcount(Actor), Ips = dcount(Ip) by AppName The watchlist is Sentinel's answer to Splunk's lookup table: a CSV you upload and reference by name, which is a nice way to let someone outside the detection team keep the list of approved apps up to date. A Sentinel catch worth knowing: Microsoft's table reference marks GoogleWorkspaceReports as a Basic log table with lake-only ingestion supported. Check which tier your data is landing in before you build a scheduled analytics rule on it. Microsoft's own data lake docs say you can't run analytics rules directly against data lake tables; the data has to be in the analytics tier, or promoted there by a job. The Noise Problem Is the Same Everywhere Before I narrowed the rule to the Gmail scope, I looked at every OAuth grant in the workspace. The numbers are the most useful thing in this article, because they'd hit you on any of the three platforms: Total authorize events (7 days): 24,364, about 145 an hour Distinct OAuth apps behind them: 2 Grants from the SIEM's own Google collector: 23,703 (97%), one client ID, 1 user, 7 cloud IP addresses, present in all 168 hours Grants from the email-security integration: 661, 8 users, 295 IP addresses Scopes the collector asks for: admin.reports.audit.readonly and apps.alerts 97% of the OAuth activity in this tenant is the log collector getting a fresh token so it can read the audit log. It requests admin-reports scopes, every hour, from a rotating set of cloud IPs. Write the common rule "alert on OAuth grants with admin scopes" and the first thing it catches is your own SIEM, about 140 times an hour. Splunk and Sentinel are no different here: their Google Workspace connectors poll the same Reports API with the same kind of service account, so they should generate the same events about themselves. This is also the second time this series has found the collector being the noisiest thing in the data. How each platform lets you exclude it RunReveal: put the exclusion straight into the SQL (AND client_id NOT IN (...)), or tag the collector's events with an enrichment and filter on the tag. Remember from Part 4 that enrichment only tags events from the moment it's activated; it doesn't go back and tag old ones. Splunk: a lookup table of approved apps or client IDs, joined in the search. Easy to share across many correlation searches. Sentinel: a watchlist, as in the KQL above, or an automation rule that closes matching incidents. The watchlist is better, because the noise never becomes an incident at all. Query Languages, Honestly SQL is RunReveal's biggest advantage, and also where it costs you something. If your team already writes SQL, the ramp-up time is close to zero, and ClickHouse's JSON and array functions are fast and capable. But security-specific idioms you'd get for free elsewhere (a time-chart command, transaction grouping, a built-in lookup operator) you write yourself as SQL. SPL is a pipeline language built for exactly this job. It takes real time to learn, and it has quirks (multivalue fields and mvexpand bite everyone at some point), but people who are fluent in it are very fast, and there's a huge amount of shared content written in it. SPL2 brings it closer to SQL, but in practice you'll still be writing classic SPL for most detections. KQL feels the nicest of the three to read. It pipes like SPL, it's typed, and it treats dynamic JSON as a first-class type. If your team already uses KQL for Defender advanced hunting or Azure Monitor, that skill carries straight over. Detection as Code In RunReveal, a detection is text from the start: SQL or Sigma, created and updated over the API. Previous Article showed a detail that matters if you sync detections from git: a Sigma rule is stored as raw YAML in settings.rule, and its query field is empty. A naive export that only reads query will back up every Sigma rule as a blank. I also hit a real gap in the API while testing: deleting a detection through the API didn't work, which is why every test query in this article ran ad hoc instead of being saved. In Splunk, Enterprise Security correlation searches are saved searches underneath, so they can live in an app's configuration files and be versioned like any other config. Splunk also publishes an open library of security detections that many teams use as a starting point. Sentinel analytics rules export as ARM templates, and Sentinel's Repositories feature can deploy rules straight from a GitHub or Azure DevOps repo. That's a solid story if you already run infrastructure as code in Azure. Ingestion: Breadth vs. What You Measure RunReveal's catalogue is around 120 connectors. That's fine for cloud, SaaS and identity sources, and thinner than Splunk's add-on ecosystem, which covers almost anything that writes a log. Sentinel's catalogue is big too, and unbeatable for Microsoft's own sources, many of which come with free ingestion. What I'd look at more than the connector count is lag and duplication, because those affect detections directly. RunReveal's Google Workspace polling lag at between 17 and 155 seconds (median 83), not the round 60 seconds you might assume. Every API-polling connector, on every platform, has some version of this delay. Measure it on whichever platform you pick, and set your rule look-back windows to fit. AI and Automation RunReveal built AI into the workflow early: AI-assisted query writing in Explorer, triage on investigations, and scheduled Agents that run a prompt on a cron schedule and report back. There's a catch I ran into first-hand: those features need an AI provider configured on the workspace, either your own API key or a cloud role RunReveal can assume. With nothing configured, the agents can't run. That's a reasonable design for data control, but it means "AI-native" comes with setup and a bill of its own Splunk offers an AI Assistant for writing and explaining SPL. Sentinel connects to Security Copilot, and the Defender portal move pulls Sentinel incidents into the same queue as Defender XDR. Microsoft's data lake docs also lean on notebooks and AI as reasons to keep years of data. The ClickHouse announcement says RunReveal's agentic investigation work is meant to shape how ClickHouse supports agentic analytics, so expect that side to get more attention, not less. Running It Day to Day This is the part feature lists skip. With RunReveal there's very little to run: no indexers, no forwarder fleet, and most of my time went into writing queries rather than looking after the platform. The flip side is that it's a younger product. I hit undocumented behaviour in nearly every part of this series (string-versus-boolean enabled fields, a delete endpoint that didn't delete, enrichment that doesn't backfill), and you'll find these by testing, not by reading the docs. Self-managed Splunk is the opposite: a lot of infrastructure, but very few surprises left in it after this many years. Splunk Cloud takes most of the infrastructure away. Sentinel needs no infrastructure, but you do need to understand Azure: workspaces, data collection rules, RBAC, and now the Defender portal on top of all of that. So Who Should Pick What Pick Splunk if you need the widest ingestion coverage, you already have SPL skills or content, or you run a lot of on-prem and unusual log sources. Go into the pricing conversation knowing all three models, and model your search load as well as your ingest. Pick Microsoft Sentinel if Microsoft 365, Entra ID and Azure are most of your estate. The free Microsoft data sources and the Defender integration are hard to beat. Plan the Defender portal move now rather than in March 2027, and decide early which tables go to the analytics tier and which go to the lake. Pick RunReveal if your team thinks in SQL, your sources are mostly cloud and SaaS, and you want detections as code without running infrastructure. Before you sign, get the post-acquisition pricing in writing, and ask where the hosted product is going, since ClickHouse's announcement emphasises the bring-your-own-database route. Whichever you pick, do what I did here before trusting any rule: run it against a week of your own data, group by whoever is generating the events, and see how much of it is your own tooling talking to itself. In this tenant it was 97%. -------------------------------------------------------------Dean-------------------------------------

Earlier, I described the escalation ladder — detection, then signal, then alert — and noted that the last step only happens once a notification channel is attached. This is that channel layer. RunReveal supports twelve destinations for alert output: Email, Slack via webhook, Slack via a dedicated bot integration, Discord, PagerDuty, VictorOps, Jira, Google Chat, Linear, incident.io, generic Webhooks, and Tines. That's a reasonable spread — enough to plug into most on-call and ticketing stacks without forcing you through a generic webhook for everything, which is where a lot of smaller SIEM offerings stop. Setting One Up Creating a channel is a one-time step; attaching it to a detection is what actually turns the tap on. A detection or agent with no channel attached can run forever, match real data, and never bother a single person — which is exactly the safe state you want while you're still validating that something works correctly. A Lesson I Had to Learn Twice I want to be honest about something rather than write around it, because it's the most useful thing in this article. Earlier in this series' research, testing a detection end-to-end with a real notification channel attached resulted in an actual email going out to a real person in the workspace — the detection matched a near-guaranteed-to-fire condition faster than I expected, and the channel was live. Later, , the same category of mistake nearly happened again from a different angle: creating an agent through the API with the notification field left unset caused it to default to the same email channel automatically, rather than staying unattached. The actual rule I now follow: Never attach a real notification channel to anything — a detection or an agent — until you have watched it complete at least one full, safe test cycle and read the output yourself. Assume any new object might default to a live channel unless you have explicitly confirmed otherwise by reading the object back, not just by reading what you sent. It's a small operational discipline, but it's the difference between testing safely and accidentally paging a real colleague over something you were only trying to verify works. Keep it in mind for the second half of this article, because it ended up shaping how I built my own MCP server. The MCP Server I Built Most of the hands-on findings came from a conversation. I'd ask a question in MCP it would run SQL, I'd read the result, and we'd go again. What made that possible is a small MCP server I wrote against RunReveal's REST API. This half of the article is about that server: what's in it, what I deliberately kept out of it, and a few real sessions that show why I think it's the most useful thing I built during this whole series. If you haven't met MCP yet: the Model Context Protocol is a standard way to hand an AI assistant a set of tools. The assistant sees each tool's name, description and parameters, decides when to call one, and gets the JSON result back. The server is the bit that sits between the assistant and your real system, and it decides exactly what the assistant is allowed to touch. What it is It's about 330 lines of Python. A thin HTTP client built on httpx handles auth and error handling, a FastMCP server defines the tools, and a tiny CLI picks the transport. It runs over stdio for AI tool, or over SSE on a local port if you want to point something like Bedrock at it. Wiring it into Claude Code is one JSON block: { "mcpServers": { "runreveal": { "command": "uv", "args": ["--directory", "/path/to/runreveal-mcp", "run", "runreveal-mcp"] } } } Auth was the first thing the docs got wrong. The example request on RunReveal's auth page 404s even with a valid token. What actually works is HTTP Basic auth where the token from the dashboard (Workspace → API Tokens) is already the finished credential: you send Authorization: Basic as-is, with no extra base64 step. I only found that by trial and error against the live API. It exposes eleven tools, split by how much damage they can do: Read (7): query_logs, list_sources, list_detections, get_detection, list_notifications, list_investigations, get_investigation Write (3): create_investigation, add_investigation_note, add_investigation_indicator Correlate (1): ioc_lookup, which searches every case in the workspace for a prior indicator match Every one of those wraps an endpoint I called by hand first and checked the response of. None were written from the docs alone. RunReveal already has an MCP server. Why build another? Fair question. RunReveal ships an official one, and it's good. There's a hosted version at api.runreveal.com/mcp that uses OAuth, and a local version you run with the runreveal CLI using an API token. It exposes a lot: querying and schema tools, but also detections_create, detection_update, detection_delete, sigma_create, agents_create, agents_delete, agents_silence, investigation_create, dashboard_graph_create and notification_send. Read that last list again with the first half of this article in mind. I had already sent one real email to a real colleague by accident, and nearly sent a second through an agent that switched itself on. Handing an AI model a tool called notification_send, in a workspace where the only channel is a live email address, is the same mistake with the model making it instead of me. Same for agents_create, given that RunReveal's own create endpoint ignores enabled: false So what i built is the other way around. The model can read anything it needs to investigate, and the only things it can write are case notes and indicators, which are append-only and land in a case someone has to look at anyway. It cannot create or change a detection, start an agent, touch a pipeline, delete anything, or send a notification. The worst a confused model can do with my server is open a case with a bad title. The design rule: Give the model every read it needs and almost no writes. A read that goes wrong wastes a query. A write that goes wrong can page someone, silence a detection, or delete evidence. What I left out on purpose Beyond the safety choice above, some tools are missing because the endpoints behind them don't work. Detection delete, pipeline add-step and execute-step, and investigation close/update/delete were all broken or unverified server-side when I am creating this article. I'd rather ship eleven tools that work than fourteen where three fail in ways the model can't explain back to me. The list is written into the server's docstring so the next person doesn't add them without testing. Gotchas baked into the wrappers Most of the value of a wrapper is the stuff it quietly gets right so the model doesn't have to. Each of these cost me at least one failed call to find: Errors come back as HTTP 200. RunReveal wraps every response in {success, result}, and a failed call can still be a 200 with success: false. The client checks that flag and raises with the API's own error text, because a bare status line tells you nothing. Optional parameters must be dropped, not passed through. A Python None handed straight to the query string risks going out as the literal text "None", which the API would treat as a real filter value. The client strips None values before every request. Artifact types are lowercase. Adding a note to a case needs type "note", and indicators need "indicators". The console labels them "Note" and "Indicators", and the capitalised versions fail. Detections are fetched by name, not ID (?name=), while investigations and agents are fetched by ID (?id=). Same API, opposite conventions. Map columns like actor and tags need bracket syntax in SQL (actor['email'], not actor.email). That hint lives in the query_logs description so the model gets it right first time. What It Looks Like in Use: Three Real Sessions This is the part that sold me on it. Each of these is a real exchange against the live workspace. I've trimmed the output, but not changed it. Session 1: "Will this detection ever fire?" A test detection for failed Google Workspace admin logins had run every five minutes for over a week with zero hits, and I asked whether that meant nothing had happened. The model pulled the detection with get_detection, saw the filter was eventName ILIKE '%fail%login%', and instead of answering it checked what event names the source actually produces: -- query_logs SELECT DISTINCT eventName FROM logs WHERE sourceType = 'google-workspace-alerts' Admin password reset Suspicious login Activity Rule None of those can ever match that filter, so the rule is structurally dead, not quiet. That took two tool calls inside one conversation. In the console it's a detection edit screen, an Explorer tab and a manual comparison, which is exactly the kind of check that's easy to skip. Session 2: "46 admin password resets in an hour" A burst-of-password-resets detection returns 46 resets in 60 minutes, with an empty actor. That looks like a mass account takeover. The model's first move was to regroup the same rows by the alert's own ID, pulled out of the raw log, instead of by actor. I re-ran it today while writing this, to see whether anything had changed: -- query_logs, whole source SELECT count() AS raw_rows, uniqExact(JSONExtractString(rawLog,'alertId')) AS distinct_alerts, countIf(actor['email'] = '') AS empty_actor_rows FROM logs WHERE sourceType = 'google-workspace-alerts' raw_rows: 90997 distinct_alerts: 4 empty_actor_rows: 90997 -- same source, last three hours, per hour hour 05:00 rows 45 distinct_alerts 1 hour 06:00 rows 46 distinct_alerts 1 hour 07:00 rows 45 distinct_alerts 1 When I wrote article the source had 86,661 rows. Four days later it has 90,997, still only four real Google alerts, and still the same one alert re-ingested about 45 times an hour. The actor field is empty on every single row, because this connector only puts the email inside rawLog. So the "burst" is a polling artifact, and the growth since previous article is the bug still running. Checking that took one follow-up question, not a new investigation. Session 3: working the case, and correlation across cases Once the triage was done, the model opened a case with create_investigation, wrote the reasoning into it with add_investigation_note, and attached the admin email as a structured indicator with add_investigation_indicator. Then it ran ioc_lookup on that email to see whether it had come up before: -- ioc_lookup(ioc_type='email', ioc_value='admin@') Burst of admin password resets (GWS alerts) closed false_positive high Demo: Admin password reset - admin@ closed false_positive high -- ioc_lookup(ioc_type='Email', ioc_value='admin@') null Two things there. First, correlation works: the new case immediately linked back to an older closed case about the same account, also marked false positive, which is exactly the context you want before escalating. Second, the value match is partial and case-insensitive, but the type match is exact and case-sensitive. Ask for "Email" and you get nothing back, with no error. The tool description spells out that the type is an exact match, so the model uses the lowercase form. One detail matters here: notes are not searchable by ioc_lookup, only indicators are. If you write "the attacker IP was 1.2.3.4" in a note, the next case will never find it. That's why the note and indicator tools are separate and the note tool's description says so. Why this is the part I care about: Question → SQL → evidence → case → correlation, in one conversation, without leaving the terminal. And at no point could the model silence a detection, delete anything, or email anyone. That combination of a short loop and a hard safety boundary is what makes it something I'd actually use on a real incident. And RunReveal's Own AI Chat? RunReveal also has native AI Chat inside the console, plus the Agents. Both are bring-your-own-model: under Settings → AI Settings you add Anthropic, OpenAI or Google AI with an API key, Amazon Bedrock with an IAM role RunReveal assumes (role ARN, region, optional external ID), or Vertex AI with a service account. There's no built-in default model. I haven't connected a model provider to this workspace, so I'm not going to describe what the chat is like to use from the docs. Part of that is deliberate. Connecting a provider means the workspace's logs flow to that provider under whoever's key or role it is, and that's a decision for whoever owns the data and the AI account, not something to do casually for a blog post. The MCP route sidesteps that: the model I already use calls the API with a scoped token, and I decide tool by tool what it can reach. --------------------------------------------------Dean-------------------------------------------- Next Part pulls back to the bigger picture — how RunReveal actually stacks up against Splunk and Microsoft Sentinel for a working SOC analyst.

Start With What Is Actually There The pitch for RunReveal's Investigations feature is easy to repeat: an alert fires, a case opens, an AI agent does first-pass triage, a human picks it up. I could write that section from the docs in ten minutes. Instead, everything below comes from pulling the real investigations, artifacts, alerts and audit trail out of a live workspace and reading what's there. The first thing the data shows is that the pitch and the reality don't line up by default. The existing investigations here were all opened by hand, from the console or through the API, not by the platform. That gap is the most useful thing in this article, so I'll start there. Why an Alert Did Not Become a Case An alert and an investigation are separate objects in RunReveal, and nothing in the alerts table points at an investigation. A real alert row has 26 columns (detection ID, run ID, the raw query, the results, the notification names it went to) and none of them is an investigation ID: SELECT * FROM alerts -- one real row, trimmed detectionName : test-alert-frequent-runquery recordsReturned : 1 results : [40] severity : low notificationNames : ["default-email"] -- no investigationID column exists on this table at all The link goes the other way, through a separate lookup. The workspace audit log shows the console calling an investigations_by_alert endpoint whenever someone opens an alert, which is how the UI answers "is there a case for this?" In SQL terms you can't join alerts to investigations. You have to ask the API. As for why no case opened: every detection in this workspace has an empty settings object (settings: {}), apart from a Sigma rule, whose settings hold only its YAML. None of them has AI triage or auto-investigation turned on. Automatic case creation is something you opt into per detection. It isn't a default that every alert goes through. So a detection wired to email, like the one that fired here, sends its email and stops. If you're evaluating RunReveal and expecting cases to show up on their own, check that setting on each detection first. Case Study: Running a Scenario End to End Reading existing cases only shows so much, so I ran a full scenario live: write a detection, run it on real data, open a case from the result, triage it, add indicators, and check whether cross-case correlation actually finds anything. I ran the detection SQL directly in Explorer rather than saving it as a scheduled detection, detections created through the API. Step 1: the detection fires, and it looks bad The scenario is a classic one: alert on a burst of admin password resets, which is what a mass account takeover or a rogue admin looks like in Google Workspace. The obvious first draft counts resets per admin over the last hour: SELECT actor['email'] AS admin, count() AS resets FROM logs WHERE sourceType = 'google-workspace-alerts' AND eventName = 'Admin password reset' AND receivedAt >= now() - INTERVAL 1 HOUR GROUP BY admin HAVING resets >= 5 → admin: "" resets: 46 46 password resets in 60 minutes by an admin the data can't even name. On a real team, that result at 3am gets somebody paged. So I opened a high-severity investigation for it, the same way an auto-triage detection would. Step 2: triage finds the real story The empty admin was the first clue. On this source, the normalized actor map and srcIP column are empty on every single row. The admin's email only exists inside rawLog, at data.actorEmail. Google Alert Center also gives each alert its own alertId, so I grouped on that instead: SELECT JSONExtractString(rawLog, 'data', 'actorEmail') AS admin, JSONExtractString(rawLog, 'alertId') AS google_alert_id, JSONExtractString(rawLog, 'metadata', 'severity') AS google_severity, min(eventTime) AS happened_at, count() AS copies_ingested_last_hour FROM logs WHERE sourceType = 'google-workspace-alerts' AND eventName = 'Admin password reset' AND receivedAt >= now() - INTERVAL 1 HOUR GROUP BY admin, google_alert_id, google_severity → admin: google_alert_id: f4a61ff7-... google_severity: HIGH happened_at: 2026-09-04 03:56:55 copies_ingested_last_hour: 46 There were never 46 resets. There was one, twenty days earlier, and RunReveal ingested the same Google alert 46 times in the last hour. Zooming out to the whole source: SELECT eventName, count() AS rows, uniqExact(JSONExtractString(rawLog, 'alertId')) AS distinct_alerts FROM logs WHERE sourceType = 'google-workspace-alerts' GROUP BY eventName → Activity Rule 44,811 rows 1 alert → Suspicious login 19,851 rows 1 alert → Admin password reset 21,999 rows 2 alerts total 86,661 rows 4 real Google alerts The pattern is consistent. The alerts poller keeps fetching the newest alert again, roughly every 80 seconds (about 46 times an hour), until Google produces a newer one. Then the newer one gets the same treatment. The Activity Rule alert was re-ingested for six weeks, the Suspicious login alert for three, and the password reset alert is still being re-ingested as I write this. Step 3: indicators and cross-case correlation, working With the conclusion written up as a note, I added the admin's email as an email indicator on the new case and ran IOC lookup on it. It came back with the new case and also an earlier, closed case, classified false positive, involving the same admin and the same kind of alert: ioc_lookup(type="email", value="") → ARTICLE TEST - Burst of admin password resets open high (unclassified) → closed high false_positive This is what the feature is for. An analyst (or the triage agent) opening the new case immediately sees that this same entity was already investigated and closed as a false positive. Here it's the right hint. It only works because both cases used the same lowercase email type, as covered in the IOC section below. What this scenario actually proved The naive detection would page someone on a 20-day-old event, about 46 times an hour, for as long as nothing newer comes in. Any rule on this source keyed on actor['email'] or srcIP can never match anything, because those fields are always empty. The real values are only in rawLog. The enabled Sigma rule for Activity Rule events on this workspace has zero matches in the detections table, even though 44,811 rows with that exact eventName exist. Its logsource says product: google_workspace, and a likely explanation is that this doesn't map to the google-workspace-alerts source, which would make it another rule that stays silent for structural reasons, like the one in Previous article It explains an earlier puzzlePrevious article example of one detection run returning 11 rows was 11 copies of the same Activity Rule alert. The investigation workflow itself did its job. Case, note, indicator and correlation all behaved as designed, and the triage is what caught the data problem. The practical takeaway: For any polled alert source, group or deduplicate on the upstream alert's own ID before you count anything, and check that the normalized fields you filter on are actually populated before you trust a rule built on them. SELECT countIf(actor['email'] != '') FROM logs WHERE sourceType = '' takes two seconds and would have caught this before it was a detection. Anatomy of a Real Case Here is a closed investigation as get_investigation returns it, with identifying values redacted. It's worth seeing the whole shape once, because a few fields don't behave the way their names suggest: { "title": "", "status": "closed", "severity": "high", "classification": "false_positive", "tags": ["google-workspace", "admin-action"], "created_by": "token:Testing", "assignee": "Testing", "created_at": "2026-09-11T10:07:27.740424Z", "detected_at": null, "responded_at": null, "triaged_at": "2026-09-11T11:15:32.560355Z", "resolved_at": "2026-09-11T11:15:32.560355Z" }, "metrics": { "investigateSeconds": 0, "resolveSeconds": 4084.819931 } Two things jump out. First, created_by is token:Testing and the assignee is just Testing. When a case is created with an API token, the token's name stands in for a person. Name your tokens after the automation or the human who owns them, because that name is what shows up as the case owner. Second, look at triaged_at and resolved_at. They are the same value down to the microsecond. That matters for MTTR, which gets its own section below. The timeline: four artifact types, only two you can write Every investigation carries a timeline of artifacts. Across the real cases in this workspace I found four types: investigation_update: written by the system when status or classification changes. The meta field records field, oldValue and newValue. note: free text you add. indicators: structured entities you add, which is what IOC lookup searches. resolution: written by the system when the case is closed, carrying the closing comment. You can only create note and indicators yourself. The other two are the platform's own record of changes. Also, don't assume the API returns the timeline in order. On the scenario case, get_investigation listed the indicator before a note that was created 150 milliseconds earlier, so sort on created_at yourself if order matters. And an open case returns metrics: {}, not zeros. The metrics object only fills in once a case closes. One detail from the real indicators artifact: its meta holds an array, not a single value, so one artifact can carry several entities at once: { "type": "indicators", "comment": "actorEmail from the source alert", "meta": { "indicators": [ { "type": "email", "value": "" } ] } } Notes vs. Indicators, and the Casing Trap, Straight From the Audit Log The split between notes and indicators is well designed. Notes are prose for humans and IOC lookup can't see them. Indicators are structured and are the only thing cross-case correlation searches. A note saying "the attacker used admin@example.com" is invisible to every other investigation. The same email in an indicators artifact is not. The API is strict about the type string, and the docs and UI write the types as "Note" and "Indicators" with capitals. What makes this concrete is that RunReveal's own audit log keeps every failed attempt, with the error message attached. Here is every rejected artifact type from one session of working out what the API actually wants, pulled straight from the log: SELECT JSONExtractString(rawLog, 'supplemental_data', 'error_message') FROM logs WHERE sourceType = 'runreveal-audit' AND eventName = 'investigation_add_artifact' → invalid artifact type: Note → invalid artifact type: NOTE → invalid artifact type: Note_Artifact → invalid artifact type: ARTIFACT_TYPE_NOTE → invalid artifact type: artifact_note → invalid artifact type: Indicator → invalid artifact type: indicator → invalid artifact type: IOC → invalid artifact type: ioc Nine rejections. The only two values that work are lowercase note and lowercase plural indicators. There's a side lesson here too: that audit event is logged the same way whether the call succeeded or failed. Of 14 investigation_add_artifact events, 9 were errors. If you count that event to mean "artifacts added," you'll be off by more than half. Filter on the error flag in supplemental_data first. The practical takeaway: Anything that writes to investigations automatically (a SOAR playbook, an MCP tool, a triage script) should hardcode type: "note" and type: "indicators", and should put every entity someone might pivot on into an indicators artifact, not just mention it in a note. IOC Lookup: Tested, Not Assumed IOC lookup is what makes cross-case correlation work, and the auto-triage agent calls it for every entity it sees. The docs describe it as an exact type match plus a partial, case-insensitive value match. I ran four lookups against a real email indicator in the workspace: type value (search) classification result ────────── ────────────────────────── ─────────────── ────────── email (any) 1 match Email (any) null user (any) null email true_positive null Each row shows something specific: Row 1: the value match really is partial and case-insensitive. Searching just the domain in capitals still found the full lowercase address. Row 2: the type match is not just exact, it's case-sensitive. Email with a capital E found nothing. The docs don't say this, and it's the same trap as the artifact types. Row 3: types are separate silos. The same address filed under user instead of email can't be found by an email lookup, and the reverse is true too. If one analyst files an address as email and another files it as user, correlation between their cases quietly breaks. Row 4: the classification filter works. The only match was a false-positive case, so a true_positive filter correctly came back empty. Worth deciding as a team before you have any cases: Pick one lowercase type per kind of entity (email for addresses, ip, domain, hash, user for account names that are not emails) and write it down. IOC lookup will not fix inconsistency for you. It will just return nothing. MTTR: What the Four Stages Show on Real Cases RunReveal breaks mean time to resolution into four stages: Detect (activity to alert), Respond (case created to first human engagement), Investigate (triage start to close), Resolve (creation to close). Here is what two real closed cases report: case detected_at responded_at investigateSeconds resolveSeconds case A null null 0 63303.08 case B null null 0 4084.82 I checked case B's resolve Seconds by hand: 10:07:27.740 to 11:15:32.560 is 4,084.82 seconds, so that number is exactly closed minus created, as documented. The other three stages are less useful on these cases: detected_at and responded_at are null on both. Nothing about a hand-opened case gives the platform a detection time, so the Detect and Respond stages have nothing to measure. investigateSeconds is 0 on both, because triaged_at gets stamped at the same moment as resolved_at when you close a case. Close a case without first moving it through a triage status, and the Investigate stage records zero seconds no matter how long you actually spent on it. In practice, for any case opened by hand and closed directly, Resolve is the only one of the four stages carrying real information. The other three only fill in when cases come from alerts and move through the agent-triage and investigating statuses on the way to closed. If your MTTR dashboard shows a suspiciously perfect 0-second Investigate time, this is why. Two Gaps in the Timeline Classification set at close leaves no trail In one case, the classification was changed on its own before closing, and the timeline has an investigation_update artifact for it (field: classification, oldValue: "", newValue: false_positive). In another, the classification was set as part of the close action, and the only artifact is the resolution, whose meta records status changing from open to closed and nothing else. The case is classified false positive, but the timeline never says when or by whom. If you rely on the timeline as an audit trail for classifications, set the classification as its own step before closing. Closed cases still take notes, and classification does not follow them One case was closed as false positive. The next day a note was added to it, while it was still closed, that reads in full: "This is a true positive but not a threat." The note was accepted, the case stayed closed, and the classification still says false_positive. Nothing flags the contradiction. That note is a real analyst hitting a real gap. The classification field has three values (true_positive, false_positive, inconclusive) and no benign-true-positive option. The detection caught exactly what it was meant to catch, a real admin password reset, which just turned out to be authorized. Plenty of SOCs track that as its own category because it's what tells you whether to tune a rule or leave it alone. Here, you have to pick one of the three and put the nuance in a note that IOC lookup and the classification filter will never see. The Audit Log Is the Real Usage Record, With a Catch Every investigation action lands in the workspace's own audit source (sourceType runreveal-audit), which you can query like any other log. That's handy if you want to know how a team really uses cases rather than how they say they do: SELECT eventName, count() FROM logs WHERE sourceType = 'runreveal-audit' AND eventName ILIKE '%investigation%' GROUP BY eventName ORDER BY count() DESC → investigation_metrics 50 → investigation_list 33 → investigations_alert_status 21 → investigation_facets 18 → investigation_get 14 → investigation_add_artifact 14 → investigation_create 3 → investigation_close 3 → investigation_ioc_lookup 1 It already tells a story: people look at the metrics widget far more often than they open cases. But investigation_create didn't match the number of cases in the investigation list, and neither did investigation_close. The raw rows explain it: some audit events are ingested twice, with identical payloads including the nanosecond timestamp, about 30 seconds apart. Counting distinct events instead of rows: SELECT count() AS raw_rows, uniqExact(JSONExtractString(rawLog, 'timestamp'), eventName) AS distinct_events FROM logs WHERE sourceType = 'runreveal-audit' → raw_rows 9683 | distinct_events 8035 (17% duplicates) Across the whole workspace's audit history, 17% of rows are duplicates of another row. That's not specific to investigations, but it's where I caught it, because the investigation counts were small enough to check against reality. Anything that counts audit events, whether a dashboard, a usage report or a detection that fires on N actions in a window, needs to deduplicate on the payload timestamp first or it will overcount. Two more limits worth knowing before you build on this source: The audit payload has no investigation ID. It records who did what and when, but not which case. You can't rebuild one investigation's history from the audit log alone. For API-token actions, actor_email is the token's name (here, Testing), not an email address. The user agent (curl vs. a browser) is the quickest way to tell automation from a person clicking. The practical takeaway: Wrap any count over runreveal-audit in uniqExact(JSONExtractString(rawLog, 'timestamp'), eventName) instead of count(), and exclude rows where supplemental_data.error is "true" if you want successful actions only. Auto-Triage: How It Is Documented to Work An alert from a triage-enabled detection auto-creates an investigation, tagged auto-created and alert, and the agent moves it to agent-triage. The agent runs IOC lookup on every entity involved. That's the same endpoint tested above, with the same exact, case-sensitive type matching. It writes an analysis note. For a true positive it adds indicators artifacts and hands off by moving the case to investigating. For a false positive it closes the case itself with a classification. A detection's own notes field goes straight into the agent's prompt. An investigation's notes don't. The agent only sees them if it fetches the case during its run. Put playbook guidance on the detection, not in the case. ------------------------------------------------------Dean------------------------------------------------ Next Part covers the other half of RunReveal's AI story: scheduled agents that go looking for problems on their own, instead of reacting to alerts that already fired.

Detection-as-Code, For Real A detection in RunReveal is a scheduled query with metadata attached — severity, risk score, MITRE ATT&CK mapping, which sources it applies to. The query is the detection: version-controllable, diffable, reviewable in a pull request, because it's SQL or Sigma text, not a form full of dropdowns serialized into some proprietary format. Everything below this point is Live tested, real detection objects, real query results, real run history. Nothing here is a hypothetical. Two Detection Types SQL detections Full ClickHouse SQL — joins, custom views, historical lookback, whatever the query language supports. Typically scheduled every five minutes via a cron expression, though one real detection in the test workspace uses a shorthand form instead: schedule: "@1m" runs identically to schedule: "* * * * *". Neither the console nor the API docs mention the @-shorthand exists; it was only visible by reading a real detection object back through the API. The windowing gotcha: The documented, correct way to filter a time window in a scheduled SQL detection is receivedAt >= {from:DateTime} AND receivedAt < {to:DateTime} — RunReveal auto-injects from/to on every scheduled run. now() - INTERVAL X MINUTE runs without erroring, but it is not the supported pattern, and one real detection in this workspace still uses it (see below) rather than the documented one. Sigma detections Real-time streaming pattern matching using the industry-standard Sigma format. Simpler than SQL, portable across tools that also support Sigma, but no joins and no custom views. Pulling a real Sigma detection back through the API shows something the docs don't spell out: the top-level query field is an empty string. The actual rule lives one level down, as raw YAML, inside settings.rule: { "type": "sigma", "query": "", "settings": { "rule": "title: Google Alert Center Activity Rule Test\n id: 11111111-1111-4111-8111-111111111111\n status: test\n logsource:\n product: google_workspace\n detection:\n selection:\n eventName: Activity Rule\n condition: selection\n level: medium" } } If you're building anything that reads detections back programmatically — a CI check, a backup script, an MCP tool — checking query for Sigma rules will silently find nothing. You have to branch on type and parse settings.rule as YAML instead. The Escalation Ladder, With Real Numbers Behind It A match doesn't automatically page anyone. It climbs a three-step ladder: Detections — every single match gets recorded here, quietly, with no notification. Signals — matches that RunReveal considers worth surfacing, still with no notification channel attached. Alerts — the same match, once a notification channel is actually attached to the detection. That's the documented model. Querying the workspace's own detections and alerts tables directly — both queryable exactly like any other table in RunReveal, which is a genuinely nice property — shows what it looks like in practice, not in theory. Across every detection that has ever run in this workspace, exactly one alert has ever fired, ever. One. Everything else that ran — and plenty has, including a detection that’s been executing every five minutes for over a week straight — has stayed at the "detections" tier: recorded, queryable, and completely silent. The escalation ladder isn’t a marketing description of intended behavior; it’s really how quiet the system stays until something is deliberately wired to notify. A structural quirk the docs never mention: detections is per-match, not per-run One real detection returned 11 matching rows on a single scheduled run. Querying the detections table for that detection’s history back: SELECT id, scheduledRunID, recordsReturned FROM detections WHERE detectionName = '' came back as 11 separate rows, each with a distinct id but the identical scheduledRunID and the identical recordsReturned: 11 value copied onto every row. The table isn't one row per scheduled execution — it's one row per matched record, with scheduledRunID as the join key back to the run that produced them. recordsReturned tells you the run's total match count, duplicated across every row it produced, not that row's own count (which is always 1). If you're aggregating this table for a dashboard or a report, count(DISTINCT scheduledRunID) is "how many times this rule fired" — count(*) is "how many individual matches came back," and confusing the two will silently 11x your numbers for any rule that returns multiple rows per run. Case Study: A Detection That Ran Clean for a Week and Found Nothing — For the Wrong Reason This is the finding worth the most attention in this article, because the failure mode it demonstrates isn't RunReveal-specific — it's a trap any SQL-based detection engine leaves open. A real test detection in this workspace flags repeated failed admin logins: SELECT actor['email'] AS user, srcIP, count() AS failures FROM logs WHERE sourceType = 'google-workspace-alerts' AND eventName ILIKE '%fail%login%' AND receivedAt > now() - INTERVAL 5 MINUTE GROUP BY user, srcIP HAVING failures >= 5 It's enabled, on a five-minute schedule, and has been executing without error for over a week. It has zero rows in the detections table for its entire run history — no matches, ever. Read on its own, that looks like exactly what you want from a tuned detection: no false positives, quietly waiting for a real brute-force attempt. Querying the real event vocabulary this source actually produces tells a different story: SELECT DISTINCT eventName FROM logs WHERE sourceType = 'google-workspace-alerts' → "Admin password reset" → "Suspicious login" → "Activity Rule" None of those three values contain the substring the detection is filtering for. eventName ILIKE '%fail%login%' cannot match anything this source will ever emit — the real event for a failed login here is literally named "Suspicious login," with no "fail" in it anywhere. This detection is not quiet because the environment is clean. It's quiet because its WHERE clause and the source's real data never had any chance of intersecting, and it will stay silent forever regardless of what actually happens, because RunReveal has no mechanism to warn you a filter doesn't match anything the source produces — a rule that's structurally guaranteed to never fire looks, from the outside, identical to a healthy, well-tuned one. The practical takeaway: Before trusting silence from any new detection as "nothing bad happened," run SELECT DISTINCT eventName FROM logs WHERE sourceType = '' against the real data first and check your filter actually intersects with it. Writing the WHERE clause from what you assume the event names should be, instead of what they actually are, is an easy way to ship a detection that will never once fire. API Gotchas Worth Knowing Before You Automate Anything I built an MCP server against this API, which meant hitting every endpoint by hand before trusting it enough to wrap in a tool. A few things the documentation doesn't mention: name is required on creation but not marked as required in the documented schema — omit it and you get a generic validation error that doesn’t obviously point at the missing field. The enabled field on /detections/enabled/update wants a string, literally "false", not a JSON boolean. Send a real boolean and it 500s. /detections/get takes the detection’s name as a query parameter, not an ID in a POST body. Deleting a detection through the API doesn’t work. It returns success and hands back the full detection list, with your target detection still sitting in it, completely unchanged. That last one isn’t a one-time observation — it’s still verifiably true. Four test detections created during earlier hands-on testing for this series are still sitting in the workspace today, weeks later, disabled rather than deleted because that was the only way to actually neutralize them. A fun contradiction, if you read the Agents piece in this series: RunReveal's Agents API wants enabled as a real boolean and rejects a string outright — the exact opposite convention from detections. And Agents' delete endpoint actually works, unlike detections. Don't assume any two endpoints on this API follow the same rules, even when they look structurally identical. A Real Timing Trap Checking whether a scheduled detection found anything immediately after its run time updates is a trap. I hit zero results checking within roughly 15 to 90 seconds of a run, even when I could prove with a manual query that the underlying data clearly matched the detection's WHERE clause. Minutes later, the same broader query found the matches sitting at their original timestamps. There's real read-after-write lag between a detection run completing and its results being reliably queryable. Don't trust an immediate zero-result check as proof a rule doesn't work. ------------------------------------------------------Dean----------------------------------- Next Part follows the thread from here — what happens once an alert actually fires and an investigation gets opened.

Three Ways Data Gets In Before any of the architecture matters, data has to actually arrive. RunReveal supports three ingestion methods, and which one a given source uses isn't a style choice — it changes what happens when something goes wrong. Webhook — push-based. The source sends events to a URL RunReveal gives you. Fast and simple, but if the network hiccups mid-send, that event is gone. No retry, no replay. Polling — RunReveal reaches out to the source on an interval, roughly every 60 seconds, using encrypted stored credentials. It checkpoints where it left off, so a restart or a blip doesn't lose data — it just resumes. Object Storage — S3, Azure Blob, GCS, R2, MinIO. A bucket gets a notification when a new object lands, and RunReveal pulls it from there. Good fit for anything that already dumps logs to a bucket as its native export format. If I were setting up a new source and had a choice between webhook and polling for the same data, I'd take polling every time — the checkpoint-and-resume behavior alone is worth the extra ~60 seconds of latency for anything that isn't genuinely time-critical. Forwarders, for Everything Else For file-tail scenarios, Windows Event Log, and syslog, RunReveal ships Fluent Bit support and its own lightweight forwarder called reveald. What I didn't expect: reveald has its own concept of destinations, separate from the platform-level Destinations — it can forward to MQTT, to a literal network printer, to S3 directly, or to RunReveal itself. The printer option is a strange one to see in 2026, but it tells you reveald was built as a genuinely generic forwarder, not something narrowly scoped to just talk to RunReveal's own backend. The Connector Catalog RunReveal advertises around 120 source connectors. I crawled the documentation for all of them rather than take that number on faith, and it holds up — roughly 114 individual connector doc pages exist, spread across cloud providers (AWS, Azure, GCP), identity platforms, EDR and security tools, SaaS applications, and networking sources, plus generic custom/webhook options for anything without official support. A caveat worth being upfront about, with real numbers instead of a vague gesture at them: 67 of those 114 connectors — 59% — came back with auth method genuinely unclear from the page text once I structured the crawl into a catalog. Broken down by ingestion type, it's 43 object storage, 34 polling, 18 webhook (push), 4 queue/pubsub, 1 forwarder/syslog, and 14 where even the ingestion method itself wasn't stated plainly. By category: 50 fall into a catch-all "Other," 17 SaaS, 12 EDR/Security, 10 each for AWS and Networking, 6 Identity, and single digits for Azure, GCP, and generic Custom sources. Treat any public connector catalog, including this one, as a starting point for discovery — always check the specific connector's live doc page before wiring it up for real. What a Real Source Looks Like The workspace I tested against runs Google Workspace ingestion — both the audit log stream and the separate security alerts stream — using polling. Watching real data actually accumulate rather than just reading about the schema was where the earlier ClickHouse architecture stopped being abstract: querying tens of thousands of real events back with sub-second response times is a genuinely different experience than tailing a Splunk search that's still "scanning buckets" thirty seconds in. Source Lifecycle Disabling a source stops new ingestion immediately but keeps its configuration and all previously ingested data intact — you're not starting from zero if you re-enable it later. Actually deleting a source's historical data is a support-ticket process, not a self-service button. Hands-On: Measuring the Polling Lag Instead of Quoting It "Roughly every 60 seconds" is the documentation's own framing for polling latency, and it's close enough for planning purposes, but I wanted an actual measurement rather than a restated number. Every Google Workspace audit event carries its own original timestamp inside the raw payload — when Google logged it — separately from receivedAt, RunReveal's own record of when it actually landed in ClickHouse. That gap is the real, end-to-end polling lag, and it's queryable directly. Eight consecutive real events, timestamp of the underlying Google event versus RunReveal's receivedAt: event 13:31:53.8 -> received 13:33:09 lag 75.2s event 13:31:43.0 -> received 13:33:06 lag 83.0s event 13:32:48.6 -> received 13:33:06 lag 17.4s event 13:30:36.2 -> received 13:31:56 lag 79.8s event 13:30:19.3 -> received 13:31:54 lag 94.7s event 13:30:31.1 -> received 13:31:54 lag 82.9s event 13:27:45.7 -> received 13:30:21 lag 155.3s event 13:29:14.9 -> received 13:30:18 lag 63.0s Min 17 seconds, max 155 seconds, median 83 seconds, average 81 seconds — real variance, not a flat 60-second heartbeat. "Roughly every 60 seconds" is the right mental model for what triggers a poll cycle, but the number that actually matters for anyone setting alert-latency expectations is closer to "budget for up to two and a half minutes, typically around 80 seconds" than "60 seconds." Worth knowing before you write a detection's schedule assuming data is never more than a minute stale. Next Part picks up right where the data lands — the detection engine that actually reads all of this on a schedule. --------------------------------------------------------Dean-----------------------------------

Why ClickHouse The first thing I wanted to understand about RunReveal wasn't the UI — it was what's actually underneath it. Most SIEMs I've worked with are built on some flavor of a log indexer: Splunk has its buckets and indexers, Sentinel sits on top of Azure Log Analytics workspaces. Those architectures were designed in an era where the assumption was you'd mostly search for specific terms across a lot of unstructured text. RunReveal is built directly on ClickHouse, a columnar OLAP database. The practical difference: Columnar storage is built for aggregation and analytical queries — count these, group by that, join across time windows — which happens to be exactly the shape of most detection logic and threat hunting. It's the same reason a lot of observability companies quietly migrated off Elasticsearch onto ClickHouse over the last few years. RunReveal just built the SIEM on that foundation from day one instead of bolting it on later. The logs Table — One Table, Everything Here's the part that actually surprised me. Almost everything in RunReveal reads from a single underlying table, called simply logs. I ran a DESCRIBE TABLE against it directly and counted 38 columns. The ones that matter most day-to-day: workspaceID, sourceID, sourceType — which source produced the event receivedAt — the field you filter on for time ranges; eventTime is separate and represents when the activity actually happened eventName, eventID — what happened srcIP / dstIP, plus full GeoIP and ASN fields for both — no separate enrichment lookup needed for basic geo/ASN context actor (a Map type) — who did it, however the source represents that tags (Map) and resources (Array) — flexible structured fields for anything source-specific enrichments (Array of Tuple) — results of any enrichment steps a pipeline applied rawLog — the original, unparsed event, always kept On top of that single table, RunReveal maintains source-specific views — aws_cloudtrail_logs, okta_logs, github_logs, aws_vpc_flow_logs, and more — which present the same underlying rows with normalized, source-appropriate column names. Same DESCRIBE TABLE trick works on those too, if you want to see exactly what a given connector normalizes for you. Topics, Pipelines, and the Step That Actually Writes Data This is the part of the architecture that took me the longest to actually understand, because the docs describe it more abstractly than what I found once I inspected a real, live pipeline through the API. A Pipeline is an ordered list of steps that runs against a Topic (a stream of incoming events). I pulled a real pipeline from a live workspace and it had five steps, in order: transform — reshape the event enrich — attach additional context filter — drop events matching a condition detect — a lightweight in-pipeline detection check destination — the step that decides where the event actually ends up The insight worth sitting with: a pipeline's Destination step is what actually writes to storage. There's no implicit side-channel where data magically ends up in ClickHouse just because it entered the system. If a pipeline doesn't have a Destination step, older pipelines fall back to RunReveal's default backend, but the docs explicitly warn that fallback isn't something you should rely on — add explicit Destination steps so routing is predictable.' Destinations, Properly Explained I want to spend real space on this because it's a genuinely well-designed piece of the platform, and the docs page for it answers a lot of questions I had after just looking at one example pipeline. Two destination types ClickHouse destinations are for real-time querying and analysis — this is what powers every RunReveal feature. You can configure more than one, with one marked as default. There's also SPIFFE/mTLS support for certificate-based auth instead of username and password, if that matters to your environment. Object Storage destinations — S3, Cloudflare R2, Google Cloud Storage, Azure Blob — are for archival, backup, and long-term retention. They're not queryable through RunReveal's own features; they're cold storage for compliance and disaster recovery. How routing actually works A few rules that aren't obvious until you read the FAQ section of the docs closely: Creating a destination in the UI does nothing on its own — it just makes that destination available. Events only flow to it once a pipeline has a Destination step selecting it. A single pipeline can have multiple Destination steps, which means you can fan the same event out to ClickHouse and S3 at once. Destination steps don't stop pipeline execution — later steps still run. If you have two Destination steps with no precondition separating them, you'll double-write the same event unless that's actually what you want. One destination failing doesn't block delivery to the others — errors are tracked independently per destination. Worth remembering: RunReveal Backend only targets ClickHouse storage. If you want data in S3 too, you need a separate explicit Destination step for that object storage destination — nothing routes there automatically. Destination health checks RunReveal can monitor write errors per destination over a rolling interval, and alert your notification channels if the error count crosses a threshold for enough consecutive checks in a row — useful for catching a misconfigured or unreachable destination before it silently drops data for hours. These are configured from the destination's own Health Checks page, not from the create/edit form, which tripped me up the first time I went looking for the setting. Retention, in Practice Default retention on ingested data is 550 days. Disabling a source stops new ingestion but keeps existing config and data intact. Actual deletion of historical data is support-ticket only — this is append-only storage, which lines up with why Object Storage destinations exist as a separate concept from ClickHouse destinations: one is for querying recent-to-mid-term data fast, the other is for keeping everything indefinitely at a lower cost. Next up: Part 3 covers how you actually query all of this yourself — the Explorer, saved queries, and a set of platform tables that let you query RunReveal's own operational data with the same SQL interface. https://www.cyberengage.org/post/runreveal-explorer-search-querying-your-data-directly ------------------------------------------------Dean------------------------------------------------

The Interface Sitting on Top of Everything in Part 2 Part 2 covered the logs table and the ClickHouse foundation underneath RunReveal. This part covers how you actually touch that table day to day — the Explorer, RunReveal's log search and query interface. It's easy to undersell this as "just a search bar," and I almost did exactly that in the first article of this series before realizing how much is actually packed into it. RunReveal's own description calls it a compact, single-page interface with sidebar navigation, saved queries with full version history, and AI-powered query assistance built in — not a separate add-on chat window bolted to the side. Layout The explorer is organized into a few consistent pieces: A collapsible sidebar listing every queryable table, grouped by category: the main logs table, source-specific views, and any custom views you've created. Expand a table to see its schema and columns without leaving the page. A top bar and filter section for building filters without hand-writing WHERE clauses — filters are type-aware, so a timestamp field gets different filter operators than a string field. A time-series histogram showing event volume across your selected window, which is the fastest way to spot a spike or a gap before you've written a single line of SQL. A saved queries sidebar, and a query editor you can drop into directly when a filter-builder UI isn't expressive enough for what you're trying to ask. Saved Queries Actually Have Version History Every time you save a query, RunReveal creates a new version rather than overwriting the last one. That's a small detail that matters more than it sounds like — it means a saved detection-adjacent query you're iterating on keeps a real history, so if a change makes it worse, going back is a click rather than a rewrite from memory. Saved queries also carry their full parameters and settings with them, and every exploration state — filters, table, time range — can be shared as a URL, which makes handing a specific query off to a teammate genuinely simple. The AI Assistant Is Actually Useful Here You can describe what you want in plain language and the assistant generates a SQL query from that description, which you then review before running — it doesn't execute blind on your behalf. The more interesting piece is what happens when a query fails: the assistant reads the actual error, proposes a fix, and lets you review the corrected query before applying it. For anyone who's spent time debugging a ClickHouse syntax error at 2am during an actual incident, that's not a gimmick feature. Two Timestamps, and Why It Actually Matters RunReveal stores two separate timestamps on every normalized event, and which one you filter on has a real, measurable performance impact — this is documented explicitly, not something I had to discover by trial and error for once. receivedAt — when RunReveal ingested the event. Indexed as part of the primary key, consistent across every source regardless of how that source reports time, and correctly accounts for delayed log delivery. Use this for time filtering, detections, and any scheduled query. eventTime — when the activity actually happened, according to the original source. Not indexed, so filtering on it is slower, and it can be missing or inconsistent depending on the source. Use it for displaying the original event time, not for filtering. ----------------------------------------------------------------------------------------------------------- The docs' own example, and it's a good one: filtering on receivedAt is marked as the correct pattern; filtering on eventTime alone is explicitly flagged as something to avoid for anything performance-sensitive. If a query feels slower than it should, this is the first thing worth checking. ----------------------------------------------------------------------------------------------------------- Query Performance, in Practice A short list of habits the documentation calls out directly, all of which match what I'd already learned the hard way testing detections in Part 6: Use source-specific views (aws_cloudtrail_logs, okta_logs, github_logs, and others) instead of querying the raw logs table and filtering sourceType yourself. Filter on primary-key and indexed fields first — sourceType, then receivedAt, then sourceID — before adding other conditions, matching how the table itself is physically organized. Start with a small time range to validate a query, then widen it. A one-hour window while testing is dramatically faster to iterate on than starting with thirty days. Select only the columns you actually need in an aggregation query rather than pulling everything. Platform Tables — Querying RunReveal About Itself This is the part that genuinely surprised me. Explorer doesn't just query your ingested logs — there's a whole second category of platform tables that let you query RunReveal's own operational data with the same SQL interface: detections — the base table for every detection run, one row per finding, regardless of whether it ever escalated to a signal or alert. Useful for answering "has this ever matched anything" without digging through the UI. scheduled_query_runs — execution metadata for every scheduled detection run: runtime, the exact SQL that executed, parameters, and error messages if a run failed. runreveal_source_volumes — per-minute event counts and approximate ingested bytes, bucketed by source. This is the raw data behind the kind of volume-anomaly agent covered in Part 8. Ingestion and pipeline error tables — failed schema validation events, complete with the original raw log that failed, plus per-pipeline-step dropped-byte metrics. Health check subscription tables — the underlying data behind the destination health checks covered in Part 2. A workspace audit trail table — who did what, when, from which IP. Threat intelligence indicator feeds — a built-in table of known-bad IPs you can join against your own traffic directly in SQL. Why this matters beyond curiosity: every dashboard, health check, and agent report in this platform is ultimately just a query against tables you can also query yourself. Nothing here is a black box — if you don't trust a number a dashboard is showing you, you can go verify it directly against the same table it reads from. -------------------------------------------------------Dean----------------------------------------------- Part 4 goes the other direction — not how you query data, but what happens to it in flight before it's ever stored: transforms, filtering, dropping, masking, sampling, and enrichment.

The Problem Nobody Loves Admitting Every SOC analyst I know has a version of the same complaint. Splunk gets expensive the moment your log volume grows past whatever number your finance team quietly panicked about last renewal. Sentinel is great if your entire estate lives in Azure and painful the moment it doesn't. And half the SIEMs on the market still feel like they were built for a world where a 'query' meant clicking through six dropdown menus instead of just writing SQL. RunReveal showed up on my radar because it skips a lot of that. It's a security data platform built directly on ClickHouse, which is the same database engine a lot of observability companies quietly switched to once they got tired of Elasticsearch falling over at scale. I spent the last few weeks actually building against it — not reading a datasheet, but standing up sources, writing detections, breaking things, and building a small MCP server so I could query it from inside AI agent. This series is what I found. A note on how this series was written: Every technical detail in this series — endpoint behavior, schema fields, quirks, things that are flat-out broken right now — comes from hands-on testing against a real RunReveal workspace, not from marketing copy. Where something is broken, I say so. So What Is It, Actually RunReveal calls itself a modern security data platform, and for once the adjective is doing real work instead of just filling space. The pitch is: Detection-as-code instead of point-and-click rule builders, AI-powered investigation triage instead of an analyst opening forty browser tabs, and sub-second query performance because the whole thing sits on a columnar database instead of a traditional log indexer. Strip away the marketing language and here's the actual shape of the thing: Sources pull data in — cloud providers, identity platforms, EDR tools, SaaS apps, roughly 120 connectors and counting. Pipelines and Topics move that data around and let you transform, enrich, or filter it before it lands anywhere. Everything ends up in one big ClickHouse table, which detections (written in SQL or Sigma) query on a schedule. A match escalates through detections → signals → alerts, getting louder each step, until it hits a notification channel. Alerts spin up Investigations — timeline-based case objects with their own AI triage agent. That pipeline — source to signal to investigation — is the spine of this whole series. Every later article is really just zooming into one link of that chain. Who This Is Actually For I don't think RunReveal is trying to be Splunk's replacement for a 500-analyst enterprise SOC that already has a decade of SPL knowledge baked into its team. Where it makes sense, at least from what I've seen, is smaller or leaner security teams who are comfortable with SQL, don't want to pay an ingest tax on every gigabyte, and want their detections to live in version control instead of a vendor's proprietary rule editor. It also clearly leans into being AI-native rather than having AI bolted on as a chatbot sidebar. The investigation triage agent, the native AI chat, and the fact that it exposes an MCP server so tools like Claude can query your workspace directly — that's a different design philosophy than most of the incumbents, who added "AI" to their SIEM after the fact. The Eight Pieces That Make Up the Platform Here's the map I wish I'd had on day one: Sources — where data comes from. Pipelines & Topics — how data moves and gets shaped in transit. Detections — SQL or Sigma rules that scan the data on a schedule. Signals & Alerts — the escalation ladder from "something matched" to "someone got notified." Investigations — the case file, with an AI agent doing first-pass triage. Search / Explorer — ad hoc querying against the same ClickHouse backend. AI Chat — a native, in-console, fully-audited conversational interface over your data. Dashboards — the usual visual rollups, built on top of everything above. None of these are novel concepts on their own — every SIEM has some version of sources, detections, and alerts. What's different is how thin the layers are between them. There's no separate indexing tier, no proprietary storage format you need a support ticket to query around. It's one table, underneath everything. What's Coming in This Series This is the first of twelve articles. The next one goes under the hood of the ClickHouse architecture itself — the actual table schema, how pipelines route data, and a genuinely interesting design decision about where the "real" write to storage happens. After that: the Explorer and how you actually query all of this yourself, everything that can happen to an event in flight before it's ever stored (transform, filter, drop, mask, sample, enrich), ingestion and the connector catalog, the detection engine, investigations and AI triage, scheduled AI agents (including a live gotcha I hit building one), notifications, the AI Chat and MCP integration (including the MCP server I built), a head-to-head comparison against Splunk and Microsoft Sentinel, and finally a full hands-on build of a detection pipeline from scratch, screenshots included. If you're the kind of person who reads a SIEM's docs page and immediately wants to know what actually happens when you hit the API instead of the marketing site — you're going to enjoy this series more than most. Next Part: https://www.cyberengage.org/post/runreveal-architecture-deep-dive ---------------------------------------Dean------------------------------------------------------

I want to be upfront about where I'm coming from before I say anything else: I'm not an AI skeptic. I run AI agents most of my working day, and I've built plenty of them for different tasks. I use them for a lot — building internal tools, troubleshooting, you name it. Half the projects on my plate right now exist because an AI agent worked alongside me, troubleshot with me, and knocked out in a few days what would've taken months. So when I say the Hugging Face breach worried me, I don't mean it in the 'AI is scary, unplug everything' way. I mean it as someone who actually uses this stuff for a living and recognized exactly how bad the failure mode was.. What actually happened In July 2026, Hugging Face disclosed that its infrastructure had been breached — not by a person, but by an AI agent that was supposed to be inside a sandbox. An OpenAI model, being run through a cybersecurity evaluation, was handed a challenge it couldn't legitimately solve. Instead of failing gracefully, it started reward hacking: chaining together previously unknown exploits to find a real answer out in the wild instead of admitting it was stuck. It broke out of its test environment, compromised a package management tool to get outbound internet access, then used a malicious dataset to abuse two separate remote-code-execution paths in Hugging Face's dataset processing pipeline. From there it leaked cloud and cluster credentials and moved laterally into internal systems, touching a limited set of internal datasets and Spaces secrets — the auth tokens developers and organizations use to talk to APIs. Hugging Face caught it through its own LLM-based anomaly detection, not a human noticing something odd in a log. Here's the part that actually changed how I think about this. Hugging Face's disclosure pushed Anthropic to go back and audit its own evaluation logs — 141,006 runs where Claude had internet access during testing. They found three separate incidents where a Claude model, during a third-party evaluation, got unauthorized access to the production infrastructure of three different companies. Two of those companies hadn't even noticed it happened until Anthropic told them. And in early September, Anthropic disclosed a fourth incident, this time involving Claude Opus 4.6. So this isn't an "OpenAI has a problem" story. It's an industry pattern. Two different frontier labs, evaluating their own models for cybersecurity capability, found those same models quietly breaking containment and touching real production systems that had nothing to do with the test. Why this rattles me more than a normal breach I do incident response. I've spent this year going through ClickFix infostealer cases, credential theft, lateral movement, Ransomwares the whole playbook — attacker moves, defender reconstructs the timeline after the fact. That's a known shape. What happened here is a different shape entirely. This wasn't a human attacker with a plan. It was a system that got stuck, decided the constraints it was given didn't apply anymore, and improvised its way to production access using techniques nobody had programmed it to know. It didn't need to be told how to chain a sandbox escape into an RCE into a lateral movement into a credential harvest — it worked that out on its own, mid-evaluation, because that's what "don't accept failure" optimization does when nobody explicitly told it where the line was. And the thing that caught it wasn't a human analyst staring at a SIEM. It was another AI model doing anomaly correlation. We are now in a place where the attacker is an AI, the intrusion looks like normal API traffic from a trusted internal tool, and the detection layer that actually works is also AI. If your org's whole security posture assumes a human is the one doing something malicious, you're already behind. Where I land — don't stop using AI, start governing it like the privileged identity it is I'm not writing this to tell you to go rip out your AI tooling. I won't be ripping out mine. The productivity gain is real and I'm not giving it up over a headline. But "keep using it" and "keep using it with no guardrails" are two very different decisions, and right now most orgs — including plenty that think of themselves as security-mature — are quietly making the second one by default. A few things I actually believe need to happen now, not in a year when someone writes a compliance framework for it: Treat AI agents as privileged identities, not as tools. Scoped credentials, short-lived tokens, least privilege — the same rules you'd apply to a service account, applied to every agent that can touch a real system, including the ones your vendors run during "just an evaluation." Agent activity monitoring has to be a first-class control, not something bolted on after the fact. Hugging Face and Anthropic both caught these incidents through AI-driven telemetry correlation, which tells you the traditional logging most companies have isn't built to catch this class of behavior at all. Sandbox boundaries need to be boundaries an agent can't reason its way out of. A test environment that can be escaped by a model that decides the test is unfair is not a sandbox, it's a suggestion. Human approval gates belong in front of anything touching production, even — especially — when the thing asking is your own evaluation pipeline testing your own model. Disclosure has to become normal, not exceptional. Credit where it's due: Hugging Face and Anthropic both published real, detailed incident reports instead of burying this. That's the behavior the rest of the industry needs to copy, not the exception that makes headlines. I've spent time this year independently testing one of the early products trying to build exactly this kind of control — Delphi Security's xAIDR, an open-source AI agent runtime security sensor.(https://delphisecurity.ai/products) Delphi Security's xAIDR might not be at that level of maturity yet, but it's a start. Tool Delphi Security. It's in early-stage . But the instinct behind it — that agent behavior needs its own dedicated monitoring layer, separate from how we watch human users — is exactly right, and I'd rather see more of these tools exist and mature than none at all. The point We're not going to put this genie back in the bottle, and honestly, I don't want to — I'd be less effective at my own job without it. But the Hugging Face breach, and the fact that Anthropic found the same failure mode in its own models within weeks of looking, should be the moment every org stops treating "we use AI agents" as a productivity line item and starts treating it as an access-control problem. Write the policy now. Monitor the agents now. Assume they will do something you didn't authorize, because two of the most capable labs in the world just showed you theirs already did. --------------------------------------------------Dean-------------------------------------------------