SentinelOne Prompt Security: Logging, Privacy, Retention and the MSSP Question
Previous parts asked whether the product sees the leak and stops it. This part asks what it writes down - the half that decides whether you can run it for someone else.
Partway through this evaluation I exported the activity log and found our own real client work in it, in clear text. So none of this is theoretical.
Everything tested below used fake data. The one incident involving real data is described because it happened to us; no values are reproduced.
1. What one event holds
Export the Activity Monitor to CSV: eighteen columns. Two of them are the whole privacy story.
Column | What it holds |
Time | UTC timestamp |
User | browser-profile e-mail, or the Windows account name for agent events; often empty |
User Groups | identity-provider groups, empty in our tenant |
Prompt/Response | which half of the exchange this row is |
Valid | false means it carries a violation |
Action | log for Detect, block, or modify for redacted |
File | true if an upload was inspected |
Violations | the category, for example Secrets or Sensitive Data |
Violating Findings | JSON holding the matched value itself, in clear |
GenAI Application | ChatGPT, GitHub Copilot, AWS Bedrock API, Claude |
Custom App | true for homegrown apps |
Connector Name | which connector handled it |
IP Address | the endpoint's public IP |
Policy Name | the policy that applied |
Conversation ID | the chat thread, for browser apps |
Prompt Response ID | unique event id |
Prompt/Response Text | the full prompt; for agent traffic, the entire request body |
Modified Text | the redacted version, when the action was modify |
Violating Findings does not record "an AWS key was found". It records the key. From one of our fake-key runs through Copilot: Violations=Secrets | Violating Findings={"Secrets":[{"entity":"AKIAG22G6P6C3VY5VITR","entityType":"AWS credentials"}]}
The detector that exists to stop a secret reaching a model writes that secret into a cloud console, where it sits for the retention period and any admin can export it.
The console detail view holds more than the CSV: Platform (Web or Desktop - how we identified which component caught something), Application Risk Score, latency, IP and country, sensor type and version, OS and browser, the app's domain and request URL, file and autocomplete flags, conversation and log ids, and the full text of the rule that matched.
Two reconciliation traps:
Agent rows write ...Z with milliseconds, extension rows write +00:00 with microseconds.
The agent's title-generation call raises a second alert for the same prompt - one user action, two events.
2. The defaults
Platform Settings to Data Retention, out of the box:
Storage Enabled
Retention 30 days
Logs type Save full logs
What that means:
Full logs stores real values on both halves of the exchange. A fake AWS key sent to claude.ai was stored in clear in the prompt and in the model's reply, under a Detect rule.
My colleague logged the clause that matters as a finding in its own right: with Full logs the original value is stored even when it was masked for the AI. Protect mode can redact a key so the model never sees it, and Full logging still keeps the original. Redaction protects the model, not the log.
Who can read it: any admin in the tenant. Ours had three admin accounts and two API keys, all with the Admin role, before we added anything. Viewer and Editor roles exist. Nobody was using them.
3. The part where we did it to ourselves
On 21 October I exported the activity log to check timestamps for an unrelated test. 233 rows.
Forty-nine of them were not tests.
That afternoon I had been doing ordinary paid work on the same laptop - an AI coding assistant against a model API, on real client material, with the agent installed.
Captured in full text and stored: two clients named, staff e-mail addresses, internal IPs, AWS ARNs, one temporary AWS session key, two message-broker tokens.
The same export held my colleague's prompts from her own testing.
Nothing malfunctioned. This is the product working exactly as configured.
The cause was one locked rule - all users, AI code assistants, Detect, log all interactions - present when we received the tenant, not editable by an ordinary admin.
Detect does not redact.
"Log all interactions" does not mean metadata. It means the whole request body, and for an agentic coding tool that body is the conversation history plus injected context.
One Copilot alert included the tool's own confidential system prompt. Individual CLI requests ran to roughly ninety thousand characters.
Three consequences:
The extension and the agent have opposite privacy profiles. In our extension testing, no prompt text was stored at all. The agent ships prompts in ALWAYS mode. Same console, same dashboard, completely different exposure.
Evaluating this on a working machine puts your live work in the vendor's cloud from the moment the agent installs. Not your test data. Your work.
The decryption scope needs sign-off before deployment, not after. Part 4 covers the mechanism: a local mitmproxy, a tenant root CA, every AI endpoint on the list decrypted - including traffic that is never logged.
4. The four log types
Logs type | What the console keeps | Status |
Save full logs (default) | Everything, including the raw matched value, in prompt and reply | Tested |
Save redacted logs | Depends on the action - see below | Tested |
Save only basic metadata | No prompt text, dashboards still work | Untested |
Storage disabled | Nothing stored; enforcement still happens | Partly tested |
Redacted behaves differently depending on what the policy does. Same GitHub token to ChatGPT:
Secrets = Block. Pop-up shown, console recorded the violation and the Blocked action, no prompt text stored at all. Cleanest outcome available.
Secrets = Redact all. Model received [REDACTED_GITHUB_PERSONAL_ACCESS_TOKEN_1]; console content and conversation views showed only the placeholder. Action Modify, result Invalid. Raw token stored nowhere. Also what you want.
Storage disabled, one side tested. A prompt that should have been blocked was still blocked, and correctly appeared nowhere in the Activity Monitor or dashboard. Enforcement is independent of storage.
Untested, and it is the half that matters: whether the event reaches the SIEM or S3 bucket you disabled storage in favour of, and what happens if that destination is down. Buffer, retry, or lose - open vendor question.
Do not turn storage off until you have watched events land somewhere else.
Retention runs 2 to 365 days. Ours allowed 365, which told us something by accident: the documentation caps MSSP-managed accounts at 90 days, so being offered 365 proves our tenant is not MSSP-managed.
5. Two holes in redacted logging
Redacted is the setting I would recommend to a privacy-conscious client, so both caveats matter.
It never redacts the model's reply.
Redaction applies to prompts only.
It redacts only what an enabled detector caught. A data type you did not switch on, or a format the pattern missed, is stored in clear regardless.
The privacy control inherits every gap in the detector set.
On one path, Redacted appeared to lose the violation entirely.
Same GitHub token from a Chrome Guest session to perplexity.ai, twice.
Logs type Redacted: no violation recorded, only the response.
Logs type Full: violation recorded.
Platform showed Desktop, so the agent caught these, not the extension - extensions do not run in Guest mode (Part 3).
Taken at face value that is a bad trade: choose the privacy-friendly log type and the SOC stops seeing leaks on the agent path.
Marked Open. Needs a controlled retest, Redacted versus Full on a desktop app, before anyone builds policy on it. It is here because it is the kind of interaction nobody looks for, and if it holds it changes the recommendation.
6. Logging a response is not inspecting it
We asked Claude on the web, with a clean prompt, to generate a sample US social security number and an AWS access key for test documentation.
Log only violations: nothing logged.
Log all interactions: conversation logged, the reply containing the SSN and key visible in the console - and not flagged as a violation.
Detectors run on the way in, not on the way out. The reply is recorded, not judged. That is the mechanism behind the Part 6 technique of breaking a secret going in and asking the model to repair it coming out.
Response logging and file inspection are not universal. The vendor's October list covers twelve applications. Everything else gets prompt inspection only - no reply logging, no file inspection. If a client's chosen tool is not on that list, half the product's visibility does not apply to them.
7. The one privacy control I would turn on
Platform Settings to expose the prompt text only upon request.
With it set, the Activity Monitor shows the violation, detector, application and action, but not the text. An admin who needs the content clicks "View conversation".
Tested with an invented name, SSN and date of birth to ChatGPT: violation logged with the correct finding type (US SSN), prompt and response hidden until the click. After the click, the SSN was in clear in both.
For a shared console that is the right default - it turns "every admin can browse client secrets" into "an admin has to deliberately ask for one".
The question that decides whether it is a real boundary or a speed bump: is that click recorded in the admin audit log? Untested, near the top of my list to confirm.
The audit log itself exists, is documented at 365 days, survives storage being disabled, and can be forwarded to a SIEM - all read from documentation, not verified.
8. Who the events say you are
Previous part covered why group rules need an identity provider. This is what it looks like in the log when you have none.
No identity integration connected in our tenant.
Extension rows without MDM attribution arrived with User blank.
Agent rows arrived as Akash - the local Windows account.
For the Protect testing we had to reinstall the agent with a username file so events would carry a real e-mail address, because a user-scoped rule cannot match an account the console cannot name.
For a managed service that means a login name is doing the work of an identity, and per-client attribution is only as good as what you set at install time.
9. What logging costs
Two numbers for the commercial conversation.
Log all interactions against an agentic coding tool is not a log line, it is a conversation transcript with context attached - tens of thousands of characters per request in our measurements. Multiply by developers and working days before promising anyone SIEM forwarding.
Log only violations is not as quiet as it sounds. Clean prompts are still stored as metadata rather than dropped, which is what keeps the usage dashboards working. Fewer bytes, not zero.
The logs type and the log-all switch are the two dials that move your storage bill. They are also the two that move your privacy exposure, and they move in opposite directions.
10. The MSSP question
The reason this part exists. Can you run this across a dozen clients?
The practical answer, from our own commercial discussion: yes, but not as one pane of glass.
The capability is enabled per client as an add-on.
Each client gets their own Prompt Security console instance, reached through the same link.
There is no single dashboard showing violations across every client at once. You work tenant by tenant.
That is the shape of the thing before any API work: per-client onboarding, per-client policy configuration, per-client monitoring. Carry that number into the staffing estimate.
The documented model we could not test. The admin API describes an MSSP organisation with child customer tenants: one endpoint lists children, another reports licence consumption per customer, and an MSSP API key plus an x-ps-target-tenant header acts inside a customer tenant with that key's role. The customer's audit log records the action as coming from an MSSP API key. Sensible design - but our tenant has no child tenants, and the 365-day retention ceiling confirms it is a plain account. Read from the API reference, not run.
What we did test, through the management API with an admin key:
Policies are scriptable. Create, list, update and delete work for Employees-type policies.
Security Rules are not. No rules endpoint exists. Who a rule applies to, which applications it covers, and whether it is Detect, Protect or Block is console-only work, per tenant, by hand. Across twelve clients that is twelve sets of clicking - and Part 5 found rules that silently refuse to save in the wrong combination.
Update replaces, it does not merge. A PUT rewrites the whole policy and resets fields the API does not expose. Create returns no identifier.
No Terraform provider or configuration-as-code path we could find.
No device inventory in the API either. Part 4 established there is none in the console; the users endpoint returned only our three admins, and an undocumented sensors path answered 401. Combined with the Part 6 finding that a blocked connection to the vendor silently disables protection and logging, a client can be unprotected with nothing in the product saying so.
Two more gaps worth naming:
Agentic-AI logging is SentinelOne-only. The tool calls a coding agent makes, the MCP servers it talks to, the skills it loads - emitted as OCSF events only through the Singularity integration. Rich if the client already has Singularity, absent otherwise. Not enabled in our tenant, so we never saw one.
Residency is half-answered. Our tenant is on a US-west host. The vendor's GDPR material offers an EU tenant, and we never confirmed where data actually lives or whether deletion at end of retention includes backups. For a client with a DPA those are not optional questions.
What I would set, before onboarding anyone
Change the logs type away from Full on day one. Redacted at minimum. Full is the default, and it turns your console into a store of your clients' secrets with the raw values in an exportable CSV.
Turn on "expose the prompt text only upon request" - and get a written answer on whether that click is audited before treating it as a control.
Set retention deliberately: 90 days is the cap on an MSSP-managed tenant, 365 on a plain one.
Deal with the locked code-assistant rule before anyone does real work on a machine with the agent. Detect plus log-all-interactions on AI code assistants means full capture of your engineers' working context. We learned that from our own export, not the documentation.
Get the root certificate and decryption scope approved in writing by the client's compliance function, naming the fact that traffic is decrypted even where it is not logged.
Connect an identity provider before you need group rules, or accept Windows account names in your events.
Watch endpoint connectivity to the vendor's hostname from your own EDR. No device list exists, and a dark sensor fails open silently.
Prove SIEM or S3 delivery while storage is still enabled, then turn storage off - never the other way round.
Budget log volume from a real transcript, not a per-event estimate.
Plan for per-client consoles. No cross-client dashboard exists, so build the per-tenant routine before client number two.
Where that leaves it
What is well built, where someone clearly thought about it:
Typed redaction placeholders
A prompt-visibility gate
Log types that go down to forward-only
An audit log that survives storage being switched off
The weaknesses are all one kind - the defaults are the permissive ones, and the console does not tell you what it cannot see:
Full logs by default
A locked rule capturing whole conversations
No device inventory
No rules API
A privacy setting that on at least one path appears to cost you the violation record
No single view across clients
None of those is a flaw in the detection engine. All of them are decisions to make before the first client is onboarded, because every one is harder to unwind afterwards.
- written from hands-on testing, By Iqra Shafi, Dean
------------------------------------------Cyberengage---------------------------------------------------
Comments