We evaluate / ml inline redaction under three operational configurations: off (default pass-through), memorize_and_recall (static write+read redaction for zero-persistence vaults), and recall_only (dynamic data masking at retrieval egress). All 13 canonical memory benchmarks are run with enterprise_strict active simultaneously across all seven protected span classes: SECRET, PCI_FINANCIAL, PII, PHI, COMMERCIAL_MNPI, LEGAL_HR, and SECURITY_EXPLOIT. Across 137,271 recalled items, dynamic recall_only masking redacts 11,093 sensitive egress spans with zero write or query edits, recovering +7.20 pp on 6-vertical Enterprise Policy Enforcement (95.00%), +6.67 pp on HumanEval+ Recall@5 (81.67%), +5.00 pp on Repository Debug Hit@1 (100.0%), and 82.08% on BEAM @ 10M tokens, while supporting per-agent clearance bindings via POST /v1/agents/.
1. Why redaction in memory operations is a 3D policy problem
Persistent agent memory sits directly in the path of live engineering, support, clinical, and financial operations. Within a single afternoon, an agent session ([1]) observes shell exports containing live bearer tokens, stack traces referencing internal vulnerabilities, customer tickets holding payment cards and emails, clinical allergy notes, HR compensation bands, and unannounced M&A terms.
Treating redaction as a single workspace-wide on/off toggle breaks down in practice because sensitivity depends simultaneously on what the span is, when the memory operation runs, and which agent is reading:
- Category selectivity: Coding harnesses want live API keys and bearer tokens (
SECRET) redacted immediately, yet depend on keeping CVE identifiers, internal hostnames, ports, and stack-trace payloads (SECURITY_EXPLOIT) intact to debug issues. - Operation lifecycle boundary (Static vs Dynamic Masking): Zero-persistence vaults require static masking at write ingress so raw literals never touch storage, whereas encrypted-vault deployments benefit from full-fidelity storage with dynamic masking enforced strictly when memories exit into an LLM prompt or tool response.
- Agent identity & least privilege: In a multi-agent workspace, a billing reconciliation agent, a customer-support triage agent, and an untrusted cloud PR sandbox agent should be able to query the same shared organizational memory while each receiving a differently redacted view matched to its role.
Redaction in memory operations has three independent axes: which categories to mask, which operation stage (write + recall vs recall-only) enforces the mask, and which
agent_id the policy is scoped to.
2. The 7 sensitivity categories: What gets masked vs what stays intact
To support fine-grained compliance presets and per-agent clearance without corrupting benign operational context, / ml partitions sensitive data into seven composable span categories covering both structured credentials and unstructured enterprise disclosures:
- Credentials & direct identifiers (
SECRET,PCI_FINANCIAL,PII): Tokens, payment instruments, and personal identifiers often contain partial formatting cues, yet still require surrounding context to separate real production secrets and customer records from benign engineering look-alikes such as Git commit SHAs, UUIDs, test-card fixtures, and shared role aliases. - Unstructured enterprise disclosures (
PHI,COMMERCIAL_MNPI,LEGAL_HR,SECURITY_EXPLOIT): Clinical history, unannounced M&A or revenue terms, privileged legal/HR records, and weaponizable exploit paths are written in ordinary natural language or code with zero fixed character templates, making them invisible to pattern and entropy scanners.
Rather than dropping or blocking an entire conversational turn when a sensitive detail appears, / ml performs surgical span-level replacement: only the protected substring is replaced with its typed placeholder ([REDACTED:<CATEGORY>]), preserving surrounding operational rules, debugging clues, and workflow preferences intact.
Table 1: The 7 Protected Sensitivity Categories in / ml Memory Operations
Each category can be enabled via built-in compliance presets (Table 3) or cherry-picked individually per workspace, per agent_id, or per API/MCP call.
| Category & Placeholder | What We Consider Sensitive (Masked) | Benign Look-Alikes Kept Intact |
|---|---|---|
SECRET
[REDACTED:SECRET]
Tier-1 · Authentication & Infra
|
Live and staging API keys (sk_live_*, rk_*, ghp_*, AKIA*), OAuth/JWT bearer tokens, PEM/SSH private key blocks, webhook signing secrets, and database or cache URIs containing embedded passwords. |
Git commit SHAs (9f82c10e4b), UUIDs, content hashes, environment variable names without values ($STRIPE_API_KEY), and public key fingerprints. |
PCI_FINANCIAL
[REDACTED:PCI_FINANCIAL]
Tier-1 · Payment & Banking
|
Payment card primary account numbers (PANs), CVV/CVC security codes, card expiration pairings, bank account and ABA routing numbers, IBAN/SWIFT codes, and wire settlement instructions. | Documented test-card fixtures (4111-1111-1111-1112, 4242), invoice and order IDs, SKU codes, port numbers, and public SaaS pricing tiers. |
PII
[REDACTED:PII]
Tier-1 · Personal Identifiers
|
Direct personal identifiers tied to a natural person: personal and customer email addresses, phone numbers, SSNs, passport, driver's license, and national tax IDs, residential street addresses, and dates of birth. | Shared functional mailboxes (support@, billing@, security@), RFC 2606 example domains (example.com), loopback IPs, and non-identifying role references. |
PHI
[REDACTED:PHI]
Contextual · HIPAA / Clinical
|
Protected health disclosures linked to a patient or user: clinical diagnoses, symptoms, drug allergies (e.g., anaphylaxis to amoxicillin), prescriptions and dosages, lab results, medical record numbers (MRNs), and treatment notes. | General biomedical concepts without patient linkage, software "health check" or CLI "doctor" diagnostics, and adjacent scheduling rules (no meetings before 11:00 AM). |
COMMERCIAL_MNPI
[REDACTED:COMMERCIAL_MNPI]
Contextual · Finance & Corp Dev
|
Material non-public financial and strategic disclosures: unannounced M&A targets, codenames, and deal valuations (e.g., Silverline $95M acquisition), pre-release quarterly revenue/ARR/churn numbers, cap-table terms, and confidential board pricing plans. | Publicly reported earnings and SEC filings, published pricing pages, routine cloud spend budgets, and operational directives (freeze vendor onboarding until Q3). |
LEGAL_HR
[REDACTED:LEGAL_HR]
Contextual · Counsel & People Ops
|
Privileged legal counsel and confidential personnel records: attorney-client litigation strategy, settlement figures, internal investigation notes, individual salaries and equity grants, performance improvement plans (PIPs), and termination records. | Public terms of service, open-source licenses (MIT, Apache-2.0), published job-posting pay bands, and company-wide engineering or PTO handbook policies. |
SECURITY_EXPLOIT
[REDACTED:SECURITY_EXPLOIT]
Contextual · AppSec & Infra
|
Weaponizable vulnerability details and internal attack paths: unpatched zero-day reproducers, SQLi/SSRF/RCE exploit payloads, internal bastion hostnames/IPs paired with exposed ports, and authentication bypass steps. | Ordinary application stack traces, unit-test assertion failures, remediated public CVE IDs, dependency version pins, and defensive coding rules (also left unmasked under tier1_secrets for local IDE debugging). |
3. Why regex filters and static write-time scrubbing break agent memory
To measure why conventional regex and entropy filters fail inside memory operations, we evaluated held-out agent turns across seven sensitivity categories (SECRET, PCI_FINANCIAL, PII, PHI, COMMERCIAL_MNPI, LEGAL_HR, SECURITY_EXPLOIT), paired with realistic benign developer strings (such as test fixtures, commit hashes, and UUIDs).
4111-1111-1111-1112 on commit 9f82c10e4b. "Regex + Entropy: Redacts benign commit
9f82c10e4b (FP) · Misses anaphylaxis to amoxicillin (0% PHI recall)/ ml Redaction:
[REDACTED:SECRET] + [REDACTED:PHI] · Preserves fixture & commit hash intact
/ ml inline redaction closes the 0% contextual blind spot of regex while cutting benign developer false alarms by 4.4x
Left: Aggregate Tier-1 recall, 7-category exact span F1, and benign developer-string false-redaction rate. Right: Per-category recall comparison showing complete regex blindness on non-pattern enterprise categories.
off overhead: None (0.0% FP)
Two practical lessons stood out immediately from these measurements:
- Opt-in defaults protect standard engineering workflows: Even after cutting false alarms on realistic code fixtures by 4.4x relative to regex (
2.50%vs11.00%), unconditionally rewriting every turn would still alter ~1 in 40 borderline fixture strings. Keeping redaction"off"by default guarantees zero runtime overhead and0.00%unintended mutation until a workspace or agent explicitly opts in. - Static write-time masking collides distinct records: Replacing clinical, legal, financial, or security details with
[REDACTED:...]before storing a memory erases the exact details that distinguish two services, two customer accounts, or two project rules during later retrieval.
4. Redaction vs data hygiene vs erasure: Separating inline masking, retirement, and purge
In security and database engineering, redaction (inline DLP / data masking), data hygiene (retiring stale or mistaken operational state), and data erasure (NIST SP 800-88 [8] and GDPR Art. 17 [7] permanent removal) solve completely different problems. Most agent memory SDKs collapse all three into a single delete(id) call ([2]). / ml exposes them as three separate runtime verbs:
Separating inline span redaction (redact), reversible memory hygiene (forget + undo), and permanent compliance erasure (purge)
Evaluated on held-out test cases covering span redaction, selective fact retirement, conversational corrections, and full compliance erasure.
POST /v1/memory/redact & Inline Policies)
99.53% Tier-1 Recall · 90.83% Span F1 · 0.00% Raw Leak
agent_id, or workspace category rules.
[REDACTED:<CATEGORY>] while preserving surrounding prose.
memorize_and_recall (static) or recall_only (dynamic egress).
POST /v1/memory/forget + /undo)
100.0% Selective Retirement · 100.0% Correction Accuracy · 100.0% Lossless Undo
undo_token for instant, lossless restoration when needed.
POST /v1/memory/purge)
100.0% Zero-Residue Verification · 0.00% Export or Read Leak
dry_run=true impact inspection prior to execution.
audit_id receipt confirming 0.00% residual plaintext.
5. Static write+read masking vs dynamic recall-only masking
Production data platforms distinguish Static Data Masking (permanently overwriting sensitive fields on write) from Dynamic Data Masking (storing records inside encrypted storage and masking output dynamically at read time). We evaluated both operational modes across all 13 canonical / ml benchmarks with all 7 categories simultaneously forced active (enterprise_strict):
- Static Write + Read Redaction (
redact_scope="memorize_and_recall"): Masks incoming text on write, masks incoming search queries, and masks outbound recalled memories. Sensitive literals are never stored at rest. - Dynamic Recall-Only Masking (
redact_scope="recall_only", alias"retrieve_only"): Preserves full internal retrieval fidelity inside the encrypted tenant vault (0write edits,0query edits), while masking every outbound memory string before it enters an LLM prompt, ambient hook context, MCP tool output, or HTTP response.
Why egress redaction is essential in both modes, and how dynamic recall_only masking avoids write-time information loss
Left: Span redaction counts in static memorize_and_recall (18,329 total spans). Right: Egress-only category breakdown in dynamic recall_only (0 write/query edits; 11,093 egress spans masked across 137,271 recalled items).
Write-only scrubbing cannot retroactively protect existing vault items when a team updates its compliance policy, nor can it serve different reader clearances. By enforcing
/ ml redaction at the recall boundary, newly tightened policies protect historical context immediately and adapt per calling agent.
6. Full 13-benchmark matrix: off vs memorize_and_recall vs recall_only
Figure 4 highlights the benchmarks where moving from static write+read redaction (memorize_and_recall) to dynamic recall-only masking (recall_only) restores retrieval and policy accuracy while keeping all 7 categories strictly masked on output, followed by the complete 13-suite table.
memorize_and_recall → recall_only)
Dynamic recall_only masking restores code localization, multi-hop QA, and enterprise policy enforcement while blocking 11,093 egress spans
All pairs evaluated under identical runtime configuration with enterprise_strict (all 7 categories active).
Table 2: Complete 13-Suite Empirical Matrix (off vs memorize_and_recall vs recall_only)
Evaluated in standard runtime configuration (lifecycle hooks + explicit MCP/REST tools) with redact_policy="enterprise_strict" (all 7 sensitive categories active simultaneously).
| Benchmark Suite | Primary Metrics | off(Default) |
memorize_recall(Static Write+Read) |
recall_only(Dynamic Recall) |
recall_only vsmemorize_recall |
|---|---|---|---|---|---|
| 1. Conversational Assistant 200 scenarios · 2,800 turns · 8 consumer domains | Turn Acc · Delayed Recall · Safety Block · Supersede | 96.21% · 100% · 100% · 93.5% | 96.71% · 100% · 100% · 93.5% | 96.71% · 100% · 100% · 93.5% | 0.00 pp (78 egress spans) |
| 2. Multi-Project Isolation & Coref 500 isolated projects · 9,952 turns · 10 verticals | Recall · Coref E2E · Supersede · Cross-Tenant Bleed | 100% · 99.8% · 100% · 0.0% | 100% · 99.2% · 98.0% · 0.0% | 100% · 100.0% · 99.0% · 0.0% | +0.80 pp Coref · +1.00 pp Supersede |
| 3. Production Mixed Workload 2,500 code definitions + 25 standing rules | Signal Precision · Rule Delivery · Policy Check · Recall@5 | 100% · 100% · 100% · 100% | 100% · 100% · 100% · 100% | 100% · 100% · 100% · 100% | 0.00 pp across all 4 tracks |
| 4. Embodied Spatial Memory E2E 100 objects · 5 timesteps · 50 hazard checks | Active Hit@1 · State Tracking · Pre-Action Safety | 100% · 100% · 100% | 100% · 100% · 100% | 100% · 100% · 100% | 0.00 pp across all 3 tracks |
| 5. High-Clutter Spatial & Affordance 50 relocated objects + 2,000 distractors | Supersession · Haystack Needle R@1 · Affordance Match | 100% · 100% · 100% | 100% · 100% · 100% | 100% · 100% · 100% | 0.00 pp across all 3 tracks |
| 6. Enterprise Policy Enforcement (6 Verticals) Coding, CRM, Clinical, Exec, Legal, Finance | Policy F1 · Extraction F1 · Supersede · Noise Filtering | 92.31% · 99.58% · 90.0% · 100% | 87.80% · 98.56% · 85.0% · 100% | 95.00% · 99.23% · 88.33% · 100% | +7.20 pp Policy · +3.33 pp Supersede |
| 7. Real-World Agent & Tool Stream 2,797 held-out multi-step agent & tool turns | Capture F1 · Rule Updates · Compliance Accuracy (False-Alarm) | 92.04% · 100% · 86.25% (8.54%) | 91.89% · 100% · 84.24% (10.47%) | 92.04% · 100% · 86.25% (8.54%) | +2.01 pp Compliance · -1.93 pp False-Alarm |
| 8. DolphinBench Track 2 E2E [3] 600 multi-tool tasks · 1,920 mechanical & rubric checks | Task Pass Rate · Check Pass Rate (vs 65.5% Mem0 #1) | 90.50% (543/600) · 91.56% | 79.17% (475/600) · 81.04% | 80.00% (480/600) · 82.19%1,507 egress spans blocked | +0.83 pp tasks · +1.15 pp checks (+14.5 pp vs Mem0) |
| 9. DolphinBench Track 1 Retrieval [3] 13,539 sessions across Morgan, Alex & Riley · 600 queries | Fact R@5 · Fact R@10 · Fact R@20 · MRR | 58.60% · 68.60% · 75.40% · 48.00% | 57.04% · 67.45% · 75.56% · 46.18% | 57.23% · 68.41% · 75.01% · 47.18%4,162 egress spans blocked | +0.96 pp R@10 · +1.00 pp MRR |
| 10. Open-Domain RAG & Official LoCoMo [4] SQuAD 1-hop · HotpotQA 2-hop bridge · LoCoMo multi-session | LoCoMo QA@5 (R@20) · SQuAD R@5 · HotpotQA 2-Hop R@5 | 86.89% (87.27%) · 96.00% · 85.00% | 87.89% (87.10%) · 96.00% · 85.00% | 87.87% (87.27%) · 97.33% · 87.50%1,084 egress spans blocked | +1.33 pp SQuAD · +2.50 pp HotpotQA · +0.17 pp R@20 |
| 11. LongMemEval-v2, LME & 10M BEAM [5] 451 LME-v2 · Official LongMemEval · BEAM 1M & 10M tokens | LME-v2 Acc (R@5) · LME Hit@5 · BEAM 1M · BEAM 10M | 93.79% · 91.67% · 88.40% · 84.34% | 92.68% (94.01%) · 89.58% · 84.21% · 80.99% | 92.68% (94.01%) · 91.67% · 84.75% · 82.08% | +2.09 pp LME Hit@5 · +0.54 pp 1M · +1.09 pp 10M |
| 12. Code Memory & SWE-bench Lite [6] 350 code chunks + 300 real GitHub issue localization tasks | SWE File Top-5 · Repo Debug Hit@1 · HumanEval+ R@5 · MBPP+ R@5 | 82.67% · 100.0% · 81.67% · 69.33% | 83.33% · 95.00% · 75.00% · 66.67% | 84.67% · 100.0% · 81.67% · 69.33%2,763 egress spans blocked | +1.34 pp SWE · +5.00 pp Debug · +6.67 pp HE+ |
| 13. Lifecycle Governance Suite Held-out test cases across redaction, hygiene & erasure | Tier-1 Redact · Selective Forget · Corrections · Compliance Purge | 99.53% · 100.0% · 100.0% · 100.0% | 99.53% · 100.0% · 100.0% · 100.0% | 99.53% · 100.0% · 100.0% · 100.0% | 0.00 pp (722 egress spans) |
Note: All three columns are reported on the unified evaluation harness so differences reflect redaction scope alone.
Why DolphinBench E2E is 80.00% under recall_only vs 90.50% under off
Comparing Suite 9 (Track 1 Retrieval) and Suite 8 (Track 2 End-to-End Execution) shows egress DLP working as intended. Under recall_only, internal retrieval over the 13,539-session vault stays virtually unchanged (68.41% Fact R@10 vs 68.60% baseline). When those retrieved memories exit into the agent's prompt or search tool output, / ml masks 865 recalled messages (1,507 sensitive spans) into typed placeholders ([REDACTED:PII], [REDACTED:PCI_FINANCIAL], [REDACTED:PHI], [REDACTED:LEGAL_HR], [REDACTED:SECRET]).
Because DolphinBench grades 1,194 of its 1,920 checks with exact field_equals assertions on outbound tool calls (send_email, create_doc, send_sms), withholding those protected literals prevents the downstream LLM from copying raw persona emails, phone numbers, card numbers, clinical notes, or tokens into external tool arguments on ~60 tasks, while still outperforming Official Leaderboard #1 Mem0 (65.50%, unredacted) by +14.50 percentage points.
7. Scoping redaction to individual agents over a shared memory vault
Dynamic recall-only masking enables a multi-agent governance pattern that is impossible under static write-time scrubbing: per-agent redaction scoping over a single shared workspace vault.
In multi-agent systems, specialized agents collaborate over shared organizational context. If an ingestion pipeline masks PII and PCI_FINANCIAL permanently on write, downstream billing and support agents lose the customer email or card reference they need to do their job. Conversely, leaving the vault unmasked exposes payment cards, clinical records, and M&A terms to untrusted background coding sandboxes or external MCP connectors.
With recall_only, organizations bind redaction policies and read/write access modes ("read_write" vs "read_only") directly to individual agent_id identities via POST /v1/agents/ (or workspace settings in the web portal). Every memory operation resolves its active mask using a 3-tier precedence order:
- Per-operation override: Explicit
redact_policy/redact_scopepassed on an individual API or MCP request. - Agent-scoped clearance policy: Policy registered on the calling
agent_id(orcaller_agent_id/X-SlashML-Agent-Idwhen searching shared cross-agent memories without restricting authorship filters). - Workspace default: Falls back to workspace default policy (
"off"by default, zero runtime overhead).
redact_scope="recall_only")
One shared memory record, three agent-scoped redaction views at recall egress
Shared vault entry: "Enterprise renewal for alice@acme.io (card 4532-0158-7741-9902, webhook key sk_test_EXAMPLE_9999ab) is tied to the Silverline $95M deal."
agent_id: "billing-reconciler"
Policy: ["SECRET", "COMMERCIAL_MNPI"] · Scope: recall_only
agent_id: "support-triage"
Policy: ["SECRET", "PCI_FINANCIAL", "COMMERCIAL_MNPI"] · Scope: recall_only
agent_id: "coding-sandbox" (Untrusted Cloud PR / MCP Subagent)
Access: read_only · Policy: "enterprise_strict" · Scope: recall_only
In standard memory APIs, passing
agent_id on search restricts results to memories written by that agent. Passing caller_agent_id on Mem0-compatible search/read routes and MCP tools (or agent_id on /v1/memory/query and /v1/memory/intervene) lets any subagent search shared cross-team memory while enforcing that caller's redaction policy on every returned result.
8. Production contract: Composable presets, cherry-picked categories, and API
Workspaces and individual agents can choose a named compliance preset from Table 3 or cherry-pick any subset of the 7 categories defined in Table 1 (for example, redact_policy=["SECRET", "PCI_FINANCIAL", "COMMERCIAL_MNPI"]) paired with either "recall_only" or "memorize_and_recall".
Table 3: Built-In Redaction Policy Presets & Custom Category Cherry-Picking
Named presets are combinations of the 7 protected span categories; callers or agent registrations can pass either a preset string or a custom array of category names.
| Preset / Policy Value | Active Span Categories | Recommended Agent / Workspace Role |
|---|---|---|
off Default policy |
None (zero overhead) | General workspaces & trusted single-tenant agents requiring 100% literal pass-through. |
tier1_secrets Alias: tier1 |
SECRET
PCI_FINANCIAL
|
Local IDE coding harnesses & checkout bots: masks API keys, JWTs, PEM blocks, DB URIs, PANs, and CVVs while keeping stack traces and hostnames intact. |
soc2_coding |
SECRET
PCI_FINANCIAL
SECURITY_EXPLOIT
|
Cloud CI/CD bots & external code-review agents where credentials, cards, and exploit payloads must not enter third-party model prompts. |
gdpr / hipaa |
SECRET
PCI_FINANCIAL
PII
PHI
|
Healthcare, consumer support, and EU privacy-regulated agents masking diagnoses, allergies, emails, phones, SSNs, and cards at LLM egress. |
enterprise_strict Alias: all |
SECRET
PCI_FINANCIAL
PII
PHI
COMMERCIAL_MNPI
LEGAL_HR
SECURITY_EXPLOIT
|
Untrusted sandboxes, vendor MCP connectors, and zero-trust multi-agent boundaries requiring full 7-category DLP. |
| Custom Array Cherry-picked list |
Any subset, e.g. ["SECRET", "COMMERCIAL_MNPI"]
|
Role-specific agents (such as billing-reconciler or clinical-triage) that require domain fields while blocking credentials and M&A data. |
Configure per-agent dynamic redaction, read/write access, selective forgetting, and compliance purge
Bind redaction policies and read/write permissions to individual agents via POST /v1/agents/ (or in the web portal), or pass redact_policy and redact_scope directly on any REST, Mem0-compatible, hook, or MCP operation:
# 1. Register a read-only agent with dynamic recall-time redaction over a shared vault
curl -sS -X POST "https://api.slashml.ai/v1/agents/" \
-H "Authorization: Bearer $SLASHML_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"agent_id": "support-triage",
"name": "Support Triage Agent",
"access_mode": "read_only",
"redact_policy": ["SECRET", "PCI_FINANCIAL", "COMMERCIAL_MNPI"],
"redact_scope": "recall_only"
}'
# 2. Query shared memories as support-triage (keeps PII email, dynamically masks PCI + SECRET + MNPI)
curl -sS -X POST "https://api.slashml.ai/v1/memories/search/" \
-H "Authorization: Bearer $SLASHML_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"query": "Enterprise renewal status for Acme",
"user_id": "org_shared_vault",
"caller_agent_id": "support-triage"
}'
# 3. Reversible memory retirement (/undo supported) & permanent compliance purge
curl -sS -X POST "https://api.slashml.ai/v1/memory/forget" \
-H "Authorization: Bearer $SLASHML_API_KEY" \
-H "Content-Type: application/json" \
-d '{"memory_ids": ["mem-stale-region-rule"]}'
References & citation
-
SlashML Research (2026). "Why Agents Forget to Use Their Own Memory Tools: A 12-Benchmark Study of Ambient Runtime Memory."
/ mlTechnical Report. https://slashml.ai/research/ambient-memory.html -
SlashML Research (2026). "When Remembering Everything Breaks AI Agents: Why Selective Erasure Is Fundamental, and Hard."
/ mlTechnical Report. https://slashml.ai/research/forgetting.html - Rathi, S., Yadav, D., & Singh, T. (2026). "DolphinBench: Mapping the Pareto Frontier of Agent Memory." Official 600-task action-execution evaluation harness and leaderboard. https://dolphinbench.ai
- Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbieri, F., & Fang, Y. (2024). "Evaluating Very Long-Term Conversational Memory of LLM Agents (LoCoMo)." Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024). https://arxiv.org/abs/2402.17753
-
Tavakoli, M., Salemi, A., Ye, C., Abdalla, M., Zamani, H., & Mitchell, J. R. (2026). "Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMs (BEAM)." International Conference on Learning Representations (ICLR 2026),
arXiv:2510.27246. https://arxiv.org/abs/2510.27246 - Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., & Narasimhan, K. (2024). "SWE-bench: Can Language Models Resolve Real-World GitHub Issues?" International Conference on Learning Representations (ICLR 2024). https://arxiv.org/abs/2310.06770
- European Parliament and Council (2016). Regulation (EU) 2016/679 (General Data Protection Regulation), Article 17: Right to erasure ('right to be forgotten'). Official Journal of the European Union. https://gdpr-info.eu/art-17-gdpr/
- Kissel, R., Regenscheid, A., Scholl, M., & Stine, K. (2014). "Guidelines for Media Sanitization." National Institute of Standards and Technology (NIST) Special Publication 800-88 Revision 1. https://csrc.nist.gov/pubs/sp/800/88/r1/final
@techreport{slashml2026redaction,
title = {Scrub on Write, or Leak on Read: The AI Memory Redaction
Dilemma Across 13 Benchmarks},
author = {SlashML Research},
institution = {/ ml (SlashML)},
year = {2026},
month = {November},
url = {https://slashml.ai/research/redaction.html}
}