MemCut evaluates whether deletion leaves enough clues for an AI agent to reconstruct a secret. The preprint follows evidence trails across 456 fictional secrets and distinguishes ordinary retrieval from recovery using surviving memory objects. In the tested systems, additional deletion still leaves recoverable information. The single study concerns text memories rather than real users’ leaked data.
MemCut, a benchmark for requested forgetting in AI agents’ long-term memory, tests whether a secret remains inferable after deletion. The October 15 arXiv preprint uses 456 fictional secrets spread through fifteen long conversation episodes. Researchers design the evidence relationships and use a model to render them as dialogue. This single study contains no real personal data.[1]
Native recovery asks a question through the memory system’s own retrieval. Residual recovery instead gives the same reader model surviving memory objects traced to annotated evidence. Ordinary retrieval can fail even when the remaining fragments still allow an inference. Testing combines four language models with five memory designs. It tracks direct disclosures alongside indirect clues and distracting material that resembles useful evidence.[1]
The authors report 39–76 per cent residual recovery after removing the explicit disclosure and using a model to delete ten additional turns chosen from twenty candidates. A smaller experimental subset finds benefits from widening the candidate pool with the deletion budget fixed. The figures describe the tested simulations. Its scope is confined to textual memories. Forgetting within model weights and memories spanning several media types fall outside the experiments, which also leave unrelated content lost through deletion incompletely measured.[1]
