Security: Agent self-modification patterns in Canvas memory module (ASI10)

[dockfixlabs]

Description

The Canvas memory module at autogen_ext/memory/canvas/_canvas.py contains patterns where the agent can modify its own code and state during execution. Self-modification is a critical security boundary violation — it allows an agent to evolve its behavior beyond its original constraints, potentially escaping safeguards or executing arbitrary operations.

Location

python/packages/autogen-ext/src/autogen_ext/memory/canvas/_canvas.py
python/packages/autogen-ext/src/autogen_ext/memory/canvas/_text_canvas.py

Impact

An agent with memory modification capability can alter its own operating parameters, tool access patterns, or behavioral constraints. This is particularly dangerous in multi-agent systems where one compromised agent can modify shared memory state affecting all agents.

Severity

HIGH — self-modifying agent code violates security invariants.

Suggested Mitigation

1. Separate read-only memory from mutable agent state
2. Audit all self.* modifications within agent execution context
3. Consider append-only memory patterns for auditability
4. Add agent integrity checks before and after memory modifications

Scanner

Found by AgentGuard v0.6.1 — open-source AI agent security scanner.

[Tuttotorna]

The important boundary here is:

memory mutation authority != agent self-modification authority

An agent may be allowed to write memory, but that should not automatically authorize mutation of its own execution constraints, tool access, operating parameters, or shared governance state.

I would separate memory writes into explicit transition classes:

1. append_only_observation

   • safe audit/event memory
   • no overwrite of prior state

2. mutable_working_state

   • allowed only inside a scoped workspace
   • no policy/tool/identity mutation

3. governance_state_change

   • changes constraints, tool access, permissions, routing, identity, or execution policy
   • requires explicit external authorization

The dangerous case is when a normal memory write can silently become class 3.

A deterministic guard could require every memory mutation to declare:

{
"transition_id": "...",
"agent_id": "...",
"memory_target": "...",
"mutation_type": "append | update | delete | policy_change | tool_access_change",
"affected_scope": "local | shared | governance",
"purpose": "...",
"policy_ref": "...",
"pre_state_hash": "sha256:...",
"post_state_hash": "sha256:...",
"authorization_ref": "..."
}

Then allow the mutation only if:

Required(memory_mutation_transition)
⊆
Supported(policy_ref + scope_authorization + mutation_class + pre_state_hash + post_state_hash)

If the mutation affects governance state, tool access, routing, identity, or execution constraints, the default should be fail-closed unless there is explicit external authorization.

Regression matrix I would add:

• append-only observation => allow
• local working-state update within scope => allow
• overwrite of audit history => reject
• mutation of tool access without authorization => reject
• mutation of system/developer constraints => reject
• shared memory modification without scope binding => reject
• policy/governance state change from inside agent execution => reject
• post-state hash missing after mutation => reject or quarantine

The key fix is not only “audit self.* modifications”.

The stronger invariant is:

agent memory writes are allowed only when the mutation class is explicitly supported for that exact transition.

Use-first diagnostic profile for this transition class:
https://github.com/Tuttotorna/PHI-OMEGA-RUNTIME

Apply it to:

Required(τ) ⊄ Supported(τ)

where τ = agent memory mutation that changes execution constraints, tool access, shared state, or governance state.