Skip to content

Enterprise Operational Model & The Golden Path

This document defines the Operational Specificity required to run the Agent Safety Framework in a mission-critical environment. It moves beyond "tooling" and into the constraints and workflows that ensure architectural integrity.

1. The "Golden Path" for Automated Remediation

A. Delivery Flow (Signal to Consensus)

sequenceDiagram
    participant Log as Cloud Logging
    participant Az as SafetyAnalyzer
    participant Am as Model Armor
    participant Ag as Agent Fleet
    participant Cg as ConsensusGuardian

    Log->>Az: Trigger (SLO Burn)
    Az->>Am: Raw Finding
    Am->>Am: Redact Secrets/PII
    Am->>Az: Sanitized Finding
    Az->>Ag: Broadcast Proposal
    Ag->>Ag: Verify Finding
    Ag->>Cg: RSA-PSS Signed Payload
    Cg->>Cg: Verify Quorum (2/3)
    Cg->>Az: Verified Quorum Payload

B. Governance & Actuation (Consensus to Verification)

sequenceDiagram
    participant Az as SafetyAnalyzer
    participant Sg as Safety Gate
    participant Ac as Remediator (Actuator)
    participant Log as Cloud Logging

    Az->>Sg: Verified Proposal
    Sg->>Sg: Check Quotas & Cost
    alt Pass
        Sg->>Ac: Authorize Action
        Ac->>Ac: Patch GKE/Cloud Run
        Ac->>Log: Log Success (Audit)
    else Fail (Failure Path)
        Sg->>Log: Log Blocked Event
        Sg->>Az: Return 403 Forbidden
    end

2. Failure Path Engineering: Control Semantics

A governable architecture is defined by how it handles the edge cases where autonomy fails. We define four Exceptional States with explicit actor-trigger-exit conditions and IAM-bound permission boundaries.

State Actor Trigger Exit Mechanism Audit Artifact
Frozen Safety Gate Cost/Quota violation SRE_APPROVAL_TOKEN (via Secret Manager) GATE_BLOCK_EVENT
Force-Apply Human SRE Critical system block BREAK_GLASS_IAM_ROLE (Signed Override) SEC_ADMIN_OVERRIDE_LOG
Quarantine Model Armor > 10% Redaction rate GitOps POLICY_COMMIT (2-Person Review) MODEL_QUARANTINE_ALERT
Fail-Safe Consensus Quorum timeout CLOUD_CONSOLE_RESET (Org Admin only) SYSTEM_FAILSAFE_LOCK

3. Recovery Orchestration & Audit Chains

We distinguish between automated self-healing and mandatory human intervention, ensuring every transition is backed by a non-repudiable audit record.

Scenario Automatic Recovery Human-in-the-Loop Audit Artifact Chain
Minor SLO Burn Scale-up via RSA-PSS Quorum Payload. Observability notification only. CONSENSUS_COMPLETE -> ACTUATION_SUCCESS
Rollback Failure System Lock (Read-Only mode). Manual state reconciliation. ROLLBACK_FAILURE_ALERT -> SRE_INCIDENT_LOG
Identity Drift Automated Key Rotation (30d). Emergency rotation on leakage. SECRET_MANAGER_ROTATION_EVENT
High-Risk Op Proposal generation + Verification. Mandatory Console Approval. APPROVAL_REQUEST -> IAM_IDENTITY_TOKEN
Authenticity Risk Advisory signaling (Low/Med). Mandatory SRE Approval (High). AUTH_RISK_AUDIT -> SRE_VERIFICATION_LOG

4. Automated Rollback Strategy

Autonomous actions are only traceable if they are reversible. - State Snapshots: Before the Remediator applies a patch, it stores the Pre-Action Configuration in a versioned GCS bucket. - Verification Window: After actuation, the system enters a 10-minute Stabilization Window. - If the SLO burn rate does not decrease, the system triggers an Automated Rollback to the snapshot. - Fail-Safe Mode: If the rollback itself fails, the system triggers a PagerDuty Escalation and locks the Safety Gate.

3. Human-in-the-Loop (HITL) Approval Gates

Not all actions are suitable for full autonomy. - High-Risk Thresholds: Any remediation with a GateResult.risk_score > 0.8 (e.g., database schema changes, global traffic shifts) requires a manual approval in the GCP Console or via a signed Slack integration. - Override Protocol: SREs can manually "Freeze" the Safety Gate via a single environment variable (SAFETY_MODE=READ_ONLY), which immediately suspends all autonomous actuation fleet-wide.

4. Multi-Tenancy & Identity Propagation

  • Isolation: Each business unit (e.g., "Payments," "Search") runs its own isolated Consensus Plane.
  • Identity: Agents use OAuth 2.0 scopes limited strictly to the resources they govern. A "Payments Agent" has zero IAM permissions to view or modify "Search" resources.
  • Source of Truth: IAM is managed via Terraform with VPC Service Controls (VPC-SC) enforced to prevent data exfiltration.

5. Reliability Metrics (SLOs)

We measure the Safety Framework with the same rigor as production services:

SLI Description Target
Safety Latency Time from log event to Actuation pass/fail < 120s
RCA Accuracy % of AI-identified root causes that match human audit > 85%
False Rejection Rate % of safe remediation proposals blocked by the Gate < 5%
Unauthorized Action Rate % of state-changes without valid consensus 0%