A3 Semantic Assist Milestone

Date: 2026-05-26

1. Milestone Status

The A3-side semantic assist decision path has been implemented and tested for the current v1 scope.

Current implemented entrypoints:

Entrypoint	Purpose	Status
`POST /api/v1/a2-a3/a3/assist`	Receive an A2 semantic packet and return the standard A3 response.	Implemented
`GET /api/v1/a2-a3/skills`	List backend-hosted A3 skills for OpenClaw or other agents.	Implemented
`POST /api/v1/a2-a3/skills/run`	Run `a3.assist_decision` through the backend skill registry.	Implemented
`POST /api/v1/a2-a3/demo`	Return offline A2 packets with A3 responses for demonstration.	Implemented

Primary code paths:

Area	File
Request/response schema	`A2A/app/schemas/a2_a3.py`
A3 deterministic policy	`A2A/app/services/a3_decision_policy.py`
A3 skill registry	`A2A/app/services/a3_skill_registry.py`
A2-A3 API router	`A2A/app/api/v1/a2_a3.py`
Existing tests	`A2A/tests/test_a3_decision_policy.py`
Offline experiment output	`gray_event_experiment/a2_a3_outputs/`

2. Point-by-Point Functional Analysis

Requirement	Implementation status	Evidence
1. Receive A2 semantic packet	Implemented. `A2SemanticPacket` accepts the requested packet shape. `timestamp`, `cycle`, `num`, and `sequence` are optional, so the minimal packet from A2 is valid.	`A2A/app/schemas/a2_a3.py`, `POST /api/v1/a2-a3/a3/assist`
A2 does not send full sensor logs	Implemented by contract. A3 only consumes risk level, pose, object state, evidence summaries, requested action, and time budget.	`A2SemanticPacket`, `docs/ontology_gray_events.md`
2. Reachability judgment	Implemented. A3 computes distance from current A3 pose to A2 end-effector pose, checks workspace bounds, max reach, safety distance, and configured blocked zones.	`decide_a3_assist`, `WorkspaceBounds`, `A3Capability`, `BlockedZone`
`reachable` and `distance_to_a2_mm` output	Implemented. The sample packet returns `reachable=true` and `distance_to_a2_mm=82.614`.	Unit test `test_sample_packet_returns_expected_assist_decision`
3. Time budget judgment	Implemented. A3 estimates time as `420 + distance_mm * 0.45` ms and compares it with `time_budget_ms`.	`decide_a3_assist`
Overtime should not blindly assist/takeover	Implemented. If assist/takeover would exceed budget, A3 returns `decision=manual_check`, `strategy=manual_check`, `result=needs_review`.	Unit test `test_overtime_assist_is_manual_check`
4. Assist/takeover decision by risk level	Implemented. `normal -> observe`, `gray -> assist`, `high -> takeover`, `critical -> takeover`.	Unit test `test_risk_level_mapping`
Unreachable case returns reject	Implemented. Outside workspace or max reach returns `decision=reject`, `strategy=manual_check`, `result=rejected`.	Unit test `test_unreachable_pose_is_rejected`; offline regression has 4 rejected packets
5. Concrete assist strategy	Implemented. Strategies include `continue_monitoring`, `reorient_and_stabilize_grasp`, `support_object`, `handover_and_continue_line`, `emergency_stop_or_safe_pose`, and `manual_check`.	`A3Strategy` enum and `_base_action`
6. Standard response format	Implemented. Responses are validated by `A3AssistResponse` and include all required fields.	`A3AssistResponse`; offline 24-packet schema regression
OpenClaw+skills access	Implemented as backend-hosted deterministic skill. OpenClaw Docker can call `a3.assist_decision` through HTTP and does not directly control robot action.	`A2A/app/services/a3_skill_registry.py`, `/skills/run`

3. Test Results

Commands run on /home/find-agent/a2a/A2A:

bash
python -m unittest tests.test_a3_decision_policy -v

Result:

text
Ran 6 tests in 0.001s
OK

Covered cases:

Test case	Result
Minimal sample packet returns expected assist decision.	Passed
Unreachable pose returns `reject`.	Passed
Overtime assist returns `manual_check`.	Passed
Risk mapping for `normal/gray/high/critical`.	Passed
Assist endpoint accepts the minimal packet without `timestamp/cycle/num`.	Passed
Skill endpoint `a3.assist_decision` returns the same decision shape.	Passed

Offline regression over gray_event_experiment/a2_a3_outputs/a2_semantic_packets.json:

Metric	Value
Packet count	24
Accepted	20
Rejected	4
Needs review	0
`assist` decisions	17
`takeover` decisions	3
`reject` decisions	4

Sample endpoint result for F004-A2-1-2:

json
{
  "request_id": "F004-A2-1-2",
  "agent": "Arm_A3",
  "reachable": true,
  "distance_to_a2_mm": 82.614,
  "estimated_time_ms": 457,
  "time_budget_ms": 1000,
  "within_time_budget": true,
  "decision": "assist",
  "strategy": "reorient_and_stabilize_grasp",
  "result": "accepted",
  "reason": "Gray F004 risk is reachable within the time budget; A3 can assist A2."
}

The direct assist endpoint and the skill endpoint returned identical output for the sample packet.

4. Milestones

Milestone	Scope	Status
M1: A2 semantic packet contract	Define and validate A2-to-A3 minimal semantic packet.	Complete
M2: A3 deterministic decision policy	Implement reachability, time budget, safety gate, risk-level decision, and strategy selection.	Complete
M3: Standard A3 response	Fix response fields and enums for `decision`, `strategy`, and `result`.	Complete
M4: OpenClaw skills access	Expose backend-hosted `a3.assist_decision` for OpenClaw HTTP calls.	Complete
M5: Offline regression data	Recompute and validate 24 A2 semantic packets with the shared A3 policy.	Complete
M6: Real A3 state integration	Replace static A3 pose/workspace/blocked-zone defaults with live robot state and motion-planning feedback.	Pending
M7: Collision model integration	Connect real obstacle, A2 arm geometry, object size, and scene occupancy to the safety gate.	Pending
M8: Runtime graph integration	Feed the A3 decision result back into the running multi-agent runtime state, not only the HTTP/offline path.	Pending

5. Remaining Engineering Notes

The v1 implementation is intentionally deterministic and conservative. It does not parse raw A2 sensor logs, and it does not let OpenClaw directly control robot motion.

Current limitations: