Actor Service API
The Actor service trains the policy model. It is deployed as a Ray Serve deployment with a FastAPI ingress, exposing HTTP endpoints for lifecycle management and recovery.
Overview
| Property | Value |
|---|---|
| Module | relax.components.actor |
| Deployment | @serve.deployment(max_ongoing_requests=10, max_queued_requests=20) |
| Ingress | FastAPI |
Execution Modes
- fully_async — Asynchronous training without waiting for rollout data. Weights are pushed to rollout engines after each step.
- sync — Waits for rollout data before each training step. Used in colocated mode.
Lifecycle
The Actor runs a background training loop that:
- Fetches data from
TransferQueueClient - Executes forward/backward/optimizer step via
RayTrainGroup - Pushes updated weights to rollout engines
- Saves checkpoints at configured intervals
HTTP Endpoints
Source
- Implementation:
relax/components/actor.py - Base class:
relax/components/base.py