Navigate Harbor
Harbor runs post-training work inside your environment. It observes each candidate and the infrastructure carrying it, recovers failed jobs, applies your evaluation gates, and returns a decision record for human sign-off.
Read run state, model signals, cluster health, and approved new work.
Turn the experiment contract into candidate jobs, policies, and evaluation work.
Run training and evaluation jobs on infrastructure inside the customer environment.
Recover infrastructure failures, restore checkpoints, or prune a bad trajectory.
Invoke the customer evaluation harness and apply task, safety, and latency gates.
Return an accepted candidate and complete operations record for human sign-off.
Autonomy is defined in policy, not implied by the word operator. The inspected implementation can recommend actions or operate within policy, while deployment remains gated on a person.
Experiment policy
The operator coordinates the systems already responsible for training, infrastructure state, checkpoints, evaluation, and identity. Harbor adds the decision logic and the durable operating record between them.
Harbor reads experiment state beside Kubernetes, Slurm, DCGM, NCCL, Prometheus, and run metrics. The operator can distinguish a bad training trajectory from an infrastructure fault because both are in the same record.
The reference operator identifies a failed GPU, quarantines its node, validates the last checkpoint, rebuilds the training allocation, and resumes the candidate. Every action is written to an actor-attributed operations log.
Harbor invokes the customer evaluation command, applies blocking criteria, and returns the accepted and rejected candidates with evidence. The implementation keeps deployment behind a required human sign-off.
The reference implementation keeps training data, checkpoints, and model weights local. Harbor connects with scoped permissions to the systems that observe jobs, execute approved actions, store state, and run evaluations.
AI Infrastructure Operator
