Haru: keep self-hosted inference up, without idle GPUs.

Haru is a free, open-source GPU orchestrator for teams serving LLMs on their own GPUs. It solves the standby problem: staying available when a region fails usually means paying for a second set of GPUs that sit idle. With Haru, they train while they wait.

Alpha · Free & open source on GitHub

View on GitHub

What Haru does.

Hot failover

Haru runs your GPUs as an Active/Standby pair in two failure domains: different regions, or different clouds. When the active side fails, the standby takes over serving in seconds.

No idle GPUs

A classic hot standby bills you for GPUs that do nothing. On Haru, the standby keeps its inference servers asleep with weights parked in CPU RAM, and uses the freed VRAM to run LoRA training until it is needed.

Verified, not scripted

Failover is a sequence of checked state transitions: stop training, verify the VRAM is free, wake the servers, probe every model, then move routing in one atomic step. A failed step never moves traffic.

Haru Cloud: fully managed fleets.

Soon

The same open-source Haru, operated by Arkor: provisioning, supervision, and failover watched around the clock, without running the control plane yourself.

More from the toolkit.

Arkor

Get a hosted endpoint, collect traffic, fine-tune, serve your weights.

Learn more

Framework

Code-first SDK for fine-tuning open-weight models.

Learn more

Code Review

LLM code review that catches stale comments and drifted docs.

Learn more

Confidential ComputingSoon

Cloud inference and training with the confidentiality of running locally.

Learn more