Kimi AI and kvcache-ai Open Source AgentENV for Agentic RL Training

Agentic reinforcement learning (RL) extends far beyond simple text sampling by requiring models to take actions inside an actual computer system. Each rollout demands an isolated Linux environment complete with its own filesystem, network stack, and live processes. This requirement forces a difficult compromise between containers, which start rapidly but share the host kernel and weaken isolation, and full virtual machines, which isolate properly but boot slowly and hold memory while idle.

AgentENV targets this exact operational gap by running Firecracker microVMs and making idle states, restarts, and branching inexpensive enough for large-scale training workloads. Every sandbox functions as an independent microVM equipped with a dedicated Linux kernel, filesystem, and network namespace, managed via an Axum HTTP API that forwards requests to an underlying orchestrator.

Stock photo for illustration only, not from the actual event

Storage architecture leverages a ublk userspace block device backed by overlaybd layered images, where read-only base layers are shared and individual sandboxes write to distinct upper layers. A standout feature tailored specifically for RL is the Fork capability, allowing a running sandbox to clone itself into up to 16 independent child sandboxes on the same node instantly, inheriting all filesystem and memory configurations from the parent source.

Prerequisites for server deployment require Linux kernel 6.8+ alongside /dev/kvm access, with the installation script specifically requiring Ubuntu 24.04. The native CLI supports both Linux and macOS across x86_64 and arm64 architectures. Furthermore, AgentENV provides an E2B-compatible HTTP API, enabling engineering teams already utilizing E2B to self-host the runtime environment without making any modifications to their existing agent codebase.

Source: MarkTechPost