A GPU-efficient notebook platform for interactive deep learning training
Scroll to explore
The Problem
GPUs sit idle while notebooks run
Interactive notebook programming is universal in modern ML and AI workflows. To ensure responsiveness, platforms like Jupyter and Colab reserve GPUs for entire notebook sessions — even though GPU usage is intermittent and sporadic. The result: extremely low GPU utilization and prohibitively high costs.
The Root Cause
Long sessions, sporadic usage
Reserved but Unused
Notebook sessions reserve GPU resources for their entire lifetime, but actually use them for only brief training bursts.
Low Utilization
During most of their lifetimes, notebook sessions do not use the reserved GPUs — utilization can drop below 10%.
Prohibitive Costs
Cloud GPU time is expensive. Paying for idle reservations makes interactive deep learning training inaccessible at scale.
The Solution
NotebookOS
A GPU-efficient notebook platform tailored for the unique requirements of interactive deep learning training. NotebookOS decouples GPU allocation from notebook session lifetime, allocating GPUs only during active cell execution.
Architecture
Four pillars of GPU efficiency
Replicated Kernels
Each notebook kernel consists of 3 distinct replicas scheduled across multiple GPU servers and synchronized via the Raft consensus protocol.
On-Demand GPU Allocation
GPUs are dynamically allocated to kernel replicas only while they are actively executing notebook cells — not for the entire session.
Resource Oversubscription
NotebookOS oversubscribes server resources via kernel replication, leveraging the relatively high task inter-arrival times in IDLT workloads.
Replica Migration
Under high load, NotebookOS migrates kernel replicas and automatically scales the GPU cluster to maintain interactive responsiveness.
Results
Proven on production workloads
Evaluated on real-world interactive deep learning training workloads, NotebookOS delivers massive resource savings while improving the interactive experience.
GPU hours saved
Real-world IDLT workload
Raft-synchronized replicas
Implementation
Built on Kubernetes
NotebookOS is built on Kubernetes with custom operators that manage the lifecycle of replicated notebook kernels. The kernel replicas, scheduler, and migration controller are written in Go for performance, while the user-facing notebook interface remains fully compatible with standard Jupyter environments.