Active STM Tip Stabilisation with Reaction-Wheels + Residual RL

Scanning Tunnelling Microscopes rely on a ~10 nm tip hovering ~2 nm above a conductive surface. The smallest bump or drift damages the tip, forcing frequent, costly replacements and downtime. In some labs, tips last only hours and cost up to €500 each, which makes high-end STM work fragile and expensive. Current setups rely on passive isolation, which struggles with low-frequency room and building vibrations and with disturbances caused by operators. We set out to counter those disturbances actively, to extend tip life, improve stability and keep microscopes working instead of being re-tipped.

Space Needle adapts proven spacecraft attitude-control concepts to STM: a compact, multi-axis reaction-wheel head that generates precise counter-torques at the tip assembly, paired with a modern control stack that blends classical control with reinforcement learning.

On the software side, we built a physics-faithful simulator and training environment:

• Plant and actuation. Two-axis gimbal with four miniature reaction wheels; actuator bandwidth and slew-rate limits are modelled to reflect real hardware behaviour. Disturbances include randomised low-frequency sinusoidal torques to mimic lab vibrations.
• Controller architecture. A PD-like stabiliser provides a safe baseline; a PPO policy learns a small corrective “residual” on top for robustness to actuator lags and unmodelled effects. Either residual-RL or end-to-end learning can be selected.
• Engineering ergonomics. 1 kHz physics with 100–200 Hz control; deterministic seeding; termination on excessive tilt or wheel speed; a live MuJoCo viewer for interactive “poke” tests.

On the hardware strategy, Space Needle targets a down-scaled, gas-bearing reaction-wheel assembly. Gas bearings avoid magnetic interference near the STM junction while enabling ultra-low-friction, high-precision torqueing at the tip. The concept draws on ESA gas-bearing reaction-wheel know-how and is designed for transfer into STM head-stacks.

In software, Space Needle delivers an end-to-end workflow a lab can pick up quickly:

• Train a robust residual policy on a safe classical baseline.
• Visualise recovery after deliberate “torque pokes” and compare baseline vs. controlled RMS/peak tip motion.
• Enforce wheel-speed limits and anti-windup so the design stays physically realisable in a prototype.

Even a modest reduction in tip-to-sample contacts adds up quickly, because longer tip life cuts consumables and operator interruptions. For high-end instruments, doubling tip lifespan can mean material annual savings per device and more stable, predictable experiment time.