Ideaticon / Case Study / Softmax

Infrastructure that lets RL researchers move fast.

Softmax runs Metta, an open multiagent reinforcement learning research project. I did the engineering that research speed depends on: refactored the core training environment from a 2,000 line monolith into a clean class hierarchy with C++ bindings and standard framework adapters, built replay mining pipelines for offline RL, and stood up AWS sandboxes so external researchers can contribute without touching internal systems.

2,000 Line Monolith Refactored
3 Framework Adapters
AWS Researcher Sandboxes

A monolith was slowing down research.

A research lab lives or dies by iteration speed. When the core environment is a monolith, every experiment fights the codebase, and every external collaborator needs a guided tour before they can contribute. I rebuilt the foundations so researchers inside and outside the lab could just work.

A core environment researchers can actually use

The mettagrid training environment had grown to 2,000 lines. I refactored it into a deliberate class hierarchy: MettaGridCore wrapping the C++ bindings, framework adapters for Gym and PettingZoo, and MettaGridPufferBase for training. External researchers now get a clean, standard interface, and internal training logic stays isolated where it belongs.

Replay mining for offline RL

I built replay mining and dataset extraction pipelines over agent simulations, turning raw episodes into datasets for offline RL and behavioral analysis. Researchers can study emergent agent behavior from recorded play instead of burning compute to reproduce it.

Sandboxes for external researchers

I designed the AWS infrastructure that gives external researchers a safe place to work: OIDC authentication, SES email integration, and system health monitoring dashboards. Collaborators get real infrastructure on day one, and the lab keeps its internal systems private.

Packaging and CI that stay out of the way

I optimized Docker images for faster deployment, repaired the CI/CD pipelines, and moved the PufferLib dependency from a forked repository to PyPI. Routine work that determines whether researchers trust the build.

Impact

  • Core environment restructured into a maintainable class hierarchy with C++ bindings and Gym, PettingZoo and PufferLib interfaces
  • External researchers onboard against a clean, standard interface
  • Offline RL and behavioral analysis unlocked through replay mining and dataset extraction
  • Researcher sandboxes on AWS with OIDC authentication, email flows and health dashboards
  • Faster deployments from leaner Docker images and repaired CI/CD
  • Dependency management cleaned up by moving PufferLib from a fork to PyPI