@sharknice showed how to train StarCraft 2 unit micro from scratch using reinforcement learning with Sharky RL Matrix Training, a proof-of-concept project built on the Sharky framework. Starting with a simple Stalker vs two Roaches scenario, he walked through defining observations (cooldown, distance, velocity), balancing reward functions (end reward, cooldown reward, kite reward), and scaling up to training individual units in group battle contexts and even squad-level aggression strategy. The key insight: scope your action space down to preset functions your bot already knows, and RL becomes fast, affordable, and achievable.
Key Takeaways:
Scope your action space to preset bot functions — Instead of giving the AI infinite map positions to choose from, feed it the actions your bot already knows (attack closest, retreat, kite forward). A 2-action Stalker trains in under a minute; 5 actions takes 300 episodes
Rewards and observations are equally important — Observations tell the AI what’s happening; rewards tell it what’s good. Change rewards mid-training and it adapts; change observations and you start over from scratch
Use your existing framework as the training backbone — SharkNice uses his Sharky framework’s built-in micro modes (aggressive, passive, retreat) as the AI’s action set. If you’re on any framework, your existing functions become the AI’s vocabulary
Behavior cloning beats starting from scratch — @mindme 's best results came from having the AI watch an existing bot’s actions first, then further training with reinforcement learning. This mirrors how AlphaStar uses imitation learning as a baseline before RL
Start small, scale up — A single Stalker kiting two Roaches is enough to prove the concept. Individual unit training is fast; squad-level strategy training with full observation spaces takes much longer
Negative rewards shape behavior — Penalize the AI for not firing when it can (cooldown reward) and reward it for keeping distance while kiting. The combination of positive and negative rewards is what produces smart micro
Network multiple instances for faster training — RL Matrix supports networked agents sharing data across machines. One core bottlenecks StarCraft; spin up Docker containers to run 10+ instances simultaneously for a 10X speedup