Reinforcement Learning Breakthrough Optimizes AI Chip Network Performance
A team of computer engineers has tackled the challenges of current Network-on-Chip (NoC) interconnect designs for large-scale AI models, focusing on Mixture-of-Experts (MoE) architectures. Their research introduces a novel approach to NoC synthesis and a new scoring system to predict performance under heavy loads.
The researchers found that large-scale model inference causes substantial data movement, leading to delays and potential performance violations in NoCs. To address this, they introduced an Interference Score (IS) to quantify worst-case interference between memory and compute traffic. They also developed a multi-objective optimization (MOO) approach to balance performance isolation and overall system throughput.
The team's topology generator, PARL, successfully reduces contention at the memory cut, meeting Service Level Agreements (SLAs) while minimizing worst-case slowdown. PARL, a Reinforcement Learning (RL) agent, discovers topologies that balance throughput and isolation, outperforming traditional NoCs under mixed MoE workloads. The team's optimization techniques, including genetic algorithms and simulated annealing, led to improvements in speed and efficiency, evaluated through metrics like latency, throughput, and energy efficiency.
The research, conducted on synthesizing NoC topologies for mixed deep learning workloads on chiplet-based accelerators, offers significant improvements in speed and efficiency. The team's novel Interference Score, MOO approach, and PARL topology generator successfully address the challenges of large-scale AI model inference in NoCs, paving the way for more efficient and performant systems.
