In partnership with
Earn a master's in AI for under $2,500
AI skills are no longer optionalātheyāre essential for staying competitive in todayās workforce. Now you can earn a fully accredited Master of Science in Artificial Intelligence from the Udacity Institute of AI and Technology, awarded by Woolf, an accredited higher education institution.
This 100% online, flexible program is designed for working professionals and can be completed for under $5,000. Youāll build deep, practical expertise in modern AI, machine learning, generative models, and production deployment through real-world projects that demonstrate job-ready skills.
Learn on your schedule, apply what you build immediately, and graduate with a credential that signals serious AI capability. This is one of the most accessible ways to earn a graduate-level AI degree and accelerate your career.
Paper: mHC: Manifold-Constrained Hyper-Connections
Authors: Zhenda Xie, Yixuan Wei, Huanqi Cao, et al. (DeepSeek-AI)
Published: December 31, 2025 | arXiv:2512.24880v1
Why Hyper-Connections Crashed at Scale
(and How DeepSeek Fixed It)
A visual guide to understanding mHC
ā” 30-Second Summary
DeepSeek tried adding more "lanes" for information to flow through neural networks. It made models smarter but kept crashing. They discovered the lanes were flooding each other. The fix: a rule that keeps every lane balanced. Result: +7 points on reasoning, no more crashes.
š Think of It Like a Highway
Every AI model (GPT-4, Claude, Gemini) processes information through layers. Since 2016, all models use a single "lane" for information to travel through. It works, but it's limited.
š„ The Problem: Signals Explode
When DeepSeek tried Hyper-Connections on a 27-billion parameter model, it crashed at step 12,000. Why? Without rules, small imbalances compound across 60 layers.
That's a 2,000Ć difference! The unconstrained version lets signals grow exponentially. The constrained version keeps them nearly flat.
ā The Fix: A Simple Balancing Rule
DeepSeek's insight: you need traffic rules. Their rule is elegant ā think of it like a budget:
This is what mathematicians call "doubly stochastic" ā but you don't need to remember that. Just remember: what goes out = what comes in.
š Training: Before vs After
Here's what training actually looked like:
š The Results
On 27-billion parameter models, mHC consistently outperformed the baseline:
| Benchmark | Baseline | mHC | Gain |
|---|---|---|---|
| BBH (reasoning) | 43.8 | 51.0 | +7.2 |
| DROP (comprehension) | 47.0 | 53.9 | +6.9 |
| GSM8K (math) | 46.7 | 53.8 | +7.1 |
| MMLU (knowledge) | 59.0 | 63.4 | +4.4 |
| HellaSwag | 73.7 | 74.7 | +1.0 |
| PIQA | 78.5 | 80.5 | +2.0 |
| TriviaQA | 54.3 | 57.6 | +3.3 |
š° The Cost-Benefit
šÆ Key Takeaways
Residual connections haven't changed since 2016.
Hyper-Connections crashed because signals exploded 3000Ć.
A simple balancing rule prevents all overflow/starvation.
+7 points on reasoning, stable training throughout.
Likely powering their next generation of models.
Read the Paper: arXiv:2512.24880
Related: Hyper-Connections | DeepSeek-V3