Archive of posts tagged: Machine Learning
To Sparsify or To Quantize: A Hardware Architecture View
The debate of sparsity versus quantization has made its rounds in the ML optimization community for many years. Now, with the Generative AI revolution, the debate is intensifying. While these might both seem like simple mathematical approximations to an AI researcher,...
All in on MatMul? Don’t Put All Your Tensors in One Basket!
Matrix multiplication dominates AI hardware and research. Betting everything on MatMul risks an innovation monoculture — it’s time to diversify our compute bets.
A Computer Architect’s Guide to Designing Abstractions for Intelligent Systems
“The growing complexity of intelligent systems can outpace the ability of conventional computing abstractions to support them effectively.” IET REACH 2024, Amir Yazdanbakhsh. I came across this observation by Edsger Dijkstra recently while scrolling X, and...