Archive of posts tagged: Accelerators
To Sparsify or To Quantize: A Hardware Architecture View
The debate of sparsity versus quantization has made its rounds in the ML optimization community for many years. Now, with the Generative AI revolution, the debate is intensifying. While these might both seem like simple mathematical approximations to an AI researcher,...
PipeOrgan: Modeling Memory-Bandwidth-Bound Executions for AI and Beyond
TL;DR: Latency-tolerant architectures, e.g., GPUs, increasingly use memory/storage hierarchies, e.g., for KV Caches to speed Large-Language Model AI inference. To aid codesign of such workloads and architectures, we develop the simple PipeOrgan analytic model for...
Efficient GEMM Kernel Designs with Pipelining
General Matrix Multiplication (GEMM) is a fundamental operation in machine learning and scientific computing. It is the classic example of an algorithm that benefits greatly from GPU acceleration due to its high degree of data parallelism. More recently, efficient...
Reducing Embodied Carbon is Important
A recent post raises awareness of the challenges of reducing operational carbon, while also controversially challenging the importance of embodied carbon. We rebut the arguments raised against using embodied carbon as a design metric and conclude by advocating for more research on reducing embodied carbon.
Subscribe
