Hardware costs to drop by 8x after bitnet and Matmul free are adopted
Just my shower thought. What do you think?
https://arxiv.org/html/2406.02528v5
https://arxiv.org/html/2402.17764v1
Here is list of improvements:
- Less memory required, and or you can handle larger models
- 8x lower energy consumption
- Lower cost to train?
- Lower cost to serve a model
- Lower cost of hardware
- Lower Latency
- Improved throughput for model serving
- Quick speed of answers, similar to latency and throughput.
Posted Jun 27, 2024, 3:16 PM UTC
264101