支持BLOOM模型在消费级PC上的inference

[lockmatrix]

https://bigscience.huggingface.co/blog/bloom

[strint]

Is there a strong reason to run such a big model (176 billion parameters) on PC with CPU ?

[lockmatrix]

Is there a strong reason to run such a big model (176 billion parameters) on PC with CPU ?

I thought that supporting models beyond GPU memory or system memory capacity was the selling point of OneFlow.

分布式性能（高效性）是深度学习框架的核心技术难点，OneFlow围绕性能提升和异构分布式扩展，秉持静态编译和流式并行的核心理念和架构，解决了集群层面的内存墙挑战，技术水平世界领先。

I think if OneFlow can be the first to support the 100B model on PC, it will have a big advertising effect.

Any tool (likes BLOOM), if the cost of use is reduced by more than an order of magnitude, may have unexpected uses and widespread popularity

[lockmatrix]

Considering the price and speed, maybe the MacBook Pro M2 24G would be a good first target