We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Describe the bug Currently, HALO converts the fill op into constants by generating random data at compile time.
With the increase of batch size, the bin file will grow dramatically for some model.
To Reproduce
For example, for Bert model, ./bin/halo bert_frozen_model.pb -target cxx -disable-broadcasting -batch-size=1 -o bert1.cc ./bin/halo bert_frozen_model.pb -target cxx -disable-broadcasting -batch-size=2 -o bert1.cc
./bin/halo bert_frozen_model.pb -target cxx -disable-broadcasting -batch-size=1 -o bert1.cc
./bin/halo bert_frozen_model.pb -target cxx -disable-broadcasting -batch-size=2 -o bert1.cc
The second command will generate larger .bin file.
Expected behavior The bin file size should remain constant.
The text was updated successfully, but these errors were encountered:
No branches or pull requests
Describe the bug
Currently, HALO converts the fill op into constants by generating random data at compile time.
With the increase of batch size, the bin file will grow dramatically for some model.
To Reproduce
For example, for Bert model,
./bin/halo bert_frozen_model.pb -target cxx -disable-broadcasting -batch-size=1 -o bert1.cc./bin/halo bert_frozen_model.pb -target cxx -disable-broadcasting -batch-size=2 -o bert1.ccThe second command will generate larger .bin file.
Expected behavior
The bin file size should remain constant.
The text was updated successfully, but these errors were encountered: