generate_tflite_from_whisper.ipynb does no longer generate valid whisper-tiny.en.tflite model

[ytsnko]

https://github.com/nyadla-sys/whisper.tflite/blob/main/models/generate_tflite_from_whisper.ipynb

TFLite model is created without errors, but on the last cell where tflite is validated signature runner throws an error:

RuntimeError Traceback (most recent call last)

in <cell line: 6>()
4
5 tflite_generate = interpreter.get_signature_runner()
----> 6 generated_ids = tflite_generate(input_features=input_features)["sequences"]
7 transcription = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
8 transcription

/usr/local/lib/python3.10/dist-packages/tensorflow/lite/python/interpreter.py in call(self, **kwargs)
247 self._subgraph_index)
248
--> 249 self._interpreter_wrapper.Invoke(self._subgraph_index)
250 result = {}
251 for output_name, output_index in self._outputs:

RuntimeError: tensorflow/lite/kernels/reduce.cc:445 std::apply(optimized_ops::Mean<T, U>, args) was not true.tensorflow/lite/kernels/reduce.cc:445 std::apply(optimized_ops::Mean<T, U>, args) was not true.tensorflow/lite/kernels/reduce.cc:445 std::appl...

I've tried to reduce max_new_tokens to 223 and that fixes the error, but the resulting model is recognizing garbage now:

this this, this this, this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this this, this this this this this this this this this this this this this this this this this, this this this this this this this this this this, this this this this this this this this, this this this this this this this, this this this this this this, this this, this this, this this this this this, this this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this this, this, this, this, this this, this, this, this, this,

[jb-ahn]

Hi, have you resolved this issue?
Because, I am also trying to make tflite by "generate_tflite_from_whisper.ipynb".
But I got the same problem.

I've changed
max_new_tokens=448
because of same error:
RuntimeError: tensorflow/lite/kernels/reduce.cc:445 std::apply...

And the output is like below
"'s's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's's'soneoneoneoneoneoneoneon...

And generated_ids is like below
array([[50257, 50362, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 338, 505, 505, 505, 505, 505, 505, 505, 505, 505, 505, 505, 272, 505, 272, 505, 272, 272, 272, 272, 272, 272, 272, 272, 272, 272, 272, 272, 272, 272, 272, 272, 272, 272, 272, 272, 272, ...

[jb-ahn]

In my case,
It worked well in case of {python 3.9 + tensorflow 2,14}.
At both {python 3.8 + tensorflow 2.13} and {python 3.10 + tensorflow 2.17 ; Google Colab}, it didn't work well.
I don't know why...

[nyadla-sys]

Can you make the changes to notebook and change the PR ?

[jb-ahn]

Hi @nyadla-sys ,
The only thing I've changed is removing
max_new_tokens=450

And, I just changed the version python & tensorflow in my system.