Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

litgpt chat crash at first char that differs from english encodind with error : UnicodeDecodeError: #1953

Open
navr32 opened this issue Mar 8, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@navr32
Copy link

navr32 commented Mar 8, 2025

Bug description

Hi ! I want just try litgpt and start to just do simple inference with doc found here.

So i start with simple chat :

litgpt chat microsoft/phi-2

And at first char not english language crash of many line with :

/litgtp/lib/python3.11/site-packages/litgpt/chat/base.py", line 120, in interact
prompt = input(">> Prompt: ")
^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 in position 53: invalid continuation byte

So this is the utf8 witch is bad handle in your code I think. Because i have never this error with all the python
app i work on ..pytorch..and so and so..

I have add the header for utf8 encodind to the base.py and now this work but not for all the case.;sometime
the unicodedecodeerrror come again. Perhaps you will find a better system to handle well the utf8 encodind ?
Or perhaps add the # -- coding: utf-8 -- to all your header python files ?

So to have this work about good i have add this to line 2:

# -*- coding: utf-8 -*-

Many thanks for litgpt look like very promising.

What operating system are you using?

Linux

LitGPT Version

Using Python 3.11.11 environment at: 
Version: 0.5.7
@navr32 navr32 added the bug Something isn't working label Mar 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant