8000 Add UTF-8 Encoding in read_text. by jm12138 · Pull Request #61 · abetlen/llama-cpp-python · GitHub
[go: up one dir, main page]
Skip to content

Add UTF-8 Encoding in read_text. #61

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Apr 10, 2023
Merged

Add UTF-8 Encoding in read_text. #61

merged 2 commits into from
Apr 10, 2023

Conversation

jm12138
Copy link
Contributor
@jm12138 jm12138 commented Apr 10, 2023 •
edited
Loading
  • When compiling and installing on a Chinese language Windows system, the following error occurs, because the default encoding of the Windows system of Chinese language is "GBK" instead of "UTF-8", and specifying the "UTF-8" encoding directly when reading the file can solve this problem. Installation on Windows failed in building wheel,UnicodeDecodeError #45
Traceback (most recent call last):
  File "C:\Users\Xpk22\miniconda3\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
    main()
  File "C:\Users\Xpk22\miniconda3\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
  File "C:\Users\Xpk22\miniconda3\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
    return hook(config_settings)
  File "C:\Users\Xpk22\AppData\Local\Temp\pip-build-env-vb8urq75\overlay\Lib\site-packages\setuptools\build_meta.py", line 338, in get_requires_for_build_wheel
    return self._get_build_requires(config_settings, requirements=['wheel'])
  File "C:\Users\Xpk22\AppData\Local\Temp\pip-build-env-vb8urq75\overlay\Lib\site-packages\setuptools\build_meta.py", line 320, in _get_build_requires
    self.run_setup()
  File "C:\Users\Xpk22\AppData\Local\Temp\pip-build-env-vb8urq75\overlay\Lib\site-packages\setuptools\build_meta.py", line 335, in run_setup
    exec(code, locals())
  File "<string>", line 6, in <module>
  File "C:\Users\Xpk22\miniconda3\lib\pathlib.py", line 1267, in read_text
    return f.read()
UnicodeDecodeError: 'gbk' codec can't decode byte 0xa6 in position 4: illegal multibyte sequence

abetlen
abetlen requested changes Apr 10, 2023
View reviewed changes
Copy link
Owner
@abetlen abetlen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this bugfix!

Can you please just update this line so it matches other encode calls in the project.

setup.py Outdated
@@ -3,7 +3,7 @@
from pathlib import Path

this_directory = Path(__file__).parent
long_description = (this_directory / "README.md").read_text()
long_description = (this_directory / "README.md").read_text(encoding='UTF-8')
Copy link
Owner
@abetlen abetlen Apr 10, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can you change this to

long_description = (this_directory / "README.md").read_text(encoding="utf-8")

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@abetlen abetlen merged commit a5554a2 into abetlen:main Apr 10, 2023
@abetlen abetlen mentioned this pull request Apr 11, 2023
Closed
@gjmulder gjmulder added the windows A Windoze-specific issue label May 18, 2023
xaptronic pushed a commit to xaptronic/llama-cpp-python that referenced this pull request Jun 13, 2023
@blackhole89
Add interactive mode (abetlen#61)
96ea727 
* Initial work on interactive mode.

* Improve interactive mode. Make rev. prompt optional.

* Update README to explain interactive mode.

* Fix OS X build
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@abetlen abetlen abetlen approved these changes

Assignees
No one assigned
Labels
windows A Windoze-specific issue
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jm12138 @abetlen @gjmulder
0