8000 Fix Llama.close didn't free lora adapter by jkawamoto · Pull Request #1679 · abetlen/llama-cpp-python · GitHub
[go: up one dir, main page]
Skip to content

Fix Llama.close didn't free lora adapter #1679

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Aug 15, 2024
Merged

Fix Llama.close didn't free lora adapter #1679

merged 1 commit into from
Aug 15, 2024

Conversation

jkawamoto
Copy link
Contributor

Llama.close didn't free the LoRA, but Llama.__del__ does. This PR moves the freeing of the LoRA to the ExitStack that we already use to free the model, context, and other resources.

Additionally, this PR moves the initialization of the ExitStack to the top of __init__, ensuring that _stack is always present. As a result, we no longer need to check whether _stack exists or is None.

@abetlen abetlen merged commit 3c7501b into abetlen:main Aug 15, 2024
13 checks passed
@jkawamoto jkawamoto deleted the free-lora-adapter branch August 15, 2024 22:27
benniekiss pushed a commit to benniekiss/llama-cpp-python that referenced this pull request Aug 23, 2024
@jkawamoto
fix: Llama.close didn't free lora adapter (abetlen#1679)
d77c322
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jkawamoto @abetlen
0