added load_progress_callback to common_params #13617
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
llama internally supports load cancellation, but when using common* API it's not available from outside:
to allow load cancellation it needs a
progress_callback
, but it requires passingcommon_params
and internallyllama_model_params
builds the object always with progress_callback as null.this PR simply provides an opportunity to set it from outside. however I called it load_progress_callback because the callback is used during model loading. user will still need to call llama_set_abort_callback for compute cancellation, but it can be done after calling common_init_from_params().