8000 clip : fix confused naming ffn_up and ffn_down by ngxson · Pull Request #13290 · ggml-org/llama.cpp · GitHub
[go: up one dir, main page]
Skip to content

clip : fix confused naming ffn_up and ffn_down #13290

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
May 5, 2025
Merged

clip : fix confused naming ffn_up and ffn_down #13290

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3ee071c
clip : fix confused naming ffn_up and ffn_down
ngxson May 3, 2025
3fbf0bd
rm ffn_i/o/g naming
ngxson May 3, 2025
f3870a6
rename n_embd, n_ff
ngxson May 3, 2025
ae83229
small fix
ngxson May 3, 2025
0009f76
Merge branch 'master' into xsn/clip_ffn_up_down_fix
ngxson May 3, 2025
246a4e0
no check n_ff
ngxson May 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
no check n_ff
  • Loading branch information
@ngxson
ngxson committed May 3, 2025
commit 246a4e0a0cf7b6b9da1c6a57a589416635414fce
5 changes: 2 additions & 3 deletions tools/llava/clip.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1877,9 +1877,8 @@ struct clip_model_loader {
layer.ff_down_b = get_tensor(string_format(TN_FFN_DOWN, "v", il, "bias"), false);

// some models already exported with legacy (incorrect) naming which is quite messy, let's fix it here
if (layer.ff_up_w && layer.ff_down_w
&& layer.ff_up_w->ne[0] == hparams.n_ff
&& layer.ff_down_w->ne[0] == hparams.n_embd) {
// note: Qwen model converted from the old surgery script has n_ff = 0, so we cannot use n_ff to check!
if (layer.ff_up_w && layer.ff_down_w && layer.ff_down_w->ne[0] == hparams.n_embd) {
// swap up and down weights
ggml_tensor * tmp = layer.ff_up_w;
layer.ff_up_w = layer.ff_down_w;
Expand Down
Loading
0