8000 extend AutoModel to be able to load transformer models & custom diffusers models · Issue #11388 · huggingface/diffusers · GitHub
[go: up one dir, main page]

Skip to content
extend AutoModel to be able to load transformer models & custom diffusers models #11388
@yiyixuxu

Description

@yiyixuxu

current AutoModel implementation only support models that importable from diffusers

model_cls = getattr(library, orig_class_name, None)

it uses this field in model config.json https://huggingface.co/HiDream-ai/HiDream-I1-Full/blob/main/transformer/config.json#L2

we can use info in model_index.json instead, https://huggingface.co/HiDream-ai/HiDream-I1-Full/blob/main/model_index.json
this way we can load anything that can be load in from_pretrained, include transformer models & custom diffusers moduels (not directly importable from top level)

can reference the code/logic in from_pretrained

init_dict = {k: v for k, v in init_dict.items() if load_module(k, v)}

I will work on this shortly, but feel free to pick this up if anyone in the community is interested!

Metadata

Metadata

Assignees

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

    0