Updated the Model docs - for the ALIGN model #38072
Conversation
|
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the |
|
hey @stevhliu can you help me with this one? this is my first time making a PR here 🤗. Hit me up if you need me to make any further updates in the docs. |
docs/source/en/model_doc/align.md
Outdated
| - Pre-trained on 1.8B image-text pairs from the web | ||
|
|
||
| ```py | ||
| # Example of using ALIGN for image-text similarity |
There was a problem hiding this comment.
Indent this code example under the list item above
import torch
import requests
from PIL import Image
from transformers import AlignProcessor, AlignModel
processor = AlignProcessor.from_pretrained("kakaobrain/align-base")
model = AlignModel.from_pretrained("kakaobrain/align-base").to("cuda")
url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg"
image = requests.get(url, stream=True)
inputs = Image.open(image.raw).convert("RGB")
texts = ["a photo of a cat", "a photo of a dog"]
inputs = processor(images=inputs, text=texts, return_tensors="pt").to("cuda")
with torch.no_grad():
outputs = model(**inputs)
image_embeds = outputs.image_embeds
text_embeds = outputs.text_embeds
image_embeds = image_embeds / image_embeds.norm(dim=1, keepdim=True)
text_embeds = text_embeds / text_embeds.norm(dim=1, keepdim=True)
similarity_scores = torch.matmul(text_embeds, image_embeds.T)
print("Similarity scores:", similarity_scores)
probs = torch.nn.functional.softmax(similarity_scores, dim=0)
print("Probabilities:", probs)
most_similar_idx = similarity_scores.argmax().item()
print(f"Most similar text: '{texts[most_similar_idx]}'")
docs/source/en/model_doc/align.md
Outdated
|
|
||
| ## AlignConfig | ||
|
|
||
| [[autodoc]] AlignConfig | ||
| - from_text_vision_configs | ||
| [[autodoc]] AlignConfig - from_text_vision_configs |
There was a problem hiding this comment.
You don't need to format the [[autodoc] here, so you can revert these changes
There was a problem hiding this comment.
Changes to the [[autodoc]] haven't been reverted yet
There was a problem hiding this comment.
This still hasn't been reverted yet. It should look like:
[[autodoc]] AlignConfig
- from_text_vision_configsCo-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
|
Anything else? :) |
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
|
Now yeah, I think it looks good. |
docs/source/en/model_doc/align.md
Outdated
|
|
||
| ## AlignConfig | ||
|
|
||
| [[autodoc]] AlignConfig | ||
| - from_text_vision_configs | ||
| [[autodoc]] AlignConfig - from_text_vision_configs |
There was a problem hiding this comment.
This still hasn't been reverted yet. It should look like:
[[autodoc]] AlignConfig
- from_text_vision_configsCo-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
What does this PR do?
Fixes # (issue)
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.