8000 Merge pull request #543 from hfhoffman1144/huggingface-transformers · realpython/materials@c14b862 · GitHub
[go: up one dir, main page]
Skip to content

Commit c14b862

Browse files
brendawelesbrendaweles
authored
Merge pull request #543 from hfhoffman1144/huggingface-transformers
Huggingface transformers
2 parents cfa3804 + de47200 commit c14b862

File tree

9 files changed

+13055
-0
lines changed

9 files changed

+13055
-0
lines changed

‎huggingface-transformers/README.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Hugging Face Transformers: Leverage Open-Source AI in Python
2+
3+
This folder contains the materials for the tutorial (Hugging Face Transformers: Leverage Open-Source AI in Python)[https://realpython.com/huggingface-transformers/].
4+
5+
Transformers is available on [PyPI](https://pypi.org/) and you can install it with [pip](https://realpython.com/what-is-pip/). Open a terminal or command prompt, create a new virtual environment, and then run the following command:
6+
7+
```console
8+
(venv) $ python -m pip install transformers
9+
```
10+
11+
This command will install the latest version of Transformers from PyPI onto your machine. Throughout this tutorial, you'll also leverage [PyTorch](https://realpython.com/pytorch-vs-tensorflow/) to interact with models at a lower level. You can install PyTorch with the following command:
12+
13+
```console
14+
(venv) $ python -m pip install torch
15+
```
16+
17+
To verify that the installations were successful, start a [Python REPL](https://realpython.com/python-repl/) and import `transformers` and `torch`:
18+
19+
```pycon
20+
>>> import transformers
21+
>>> import torch
22+
```
23+
24+
If the imports run without errors, then you've successfully installed the dependencies needed for this tutorial.

‎huggingface-transformers/auto_classes.py

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
import torch
2+
from transformers import (
3+
AutoConfig,
4+
AutoModelForSequenceClassification,
5+
AutoTokenizer,
6+
)
7+
8+
model_name = "cardiffnlp/twitter-roberta-base-sentiment-latest"
9+
10+
config = AutoConfig.from_pretrained(model_name)
11+
tokenizer = AutoTokenizer.from_pretrained(model_name)
12+
model = AutoModelForSequenceClassification.from_pretrained(model_name)
13+
14+
text = "I love using the Transformers library!"
15+
encoded_input = tokenizer(text, return_tensors="pt")
16+
17+
with torch.no_grad():
18+
output = model(**encoded_input)
19+
20+
scores = output.logits[0]
21+
probabilities = torch.softmax(scores, dim=0)
22+
23+
for i, prob in enumerate(probabilities):
24+
label = config.id2label[i]
25+
print(f"{i + 1}) {label}: {prob}")

‎huggingface-transformers/cars_data/Scraped_Car_Review_dodge.csv

Lines changed: 8500 additions & 0 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)
0