You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{"payload":{"allShortcutsEnabled":false,"path":"docs","repo":{"id":634747119,"defaultBranch":"main","name":"transformers","ownerLogin":"githubhjs","currentUserCanPush":false,"isFork":true,"isEmpty":false,"createdAt":"2023-05-01T04:36:10.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/19849?v=4","public":true,"private":false,"isOrgOwned":false},"currentUser":null,"refInfo":{"name":"c2c99dc7ef5edab8f7674a1eb00cf6ac6996fd0f","listCacheKey":"v0:1682915777.8764749","canEdit":false,"refType":"tree","currentOid":"c2c99dc7ef5edab8f7674a1eb00cf6ac6996fd0f"},"tree":{"items":[{"name":"source","path":"docs/source","contentType":"directory"},{"name":"README.md","path":"docs/README.md","contentType":"file"},{"name":"TRANSLATING.md","path":"docs/TRANSLATING.md","contentType":"file"}],"templateDirectorySuggestionUrl":null,"readme":{"displayName":"README.md","richText":"\u003carticle class=\"markdown-body entry-content container-lg\" itemprop=\"text\"\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch1 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eGenerating the documentation\u003c/h1\u003e\u003ca id=\"user-content-generating-the-documentation\" class=\"anchor\" aria-label=\"Permalink: Generating the documentation\" href=\"#generating-the-documentation\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eTo generate the documentation, you first have to build it. Several packages are necessary to build the doc,\nyou can install them with the following command, at the root of the code repository:\u003c/p\u003e\n\u003cdiv class=\"highlight highlight-source-shell notranslate position-relative overflow-auto\" dir=\"auto\" data-snippet-clipboard-copy-content=\"pip install -e \u0026quot;.[docs]\u0026quot;\"\u003e\u003cpre\u003epip install -e \u003cspan class=\"pl-s\"\u003e\u003cspan class=\"pl-pds\"\u003e\"\u003c/span\u003e.[docs]\u003cspan class=\"pl-pds\"\u003e\"\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eThen you need to install our special tool that builds the documentation:\u003c/p\u003e\n\u003cdiv class=\"highlight highlight-source-shell notranslate position-relative overflow-auto\" dir=\"auto\" data-snippet-clipboard-copy-content=\"pip install git+https://github.com/huggingface/doc-builder\"\u003e\u003cpre\u003epip install git+https://github.com/huggingface/doc-builder\u003c/pre\u003e\u003c/div\u003e\n\u003chr\u003e\n\u003cp dir=\"auto\"\u003e\u003cstrong\u003eNOTE\u003c/strong\u003e\u003c/p\u003e\n\u003cp dir=\"auto\"\u003eYou only need to generate the documentation to inspect it locally (if you're planning changes and want to\ncheck how they look before committing for instance). You don't have to commit the built documentation.\u003c/p\u003e\n\u003chr\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch2 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eBuilding the documentation\u003c/h2\u003e\u003ca id=\"user-content-building-the-documentation\" class=\"anchor\" aria-label=\"Permalink: Building the documentation\" href=\"#building-the-documentation\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eOnce you have setup the \u003ccode\u003edoc-builder\u003c/code\u003e and additional packages, you can generate the documentation by\ntyping the following command:\u003c/p\u003e\n\u003cdiv class=\"highlight highlight-source-shell notranslate position-relative overflow-auto\" dir=\"auto\" data-snippet-clipboard-copy-content=\"doc-builder build transformers docs/source/en/ --build_dir ~/tmp/test-build\"\u003e\u003cpre\u003edoc-builder build transformers docs/source/en/ --build_dir \u003cspan class=\"pl-k\"\u003e~\u003c/span\u003e/tmp/test-build\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eYou can adapt the \u003ccode\u003e--build_dir\u003c/code\u003e to set any temporary folder that you prefer. This command will create it and generate\nthe MDX files that will be rendered as the documentation on the main website. You can inspect them in your favorite\nMarkdown editor.\u003c/p\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch2 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003ePreviewing the documentation\u003c/h2\u003e\u003ca id=\"user-content-previewing-the-documentation\" class=\"anchor\" aria-label=\"Permalink: Previewing the documentation\" href=\"#previewing-the-documentation\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eTo preview the docs, first install the \u003ccode\u003ewatchdog\u003c/code\u003e module with:\u003c/p\u003e\n\u003cdiv class=\"highlight highlight-source-shell notranslate position-relative overflow-auto\" dir=\"auto\" data-snippet-clipboard-copy-content=\"pip install watchdog\"\u003e\u003cpre\u003epip install watchdog\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eThen run the following command:\u003c/p\u003e\n\u003cdiv class=\"highlight highlight-source-shell notranslate position-relative overflow-auto\" dir=\"auto\" data-snippet-clipboard-copy-content=\"doc-builder preview {package_name} {path_to_docs}\"\u003e\u003cpre\u003edoc-builder preview {package_name} {path_to_docs}\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eFor example:\u003c/p\u003e\n\u003cdiv class=\"highlight highlight-source-shell notranslate position-relative overflow-auto\" dir=\"auto\" data-snippet-clipboard-copy-content=\"doc-builder preview transformers docs/source/en/\"\u003e\u003cpre\u003edoc-builder preview transformers docs/source/en/\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eThe docs will be viewable at \u003ca href=\"http://localhost:3000\" rel=\"nofollow\"\u003ehttp://localhost:3000\u003c/a\u003e. You can also preview the docs once you have opened a PR. You will see a bot add a comment to a link where the documentation with your changes lives.\u003c/p\u003e\n\u003chr\u003e\n\u003cp dir=\"auto\"\u003e\u003cstrong\u003eNOTE\u003c/strong\u003e\u003c/p\u003e\n\u003cp dir=\"auto\"\u003eThe \u003ccode\u003epreview\u003c/code\u003e command only works with existing doc files. When you add a completely new file, you need to update \u003ccode\u003e_toctree.yml\u003c/code\u003e \u0026amp; restart \u003ccode\u003epreview\u003c/code\u003e command (\u003ccode\u003ectrl-c\u003c/code\u003e to stop it \u0026amp; call \u003ccode\u003edoc-builder preview ...\u003c/code\u003e again).\u003c/p\u003e\n\u003chr\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch2 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eAdding a new element to the navigation bar\u003c/h2\u003e\u003ca id=\"user-content-adding-a-new-element-to-the-navigation-bar\" class=\"anchor\" aria-label=\"Permalink: Adding a new element to the navigation bar\" href=\"#adding-a-new-element-to-the-navigation-bar\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eAccepted files are Markdown (.md or .mdx).\u003c/p\u003e\n\u003cp dir=\"auto\"\u003eCreate a file with its extension and put it in the source directory. You can then link it to the toc-tree by putting\nthe filename without the extension in the \u003ca href=\"https://github.com/huggingface/transformers/blob/main/docs/source/_toctree.yml\"\u003e\u003ccode\u003e_toctree.yml\u003c/code\u003e\u003c/a\u003e file.\u003c/p\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch2 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eRenaming section headers and moving sections\u003c/h2\u003e\u003ca id=\"user-content-renaming-section-headers-and-moving-sections\" class=\"anchor\" aria-label=\"Permalink: Renaming section headers and moving sections\" href=\"#renaming-section-headers-and-moving-sections\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eIt helps to keep the old links working when renaming the section header and/or moving sections from one document to another. This is because the old links are likely to be used in Issues, Forums, and Social media and it'd make for a much more superior user experience if users reading those months later could still easily navigate to the originally intended information.\u003c/p\u003e\n\u003cp dir=\"auto\"\u003eTherefore, we simply keep a little map of moved sections at the end of the document where the original section was. The key is to preserve the original anchor.\u003c/p\u003e\n\u003cp dir=\"auto\"\u003eSo if you renamed a section from: \"Section A\" to \"Section B\", then you can add at the end of the file:\u003c/p\u003e\n\u003cdiv class=\"snippet-clipboard-content notranslate position-relative overflow-auto\" data-snippet-clipboard-copy-content=\"Sections that were moved:\n\n[ \u0026lt;a href=\u0026quot;#section-b\u0026quot;\u0026gt;Section A\u0026lt;/a\u0026gt;\u0026lt;a id=\u0026quot;section-a\u0026quot;\u0026gt;\u0026lt;/a\u0026gt; ]\"\u003e\u003cpre class=\"notranslate\"\u003e\u003ccode\u003eSections that were moved:\n\n[ \u0026lt;a href=\"#section-b\"\u0026gt;Section A\u0026lt;/a\u0026gt;\u0026lt;a id=\"section-a\"\u0026gt;\u0026lt;/a\u0026gt; ]\n\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eand of course, if you moved it to another file, then:\u003c/p\u003e\n\u003cdiv class=\"snippet-clipboard-content notranslate position-relative overflow-auto\" data-snippet-clipboard-copy-content=\"Sections that were moved:\n\n[ \u0026lt;a href=\u0026quot;../new-file#section-b\u0026quot;\u0026gt;Section A\u0026lt;/a\u0026gt;\u0026lt;a id=\u0026quot;section-a\u0026quot;\u0026gt;\u0026lt;/a\u0026gt; ]\"\u003e\u003cpre class=\"notranslate\"\u003e\u003ccode\u003eSections that were moved:\n\n[ \u0026lt;a href=\"../new-file#section-b\"\u0026gt;Section A\u0026lt;/a\u0026gt;\u0026lt;a id=\"section-a\"\u0026gt;\u0026lt;/a\u0026gt; ]\n\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eUse the relative style to link to the new file so that the versioned docs continue to work.\u003c/p\u003e\n\u003cp dir=\"auto\"\u003eFor an example of a rich moved section set please see the very end of \u003ca href=\"https://github.com/huggingface/transformers/blob/main/docs/source/en/main_classes/trainer.mdx\"\u003ethe Trainer doc\u003c/a\u003e.\u003c/p\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch2 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eWriting Documentation - Specification\u003c/h2\u003e\u003ca id=\"user-content-writing-documentation---specification\" class=\"anchor\" aria-label=\"Permalink: Writing Documentation - Specification\" href=\"#writing-documentation---specification\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eThe \u003ccode\u003ehuggingface/transformers\u003c/code\u003e documentation follows the\n\u003ca href=\"https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html\" rel=\"nofollow\"\u003eGoogle documentation\u003c/a\u003e style for docstrings,\nalthough we can write them directly in Markdown.\u003c/p\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch3 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eAdding a new tutorial\u003c/h3\u003e\u003ca id=\"user-content-adding-a-new-tutorial\" class=\"anchor\" aria-label=\"Permalink: Adding a new tutorial\" href=\"#adding-a-new-tutorial\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eAdding a new tutorial or section is done in two steps:\u003c/p\u003e\n\u003cul dir=\"auto\"\u003e\n\u003cli\u003eAdd a new file under \u003ccode\u003e./source\u003c/code\u003e. This file can either be ReStructuredText (.rst) or Markdown (.md).\u003c/li\u003e\n\u003cli\u003eLink that file in \u003ccode\u003e./source/_toctree.yml\u003c/code\u003e on the correct toc-tree.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp dir=\"auto\"\u003eMake sure to put your new file under the proper section. It's unlikely to go in the first section (\u003cem\u003eGet Started\u003c/em\u003e), so\ndepending on the intended targets (beginners, more advanced users, or researchers) it should go in sections two, three, or\nfour.\u003c/p\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch3 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eTranslating\u003c/h3\u003e\u003ca id=\"user-content-translating\" class=\"anchor\" aria-label=\"Permalink: Translating\" href=\"#translating\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eWhen translating, refer to the guide at \u003ca href=\"https://github.com/huggingface/transformers/blob/main/docs/TRANSLATING.md\"\u003e./TRANSLATING.md\u003c/a\u003e.\u003c/p\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch3 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eAdding a new model\u003c/h3\u003e\u003ca id=\"user-content-adding-a-new-model\" class=\"anchor\" aria-label=\"Permalink: Adding a new model\" href=\"#adding-a-new-model\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eWhen adding a new model:\u003c/p\u003e\n\u003cul dir=\"auto\"\u003e\n\u003cli\u003eCreate a file \u003ccode\u003exxx.mdx\u003c/code\u003e or under \u003ccode\u003e./source/model_doc\u003c/code\u003e (don't hesitate to copy an existing file as template).\u003c/li\u003e\n\u003cli\u003eLink that file in \u003ccode\u003e./source/_toctree.yml\u003c/code\u003e.\u003c/li\u003e\n\u003cli\u003eWrite a short overview of the model:\n\u003cul dir=\"auto\"\u003e\n\u003cli\u003eOverview with paper \u0026amp; authors\u003c/li\u003e\n\u003cli\u003ePaper abstract\u003c/li\u003e\n\u003cli\u003eTips and tricks and how to use it best\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eAdd the classes that should be linked in the model. This generally includes the configuration, the tokenizer, and\nevery model of that class (the base model, alongside models with additional heads), both in PyTorch and TensorFlow.\nThe order is generally:\n\u003cul dir=\"auto\"\u003e\n\u003cli\u003eConfiguration,\u003c/li\u003e\n\u003cli\u003eTokenizer\u003c/li\u003e\n\u003cli\u003ePyTorch base model\u003c/li\u003e\n\u003cli\u003ePyTorch head models\u003c/li\u003e\n\u003cli\u003eTensorFlow base model\u003c/li\u003e\n\u003cli\u003eTensorFlow head models\u003c/li\u003e\n\u003cli\u003eFlax base model\u003c/li\u003e\n\u003cli\u003eFlax head models\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp dir=\"auto\"\u003eThese classes should be added using our Markdown syntax. Usually as follows:\u003c/p\u003e\n\u003cdiv class=\"snippet-clipboard-content notranslate position-relative overflow-auto\" data-snippet-clipboard-copy-content=\"## XXXConfig\n\n[[autodoc]] XXXConfig\"\u003e\u003cpre class=\"notranslate\"\u003e\u003ccode\u003e## XXXConfig\n\n[[autodoc]] XXXConfig\n\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eThis will include every public method of the configuration that is documented. If for some reason you wish for a method\nnot to be displayed in the documentation, you can do so by specifying which methods should be in the docs:\u003c/p\u003e\n\u003cdiv class=\"snippet-clipboard-content notranslate position-relative overflow-auto\" data-snippet-clipboard-copy-content=\"## XXXTokenizer\n\n[[autodoc]] XXXTokenizer\n - build_inputs_with_special_tokens\n - get_special_tokens_mask\n - create_token_type_ids_from_sequences\n - save_vocabulary\"\u003e\u003cpre class=\"notranslate\"\u003e\u003ccode\u003e## XXXTokenizer\n\n[[autodoc]] XXXTokenizer\n - build_inputs_with_special_tokens\n - get_special_tokens_mask\n - create_token_type_ids_from_sequences\n - save_vocabulary\n\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eIf you just want to add a method that is not documented (for instance magic methods like \u003ccode\u003e__call__\u003c/code\u003e are not documented\nby default) you can put the list of methods to add in a list that contains \u003ccode\u003eall\u003c/code\u003e:\u003c/p\u003e\n\u003cdiv class=\"snippet-clipboard-content notranslate position-relative overflow-auto\" data-snippet-clipboard-copy-content=\"## XXXTokenizer\n\n[[autodoc]] XXXTokenizer\n - all\n - __call__\"\u003e\u003cpre class=\"notranslate\"\u003e\u003ccode\u003e## XXXTokenizer\n\n[[autodoc]] XXXTokenizer\n - all\n - __call__\n\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch3 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eWriting source documentation\u003c/h3\u003e\u003ca id=\"user-content-writing-source-documentation\" class=\"anchor\" aria-label=\"Permalink: Writing source documentation\" href=\"#writing-source-documentation\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eValues that should be put in \u003ccode\u003ecode\u003c/code\u003e should either be surrounded by backticks: `like so`. Note that argument names\nand objects like True, None, or any strings should usually be put in \u003ccode\u003ecode\u003c/code\u003e.\u003c/p\u003e\n\u003cp dir=\"auto\"\u003eWhen mentioning a class, function, or method, it is recommended to use our syntax for internal links so that our tool\nadds a link to its documentation with this syntax: [`XXXClass`] or [`function`]. This requires the class or\nfunction to be in the main package.\u003c/p\u003e\n\u003cp dir=\"auto\"\u003eIf you want to create a link to some internal class or function, you need to\nprovide its path. For instance: [`utils.ModelOutput`]. This will be converted into a link with\n\u003ccode\u003eutils.ModelOutput\u003c/code\u003e in the description. To get rid of the path and only keep the name of the object you are\nlinking to in the description, add a ~: [`~utils.ModelOutput`] will generate a link with \u003ccode\u003eModelOutput\u003c/code\u003e in the description.\u003c/p\u003e\n\u003cp dir=\"auto\"\u003eThe same works for methods so you can either use [`XXXClass.method`] or [~`XXXClass.method`].\u003c/p\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch4 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eDefining arguments in a method\u003c/h4\u003e\u003ca id=\"user-content-defining-arguments-in-a-method\" class=\"anchor\" aria-label=\"Permalink: Defining arguments in a method\" href=\"#defining-arguments-in-a-method\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eArguments should be defined with the \u003ccode\u003eArgs:\u003c/code\u003e (or \u003ccode\u003eArguments:\u003c/code\u003e or \u003ccode\u003eParameters:\u003c/code\u003e) prefix, followed by a line return and\nan indentation. The argument should be followed by its type, with its shape if it is a tensor, a colon, and its\ndescription:\u003c/p\u003e\n\u003cdiv class=\"snippet-clipboard-content notranslate position-relative overflow-auto\" data-snippet-clipboard-copy-content=\" Args:\n n_layers (`int`): The number of layers of the model.\"\u003e\u003cpre class=\"notranslate\"\u003e\u003ccode\u003e Args:\n n_layers (`int`): The number of layers of the model.\n\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eIf the description is too long to fit in one line, another indentation is necessary before writing the description\nafter the argument.\u003c/p\u003e\n\u003cp dir=\"auto\"\u003eHere's an example showcasing everything so far:\u003c/p\u003e\n\u003cdiv class=\"snippet-clipboard-content notranslate position-relative overflow-auto\" data-snippet-clipboard-copy-content=\" Args:\n input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):\n Indices of input sequence tokens in the vocabulary.\n\n Indices can be obtained using [`AlbertTokenizer`]. See [`~PreTrainedTokenizer.encode`] and\n [`~PreTrainedTokenizer.__call__`] for details.\n\n [What are input IDs?](../glossary#input-ids)\"\u003e\u003cpre class=\"notranslate\"\u003e\u003ccode\u003e Args:\n input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):\n Indices of input sequence tokens in the vocabulary.\n\n Indices can be obtained using [`AlbertTokenizer`]. See [`~PreTrainedTokenizer.encode`] and\n [`~PreTrainedTokenizer.__call__`] for details.\n\n [What are input IDs?](../glossary#input-ids)\n\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eFor optional arguments or arguments with defaults we follow the following syntax: imagine we have a function with the\nfollowing signature:\u003c/p\u003e\n\u003cdiv class=\"snippet-clipboard-content notranslate position-relative overflow-auto\" data-snippet-clipboard-copy-content=\"def my_function(x: str = None, a: float = 1):\"\u003e\u003cpre class=\"notranslate\"\u003e\u003ccode\u003edef my_function(x: str = None, a: float = 1):\n\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003ethen its documentation should look like this:\u003c/p\u003e\n\u003cdiv class=\"snippet-clipboard-content notranslate position-relative overflow-auto\" data-snippet-clipboard-copy-content=\" Args:\n x (`str`, *optional*):\n This argument controls ...\n a (`float`, *optional*, defaults to 1):\n This argument is used to ...\"\u003e\u003cpre class=\"notranslate\"\u003e\u003ccode\u003e Args:\n x (`str`, *optional*):\n This argument controls ...\n a (`float`, *optional*, defaults to 1):\n This argument is used to ...\n\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eNote that we always omit the \"defaults to `None`\" when None is the default for any argument. Also note that even\nif the first line describing your argument type and its default gets long, you can't break it on several lines. You can\nhowever write as many lines as you want in the indented description (see the example above with \u003ccode\u003einput_ids\u003c/code\u003e).\u003c/p\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch4 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eWriting a multi-line code block\u003c/h4\u003e\u003ca id=\"user-content-writing-a-multi-line-code-block\" class=\"anchor\" aria-label=\"Permalink: Writing a multi-line code block\" href=\"#writing-a-multi-line-code-block\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eMulti-line code blocks can be useful for displaying examples. They are done between two lines of three backticks as usual in Markdown:\u003c/p\u003e\n\u003cdiv class=\"snippet-clipboard-content notranslate position-relative overflow-auto\" data-snippet-clipboard-copy-content=\"```\n# first line of code\n# second line\n# etc\n```\"\u003e\u003cpre class=\"notranslate\"\u003e\u003ccode\u003e```\n# first line of code\n# second line\n# etc\n```\n\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eWe follow the \u003ca href=\"https://docs.python.org/3/library/doctest.html\" rel=\"nofollow\"\u003edoctest\u003c/a\u003e syntax for the examples to automatically test\nthe results to stay consistent with the library.\u003c/p\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch4 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eWriting a return block\u003c/h4\u003e\u003ca id=\"user-content-writing-a-return-block\" class=\"anchor\" aria-label=\"Permalink: Writing a return block\" href=\"#writing-a-return-block\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u00
A0B8
3cp dir=\"auto\"\u003eThe return block should be introduced with the \u003ccode\u003eReturns:\u003c/code\u003e prefix, followed by a line return and an indentation.\nThe first line should be the type of the return, followed by a line return. No need to indent further for the elements\nbuilding the return.\u003c/p\u003e\n\u003cp dir=\"auto\"\u003eHere's an example of a single value return:\u003c/p\u003e\n\u003cdiv class=\"snippet-clipboard-content notranslate position-relative overflow-auto\" data-snippet-clipboard-copy-content=\" Returns:\n `List[int]`: A list of integers in the range [0, 1] --- 1 for a special token, 0 for a sequence token.\"\u003e\u003cpre class=\"notranslate\"\u003e\u003ccode\u003e Returns:\n `List[int]`: A list of integers in the range [0, 1] --- 1 for a special token, 0 for a sequence token.\n\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eHere's an example of a tuple return, comprising several objects:\u003c/p\u003e\n\u003cdiv class=\"snippet-clipboard-content notranslate position-relative overflow-auto\" data-snippet-clipboard-copy-content=\" Returns:\n `tuple(torch.FloatTensor)` comprising various elements depending on the configuration ([`BertConfig`]) and inputs:\n - ** loss** (*optional*, returned when `masked_lm_labels` is provided) `torch.FloatTensor` of shape `(1,)` --\n Total loss is the sum of the masked language modeling loss and the next sequence prediction (classification) loss.\n - **prediction_scores** (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`) --\n Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).\"\u003e\u003cpre class=\"notranslate\"\u003e\u003ccode\u003e Returns:\n `tuple(torch.FloatTensor)` comprising various elements depending on the configuration ([`BertConfig`]) and inputs:\n - ** loss** (*optional*, returned when `masked_lm_labels` is provided) `torch.FloatTensor` of shape `(1,)` --\n Total loss is the sum of the masked language modeling loss and the next sequence prediction (classification) loss.\n - **prediction_scores** (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`) --\n Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).\n\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch4 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eAdding an image\u003c/h4\u003e\u003ca id=\"user-content-adding-an-image\" class=\"anchor\" aria-label=\"Permalink: Adding an image\" href=\"#adding-an-image\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eDue to the rapidly growing repository, it is important to make sure that no files that would significantly weigh down the repository are added. This includes images, videos, and other non-text files. We prefer to leverage a hf.co hosted \u003ccode\u003edataset\u003c/code\u003e like\nthe ones hosted on \u003ca href=\"https://huggingface.co/hf-internal-testing\" rel=\"nofollow\"\u003e\u003ccode\u003ehf-internal-testing\u003c/code\u003e\u003c/a\u003e in which to place these files and reference\nthem by URL. We recommend putting them in the following dataset: \u003ca href=\"https://huggingface.co/datasets/huggingface/documentation-images\" rel=\"nofollow\"\u003ehuggingface/documentation-images\u003c/a\u003e.\nIf an external contribution, feel free to add the images to your PR and ask a Hugging Face member to migrate your images\nto this dataset.\u003c/p\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch2 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eStyling the docstring\u003c/h2\u003e\u003ca id=\"user-content-styling-the-docstring\" class=\"anchor\" aria-label=\"Permalink: Styling the docstring\" href=\"#styling-the-docstring\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eWe have an automatic script running with the \u003ccode\u003emake style\u003c/code\u003e comment that will make sure that:\u003c/p\u003e\n\u003cul dir=\"auto\"\u003e\n\u003cli\u003ethe docstrings fully take advantage of the line width\u003c/li\u003e\n\u003cli\u003eall code examples are formatted using black, like the code of the Transformers library\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp dir=\"auto\"\u003eThis script may have some weird failures if you made a syntax mistake or if you uncover a bug. Therefore, it's\nrecommended to commit your changes before running \u003ccode\u003emake style\u003c/code\u003e, so you can revert the changes done by that script\neasily.\u003c/p\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch1 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eTesting documentation examples\u003c/h1\u003e\u003ca id=\"user-content-testing-documentation-examples\" class=\"anchor\" aria-label=\"Permalink: Testing documentation examples\" href=\"#testing-documentation-examples\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eGood documentation often comes with an example of how a specific function or class should be used.\nEach model class should contain at least one example showcasing\nhow to use this model class in inference. \u003cem\u003eE.g.\u003c/em\u003e the class \u003ca href=\"https://huggingface.co/docs/transformers/model_doc/wav2vec2#transformers.Wav2Vec2ForCTC\" rel=\"nofollow\"\u003eWav2Vec2ForCTC\u003c/a\u003e\nincludes an example of how to transcribe speech to text in the\n\u003ca href=\"https://huggingface.co/docs/transformers/model_doc/wav2vec2#transformers.Wav2Vec2ForCTC.forward\" rel=\"nofollow\"\u003edocstring of its forward function\u003c/a\u003e.\u003c/p\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch2 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eWriting documentation examples\u003c/h2\u003e\u003ca id=\"user-content-writing-documentation-examples\" class=\"anchor\" aria-label=\"Permalink: Writing documentation examples\" href=\"#writing-documentation-examples\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eThe syntax for Example docstrings can look as follows:\u003c/p\u003e\n\u003cdiv class=\"snippet-clipboard-content notranslate position-relative overflow-auto\" data-snippet-clipboard-copy-content=\" Example:\n\n ```python\n \u0026gt;\u0026gt;\u0026gt; from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC\n \u0026gt;\u0026gt;\u0026gt; from datasets import load_dataset\n \u0026gt;\u0026gt;\u0026gt; import torch\n\n \u0026gt;\u0026gt;\u0026gt; dataset = load_dataset(\u0026quot;hf-internal-testing/librispeech_asr_demo\u0026quot;, \u0026quot;clean\u0026quot;, split=\u0026quot;validation\u0026quot;)\n \u0026gt;\u0026gt;\u0026gt; dataset = dataset.sort(\u0026quot;id\u0026quot;)\n \u0026gt;\u0026gt;\u0026gt; sampling_rate = dataset.features[\u0026quot;audio\u0026quot;].sampling_rate\n\n \u0026gt;\u0026gt;\u0026gt; processor = Wav2Vec2Processor.from_pretrained(\u0026quot;facebook/wav2vec2-base-960h\u0026quot;)\n \u0026gt;\u0026gt;\u0026gt; model = Wav2Vec2ForCTC.from_pretrained(\u0026quot;facebook/wav2vec2-base-960h\u0026quot;)\n\n \u0026gt;\u0026gt;\u0026gt; # audio file is decoded on the fly\n \u0026gt;\u0026gt;\u0026gt; inputs = processor(dataset[0][\u0026quot;audio\u0026quot;][\u0026quot;array\u0026quot;], sampling_rate=sampling_rate, return_tensors=\u0026quot;pt\u0026quot;)\n \u0026gt;\u0026gt;\u0026gt; with torch.no_grad():\n ... logits = model(**inputs).logits\n \u0026gt;\u0026gt;\u0026gt; predicted_ids = torch.argmax(logits, dim=-1)\n\n \u0026gt;\u0026gt;\u0026gt; # transcribe speech\n \u0026gt;\u0026gt;\u0026gt; transcription = processor.batch_decode(predicted_ids)\n \u0026gt;\u0026gt;\u0026gt; transcription[0]\n 'MISTER QUILTER IS THE APOSTLE OF THE MIDDLE CLASSES AND WE ARE GLAD TO WELCOME HIS GOSPEL'\n ```\"\u003e\u003cpre class=\"notranslate\"\u003e\u003ccode\u003e Example:\n\n ```python\n \u0026gt;\u0026gt;\u0026gt; from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC\n \u0026gt;\u0026gt;\u0026gt; from datasets import load_dataset\n \u0026gt;\u0026gt;\u0026gt; import torch\n\n \u0026gt;\u0026gt;\u0026gt; dataset = load_dataset(\"hf-internal-testing/librispeech_asr_demo\", \"clean\", split=\"validation\")\n \u0026gt;\u0026gt;\u0026gt; dataset = dataset.sort(\"id\")\n \u0026gt;\u0026gt;\u0026gt; sampling_rate = dataset.features[\"audio\"].sampling_rate\n\n \u0026gt;\u0026gt;\u0026gt; processor = Wav2Vec2Processor.from_pretrained(\"facebook/wav2vec2-base-960h\")\n \u0026gt;\u0026gt;\u0026gt; model = Wav2Vec2ForCTC.from_pretrained(\"facebook/wav2vec2-base-960h\")\n\n \u0026gt;\u0026gt;\u0026gt; # audio file is decoded on the fly\n \u0026gt;\u0026gt;\u0026gt; inputs = processor(dataset[0][\"audio\"][\"array\"], sampling_rate=sampling_rate, return_tensors=\"pt\")\n \u0026gt;\u0026gt;\u0026gt; with torch.no_grad():\n ... logits = model(**inputs).logits\n \u0026gt;\u0026gt;\u0026gt; predicted_ids = torch.argmax(logits, dim=-1)\n\n \u0026gt;\u0026gt;\u0026gt; # transcribe speech\n \u0026gt;\u0026gt;\u0026gt; transcription = processor.batch_decode(predicted_ids)\n \u0026gt;\u0026gt;\u0026gt; transcription[0]\n 'MISTER QUILTER IS THE APOSTLE OF THE MIDDLE CLASSES AND WE ARE GLAD TO WELCOME HIS GOSPEL'\n ```\n\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eThe docstring should give a minimal, clear example of how the respective model\nis to be used in inference and also include the expected (ideally sensible)\noutput.\nOften, readers will try out the example before even going through the function\nor class definitions. Therefore, it is of utmost importance that the example\nworks as expected.\u003c/p\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch2 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eDocstring testing\u003c/h2\u003e\u003ca id=\"user-content-docstring-testing\" class=\"anchor\" aria-label=\"Permalink: Docstring testing\" href=\"#docstring-testing\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eTo do so each example should be included in the doctests.\nWe use pytests' \u003ca href=\"https://docs.pytest.org/doctest.html\" rel=\"nofollow\"\u003edoctest integration\u003c/a\u003e to verify that all of our examples run correctly.\nFor Transformers, the doctests are run on a daily basis via GitHub Actions as can be\nseen \u003ca href=\"https://github.com/huggingface/transformers/actions/workflows/doctests.yml\"\u003ehere\u003c/a\u003e.\u003c/p\u003e\n\u003cp dir=\"auto\"\u003eTo include your example in the daily doctests, you need to add the filename that\ncontains the example docstring to the \u003ca href=\"/githubhjs/transformers/blob/c2c99dc7ef5edab8f7674a1eb00cf6ac6996fd0f/utils/documentation_tests.txt\"\u003edocumentation_tests.txt\u003c/a\u003e.\u003c/p\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch3 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eFor Python files\u003c/h3\u003e\u003ca id=\"user-content-for-python-files\" class=\"anchor\" aria-label=\"Permalink: For Python files\" href=\"#for-python-files\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eYou will first need to run the following command (from the root of the repository) to prepare the doc file (doc-testing needs to add additional lines that we don't include in the doc source files):\u003c/p\u003e\n\u003cdiv class=\"highlight highlight-source-shell notranslate position-relative overflow-auto\" dir=\"auto\" data-snippet-clipboard-copy-content=\"python utils/prepare_for_doc_test.py src docs\"\u003e\u003cpre\u003epython utils/prepare_for_doc_test.py src docs\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eIf you work on a specific python module, say \u003ccode\u003emodeling_wav2vec2.py\u003c/code\u003e, you can run the command as follows (to avoid the unnecessary temporary changes in irrelevant files):\u003c/p\u003e\n\u003cdiv class=\"highlight highlight-source-shell notranslate position-relative overflow-auto\" dir=\"auto\" data-snippet-clipboard-copy-content=\"python utils/prepare_for_doc_test.py src/transformers/utils/doc.py src/transformers/models/wav2vec2/modeling_wav2vec2.py\"\u003e\u003cpre\u003epython utils/prepare_for_doc_test.py src/transformers/utils/doc.py src/transformers/models/wav2vec2/modeling_wav2vec2.py\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003e(\u003ccode\u003eutils/doc.py\u003c/code\u003e should always be included)\u003c/p\u003e\n\u003cp dir=\"auto\"\u003eThen you can run all the tests in the docstrings of a given file with the following command, here is how we test the modeling file of Wav2Vec2 for instance:\u003c/p\u003e\n\u003cdiv class=\"highlight highlight-source-shell notranslate position-relative overflow-auto\" dir=\"auto\" data-snippet-clipboard-copy-content=\"pytest --doctest-modules src/transformers/models/wav2vec2/modeling_wav2vec2.py -sv --doctest-continue-on-failure\"\u003e\u003cpre\u003epytest --doctest-modules src/transformers/models/wav2vec2/modeling_wav2vec2.py -sv --doctest-continue-on-failure\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eIf you want to isolate a specific docstring, just add \u003ccode\u003e::\u003c/code\u003e after the file name then type the whole path of the function/class/method whose docstring you want to test. For instance, here is how to just test the forward method of \u003ccode\u003eWav2Vec2ForCTC\u003c/code\u003e:\u003c/p\u003e\n\u003cdiv class=\"highlight highlight-source-shell notranslate position-relative overflow-auto\" dir=\"auto\" data-snippet-clipboard-copy-content=\"pytest --doctest-modules src/transformers/models/wav2vec2/modeling_wav2vec2.py::transformers.models.wav2vec2.modeling_wav2vec2.Wav2Vec2ForCTC.forward -sv --doctest-continue-on-failure\"\u003e\u003cpre\u003epytest --doctest-modules src/transformers/models/wav2vec2/modeling_wav2vec2.py::transformers.models.wav2vec2.modeling_wav2vec2.Wav2Vec2ForCTC.forward -sv --doctest-continue-on-failure\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eOnce you're done, you can run the following command (still from the root of the repository) to undo the changes made by the first command before committing:\u003c/p\u003e\n\u003cdiv class=\"highlight highlight-source-shell notranslate position-relative overflow-auto\" dir=\"auto\" data-snippet-clipboard-copy-content=\"python utils/prepare_for_doc_test.py src docs --remove_new_line\"\u003e\u003cpre\u003epython utils/prepare_for_doc_test.py src docs --remove_new_line\u003c/pre\u003e\u003c/div\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch3 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eFor Markdown files\u003c/h3\u003e\u003ca id=\"user-content-for-markdown-files\" class=\"anchor\" aria-label=\"Permalink: For Markdown files\" href=\"#for-markdown-files\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eYou will first need to run the following command (from the root of the repository) to prepare the doc file (doc-testing needs to add additional lines that we don't include in the doc source files):\u003c/p\u003e\n\u003cdiv class=\"highlight highlight-source-shell notranslate position-relative overflow-auto\" dir=\"auto\" data-snippet-clipboard-copy-content=\"python utils/prepare_for_doc_test.py src docs\"\u003e\u003cpre\u003epython utils/prepare_for_doc_test.py src docs\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eThen you can test locally a given file with this command (here testing the quicktour):\u003c/p\u003e\n\u003cdiv class=\"highlight highlight-source-shell notranslate position-relative overflow-auto\" dir=\"auto\" data-snippet-clipboard-copy-content=\"pytest --doctest-modules docs/source/quicktour.mdx -sv --doctest-continue-on-failure --doctest-glob=\u0026quot;*.mdx\u0026quot;\"\u003e\u003cpre\u003epytest --doctest-modules docs/source/quicktour.mdx -sv --doctest-continue-on-failure --doctest-glob=\u003cspan class=\"pl-s\"\u003e\u003cspan class=\"pl-pds\"\u003e\"\u003c/span\u003e*.mdx\u003cspan class=\"pl-pds\"\u003e\"\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eOnce you're done, you can run the following command (still from the root of the repository) to undo the changes made by the first command before committing:\u003c/p\u003e\n\u003cdiv class=\"highlight highlight-source-shell notranslate position-relative overflow-auto\" dir=\"auto\" data-snippet-clipboard-copy-content=\"python utils/prepare_for_doc_test.py src docs --remove_new_line\"\u003e\u003cpre\u003epython utils/prepare_for_doc_test.py src docs --remove_new_line\u003c/pre\u003e\u003c/div\u003e\n\u003cdiv class=\"markdown-heading\" dir=\"auto\"\u003e\u003ch3 tabindex=\"-1\" class=\"heading-element\" dir=\"auto\"\u003eWriting doctests\u003c/h3\u003e\u003ca id=\"user-content-writing-doctests\" class=\"anchor\" aria-label=\"Permalink: Writing doctests\" href=\"#writing-doctests\"\u003e\u003csvg class=\"octicon octicon-link\" viewBox=\"0 0 16 16\" version=\"1.1\" width=\"16\" height=\"16\" aria-hidden=\"true\"\u003e\u003cpath d=\"m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z\"\u003e\u003c/path\u003e\u003c/svg\u003e\u003c/a\u003e\u003c/div\u003e\n\u003cp dir=\"auto\"\u003eHere are a few tips to help you debug the doctests and make them pass:\u003c/p\u003e\n\u003cul dir=\"auto\"\u003e\n\u003cli\u003eThe outputs of the code need to match the expected output \u003cstrong\u003eexactly\u003c/strong\u003e, so make sure you have the same outputs. In particular doctest will see a difference between single quotes and double quotes, or a missing parenthesis. The only exceptions to that rule are:\n\u003cul dir=\"auto\"\u003e\n\u003cli\u003ewhitespace: one give whitespace (space, tabulation, new line) is equivalent to any number of whitespace, so you can add new lines where there are spaces to make your output more readable.\u003c/li\u003e\n\u003cli\u003enumerical values: you should never put more than 4 or 5 digits to expected results as different setups or library versions might get you slightly different results. \u003ccode\u003edoctest\u003c/code\u003e is configured to ignore any difference lower than the precision to which you wrote (so 1e-4 if you write 4 digits).\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eDon't leave a block of code that is very long to execute. If you can't make it fast, you can either not use the doctest syntax on it (so that it's ignored), or if you want to use the doctest syntax to show the results, you can add a comment \u003ccode\u003e# doctest: +SKIP\u003c/code\u003e at the end of the lines of code too long to execute\u003c/li\u003e\n\u003cli\u003eEach line of code that produces a result needs to have that result written below. You can ignore an output if you don't want to show it in your code example by adding a comment \u003ccode\u003e # doctest: +IGNORE_RESULT\u003c/code\u003e at the end of the line of code producing it.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/article\u003e","errorMessage":null,"headerInfo":{"toc":[{"level":1,"text":"Generating the documentation","anchor":"generating-the-documentation","htmlText":"Generating the documentation"},{"level":2,"text":"Building the documentation","anchor":"building-the-documentation","htmlText":"Building the documentation"},{"level":2,"text":"Previewing the documentation","anchor":"previewing-the-documentation","htmlText":"Previewing the documentation"},{"level":2,"text":"Adding a new element to the navigation bar","anchor":"adding-a-new-element-to-the-navigation-bar","htmlText":"Adding a new element to the navigation bar"},{"level":2,"text":"Renaming section headers and moving sections","anchor":"renaming-section-headers-and-moving-sections","htmlText":"Renaming section headers and moving sections"},{"level":2,"text":"Writing Documentation - Specification","anchor":"writing-documentation---specification","htmlText":"Writing Documentation - Specification"},{"level":3,"text":"Adding a new tutorial","anchor":"adding-a-new-tutorial","htmlText":"Adding a new tutorial"},{"level":3,"text":"Translating","anchor":"translating","htmlText":"Translating"},{"level":3,"text":"Adding a new model","anchor":"adding-a-new-model","htmlText":"Adding a new model"},{"level":3,"text":"Writing source documentation","anchor":"writing-source-documentation","htmlText":"Writing source documentation"},{"level":4,"text":"Defining arguments in a method","anchor":"defining-arguments-in-a-method","htmlText":"Defining arguments in a method"},{"level":4,"text":"Writing a multi-line code block","anchor":"writing-a-multi-line-code-block","htmlText":"Writing a multi-line code block"},{"level":4,"text":"Writing a return block","anchor":"writing-a-return-block","htmlText":"Writing a return block"},{"level":4,"text":"Adding an image","anchor":"adding-an-image","htmlText":"Adding an image"},{"level":2,"text":"Styling the docstring","anchor":"styling-the-docstring","htmlText":"Styling the docstring"},{"level":1,"text":"Testing documentation examples","anchor":"testing-documentation-examples","htmlText":"Testing documentation examples"},{"level":2,"text":"Writing documentation examples","anchor":"writing-documentation-examples","htmlText":"Writing documentation examples"},{"level":2,"text":"Docstring testing","anchor":"docstring-testing","htmlText":"Docstring testing"},{"level":3,"text":"For Python files","anchor":"for-python-files","htmlText":"For Python files"},{"level":3,"text":"For Markdown files","anchor":"for-markdown-files","htmlText":"For Markdown files"},{"level":3,"text":"Writing doctests","anchor":"writing-doctests","htmlText":"Writing doctests"}],"siteNavLoginPath":"/login?return_to=https%3A%2F%2Fgithub.com%2Fgithubhjs%2Ftransformers%2Ftree%2Fc2c99dc7ef5edab8f7674a1eb00cf6ac6996fd0f%2Fdocs"}},"totalCount":3,"showBranchInfobar":false},"fileTree":{"":{"items":[{"name":".circleci","path":".circleci","contentType":"directory"},{"name":".github","path":".github","contentType":"directory"},{"name":"docker","path":"docker","contentType":"directory"},{"name":"docs","path":"docs","contentType":"directory"},{"name":"examples","path":"examples","contentType":"directory"},{"name":"model_cards","path":"model_cards","contentType":"directory"},{"name":"notebooks","path":"notebooks","contentType":"directory"},{"name":"scripts","path":"scripts","contentType":"directory"},{"name":"src","path":"src","contentType":"directory"},{"name":"templates","path":"templates","contentType":"directory"},{"name":"tests","path":"tests","contentType":"directory"},{"name":"utils","path":"utils","contentType":"directory"},{"name":".coveragerc","path":".coveragerc","contentType":"file"},{"name":".gitattributes","path":".gitattributes","contentType":"file"},{"name":".gitignore","path":".gitignore","contentType":"file"},{"name":"CITATION.cff","path":"CITATION.cff","contentType":"file"},{"name":"CODE_OF_CONDUCT.md","path":"CODE_OF_CONDUCT.md","contentType":"file"},{"name":"CONTRIBUTING.md","path":"CONTRIBUTING.md","contentType":"file"},{"name":"ISSUES.md","path":"ISSUES.md","contentType":"file"},{"name":"LICENSE","path":"LICENSE","contentType":"file"},{"name":"MANIFEST.in","path":"MANIFEST.in","contentType":"file"},{"name":"Makefile","path":"Makefile","contentType":"file"},{"name":"README.md","path":"README.md","contentType":"file"},{"name":"README_es.md","path":"README_es.md","contentType":"file"},{"name":"README_hd.md","path":"README_hd.md","contentType":"file"},{"name":"README_ja.md","path":"README_ja.md","contentType":"file"},{"name":"README_ko.md","path":"README_ko.md","contentType":"file"},{"name":"README_zh-hans.md","path":"README_zh-hans.md","contentType":"file"},{"name":"README_zh-hant.md","path":"README_zh-hant.md","contentType":"file"},{"name":"conftest.py","path":"conftest.py","contentType":"file"},{"name":"hubconf.py","path":"hubconf.py","contentType":"file"},{"name":"pyproject.toml","path":"pyproject.toml","contentType":"file"},{"name":"setup.cfg","path":"setup.cfg","contentType":"file"},{"name":"setup.py","path":"setup.py","contentType":"file"}],"totalCount":34}},"fileTreeProcessingTime":1.923907,"foldersToFetch":[],"treeExpanded":true,"symbolsExpanded":false,"csrf_tokens":{"/githubhjs/transformers/branches":{"post":"W2W-WGaFRhxxPVIFEqP8KqNgv9TwCuDU_v4jkC8Ng2hygohLbsDnsQkcrzleS4w4YcKSI-_XOzz8hnxjKq8yIA"}}},"title":"transformers/docs at c2c99dc7ef5edab8f7674a1eb00cf6ac6996fd0f · githubhjs/transformers","appPayload":{"helpUrl":"https://docs.github.com","findFileWorkerPath":"/assets-cdn/worker/find-file-worker-263cab1760dd.js","findInFileWorkerPath":"/assets-cdn/worker/find-in-file-worker-1b17b3e7786a.js","githubDevUrl":null,"enabled_features":{"code_nav_ui_events":false,"react_blob_overlay":false,"accessible_code_button":true}}}