Fixed token.c hyperlink in compiler design

python · AlexWaygood · Dec 19, 2023 · Dec 17, 2023 · Dec 17, 2023 · Dec 19, 2023
commit 401d3657da56f4709c3eae10bed73aff6c1466f1
diff --git a/developer-workflow/grammar.rst b/developer-workflow/grammar.rst
@@ -42,7 +42,7 @@ Note: sometimes things mysteriously don't work.  Before giving up, try ``make cl
   Then run ``make regen-ast`` to regenerate
   :cpy-file:`Include/internal/pycore_ast.h` and :cpy-file:`Python/Python-ast.c`.
 
-* :cpy-file:`Parser/tokenizer.c` contains the tokenization code.
+* :cpy-file:`Parser/lexer/` & :cpy-file:`Parser/tokenizer/` contains the tokenization code.
   This is where you would add a new type of comment or string literal, for example.
 
 * :cpy-file:`Python/ast.c` will need changes to validate AST objects

diff --git a/internals/compiler.rst b/internals/compiler.rst
@@ -11,7 +11,7 @@ Abstract
 
 In CPython, the compilation from source code to bytecode involves several steps:
 
-1. Tokenize the source code (:cpy-file:`Parser/tokenizer.c`).
+1. Tokenize the source code (:cpy-file:`Parser/lexer/` & :cpy-file:`Parser/tokenizer/`).
 2. Parse the stream of tokens into an Abstract Syntax Tree
    (:cpy-file:`Parser/parser.c`).
 3. Transform AST into an instruction sequence (:cpy-file:`Python/compile.c`).

diff --git a/internals/parser.rst b/internals/parser.rst
@@ -599,7 +599,7 @@ If you are on Windows you can use the Visual Studio project files to regenerate
 
     ./PCbuild/build.bat --regen
 
-How tokens are generated and the rules governing this is completely up to the tokenizer (:cpy-file:`Parser/tokenizer.c`)
+How tokens are generated and the rules governing this is completely up to the tokenizer (:cpy-file:`Parser/lexer/` & :cpy-file:`Parser/tokenizer/`)
 and the parser just receives tokens from it.
 
 Memoization