8000 Merge pull request #47 from tecladocode/jose/cou-169-cpc-set-up-mkdoc… · code93/complete-python-course@ea11457 · GitHub
[go: up one dir, main page]
Skip to content

Commit ea11457

Browse files
jslvtrjslvtr
authored
Merge pull request tecladocode#47 from tecladocode/jose/cou-169-cpc-set-up-mkdocs-ebook-for-this-course
2 parents 07c4f72 + 275c962 commit ea11457

File tree

35 files changed

+371
-148
lines changed

35 files changed

+371
-148
lines changed

‎.github/workflows/ebook.yml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
name: ebook
2+
3+
on:
4+
push:
5+
branches:
6+
- master
7+
jobs:
8+
deploy:
9+
runs-on: ubuntu-latest
10+
steps:
11+
- uses: actions/checkout@v2
12+
- uses: actions/setup-python@v2
13+
with:
14+
python-version: 3.x
15+
- run: pip install -r requirements-docs.txt
16+
- run: cog -r mkdocs.yml
17+
- run: mkdocs gh-deploy --force

‎.gitignore

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,3 +33,24 @@ videos/
3333
**/old
3434
.venv
3535
.python-version
36+
37+
# Dependencies
38+
/node_modules
39+
40+
# Production
41+
/build
42+
43+
# Generated files
44+
.docusaurus
45+
.cache-loader
46+
47+
# Misc
48+
.DS_Store
49+
.env.local
50+
.env.development.local
51+
.env.test.local
52+
.env.production.local
53+
54+
npm-debug.log*
55+
yarn-debug.log*
56+
yarn-error.log*

‎CONTRIBUTING.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Contributing
2+
3+
## Building the docs
4+
5+
Write your documentation inside `course_contents`.
6+
7+
To build and serve the documentation, use this command:
8+
9+
```bash
10+
watchmedo shell-command --patterns="*.md" --recursive course_contents --command="cog -r mkdocs.yml"
11+
```
12+
13+
This runs the `cog -r mkdocs.yml` command every time there is a markdown file change.
14+
15+
**note**: at the moment there is [a bug](https://github.com/gorakhargosh/watchdog/issues/346) in `watchmedo` that triggers the shell command twice on file modification, or potentially a few times on file creation. It doesn't affect us majorly.
16+
17+
### Why do we need to use `cog` to build the docs before serving?
18+
19+
The documentation system we use for this e-book, `mkdocs`, has limitation regarding slugs: all slugs are calculated from the filesystem paths.
20+
21+
This means that if we simply serve the docs from `course_contents`, all our slugs would be numbered (e.g. `1_intro/lectures/3_variables_printing/`).
22+
23+
To avoid this, we first run a build step (in `content.py`, `build_and_get_yaml_contents()`) that copies all the non-hidden sections and lectures to a `build` folder, and then we can serve the documentation from there.
24+
25+
This is a bit "hacky", and we must remember to run `cog` on the `mkdocs.yml` file if we want to see our updated documentation!
26+
27+
## Writing documentation README files
28+
29+
There are a few attributes for each `README.md` file that we can use.
30+
31+
In section files:
32+
33+
- `group: str` will try to place sections with the same group name inside a tabbed navigation.
34+
- `hidden: true | false` will hide the section and its lectures, and not include any of them in the build.
35+
36+
In lecture files:
37+
38+
- `hidden: true | false` will hide the lecture and not include it in the build. Other lectures in the same section are unaffected.

‎content.py

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
import glob
2+
import json
3+
import pathlib
4+
import shutil
5+
import string
6+
import markdown
7+
from bs4 import BeautifulSoup
8+
9+
def get_grouped_sections(root: str = "course_contents") -> dict[str, list]:
10+
sections = get_all_sections_with_content(root)
11+
grouped_sections = {}
12+
for section in sections:
13+
group = section["index"]["group"]
14+
if not group:
15+
raise RuntimeError("Not all sections part of a group.")
16+
if group:
17+
grouped_sections.setdefault(group, []).append(section)
18+
else:
19+
grouped_sections.setdefault("default", []).append(section)
20+
return grouped_sections
21+
22+
def get_all_sections_with_content(root: str = "course_contents"):
23+
root_path = pathlib.Path(root)
24+
section_folders = [folder.name for folder in root_path.iterdir() if folder.is_dir() and folder.name[0] in string.digits]
25+
sections = sorted(section_folders, key=lambda e: int(str(e).split("_")[0]))
26+
for section in sections:
27+
section = root_path / section
28+
section_index = get_lecture_content(section / "README.md")
29+
if not section_index.get("title") or section_index.get("hidden"):
30+
continue
31+
lectures = get_section_lectures(section)
32+
lecture_contents = [get_lecture_content(lecture) for lecture in lectures]
33+
non_hidden_lectures = [lecture for lecture in lecture_contents if not lecture.get("hidden")]
34+
yield {"index": section_index, "lectures": non_hidden_lectures}
35+
36+
37+
def get_section_lectures(folder: pathlib.Path | str) -> list[str]:
38+
"""Return a list of pathlib.Path objects for all lectsures in a section folder"""
39+
lecture_path = pathlib.Path(folder) / "lectures"
40+
return sorted([folder for folder in lecture_path.glob("*/README.md")])
41+
42+
43+
def get_lecture_content(lecture_path: pathlib.Path, root_path: str | pathlib.Path = "course_contents") -> dict[str, str]:
44+
"""Return a dictionary of lecture content.
45+
Return a dictionary with the following keys:
46+
- title, the h1 from the markdown file
47+
- summary from the markdown metadata
48+
- group, to group sections together, optional
49+
- hidden, to hide entire sections, optional
50+
- filename, the name of the markdown file
51+
- full_path, the full path to the markdown file
52+
- order, the order of the lecture in the section (defaults to file name)
53+
- path, the path to this file relative to the root"""
54+
with open(lecture_path) as f:
55+
content = f.read()
56+
md = markdown.Markdown(extensions=["meta"])
57+
html = md.convert(content)
58+
soup = BeautifulSoup(html, "html.parser")
59+
return {
60+
"title": soup.find("h1").text,
61+
"summary": md.Meta.get("summary", ""),
62+
"group": md.Meta.get("group", [""])[0],
63+
"hidden": md.Meta.get("hidden", [""])[0] == "true",
64+
"filename": lecture_path.name,
65+
"full_path": lecture_path.absolute(),
66+
"order": md.Meta.get("order", [lecture_path.name])[0],
67+
"path": lecture_path.relative_to(root_path),
68+
}
69+
70+
71+
def get_grouped_build_sections(root: str = "build") -> dict[str, list]:
72+
sections = build_and_get_yaml_contents(root)
73+
grouped_sections = {}
74+
for section in sections:
75+
group = section["index"]["group"]
76+
if not group:
77+
raise RuntimeError("Not all sections part of a group.")
78+
if group:
79+
grouped_sections.setdefault(group, []).append(section)
80+
else:
81+
grouped_sections.setdefault("default", []).append(section)
82+
return grouped_sections
83+
84+
85+
def build_and_get_yaml_contents(build_path: str = "build"):
86+
# Delete contents of the build directory
87+
shutil.rmtree(build_path, ignore_errors=True)
88+
pathlib.Path(build_path).mkdir(parents=True, exist_ok=True)
89+
files = glob.glob("course_contents/*", recursive=True)
90+
# Copy all files (not folders) to the build directory
91+
for file in files:
92+
if pathlib.Path(file).is_file():
93+
shutil.copy(file, build_path)
94+
sections = get_all_sections_with_content()
95+
for section in sections:
96+
# Strip the leading numbers of the section folder
97+
old_section_name = section["index"]["full_path"].parent.name
98+
section_name = "_".join(old_section_name.split("_")[1:])
99+
pathlib.Path(build_path, section_name).mkdir(parents=True, exist_ok=True)
100+
# Create a directory in the build folder matching the section_name
101+
# Copy the README.md file from the original section to the new directory
102+
shutil.copyfile(section["index"]["full_path"], pathlib.Path(build_path, section_name, "README.md"))
103+
# Copy the lecture folders to the new directory
104+
section["lectures"] = list(copy_lectures_to_build_path(section, section_name, build_path=build_path))
105+
# Update the section index to point to the new path
106+
section["index"]["full_path"] = pathlib.Path(build_path, section_name, "README.md").absolute()
107+
section["index"]["path"] = pathlib.Path(build_path, section_name, "README.md").relative_to(build_path)
108+
yield section
109+
110+
111+
def copy_lectures_to_build_path(section: dict, new_section_name: str, build_path: str = "build"):
112+
for lecture in section["lectures"]:
113+
lecture_name = "_".join(lecture["full_path"].parent.name.split("_")[1:])
114+
pathlib.Path(build_path, new_section_name, lecture_name).mkdir(parents=True, exist_ok=True)
115+
shutil.copyfile(lecture["full_path"], pathlib.Path(build_path, new_section_name, lecture_name, "README.md"))
116+
lecture["full_path"] = pathlib.Path(build_path, new_section_name, lecture_name, "README.md").absolute()
117+
lecture["path"] = pathlib.Path(build_path, new_section_name, lecture_name, "README.md").relative_to(build_path)
118+
yield lecture
119+
120+
121+
if __name__ == "__main__":
122+
section_yaml = get_grouped_build_sections()
123+
print(list(section_yaml))

‎course_contents/10_advanced_python/README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
---
2+
group: Intermediate
3+
hidden: true
4+
---
15
# Advanced Python Development
26

37
In this section of the course we look at some advanced Python features, such as:

‎course_contents/11_web_scraping/README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
---
2+
group: Practical Python
3+
hidden: true
4+
---
15
# Web Scraping
26

37
In this section we look at web scraping using Python and the `requests` library.

‎course_contents/12_browser_automation_selenium/README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
---
2+
group: Practical Python
3+
hidden: true
4+
---
15
# Browser automation with Selenium
26

37
The code is very similar to last section, but now we're launching a browser instead of requesting the page with Python. We will be controlling the browser, instead of just getting HTML.

‎course_contents/13_async_development/README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
---
2+
group: Practical Python
3+
hidden: false
4+
---
15
# Async Development with Python
26

37
Python is a single-threaded language, which means that asynchronous development can sometimes be tricky.

‎course_contents/13_async_development/lectures/01_code_samples/README.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
---
2+
hidden: false
3+
---
4+
5+
# Code samples for this section
6+
7+
Hey, welcome back!
8+
9+
This section won't have code available on our online editor, but instead all the code we'll write here is available on the GitHub page.
10+
11+
Again, don't read through the code before going through some of the videos. The code is here for you to check as you progress through the section. Otherwise, it could get a bit confusing!
12+
13+
Here's a link to the code in this section: https://github.com/tecladocode/complete-python-course/tree/master/course_contents/13_async_development/sample_code
14+
15+
As always, if you have any questions please ask away in the Course Q&A.
16+
17+
Happy coding!
18+
19+
Jose—your instructor

‎course_contents/13_async_development/lectures/02_async_terms_glossary/README.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
---
2+
hidden: true
3+
---
4+
5+
# Glossary of terms used in concurrency
6+
7+
Hey there! Welcome to one of the most advanced topics in Python. A lot of developers are scared of what we're going to learn in this section.
8+
9+You shouldn't be scared!
10+
11+
We'll start at the fundamentals, and build out our knowledge of asynchronous development from the ground up.
12+
13+
Throughout the section we'll learn more about what these terms mean, but it can be helpful to have a short glossary of terms just in case you want to come back and remind yourself.
14+
15+
- **Synchronous**: actions that happen one after another. Programming as we've seen it until now is synchronous, because each line executes after the previous one.
16+
- **Asynchronous**: actions that don't necessary happen after one another, or that can happen in arbitrary order ("without synchrony").
17+
- **Concurrency**: The ability of our programs to run things in different order every time the program runs, without affecting the final outcome.
18+
- **Parallelism**: Running two or more things at the same time.
19+
- **Thread**: A line of code execution that can run in one of your computer's cores.
20+
- **Process**: One of more threads and the resources they need (e.g. network connection, mouse pointer, hard drive access, or even the core(s) in which the thread(s) run).
21+
- **GIL**: A key, critical, important resource in any Python program. Only one is created per Python process, so it's unique in each.
22+
23+
Take it slowly through this section, and ask questions in the Course Q&A as required. We're here to help!
24+
25+
Kind regards,
26+
27+
Jose

0 commit comments

Comments
 (0)
0