8000 tests/run-tests: Add general newline normalization function. · micropython/micropython@701609f · GitHub
[go: up one dir, main page]
Skip to content

Commit 701609f

Browse files
pi-anlpi-anl
committed
tests/run-tests: Add general newline normalization function.
Add a general normalize_newlines() function that handles newline variations (\\r\\r\\n, \\r\\n) to \\n while preserving literal \\r characters that are part of test content. This provides a robust solution for cross-platform test compatibility, particularly addressing PTY double-newline issues that can occur with some terminal implementations. The function is applied to all test output before comparison, eliminating platform-specific newline issues. Includes a unit test to verify the normalization behavior. Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
1 parent 576e2bc commit 701609f

File tree

5 files changed

+42
-1
lines changed

5 files changed

+42
-1
lines changed

‎.gitattributes

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
# These should also not be modified by git.
1919
tests/basics/string_cr_conversion.py -text
2020
tests/basics/string_crlf_conversion.py -text
21+
tests/micropython/test_normalize_newlines.py.exp -text
2122
ports/stm32/pybcdc.inf_template -text
2223
ports/stm32/usbhost/** -text
2324
ports/cc3200/hal/aes.c -text

‎pyproject.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,7 @@ exclude = [
7070
"tests/basics/*.py",
7171
"tests/*/repl_*.py",
7272
"tests/cmdline/cmd_compile_only_error.py",
73+
"tests/micropython/test_normalize_newlines.py",
7374
"tests/micropython/viper_args.py",
7475
]
7576
quote-style = "preserve"

‎tests/micropython/test_normalize_newlines.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# Test for normalize_newlines functionality
2+
# This test verifies that test framework handles various newline combinations correctly
3+
4+
# Note: This is more of an integration test since normalize_newlines is in the test framework
5+
# The actual testing happens when this test is run through run-tests.py
6+
7+
print("Testing newline handling")
8+
print("Line 1\r\nLine 2") # Windows-style line ending - should be normalized
9+
print("Line 3") # Normal line
10+
print("Line 4") # Normal line
11+
print("Line 5\nLine 6") # Unix-style line ending - already normalized
12+
13+
# Test that literal \r in strings is preserved
14+
print(repr("test\rstring")) # Should show 'test\rstring' not 'test\nstring'

‎tests/micropython/test_normalize_newlines.py.exp

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
Testing newline handling
2+
Line 1
3+
Line 2
4+
Line 3
5+
Line 4
6+
Line 5
7+
Line 6
8+
'test\rstring'

‎tests/run-tests.py

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,23 @@ def base_path(*p):
5858
# Set PYTHONIOENCODING so that CPython will use utf-8 on systems which set another encoding in the locale
5959
os.environ["PYTHONIOENCODING"] = "utf-8"
6060

61+
62+
def normalize_newlines(data):
63+
"""Normalize newline variations to \\n.
64+
65+
Only normalizes actual line endings, not literal \\r characters in strings.
66+
Handles \\r\\r\\n and \\r\\n cases to ensure consistent comparison
67+
across different platforms and terminals.
68+
"""
69+
if isinstance(data, bytes):
70+
# Handle PTY double-newline issue first
71+
data = data.replace(b'\r\r\n', b'\n')
72+
# Then handle standard Windows line endings
73+
data = data.replace(b'\r\n', b'\n')
74+
# Don't convert standalone \r as it might be literal content
75+
return data
76+
77+
6178
# Code to allow a target MicroPython to import an .mpy from RAM
6279
# Note: the module is named `__injected_test` but it needs to have `__name__` set to
6380
# `__main__` so that the test sees itself as the main module, eg so unittest works.
@@ -471,7 +488,7 @@ def send_get(what):
471488
)
472489

473490
# canonical form for all ports/platforms is to use \n for end-of-line
474-
output_mupy = output_mupy.replace(b"\r\n", b"\n")
491+
output_mupy = normalize_newlines(output_mupy)
475492

476493
# don't try to convert the output if we should skip this test
477494
if had_crash or output_mupy in (b"SKIP\n", b"CRASH"):

0 commit comments

Comments
 (0)
0