Add comprehensive float32 precision analysis and test suite #54
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Investigated reported issue of false negatives in overlap detection due to float32 rounding. Created extensive test suite and detailed analysis report. Changes: - Add FLOAT32_PRECISION_ANALYSIS.md: Comprehensive investigation report - Documented code-level vulnerabilities in float32 precision handling - Identified asymmetry between float64 and float32 input handling - Analyzed theoretical risks and edge cases - Provided recommendations for mitigation - Add test_float32_overlap_issue.py: Basic precision tests - Very small overlap detection - Float32 precision boundary tests - Query method verification - ULP (Unit in Last Place) separation tests - Add test_float32_refined.py: Advanced precision tests - Internal float32 representation verification - Mixed precision (float32 tree + float64 query) scenarios - Gap creation via rounding - Specific failing case analysis - Add test_float32_extreme.py: Extreme edge case tests - Minimal overlap at ULP boundaries - Negative coordinates with precision loss - Accumulated rounding errors - Subnormal float32 values - Double rounding effects Findings: - Confirmed code vulnerability: float32 input lacks refinement mechanism - Float64 input uses idx2exact for precision refinement, float32 does not - Unable to reproduce false negatives in synthetic tests - Closed interval semantics (<=) properly handles touching boxes - Issue likely requires specific real-world data to reproduce Note: All tests pass without detecting false negatives, suggesting the issue may be context-dependent or require specific coordinate patterns.
Extended float32 precision investigation based on the hypothesis that
rounding direction differences could cause false negatives. Discovered
false positive case instead and updated comprehensive analysis.
New Test Scripts:
- test_rounding_direction.py: Tests for rounding direction mismatches
- Attempts to find values that round in opposite directions
- Tests guaranteed opposite rounding at float32 precision boundaries
- Explores midpoint rounding and nextafter gaps
- test_different_sources.py: Tests values from different sources
- Computed vs literal values
- Accumulated computation effects
- Separate float32 array creation
- Binary representation differences
- **FOUND FALSE POSITIVE**: accumulated_f64 < direct_f64 (no overlap)
but both round to 100.0 in float32 (false overlap detected)
- test_false_negative_found.py: Systematic false negative search
- Accumulated overlap loss scenarios
- Different accumulation rates
- NextAfter gap exploration
- String roundtrip testing
Key Findings:
1. False Positive Discovered:
- Float64: sum(0.1 for _ in range(1000)) ≈ 99.999 < 100.0 (no overlap)
- Float32: both values round to 100.0 (overlap detected)
- This is the INVERSE of the reported issue
2. False Negative Still Not Reproduced:
- Closed interval semantics (<=) handles all boundary cases correctly
- Consistent rounding ensures internal consistency
- All touching boxes are properly detected
3. Theoretical False Negative Scenarios Identified:
- Different computation paths leading to same logical value
- Compiler optimization with intermediate precision
- FPU settings and register precision differences
- Data pipeline inconsistencies
- Platform-dependent floating point behavior
Updated FLOAT32_PRECISION_ANALYSIS.md:
- Added rounding direction investigation section
- Documented false positive discovery
- Expanded theoretical risk analysis
- Added compiler/FPU/platform considerations
- Updated recommendations and next steps
Conclusion:
While false negatives could not be reproduced synthetically, false positives
were found. The reported issue likely requires specific real-world data,
compiler settings, or platform configurations to manifest. Requesting
concrete failing examples from reporter for further investigation.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Investigated reported issue of false negatives in overlap detection due to
float32 rounding. Created extensive test suite and detailed analysis report.
Changes:
Add FLOAT32_PRECISION_ANALYSIS.md: Comprehensive investigation report
Add test_float32_overlap_issue.py: Basic precision tests
Add test_float32_refined.py: Advanced precision tests
Add test_float32_extreme.py: Extreme edge case tests
Findings:
Note: All tests pass without detecting false negatives, suggesting the
issue may be context-dependent or require specific coordinate patterns.