DOC: Mention and try to explain pairwise summation in sum (#13737)

seberg · mattip · commit 871ed9b188da · 2019-06-11T08:55:45.000+03:00
* DOC: Mention and try to explain pairwise summation in sum

Note that this behaviour is of course inherited into `np.add.reduce` and
many other reductions such as `mean` or users of this reduction, such
as `cov`. This is ignored here.
diff --git a/numpy/core/fromnumeric.py b/numpy/core/fromnumeric.py
@@ -2104,6 +2104,8 @@ def sum(a, axis=None, dtype=None, out=None, keepdims=np._NoValue,
     --------
     ndarray.sum : Equivalent method.
 
+    add.reduce : Equivalent functionality of `add`.
+
     cumsum : Cumulative sum of array elements.
 
     trapz : Integration of array values using the composite trapezoidal rule.
@@ -2120,6 +2122,23 @@ def sum(a, axis=None, dtype=None, out=None, keepdims=np._NoValue,
     >>> np.sum([])
     0.0
 
+    For floating point numbers the numerical precision of sum (and
+    ``np.add.reduce``) is in general limited by directly adding each number
+    individually to the result causing rounding errors in every step.
+    However, often numpy will use a  numerically better approach (partial
+    pairwise summation) leading to improved precision in many use-cases.
+    This improved precision is always provided when no ``axis`` is given.
+    When ``axis`` is given, it will depend on which axis is summed.
+    Technically, to provide the best speed possible, the improved precision
+    is only used when the summation is along the fast axis in memory.
+    Note that the exact precision may vary depending on other parameters.
+    In contrast to NumPy, Python's ``math.fsum`` function uses a slower but
+    more precise approach to summation.
+    Especially when summing a large number of lower precision floating point
+    numbers, such as ``float32``, numerical errors can become significant.
+    In such cases it can be advisable to use `dtype="float64"` to use a higher
+    precision for the output.
+
     Examples
     --------
     >>> np.sum([0.5, 1.5])
