8000 Merge pull request #12 from bashtage/add-box-muller · mattip/numpy@db709c1 · GitHub
[go: up one dir, main page]
Skip to content

Commit db709c1

Browse files
bashtagebashtage
authored
Merge pull request #12 from bashtage/add-box-muller
ENH: Add Box-Muller gauss
2 parents 10e984d + 61656c4 commit db709c1

22 files changed

+2710
-400
lines changed

‎_randomgen/.travis.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,11 +20,11 @@ matrix:
2020
fast_finish: true
2121
include:
2222
- os: linux
23-
env: [PYTHON=2.7, NUMPY=1.10, CYTHON=0.24]
23+
env: [PYTHON=2.7, NUMPY=1.10, CYTHON=0.26]
2424
- os: linux
2525
env: [PYTHON=3.5, NUMPY=1.11]
2626
- os: linux
27-
env: [PYTHON=3.6, NUMPY=1.13, CYTHON=0.25]
27+
env: [PYTHON=3.6, NUMPY=1.13, CYTHON=0.27]
2828
- os: linux
2929
env: [PYTHON=3.6, NUMPY=1.13, CYTHON=0.26]
3030
- os: linux

‎_randomgen/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -146,7 +146,7 @@ Building requires:
146146

147147
* Python (2.7, 3.4, 3.5, 3.6)
148148
* NumPy (1.10, 1.11, 1.12, 1.13, 1.14)
149-
* Cython (0.25+)
149+
* Cython (0.26+)
150150
* tempita (0.5+), if not provided by Cython
151151

152152
Testing requires pytest (3.0+).

‎_randomgen/README.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -163,7 +163,7 @@ Building requires:
163163

164164
- Python (2.7, 3.4, 3.5, 3.6)
165165
- NumPy (1.10, 1.11, 1.12, 1.13, 1.14)
166-
- Cython (0.25+)
166+
- Cython (0.26+)
167167
- tempita (0.5+), if not provided by Cython
168168

169169
Testing requires pytest (3.0+).

‎_randomgen/doc/source/change-log.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@ Change Log
44
Changes since v1.14
55
===================
66

7+
- Add legacy generator which allows NumPy replication
8+
- Improve type handling of integers
79
- Switch to array-fillers for 0 parameter distribution to improve performance
810
- Small changes to build on manylinux
911
- Build wheels using multibuild

‎_randomgen/doc/source/extending.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ This example shows how numba can be used to produce Box-Muller normals using
1515
a pure Python implementation which is then compiled. The random numbers are
1616
provided by ``ctypes.next_double``.
1717

18-
.. ipython:: python
18+
.. code-block:: python
1919
2020
from randomgen import Xoroshiro128
2121
import numpy as np

‎_randomgen/doc/source/index.rst

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -174,7 +174,8 @@ Random Generator
174174
.. toctree::
175175
:maxdepth: 1
176176

177-
Random Generator <generator>
177+
Random Generation <generator>
178+
legacy
178179

179180
Basic Random Number Generators
180181
------------------------------

‎_randomgen/doc/source/legacy.rst

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
Legacy Random Generation
2+
------------------------
3+
The :class:`~randomgen.legacy.LegacyGenerator` provides access to
4+
some legacy generators. These all depend on Box-Muller normals or
5+
inverse CDF exponentials or gammas. This class should only be used
6+
if it is essential to have randoms that are identical to what
7+
would have been produced by NumPy.
8+
9+
:class:`~randomgen.legacy.LegacyGenerator` add additional information
10+
to the state which is required when using Box-Muller normals since these
11+
are produced in pairs. It is important to use
12+
:attr:`~randomgen.legacy.LegacyGenerator.state`
13+
when accessing the state so that these extra values are saved.
14+
15+
.. warning::
16+
17+
:class:`~randomgen.legacy.LegacyGenerator` only contains functions
18+
that have changed. Since it does not contain other functions, it
19+
is not direclty possible to replace :class:`~numpy.random.RandomState`.
20+
In order to full replace :class:`~numpy.random.RandomState`, it is
21+
necessary to use both :class:`~randomgen.legacy.LegacyGenerator`
22+
and :class:`~randomgen.generator.RandomGenerator` both driven
23+
by the same basic RNG. Methods present in :class:`~randomgen.legacy.LegacyGenerator`
24+
must be called from :class:`~randomgen.legacy.LegacyGenerator`. Other Methods
25+
should be called from :class:`~randomgen.generator.RandomGenerator`.
26+
27+
28+
.. code-block:: python
29+
30+
from randomgen import RandomGenerator, MT19937
31+
from randomgen.legacy import LegacyGenerator
32+
from numpy.random import RandomState
33+
# Use same seed
34+
rs = RandomState(12345)
35+
mt19937 = MT19937(12345)
36+
rg = RandomGenerator(mt19937)
37+
lg = LegacyGenerator(mt19937)
38+
39+
# Identical output
40+
rs.standard_normal()
41+
lg.standard_normal()
42+
43+
rs.random_sample()
44+
rg.random_sample()
45+
46+
rs.standard_exponential()
47+
lg.standard_exponential()
48+
49+
50+
.. currentmodule:: randomgen.legacy
51+
52+
.. autoclass::
53+
LegacyGenerator
54+
55+
Seeding and State
56+
=================
57+
58+
.. autosummary::
59+
:toctree: generated/
60+
61+
~LegacyGenerator.state
62+
63+
Simple random data
64+
==================
65+
.. autosummary::
66+
:toctree: generated/
67+
68+
~LegacyGenerator.randn
69+
70+
Distributions
71+
=============
72+
.. autosummary::
73+
:toctree: generated/
74+
75+
~LegacyGenerator.beta
76+
~LegacyGenerator.chisquare
77+
~LegacyGenerator.dirichlet
78+
~LegacyGenerator.exponential
79+
~LegacyGenerator.f
80+
~LegacyGenerator.gamma
81+
~LegacyGenerator.lognormal
82+
~LegacyGenerator.multivariate_normal
83+
~LegacyGenerator.negative_binomial
84+
~LegacyGenerator.noncentral_chisquare
85+
~LegacyGenerator.noncentral_f
86+
~LegacyGenerator.normal
87+
~LegacyGenerator.pareto
88+
~LegacyGenerator.power
89+
~LegacyGenerator.standard_cauchy
90+
~LegacyGenerator.standard_exponential
91+
~LegacyGenerator.standard_gamma
92+
~LegacyGenerator.standard_normal
93+
~LegacyGenerator.standard_t
94+
~LegacyGenerator.wald
95+
~LegacyGenerator.weibull

‎_randomgen/doc/source/multithreading.rst

Lines changed: 35 additions & 61 deletions
Original file line numberDiff line numberDiff line change
@@ -14,117 +14,91 @@ xorshift2014 which is fast, has a long period and supports using ``jump`` to
1414
advance the state. The random numbers generated are reproducible in the sense
1515
that the same seed will produce the same outputs.
1616

17-
.. code-block:: python
17+
.. code-block:: ipython
1818
1919
from randomgen import Xorshift1024
2020
import multiprocessing
2121
import concurrent.futures
2222
import numpy as np
23-
23+
2424
class MultithreadedRNG(object):
2525
def __init__(self, n, seed=None, threads=None):
2626
rg = Xorshift1024(seed)
2727
if threads is None:
2828
threads = multiprocessing.cpu_count()
2929
self.threads = threads
30-
30+
3131
self._random_generators = []
3232
for _ in range(0, threads-1):
3333
_rg = Xorshift1024()
3434
_rg.state = rg.state
3535
self._random_generators.append(_rg.generator)
3636
rg.jump()
3737
self._random_generators.append(rg.generator)
38-
38+
3939
self.n = n
4040
self.executor = concurrent.futures.ThreadPoolExecutor(threads)
4141
self.values = np.empty(n)
4242
self.step = np.ceil(n / threads).astype(np.int)
43-
43+
4444
def fill(self):
4545
def _fill(random_state, out, first, last):
4646
random_state.standard_normal(out=out[first:last])
47-
47+
4848
futures = {}
4949
for i in range(self.threads):
50-
args = (_fill, self._random_generators[i], self.values, i * self.step, (i + 1) * self.step)
50+
args = (_fill,
51+
self._random_generators[i],
52+
self.values,
53+
i * self.step,
54+
(i + 1) * self.step)
5155
futures[self.executor.submit(*args)] = i
5256
concurrent.futures.wait(futures)
53-
57+
5458
def __del__(self):
5559
self.executor.shutdown(False)
5660
5761
58-
.. ipython:: python
59-
:suppress:
60-
61-
62-
In [1]: from randomgen import Xorshift1024
63-
....: import multiprocessing
64-
....: import concurrent.futures
65-
....: import numpy as np
66-
....: class MultithreadedRNG(object):
67-
....: def __init__(self, n, seed=None, threads=None):
68-
....: rg = Xorshift1024(seed)
69-
....: if threads is None:
70-
....: threads = multiprocessing.cpu_count()
71-
....: self.threads = threads
72-
....: self._random_generators = []
73-
....: for _ in range(0, threads-1):
74-
....: _rg = Xorshift1024()
75-
....: _rg.state = rg.state
76-
....: self._random_generators.append(_rg.generator)
77-
....: rg.jump()
78-
....: self._random_generators.append(rg.generator)
79-
....: self.n = n
80-
....: self.executor = concurrent.futures.ThreadPoolExecutor(threads)
81-
....: self.values = np.empty(n)
82-
....: self.step = np.ceil(n / threads).astype(np.int)
83-
....: def fill(self):
84-
....: def _fill(random_state, out, first, last):
85-
....: random_state.standard_normal(out=out[first:last])
86-
....: futures = {}
87-
....: for i in range(self.threads):
88-
....: args = (_fill, self._random_generators[i], self.values, i * self.step, (i + 1) * self.step)
89-
....: futures[self.executor.submit(*args)] = i
90-
....: concurrent.futures.wait(futures)
91-
....: def __del__(self):
92-
....: self.executor.shutdown(False)
93-
....:
94-
9562
The multithreaded random number generator can be used to fill an array.
9663
The ``values`` attributes shows the zero-value before the fill and the
9764
random value after.
9865

99-
.. ipython:: python
66+
.. code-block:: ipython
10067
101-
mrng = MultithreadedRNG(10000000, seed=0)
102-
print(mrng.values[-1])
103-
mrng.fill()
104-
print(mrng.values[-1])
68+
In [2]: mrng = MultithreadedRNG(10000000, seed=0)
69+
...: print(mrng.values[-1])
70+
0.0
71+
72+
In [3]: mrng.fill()
73+
...: print(mrng.values[-1])
74+
3.296046120254392
10575
10676
The time required to produce using multiple threads can be compared to
10777
the time required to generate using a single thread.
10878

109-
.. ipython:: python
110-
111-
print(mrng.threads)
112-
%timeit mrng.fill()
79+
.. code-block:: ipython
11380
81+
In [4]: print(mrng.threads)
82+
...: %timeit mrng.fill()
83+
4
84+
42.9 ms ± 2.55 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
11485
11586
The single threaded call directly uses the PRNG.
11687

117-
.. ipython:: python
88+
.. code-block:: ipython
11889
119-
values = np.empty(10000000)
120-
rg = Xorshift1024().generator
121-
%timeit rg.standard_normal(out=values)
90+
In [5]: values = np.empty(10000000)
91+
...: rg = Xorshift1024().generator
92+
...: %timeit rg.standard_normal(out=values)
93+
220 ms ± 27.3 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
12294
12395
The gains are substantial and the scaling is reasonable even for large that
12496
are only moderately large. The gains are even larger when compared to a call
12597
that does not use an existing array due to array creation overhead.
12698

127-
.. ipython:: python
99+
.. code-block:: ipython
128100
129-
rg = Xorshift1024().generator
130-
%timeit rg.standard_normal(10000000)
101+
In [6]: rg = Xorshift1024().generator
102+
...: %timeit rg.standard_normal(10000000)
103+
...:
104+
256 ms ± 41.8 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

‎_randomgen/doc/source/new-or-different.rst

Lines changed: 8 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -29,29 +29,24 @@ What's New or Different
2929
* The basic random number generators can be used in downstream projects via
3030
Cython.
3131

32+
3233
.. ipython:: python
3334
3435
from randomgen import Xoroshiro128
36+
import numpy.random
3537
rg = Xoroshiro128().generator
36-
%timeit rg.standard_normal(1000000)
37-
from numpy.random import standard_normal
38-
%timeit standard_normal(1000000)
38+
%timeit rg.standard_normal(100000)
39+
%timeit numpy.random.standard_normal(100000)
3940
4041
.. ipython:: python
4142
42-
from randomgen import Xoroshiro128
43-
rg = Xoroshiro128().generator
44-
%timeit rg.standard_exponential(1000000)
45-
from numpy.random import standard_exponential
46-
%timeit standard_exponential(1000000)
43+
%timeit rg.standard_exponential(100000)
44+
%timeit numpy.random.standard_exponential(100000)
4745
4846
.. ipython:: python
4947
50-
from randomgen import Xoroshiro128
51-
rg = Xoroshiro128().generator
52-
%timeit rg.standard_gamma(3.0, 1000000)
53-
from numpy.random import standard_gamma
54-
%timeit standard_gamma(3.0, 1000000)
48+
%timeit rg.standard_gamma(3.0, 100000)
49+
%timeit numpy.random.standard_gamma(3.0, 100000)
5550
5651
* Optional ``dtype`` argument that accepts ``np.float32`` or ``np.float64``
5752
to produce either single or double prevision uniform random variables for
@@ -66,8 +61,6 @@ What's New or Different
6661

6762
.. ipython:: python
6863
69-
from randomgen import Xoroshiro128
70-
rg = Xoroshiro128().generator
7164
rg.seed(0)
7265
rg.random_sample(3, dtype='d')
7366
rg.seed(0)
@@ -86,8 +79,6 @@ What's New or Different
8679

8780
.. ipython:: python
8881
89-
from randomgen import Xoroshiro128
90-
rg = Xoroshiro128(0).generator
9182
existing = np.zeros(4)
9283
rg.random_sample(out=existing[:2])
9384
print(existing)

0 commit comments

Comments
 (0)
0