[WIP] ENH/FIX Support large sparse matrices #9678

kdhingra307 · 2017-09-03T00:24:35Z

Fixes #9545
Partially addresses #4149
Partially addresses #2969

accept_large_sparse option is added in linear_model/base.py, linear_model/logistic.py and decomposition/truncated_svd,nml

while sparse_indices_check directly block all calls to liblinear

Reference Issue

What does this implement/fix? Explain your changes.

Any other comments?

…eck_array and new functon sparse_indices_check

jnothman · 2017-09-03T02:49:29Z

thanks for this! ping @rth

jnothman

Note I have made some changes to the title and description to conform with our conventions, and to mark that this resolves some issues (which allows github to automatically close them when this PR gets merged).

I am somewhat concerned that you have bitten off a bit more than you can chew. #9545 was an easy issue as long as you constrained it to fixing the SVM bug. You've tried to fix the broader issue, which will require more work. We encourage new contributors to take on small issues first so that they learn the ropes before we have to hold their hands through a large issue. If you do not feel comfortable making the following changes, perhaps you should scale this back to the smaller scope of fixing #9545:

You have some flake8 failures. Please always ensure your changes conform to PEP8.
The changes to check_array and check_X_y need tests in sklearn/utils/tests/test_validation.py, including assertions that appropriate errors are raised.
Please extend check_estimator_sparse_data in sklearn/utils/estimator_checks.py to check that either large sparse matrices are accepted or that the correct error message is raised.

jnothman · 2017-09-03T02:56:20Z

sklearn/decomposition/nmf.py

@@ -970,8 +970,7 @@ def non_negative_factorization(X, W=None, H=None, n_components=None,
    Fevotte, C., & Idier, J. (2011). Algorithms for nonnegative matrix
    factorization with the beta-divergence. Neural Computation, 23(9).
    """
-
-    X = check_array(X, accept_sparse=('csr', 'csc'), dtype=float)
+    X = check_array(X, accept_sparse=('csr', 'csc'), dtype=float,accept_large_sparse=True)


Pep8: space after comma; line length should be at most 79 characters. See Travis output for more pep8 errors

jnothman · 2017-09-03T02:56:40Z

sklearn/decomposition/nmf.py

@@ -1012,7 +1011,6 @@ def non_negative_factorization(X, W=None, H=None, n_components=None,

    l1_reg_W, l1_reg_H, l2_reg_W, l2_reg_H = _compute_regularization(
        alpha, l1_ratio, regularization)
-


Please try to avoid modifying unrelated code

jnothman · 2017-09-03T03:11:26Z

sklearn/utils/validation.py

@@ -499,14 +499,38 @@ def check_array(array, accept_sparse=False, dtype="numeric", order=None,
        msg = ("Data with input dtype %s was converted to %s%s."
               % (dtype_orig, array.dtype, context))
        warnings.warn(msg, DataConversionWarning)
+
+    ## Indices Datatype regulation
+    sparse_indices_check(array,accept_large_sparse)


This should be moved above alongside _ensure_sparse_format

jnothman · 2017-09-03T03:12:07Z

sklearn/svm/base.py

@@ -875,6 +875,9 @@ def _fit_liblinear(X, y, C, fit_intercept, intercept_scaling, class_weight,
    libsvm_sparse.set_verbosity_wrap(verbose)
    liblinear.set_verbosity_wrap(verbose)

+    ##Liblinear doesnt support 64bit sparse matrix indices yet
+    sparse_indices_check(X)


Is there a reason not to use check_array or check_X_y in this case?

jnothman · 2017-09-03T03:12:36Z

sklearn/utils/validation.py

    return array

+def sparse_indices_check(array,accept_large_sparse=False):


This should probably be _check_large_sparse or something.

jnothman · 2017-09-03T03:13:13Z

sklearn/utils/validation.py

+            if hasattr(array, "indices"):
+                if hasattr(array.indices, "dtype"):
+                    if array.indices.dtype not in supported_indices:
+                        raise ValueError("CSR arrays accepts only 32 bit integers indexing. You provided %s bit indexing"


I think just "Sparse matrices with 64bit indices are not supported"

because even 8bit or other indices are also crashing, so that's why I added supported_indices array

scipy.sparse should refuse to create <32bit integer indices. It only supports 32 and 64.

@jnothman I was able to create with int8 and it gave segmentation fault in truncatedsvd, so I think we should add this matrix only

jnothman · 2017-09-03T03:13:37Z

sklearn/utils/validation.py

+
+    """
+    if accept_large_sparse==False:
+        if type(array) ==sp.csr.csr_matrix:


Use array.format.

We also need to check for:

csc_matrix.indices;

coo_matrix.row

dia_matrix.offsets

And for completeness's sake:

bsr_matrix.indices

will do for bsr_matrix directly

jnothman · 2017-09-03T03:14:18Z

sklearn/utils/validation.py

+    """
+    if accept_large_sparse==False:
+        if type(array) ==sp.csr.csr_matrix:
+            supported_indices=["int32"]


I don't think we need the indirection of an additional variable here.

jnothman · 2017-09-03T03:18:57Z

sklearn/utils/validation.py

+        if type(array) ==sp.csr.csr_matrix:
+            supported_indices=["int32"]
+            if hasattr(array, "indices"):
+                if hasattr(array.indices, "dtype"):


please use and where possible instead of nested ifs.

rth · 2017-09-03T10:12:13Z

sklearn/linear_model/logistic.py

@@ -1213,7 +1213,7 @@ def fit(self, X, y, sample_weight=None):
            _dtype = np.float64

        X, y = check_X_y(X, y, accept_sparse='csr', dtype=_dtype,
-                         order="C")
+                         order="C",accept_large_sparse=True)


Note that this is solver dependent : as far as I could tell newton-cg and lbfgs worked while sag failed..

yes it is solver dependent, shouldn't this work this way only?

Well maybe reflect that SAG doesn't accept those with ?

accept_large_array=(self.solver not in ['sag', 'saga'])

[need to check which ones work..]

rth · 2017-09-03T10:13:07Z

Thanks for this PR @kdhingra307 !

I have made some changes to the title and description to conform with our conventions, and to mark that this resolves some issues

Fixes #9545
Fixes #4149
Fixes #2969

@jnothman I think this fixes #9545 and only starts to address the other two, so it might be too early to auto-close those..

jnothman · 2017-11-06T10:47:49Z

@kdhingra307, are you still working on this? I'd really like to see this fixed in the next release.

…into sparse_indices_check Merging changes between fork and master:

jnothman · 2017-11-26T11:10:38Z

Hi @kdhingra307, given your silence, we'll try get someone else to complete this. Let us know if we are mistaken on that. The PR still suffers from a lot of PEP8 errors.

kdhingra307 · 2017-11-26T14:38:39Z

@jnothman I had an annual exam today(CAT for my postgrad), thats why i was not online for last 2 month, I will be active from today only

jnothman · 2017-11-26T22:25:15Z

Great, thanks!

jnothman · 2017-12-12T00:10:56Z

@kdhingra307, we really need the SVM at least fixed. Are you continuing to work on this?

gxyd · 2017-12-18T06:00:09Z

@kdhingra307 a polite ping. If you intend to continue working on this in near future it is fine, otherwise I could try to get this PR complete.

…he go

jnothman · 2017-12-31T12:43:32Z

sklearn/utils/validation.py

+    Only Int32 are supported for now
+
+    """
+    if accept_large_sparse is False and type(array) == sp.csr.csr_matrix:


we need to be checking CSC matrices and COO matrices also, at least. Why not just use isspmatrix

will do this in next commit

@jnothman i checked for almost all the algorithms with CSC and COO matrices, it passed all the test cases.

I analyzed why these matrices are behaving like these and found that during preprocessing CSC and COO matrices are converted to CSR matrices using _ensure_sparse_format function

as _check_large_sparse is called after _ensure_sparse_format, therefore I don't feel the need to handle CSC and COO matrices cases.

I will do extensive check in order to find out whether any algorithm is prioritizing COO/CSC over CSR, if you know any plz share (it would ease the implementation)

I think it would be a good idea to just handle other cases here, as _check_large_sparse can and is being used as a standalone validation routine.

Please avoid is False. Just use if not accept_large_sparse

jnothman · 2017-12-31T12:43:46Z

sklearn/utils/validation.py

+            raise ValueError("CSR arrays accepts only 32 bit integer indexing."
+                             " You provided % s bit indexing"
+                             % array.indices.dtype)
+        if (hasattr(array, "indptr") and hasattr(array.indptr,  "dtype") and


what case does this handle? It's not tested.

basically if any of indices or indptr are 64bit,
sparse will crash (so to check OR condition)

jnothman · 2017-12-31T12:43:52Z

sklearn/utils/validation.py

+        supported_indices = ["int32"]
+        if (hasattr(array, "indices") and hasattr(array.indices, "dtype") and
+                array.indices.dtype not in supported_indices):
+            raise ValueError("CSR arrays accepts only 32 bit integer indexing."


arrays -> matrices

And I think what you mean is something more like "Only sparse matrices with 32-bit integer indices are accepted. Got %s indices."

arrays->matrices?
I didn't understand

will change the exception though

The correct term is "CSR matrix" not "CSR array"

jnothman · 2017-12-31T12:45:26Z

sklearn/utils/validation.py

 def check_X_y(X, y, accept_sparse=False, dtype="numeric", order=None,
              copy=False, force_all_finite=True, ensure_2d=True,
              allow_nd=False, multi_output=False, ensure_min_samples=1,
              ensure_min_features=1, y_numeric=False,
-              warn_on_dtype=False, estimator=None):
+              warn_on_dtype=False, estimator=None, accept_large_sparse=False):


This needs documentation under Parameters. I also wonder whether we should just be some alteration to accept_sparse, e.g. having accept_sparse=['csr', 'csr64', ...] means that a 32bit-indexed or a 64bit-indexed csr is accepted.

I feel this should be different variable altogether

cuz in later phases, when 64bits indices will be supported (less iteration would be required to fix)

jnothman · 2017-12-31T12:49:55Z

Btw, I would certainly not call this issue a "good first issue". In hindsight, it may be Moderate more than Easy. But I was also initially suggesting that we only fix it piece by piece: make sure SVMs don't crash, then build up from there. Your current general solution requires, in turn, a general test.

kdhingra307 · 2018-01-03T15:00:38Z

@jnothman it works fine with SVM

I will just share a modified test case file (originally by @rth), which will support this

kdhingra307 · 2018-01-03T20:33:22Z

@jnothman pfa

https://github.com/kdhingra307/scikit-learn/blob/test_files/test_64bit_csr_indices.py

jnothman · 2018-02-12T09:12:00Z

So yes, rather than removing the tests for that case, let's just throw an approppriate error in check_array. Thanks.

… its test case

kdhingra307 · 2018-02-12T16:52:36Z

@jnothman done that in _ensure_sparse_format,

it raises an issue with large sparse indices over scipy versions <0.14

kdhingra307

completed

do u think anything is missing or is there any bigger scope that yet has to be dealt with?

jnothman · 2018-02-12T21:43:02Z

sklearn/preprocessing/data.py

@@ -1263,6 +1263,7 @@ class PolynomialFeatures(BaseEstimator, TransformerMixin):
    See :ref:`examples/linear_model/plot_polynomial_interpolation.py
    <sphx_glr_auto_examples_linear_model_plot_polynomial_interpolation.py>`
    """
+


I would like you to remove these blank lines. It is not our convention. And extra changes means extra merge conflicts

jnothman · 2018-02-12T21:44:14Z

sklearn/utils/estimator_checks.py

-                raise
-        except Exception:
+                if "64" in matrix_format:
+                    raise AssertionError("Estimator % s doesn't seem to "


remove space between % and s?

jnothman · 2018-02-12T21:44:33Z

sklearn/utils/estimator_checks.py

+                                         % (name, matrix_format))
+                else:
+                    print("Estimator %s doesn't seem to fail gracefully on "
+                          "sparse data: error message state explicitly that "


message *should state

jnothman · 2018-02-12T21:45:04Z

sklearn/utils/tests/test_estimator_checks.py

+                        "Estimator doesn't support 64-bit indices")
+            elif X.getformat() in ["csc", "csr"]:
+                if X.indices.dtype == "int64" or X.indptr.dtype == "int64":
+                    raise ValueError(


codecov says this is never run. Why not?

jnothman · 2018-02-12T21:46:00Z

sklearn/utils/tests/test_validation.py

@@ -427,6 +429,48 @@ def test_check_array_accept_sparse_no_exception():
    check_array(X_csr, accept_sparse=('csr',))


+def test_check_array_accept_large_sparse_no_exception():
+


rm blank line

jnothman · 2018-02-12T21:46:54Z

sklearn/utils/validation.py

+            indices_datatype = getattr(X, key).dtype
+            if (indices_datatype not in supported_indices):
+                if not_supported_scipy_version:
+                    raise ValueError("Scipy Version %s does not support large"


lowercase "version"

Perhaps scipy -> scipy.sparse

jnothman · 2018-02-12T21:49:30Z

sklearn/utils/validation.py

@@ -547,14 +555,44 @@ def check_array(array, accept_sparse=False, dtype="numeric", order=None,
        msg = ("Data with input dtype %s was converted to %s%s."
               % (dtype_orig, array.dtype, context))
        warnings.warn(msg, DataConversionWarning)
+
+    # Indices Datatype regulation
+    _check_large_sparse(array, accept_large_sparse)


Is there a reason we are not just doing this in _ensure_sparse_format called above?

there is no reason,

I shifted there but below some of the functions, because those function may change the indices type of spmatrix

Really? Where do you mean? If it's possible to move it to _ensure_sparse_format I'd much rather it

jnothman · 2018-02-12T21:50:12Z

sklearn/utils/validation.py

+       Only Int32 are supported for now
+    """
+
+    if (not accept_large_sparse and sp.issparse(X)):


rm unnecessary parentheses

jnothman · 2018-02-12T21:50:15Z

sklearn/utils/validation.py

+def _check_large_sparse(X, accept_large_sparse=False,
+                        not_supported_scipy_version=False):
+    """Indices Regulation for CSR Matrices.
+       Only Int32 are supported for now


jnothman · 2018-02-12T21:51:03Z

sklearn/utils/validation.py

@@ -297,6 +300,10 @@ def _ensure_sparse_format(spmatrix, accept_sparse, dtype, copy,
    if isinstance(accept_sparse, six.string_types):
        accept_sparse = [accept_sparse]

+    if LooseVersion(scipy_version) < '0.14.0':


I think you should do this check once when the module is loaded and store a LARGE_SPARSE_SUPPORTED global variable.

Do you understand what I mean here?

jnothman

Despite the requested changes this is very good

jnothman · 2018-02-13T11:06:21Z

sklearn/utils/validation.py

@@ -297,6 +300,10 @@ def _ensure_sparse_format(spmatrix, accept_sparse, dtype, copy,
    if isinstance(accept_sparse, six.string_types):
        accept_sparse = [accept_sparse]

+    if LooseVersion(scipy_version) < '0.14.0':


Do you understand what I mean here?

jnothman · 2018-02-13T11:09:12Z

sklearn/utils/validation.py

@@ -345,7 +352,8 @@ def _ensure_no_complex_data(array):
 def check_array(array, accept_sparse=False, dtype="numeric", order=None,
                copy=False, force_all_finite=True, ensure_2d=True,
                allow_nd=False, ensure_min_samples=1, ensure_min_features=1,
-                warn_on_dtype=False, estimator=None):
+                warn_on_dtype=False, estimator=None,
+                accept_large_sparse=False):


I am not sure this should default to false, as that may break current behaviour in external uses of check_array

I know, this changes a lot of lines in the PR. Sorryi did not realise it sooner

jnothman · 2018-02-13T11:10:55Z

sklearn/utils/validation.py

@@ -547,14 +555,44 @@ def check_array(array, accept_sparse=False, dtype="numeric", order=None,
        msg = ("Data with input dtype %s was converted to %s%s."
               % (dtype_orig, array.dtype, context))
        warnings.warn(msg, DataConversionWarning)
+
+    # Indices Datatype regulation
+    _check_large_sparse(array, accept_large_sparse)


Really? Where do you mean? If it's possible to move it to _ensure_sparse_format I'd much rather it

jnothman · 2018-02-13T11:18:03Z

After those are fixed, please change the title to MRG

Please add an entry to the change log at doc/whats_new/v0.20.rst. Like the other entries there, please reference this pull request with :issue: and credit yourself (and other contributors if applicable) with :user:

kdhingra307 · 2018-02-13T11:23:11Z

@jnothman i did all of the changes but just wanted to push them in different segments

I understood your LARGE_SPARSE_SUPPORTED global variable, but do not know where and when to load it(if u can give some insight)

kdhingra307 · 2018-02-13T11:23:41Z

and i will push all the changes after this global initialization of LARGE_SPARSE_SUPPORTED

jnothman · 2018-02-13T11:30:02Z

You can do it right at the top of validation.py

jnothman · 2018-02-13T20:34:53Z

I'm not sure what you mean. Give an example?

…

On 14 February 2018 at 06:10, Karan Dhingra ***@***.***> wrote: @jnothman <https://github.com/jnothman> I have reduced the complexity of the whole patch, I was allowing the case when someone purposefully enforced the indices of 64bits but entries are way less. But there is no purpose as no one will do that practically and we should also block that, i guess what do u think? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9678 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEz65muzLR8-83yIHxAj5HZh99AIOC_ks5tUd4ggaJpZM4PLDF-> .

kdhingra307 · 2018-02-13T20:43:05Z

@jnothmanI have rewritten most of the stuffs to ensure any possible micro-optimization plus lesser usage per loop

also now its far easier to reverse accept_large_sparse from False to True (basically you want default value set to be True, I just wanna know any particular or specific reason??)

and I will check the commit thoroughly once for any unnecessary changes (so that you dont have to)

rth · 2018-02-13T20:44:50Z

also now its far easier to reverse accept_large_sparse from False to True (basically you want default value set to be True, I just wanna know any particular or specific reason??)

I'm also +1 on having accept_large_sparse=True by default. Support for 64 bit indices should be the norm not an exception. Sorry @kdhingra307 about all the resulting changes and thanks for your work on this PR.

kdhingra307 · 2018-02-13T20:46:57Z

@rth no issues, will migrate that change in next commit

jnothman · 2018-02-13T20:54:21Z

I just mean that existing code might be relying on check_array letting large sparse matrices through would be broken if we rejected them by default. You've fixed up the cases in scikit-learn (although perhaps not in meta-estimators) but not in custom uses

sklearn-lgtm · 2018-02-13T21:13:12Z

This pull request introduces 37 alerts and fixes 18 - view on lgtm.com

new alerts:

37 for Explicit export is not defined

fixed alerts:

6 for Mismatch between signature and use of an overridden method
3 for Missing call to init during object initialization
3 for Potentially uninitialized local variable
2 for Comparison using is when operands support eq
2 for Wrong number of arguments for format
1 for Conflicting attributes in base classes
1 for Mismatch between signature and use of an overriding method

Comment posted by lgtm.com

jnothman · 2018-02-13T21:25:31Z

(don't worry about the lgtm.com alerts. it's a glitch on their side, and they're investigating)

jnothman

Yes, the recent changes look good. Just to change the default...

jnothman · 2018-02-13T21:33:36Z

sklearn/utils/validation.py

@@ -33,6 +33,12 @@
 # performance profiling.
 warnings.simplefilter('ignore', NonBLASDotWarning)

+# checking whether large sparse are supported by scipy or not
+if LooseVersion(scipy_version) >= '0.14.0':
+    LARGE_SPARSE_SUPPORTED = True


You can directly do = LooseV... >= ...0' without an if statement

jnothman · 2018-02-13T21:34:11Z

sklearn/utils/validation.py

-       Only Int32 are supported for now
+def _check_large_sparse(X, accept_large_sparse=False):
+    """Indices Regulation of Sparse Matrices
+       for estimators


Drop "for estimators". It's also for other things

rth · 2018-03-08T22:43:51Z

sklearn/utils/validation.py

@@ -30,6 +33,9 @@
 # performance profiling.
 warnings.simplefilter('ignore', NonBLASDotWarning)

+# checking whether large sparse are supported by scipy or not


In the comment let's say "large sparse" -> "sparse arrays with 64 bit indices"

rth

This does look nice.

A few more comments below. Please ping me when the default is changed.

rth · 2018-03-08T22:47:50Z

sklearn/utils/estimator_checks.py

+
+        Parameters
+        ----------
+


Remove whitespace (also below Returns)

rth · 2018-03-08T22:50:44Z

sklearn/utils/estimator_checks.py

@@ -403,6 +404,40 @@ def pairwise_estimator_convert_X(X, estimator, kernel=linear_kernel):
    return X


+def _generate_sparse_matrix(X_csr):
+    """Generate matrices in multiple formats for CSR,CSC and COO matrices


I know it's a private function, but this could be reformulated to,

Generate 32 and 64 bit indexed sparse matrices in CSR, CSC and COO format

rth · 2018-03-08T22:58:45Z

sklearn/utils/estimator_checks.py

+    """
+
+    for sparse_format in ['dok', 'lil', 'dia', 'bsr', 'csr', 'csc', 'coo']:
+        yield sparse_format, X_csr.asformat(sparse_format)


Could we add a X_csr.copy().asformat here (and on L430) ?

Just to be sure that we are not taking in e.g. a CSR matrix, returning it as it is when sparse_format="csr" that get's then in place modified by some estimator and so the next iteration of sparse_format="csc" would use this modified version.

There should be a copy=True kwarg iirc

jnothman · 2018-06-06T14:50:54Z

We'd really like to have this merged in the next release. @kdhingra307, are you with us? Perhaps a core dev should adopt this, otherwise...

Fix to Issue #9545, it includes a new param accept_large_sparse to ch…

9de3138

…eck_array and new functon sparse_indices_check

jnothman changed the title ~~Fix to Issue #9545,~~ [MRG] ENH Support large sparse matrices Sep 3, 2017

jnothman changed the title ~~[MRG] ENH Support large sparse matrices~~ [MRG] ENH/FIX Support large sparse matrices Sep 3, 2017

jnothman changed the title ~~[MRG] ENH/FIX Support large sparse matrices~~ [WIP] ENH/FIX Support large sparse matrices Sep 3, 2017

jnothman reviewed Sep 3, 2017

View reviewed changes

rth reviewed Sep 3, 2017

View reviewed changes

jnothman added the Bug label Nov 6, 2017

Merge branch 'master' of https://github.com/scikit-learn/scikit-learn …

ecada4d

…into sparse_indices_check Merging changes between fork and master:

jnothman added help wanted Stalled labels Nov 26, 2017

jnothman added this to the 0.20 milestone Nov 29, 2017

jnothman added Stalled and removed Stalled help wanted labels Nov 29, 2017

kdhingra307 added 3 commits December 28, 2017 02:27

Cleared Flake8 issues and also chained if loop issue, test cases on t…

3651c17

…he go

Changed Naming Conventions

9e43610

test cases added, flake should not give any error now

a049743

jnothman reviewed Dec 31, 2017

View reviewed changes

More checks

9e60f95

Dhingra added 2 commits February 12, 2018 20:50

Added check for unsupported scipy version in validation.py along with…

11690ef

… its test case

modified sparse version check for test_validation

14f0384

kdhingra307 commented Feb 12, 2018

View reviewed changes

jnothman reviewed Feb 12, 2018

View reviewed changes

Dhingra added 3 commits February 13, 2018 14:27

Reverted all of the unnecessary changes

f9fe4b8

Reverted unnecessary changed, which got missed in previous commit

9f6e0f7

sorted flake8 issue in nmf.py

1d0a282

jnothman reviewed Feb 13, 2018

View reviewed changes

Revised whole algorithms from ground up plus more flatter now

0648248

jnothman reviewed Feb 13, 2018

View reviewed changes

Changed few norms, default case still needed to be changed

4e8b9a0

jnothman mentioned this pull request Mar 6, 2018

ValueError: Buffer dtype mismatch, expected 'int' but got 'long' #10758

Closed

rth reviewed Mar 8, 2018

View reviewed changes

jnothman mentioned this pull request Jun 20, 2018

[MRG] Large sparse matrix support #11327

Merged

2 tasks

rth closed this in #11327 Jun 25, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] ENH/FIX Support large sparse matrices #9678

[WIP] ENH/FIX Support large sparse matrices #9678

		@@ -1012,7 +1011,6 @@ def non_negative_factorization(X, W=None, H=None, n_components=None,

		l1_reg_W, l1_reg_H, l2_reg_W, l2_reg_H = _compute_regularization(
		alpha, l1_ratio, regularization)

		return array

		def sparse_indices_check(array,accept_large_sparse=False):

		@@ -427,6 +429,48 @@ def test_check_array_accept_sparse_no_exception():
		check_array(X_csr, accept_sparse=('csr',))


		def test_check_array_accept_large_sparse_no_exception():

[WIP] ENH/FIX Support large sparse matrices #9678

[WIP] ENH/FIX Support large sparse matrices #9678

Conversation

Reference Issue

What does this implement/fix? Explain your changes.

Any other comments?

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment