8000 Training MLP using l-bfgs limited to default l-bfgs maxiter value · Issue #9273 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]
Skip to content

Training MLP using l-bfgs limited to default l-bfgs maxiter value #9273

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
daniel-perry opened this issue Jul 3, 2017 · 1 comment · Fixed by #9274
Closed

Training MLP using l-bfgs limited to default l-bfgs maxiter value #9273

daniel-perry opened this issue Jul 3, 2017 · 1 comment · Fixed by #9274
Labels
Bug Easy Well-defined and straightforward way to resolve help wanted

Comments

@daniel-perry
Copy link
Contributor
daniel-perry commented Jul 3, 2017 •
edited
Loading

Description

Training an MLP regressor (or classifier) using l-bfgs currently cannot run for more than (approx) 15000 iterations.
This artificial limit is caused by the call site to l-bfgs passing the MLP argument value "max_iters" to the argument for "maxfun" (maximum number of function calls), but not for "maxiter" (maximum number of iterations), so that no matter how large a number you pass as "max_iters" to train for MLP, the iterations are capped by the default value for maxiter (15000).

Steps/Code to Reproduce

Fit an MLP for a problem that requires > 15000 iterations

Here is an example (tested in python 2.7):
https://gist.github.com/daniel-perry/d9e356a03936673e58e0ce47d5fc70ef

(you will need data.npy from the gist linked to above)

from __future__ import print_function
import numpy as np
from sklearn.neural_network import MLPRegressor

train = np.load("data.npy").tolist()

max_iter = 18000
clf = MLPRegressor(max_iter=max_iter, activation='relu', solver='lbfgs', verbose=True)

clf.fit(train["train_x"],train["train_y"])

print("score: ", clf.score(train["train_x"],train["train_y"]))
print("iters: ", clf.n_iter_, " / ", max_iter)

Expected Results

The training should run for 18000 iterations.

Actual Results

The training runs for 15000 iterations.

Versions

Here are my local version details, though the problem appears to exist on the current head, and so should exist for any python/sklearn versions.

'Python', '2.7.12 (default, Jul 1 2016, 15:12:24) \n[GCC 5.4.0 20160609]'
'NumPy', '1.13.0'
'SciPy', '0.19.1'
'Scikit-Learn', '0.18'
@daniel-perry daniel-perry mentioned this issue Jul 3, 2017
Merged
@jnothman jnothman added the Bug label Jul 3, 2017
@jnothman jnothman mentioned this issue Feb 28, 2018
Closed
1 task
@amueller amueller added Easy Well-defined and straightforward way to resolve help wanted labels May 22, 2018
@amueller
Copy link
Member

We should make sure that if the user passes a max_iter that's greater than the default maxiter we change maxiter as well. I think using maxfun instead of maxiter makes sense as it makes the computation more comparable to the other solvers, but we should document better.

@rth rth closed this as completed in #9274 Jul 2, 2019
@vascosmota vascosmota mentioned this issue Sep 27, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug Easy Well-defined and straightforward way to resolve help wanted
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[MRG] Adding max_fun parameter to MLP for use in lbfgs optimization daniel-perry/scikit-learn
4 participants
@jnothman @amueller @daniel-perry @raghavrv
0