Abstract
Ill-posed problems are widely existed in signal processing. In this paper, we review popular regularization models such as truncated singular value decomposition regularization, iterative regularization, variational regularization. Meanwhile, we also retrospect popular optimization approaches and regularization parameter choice methods. In fact, the regularization problem is inherently a multi-objective problem. The traditional methods usually combine the fidelity term and the regularization term into a single-objective with regularization parameters, which are difficult to tune. Therefore, we propose a multi-objective framework for ill-posed problems, which can handle complex features of problem such as non-convexity, discontinuity. In this framework, the fidelity term and regularization term are optimized simultaneously to gain more insights into the ill-posed problems. A case study on signal recovery shows the effectiveness of the multi-objective framework for ill-posed problems.
Similar content being viewed by others
References
Hadamard J. Sur les Problemes aux Derivees Partielles et Leur Signification Physique. Princeton University Bulletin, 1902, 13: 49–52
Kabanikhin S I. Inverse and Ill-Posed Problems: Theory and Applications. Berlin: Water De Gruyter, 2011
Zhang B Y, Xu D H, Liu T W. Stabilized algorithms for ill-posed problems in signal processing. In: Proceedings of the IEEE International Conferences on Info-tech and Info-net. 2001, 1: 375–380
Scherzer O. Handbook of Mathematical Methods in Imaging. Springer Science & Business Media, 2011
Groetsch C W. Inverse problems in the mathematical sciences. Mathematics of Computation, 1993, 63(5): 799–811
Rudin L I, Osher S, Fatemi E. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 1992, 60(1): 259–268
Tikhonov A N. Solution of incorrectly formulated problems and the regularization method. Soviet Math, 1963, 4: 1035–1038
Tikhonov A N, Arsenin V I. Solutions of Ill-posed Problems. Washington, DC: V. H. Winston & Sons, 1977
Landweber L. An iteration formula for Fredholm integral equations of the first kind. American Journal of Mathematics, 1951, 73(3): 615–624
Hestenes M R, Stiefel E. Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards, 1952, 49(6): 409–436
Vogel C R. Computational Methods for Inverse Problems. Society for Industrial and Applied Mathematics, 2002, 23
Hansen P C. The truncated SVD as a method for regularization. Bit Numerical Mathematics, 1987, 27(4): 534–553
Honerkamp J, Weese J. Tikhonovs regularization method for ill-posed problems. Continuum Mechanics and Thermodynamics, 1990, 2(1): 17–30
Zhang X Q, Burger M, Bresson X, Osher S. Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM Journal on Imaging Sciences, 2010, 3(3): 253–276
Deb K. Multi-Objective Optimization Using Evolutionary Algorithms. New York: John Wiley & Sons, 2001, 16
Fonseca CM, Fleming P J. An overview of evolutionary algorithms in multiobjective optimization. Evolutionary Computation, 1995, 3(1): 1–16
Coello C A C, Van Veldhuizen D A, Lamont G B. Evolutionary Algorithms for Solving Multi-objective Problems. New York: Kluwer Academic, 2002
Tan K C, Khor E F, Lee T H. Multiobjective Evolutionary Algorithms and Applications. Springer Science & Business Media, 2005
Knowles J, Corne D, Deb K. Multiobjective Problem Solving from Nature: from Concepts to Applications. Springer Science & Business Media, 2008
Raquel C, Yao X. Dynamic multi-objective optimization: a survey of the state-of-the-art. In: Yang S X, Yao X, eds. Evolutionary Computation for Dynamic Optimization Problems. Springer Berlin Heidelberg, 2013, 85–106
Lücken C V, Barán B, Brizuela C. A survey on multi-objective evolutionary algorithms for many-objective problems. Computational Optimization and Applications, 2014, 58(3): 707–756
Hwang C L, Masud A S M. Multiple Objective Decision Making- Methods and Applications. Springer Science & Business Media, 1979, 164
Girosi F, Jones M B, Poggio T. Regularization theory and neural networks architectures. Neural Computation, 1995, 7(2): 219–269
Belge M, Kilmer M E, Miller E L. Efficient determination of multiple regularization parameters in a generalized L-curve framework. Inverse Problems, 2002, 18(4): 1161
Hansen P C. Rank-deficient and discrete ill-posed problems: numerical aspects of linear inversion. American Mathematical Monthly, 1997, 4(5): 491
Eriksson P, Jiménez C, Buehler S A. Qpack, a general tool for instrument simulation and retrieval work. Journal of Quantitative Spectroscopy and Radiative Transfer, 2005, 91(1): 47–64
Giusti E. Minimal Surfaces and Functions of Bounded Variation. Springer Science & Business Media, 1984, 80
Catté F, Coll T. Image selective smoothing and edge detection by nonlinear diffusion. SIAM Journal on Numerical Analysis, 1992, 29(1): 182–193
Björck A. Numerical Methods for Least Squares Problems. Society for Industrial and Applied Mathematics, 1996
Groetsch C W. The Theory of Tikhonov Regularization for Fredholm Equations of the First Kind. Pitman Advanced Publishing Program, 1984
Hanson R J. A numerical method for solving Fredholm integral equations of the first kind using singular values. SIAM Journal on Numerical Analysis, 1971, 8(3): 616–622
Stewart G W. Rank degeneracy. SIAM Journal on Scientific and Statistical Computing, 1984, 5(2): 403–413
Hansen P C, Sekii T, Shibahashi H. The modified truncated SVD method for regularization in general form. SIAM Journal on Scientific and Statistical Computing, 1992, 13(5): 1142–1150
Van Loan C F. Generalizing the singular value decomposition. SIAM Journal on Numerical Analysis, 1976, 13(1): 76–83
Hansen P C. Regularization, GSVD and truncated GSVD. BIT Numerical Mathematics, 1989, 29(3): 491–504
Paige C C. Computing the generalized singular value decomposition. SIAM Journal on Scientific and Statistical Computing, 1986, 7(4): 1126–1146
Morigi S, Reichel L, Sgallari F. A truncated projected SVD method for linear discrete ill-posed problems. Numerical Algorithms, 2006, 43(3): 197–213
Fernando K V, Hammarling S. A product induced singular value decomposition (ΠSVD) for two matrices and balanced realization. In: Proceedings of SIAM Conference on Linear Algebra in Signals, Systems and Control. 1988, 128–140
Zha H Y. The restricted singular value decomposition of matrix triplets. SIAM Journal on Matrix Analysis and Applications, 1991, 12(1): 172–194
De Moor B, Golub G H. The restricted singular value decomposition: properties and applications. SIAM Journal on Matrix Analysis and Applications, 1991, 12(3): 401–425
De Moor B, Zha H Y. A tree of generalizations of the ordinary singular value decomposition. Linear Algebra and Its Applications, 1991, 147: 469–500
De Moor B. Generalizations of the OSVD: structure, properties and applications. In: Vaccaro R J, ed. SVD & Signal Processing, II: Algorithms, Analysis & Applications. 1991, 83–98
Noschese S, Reichel L. A modified TSVD method for discrete ill posed problems. Numerical Linear Algebra with Applications, (in press)
Dykes L, Noschese S, Reichel L. Rescaling the GSVD with application to ill-posed problems. Numerical Algorithms, 2015, 68(3): 531–545
Edo L, Franco W, Martinsson P G, Rokhlin V, Tygert M. Randomized algorithms for the low-rank approximation of matrices. Proceedings of the National Academy of Sciences, 2007, 104(51): 20167–20172
Woolfe F, Liberty E, Rokhlin V, Tygert M. A fast randomized algorithm for the approximation of matrices. Applied & Computational Harmonic Analysis, 2008, 25(3): 335–366
Sifuentes J, Gimbutas Z, Greengard L. Randomized methods for rankdeficient linear systems. Electronic Transactions on Numerical Analysis, 2015, 44: 177–188
Liu Y G, Lei Y J, Li C G, Xu W Z, Pu Y F. A random algorithm for low-rank decomposition of large-scale matrices with missing entries. IEEE Transactions on Image Processing, 2015, 24(11): 4502–4511
Sekii T. Two-dimensional inversion for solar internal rotation. Publications of the Astronomical Society of Japan, 1991, 43: 381–411
Scales J A. Uncertainties in seismic inverse calculations. In: Jacobsen B H, Mosegaard K, Sibani P, eds. Inverse Methods. Berlin: Springer-Verlag, 1996, 79–97
Lawless J F, Wang P. A simulation study of ridge and other regression estimators. Communications in Statistics-Theory and Methods, 1976, 5(4): 307–323
Dempster A P, Schatzoff M, Wermuth N. A simulation study of alternatives to ordinary least squares. Journal of the American Statistical Association, 1977, 72(357): 77–91
Hansen P C, O’Leary D P. The use of the L-curve in the regularization of discrete ill-posed problems. SIAM Journal on Scientific Computing, 1993, 14(6): 1487–1503
Hansen P C. Analysis of discrete ill-posed problems by means of the L-curve. SIAM Review, 1992, 34(4): 561–580
Xu P L. Truncated SVD methods for discrete linear ill-posed problems. Geophysical Journal International, 1998, 135(2): 505–514
Wu Z M, Bian S F, Xiang C B, Tong Y D. A new method for TSVD regularization truncated parameter selection. Mathematical Problems in Engineering, 2013
Chicco D, Masseroli M. A discrete optimization approach for SVD best truncation choice based on ROC curves. In: Proceedings of the 13th IEEE International Conference on Bioinformatics and Bioengineering. 2013: 1–4
Golub G H, Heath M, Wahba G. Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics, 1979, 21(2): 215–223
Jbilou K, Reichel L, Sadok H. Vector extrapolation enhanced TSVD for linear discrete ill-posed problems. Numerical Algorithms, 2009, 51(2): 195–208
Bouhamidi A, Jbilou K, Reichel L, Sadok H, Wang Z. Vector extrapolation applied to truncated singular value decomposition and truncated iteration. Journal of Engineering Mathematics, 2015, 93(1): 99–112
Vogel C R. Computational Methods for Inverse Problems. Society for Industrial and Applied Mathematics, 2002
Doicu A, Trautmann T, Schreier F. Numerical Regularization for Atmospheric Inverse Problems. Springer Science & Business Media, 2010
Bakushinsky A B, Goncharsky A V. Iterative Methods for the Solution of Incorrect Problems. Moscow: Nauka, 1989
Rieder A. Keine Probleme mit Inversen Problemen: Eine Einführung in ihre stabile Lösung. Berlin: Springer-Verlag, 2013
Nemirovskiy A S, Polyak B T. Iterative methods for solving linear ill posed problems under precise information. Engineering Cybernetics, 1984, 22(4): 50–56
Brakhage H. On ill-posed problems and the method of conjugate gradients. Inverse and Ill-posed Problems, 1987, 4: 165–175
Hanke M. Accelerated Landweber iterations for the solution of ill posed equations. Numerische Mathematik, 1991, 60(1): 341–373
Barzilai J, Borwein J M. Two-point step size gradient methods. IMA Journal of Numerical Analysis, 1988, 8(1): 141–148
Axelsson O. Iterative Solution Methods. Cambridge: Cambridge University Press, 1996
Van der Sluis A, Van der Vorst H A. The rate of convergence of conjugate gradients. Numerische Mathematik, 1986, 48(5): 543–560
Scales J A, Gersztenkorn A. Robust methods in inverse theory. Inverse Problems, 1988, 4(4): 1071–1091
Björck Å, Eldén L. Methods in numerical algebra for ill-posed problems. Technical Report LiTH-MAT-R-33-1979. 1979
Trefethen L N, Bau D. Numerical Linear Algebra. Society for Industrial and Applied Mathematics, 1997
Calvetti D, Lewis B, Reichel L. On the regularizing properties of the GMRES method. Numerische Mathematik, 2002, 91(4): 605–625
Calvetti D, Lewis B, Reichel L. Alternating Krylov subspace image restoration methods. Journal of Computational and Applied Mathematics, 2012, 236(8): 2049–2062
Brianzi P, Favati P, Menchi O, Romani F. A framework for studying the regularizing properties of Krylov subspace methods. Inverse Problems, 2006, 22(3): 1007–1021
Sonneveld P, Van Gijzen M B. IDR(s): a family of simple and fast algorithms for solving large nonsymmetric systems of linear equations. SIAM Journal on Scientific Computing, 2008, 31(2): 1035–1062
Fong D C L, Saunders M. LSMR: an iterative algorithm for sparse least-squares problems. SIAM Journal on Scientific Computing, 2011, 33(5): 2950–2971
Zhao C, Huang T Z, Zhao X L, Deng L J. Two new efficient iterative regularization methods for image restoration problems. Abstract & Applied Analysis, 2013
Perez A, Gonzalez R C. An iterative thresholding algorithm for image segmentation. IEEE Transactions on Pattern Analysis & Machine Intelligence, 1987, 9(6): 742–751
Beck A, Teboulle M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2009, 2(1): 183–202
Bioucas-Dias J M, Figueiredo M A T. Two-step algorithms for linear inverse problems with non-quadratic regularization. In: Proceedings of the IEEE International Conference on Image Processing. 2007, 105–108
Bioucas-Dias J M, Figueiredo M A T. A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Transactions on Image Processing. 2007, 16(12): 2992–3004
Bayram I, Selesnick I W. A subband adaptive iterative shrinkage/ thresholding algorithm. IEEE Transactions on Signal Processing, 2010, 58(3): 1131–1143
Yamagishi M, Yamada I. Over-relaxation of the fast iterative shrinkage-thresholding algorithm with variable stepsize. Inverse Problems, 2011, 27(10): 105008–105022
Bhotto M Z A, Ahmad M O, Swamy M N S. An improved fast iterative shrinkage thresholding algorithm for image deblurring. SIAM Journal on Imaging Sciences, 2015, 8(3): 1640–1657
Zhang Y D, Dong Z C, Phillips P, Wang S H, Ji G L, Yang J Q. Exponential wavelet iterative shrinkage thresholding algorithm for compressed sensing magnetic resonance imaging. Information Sciences, 2015, 322: 115–132
Zhang Y D, Wang S H, Ji G L, Dong Z C. Exponential wavelet iterative shrinkage thresholding algorithm with random shift for compressed sensing magnetic resonance imaging. IEEJ Transactions on Electrical and Electronic Engineering, 2015, 10(1): 116–117
Wu G M, Luo S Q. Adaptive fixed-point iterative shrinkage/ thresholding algorithm for MR imaging reconstruction using compressed sensing. Magnetic Resonance Imaging, 2014, 32(4): 372–378
Fang E X, Wang J J, Hu D F, Zhang J Y, Zou W, Zhou Y. Adaptive monotone fast iterative shrinkage thresholding algorithm for fluorescence molecular tomography. IET Science Measurement Technology, 2015, 9(5): 587–595
Zuo W M, Meng D Y, Zhang L, Feng X C, Zhang D. A generalized iterated shrinkage algorithm for non-convex sparse coding. In: Proceedings of the IEEE International Conference on Computer Vision. 2013, 217–224
Krishnan D, Fergus R. Fast image deconvolution using hyperlaplacian priors. In: Bengio Y, Schuurmans D, Lafferty J D, et al., eds. Advances in Neural Information Processing Systems 22. 2009, 1033–1041
Chartrand R, Yin W. Iteratively reweighted algorithms for compressive sensing. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. 2008, 3869–3872
She Y Y. An iterative algorithm for fitting nonconvex penalized generalized linear models with grouped predictors. Computational Statistics & Data Analysis, 2012, 56(10): 2976–2990
Gong P H, Zhang C S, Lu Z S, Huang J Z, Ye J P. A general iterative shrinkage and thresholding algorithm for non-convex regularized optimization problems. In: Proceedings of International Conference on Machine Learning. 2013, 37–45
Bredies K, Lorenz D A. Linear convergence of iterative softthresholding. Journal of Fourier Analysis and Applications, 2008, 14(5–6): 813–837
Kowalski M. Thresholding rules and iterative shrinkage/thresholding algorithm: a convergence study. In: Proceedings of the IEEE International Conference on Image Processing. 2014, 4151–4155
Chambolle A, Dossal C. On the convergence of the iterates of the “fast iterative shrinkage/thresholding algorithm”. Journal of Optimization Theory & Applications, 2015, 166(3): 968–982
Johnstone P R, Moulin P. Local and global convergence of an inertial version of forward-backward splitting. Advances in Neural Information Processing Systems, 2014, 1970–1978
Morozov V A. On the solution of functional equations by the method of regularization. Soviet Mathematics Doklady, 1966, 7(11): 414–417
Vainikko G M. The discrepancy principle for a class of regularization methods. USSR Computational Mathematics and Mathematical Physics, 1982, 22(3): 1–19
Vainikko G M. The critical level of discrepancy in regularization methods. USSR Computational Mathematics and Mathematical Physics, 1983, 23(6): 1–9
Plato R. On the discrepancy principle for iterative and parametric methods to solve linear ill-posed equations. Numerische Mathematik, 1996, 75(1): 99–120
Borges L S, Bazán F S V, Cunha M C C. Automatic stopping rule for iterative methods in discrete ill-posed problems. Computational & Applied Mathematics, 2015, 34(3): 1175–1197
Dziwoki G, Izydorczyk J. Stopping criteria analysis of the OMP algorithm for sparse channels estimation. In: Proceedings of the International Conference on Computer Networks. 2015, 250–259
Favati P, Lotti G, Menchi O, Romani F. Stopping rules for iterative methods in nonnegatively constrained deconvolution. Applied Numerical Mathematics, 2014, 75: 154–166
Engl H W, Hanke M, Neubauer A. Regularization of Inverse Problems. Springer Science & Business Media, 1996
Amster P. Iterative Methods. Universitext, 2014, 53–82
Waseem M. On some iterative methods for solving system of nonlinear equations. Dissertation for the Doctoral Degree. Islamabad: COMSATS Institute of Information Technology, 2012
Burger M, Osher S. A guide to the TV zoo. In: Burger M, Mennucci A C G, Osher S, et al., eds. Level Set and PDE Based Reconstruction Methods in Imaging. Springer International Publishing, 2013, 1–70
Tikhonov A N. Regularization of incorrectly posed problems. Soviet Mathematics Doklady, 1963, 4(1): 1624–1627
Nikolova M. Analysis of the recovery of edges in images and signals by minimizing nonconvex regularized least-squares. Multiscale Modeling & Simulation, 2005, 4(3): 960–991
Burger M, Osher S. Convergence rates of convex variational regularization. Inverse Problems, 2004, 20(5): 1411–1421
Hofmann B, Kaltenbacher B, Pöschl C, Scherzer O. A convergence rates result for Tikhonov regularization in Banach spaces with nonsmooth operators. Inverse Problems, 2007, 23(3): 987–1010
Resmerita E. Regularization of ill-posed problems in Banach spaces:convergence rates. Inverse Problems, 2005, 21(4): 1303–1314
Resmerita E, Scherzer O. Error estimates for non-quadratic regularization and the relation to enhancement. Inverse Problems, 2006, 22(3): 801–814
Engl H W. Discrepancy principles for Tikhonov regularization of ill posed problems leading to optimal convergence rates. Journal of Optimization Theory and Applications, 1987, 52(2): 209–215
Gfrerer H. An a posteriori parameter choice for ordinary and iterated Tikhonov regularization of ill-posed problems leading to optimal convergence rates. Mathematics of Computation, 1987, 49(180): 507–522
Natterer F. Error bounds for Tikhonov regularization in Hilbert scales. Applicable Analysis, 1984, 18(1–2): 29–37
Neubauer A. An a posteriori parameter choice for Tikhonov regularization in the presence of modeling error. Applied Numerical Mathematics, 1988, 4(6): 507–519
Engl H W, Kunisch K, Neubauer A. Convergence rates for Tikhonov regularisation of non-linear ill-posed problems. Inverse Problems, 1989, 5(4): 523–540
Scherzer O, Engl H W, Kunisch K. Optimal a posteriori parameter choice for Tikhonov regularization for solving nonlinear ill-posed problems. SIAM Journal on Numerical Analysis, 1993, 30(6): 1796–1838
Varah J M. On the numerical solution of ill-conditioned linear systems with applications to ill-posed problems. SIAM Journal on Numerical Analysis, 1973, 10(2): 257–267
Vinod H D, Ullah A. Recent Advances in Regression Methods. Danbury: Marcel Dekker Incorporated, 1981
O’Sullivan F. A statistical perspective on ill-posed inverse problems. Statistical Science, 1986, 1(4): 502–518
Grafarend EW, Schaffrin B. Ausgleichungsrechnung in linearen modellen. BI Wissenschaftsverlag Mannheim, 1993
Rodgers C D. Inverse Methods for Atmospheric Sounding: Theory and Practice. Singapore: World Scientific, 2000
Ceccherini S. Analytical determination of the regularization parameter in the retrieval of atmospheric vertical profiles. Optics Letters, 2005, 30(19): 2554–2556
Mallows C L. Some comments on C p. Technometrics, 1973, 15(4): 661–675
Rice J. Choice of smoothing parameter in deconvolution problems. Contemporary Mathematics, 1986, 59: 137–151
Hanke M, Raus T. A general heuristic for choosing the regularization parameter in ill-posed problems. SIAM Journal on Scientific Computing, 1996, 17(4): 956–972
Wu L M. A parameter choice method for Tikhonov regularization. Electronic Transactions on Numerical Analysis, 2003, 16: 107–128
Gao W, Yu K P. A new method for determining the Tikhonov regularization parameter of load identification. In: Proceedings of the International Symposium on Precision Engineering Measurement and Instrumentation. 2015
Ito K, Jin B, Takeuchi T. Multi-parameter Tikhonov regularizationan augmented approach. Chinese Annals of Mathematics, Series B, 2014, 35(03): 383–398
Jin B, Lorenz D A. Heuristic parameter-choice rules for convex variational regularization based on error estimates. SIAM Journal on Numerical Analysis, 2010, 48(3): 1208–1229
Pazos F, Bhaya A. Adaptive choice of the Tikhonov regularization parameter to solve ill-posed linear algebraic equations via Liapunov optimizing control. Journal of Computational and Applied Mathematics, 2015, 279: 123–132
Hämarik U, Palm R, Raus T. A family of rules for parameter choice in Tikhonov regularization of ill-posed problems with inexact noise level. Journal of Computational and Applied Mathematics, 2012, 236(8): 2146–2157
Reichel L, Rodriguez G. Old and new parameter choice rules for discrete ill-posed problems. Numerical Algorithms, 2013, 63(1): 65–87
Kryanev A V. An iterative method for solving incorrectly posed problems. USSR Computational Mathematics and Mathematical Physics, 1974, 14(1): 24–35
King J T, Chillingworth D. Approximation of generalized inverses by iterated regularization. Numerical Functional Analysis & Optimization, 1979, 1(5): 499–513
Fakeev A G. A class of iterative processes for solving degenerate systems of linear algebraic equations. USSR Computational Mathematics and Mathematical Physics, 1981, 21(3): 15–22
Brill M, Schock E. Iterative solution of ill-posed problems: a survey. In: Proceedings of the 4th International Mathematical Geophysics Seminar. 1987
Hanke M, Groetsch CW. Nonstationary iterated Tikhonov regularization. Journal of Optimization Theory and Applications, 1998, 98(1): 37–53
Lampe J, Reichel L, Voss H. Large-scale Tikhonov regularization via reduction by orthogonal projection. Linear Algebra and Its Applications, 2012, 436(8): 2845–2865
Reichel L, Yu X B. Tikhonov regularization via flexible Arnoldi reduction. BIT Numerical Mathematics, 2015, 55(4): 1145–1168
Huang G, Reichel L, Yin F. Projected nonstationary iterated Tikhonov regularization. BIT Numerical Mathematics, 2016, 56(2): 467–487
Ambrosio L, Fusco N, Pallara D. Functions of Bounded Variation and Free Discontinuity Problems. Oxford: Oxford University Press, 2000
Acar R, Vogel C R. Analysis of bounded variation penalty methods for ill-posed problems. Inverse Problems, 1997, 10(6): 1217–1229
Hunt B R. The application of constrained least squares estimation to image restoration by digital computer. IEEE Transactions on Computers, 1973, 100(9): 805–812
Demoment G. Image reconstruction and restoration: overview of common estimation structures and problems. IEEE Transactions on Acoustics, Speech and Signal Processing, 1989, 37(12): 2024–2036
Katsaggelos A K. Iterative image restoration algorithms. Optical Engineering, 1989, 28(7): 735–748
Katsaggelos A K, Biemond J, Schafer R W, Mersereau R M. A regularized iterative image restoration algorithm. IEEE Transactions on Signal Processing, 1991, 39(4): 914–929
Babacan S D, Molina R, Katsaggelos A K. Parameter estimation in TV image restoration using variational distribution approximation. IEEE Transactions on Image Processing, 2008, 17(3): 326–339
Wen Y W, Chan R H. Parameter selection for total-variation-based image restoration using discrepancy principle. IEEE Transactions on Image Processing, 2012, 21(4): 1770–1781
Chen A, Huo B M, Wen CW. Adaptive regularization for color image restoration using discrepancy principle. In: Proceedings of the IEEE International Conference on Signal processing, Comminications and Computing. 2013, 1–6
Lin Y, Wohlberg B, Guo H. UPRE method for total variation parameter selection. Signal Processing, 2010, 90(8): 2546–2551
Stein C M. Estimation of the mean of a multivariate normal distribution. Annals of Statistics, 1981, 9(6): 1135–1151
Ramani S, Blu T, Unser M. Monte-Carlo SURE: a black-box optimization of regularization parameters for general denoising algorithms. IEEE Transactions on Image Processing, 2008, 17(9): 1540–1554
Palsson F, Sveinsson J R, Ulfarsson M O, Benediktsson J A. SAR image denoising using total variation based regularization with surebased optimization of regularization parameter. In: Proceedings of the IEEE International Conference on Geoscience and Remote Sensing Symposium. 2012, 2160–2163
Liao H Y, Li F, Ng MK. Selection of regularization parameter in total variation image restoration. Journal of the Optical Society of America A, 2009, 26(11): 2311–2320
Bertalmío M, Caselles V, Rougé B, Solé A. TV based image restoration with local constraints. Journal of Scientific Computing, 2003, 19(1–3): 95–122
Almansa A, Ballester C, Caselles V, Haro G. A TV based restoration model with local constraints. Journal of Scientific Computing, 2008, 34(3): 209–236
Vogel C R, Oman ME. Iterative methods for total variation denoising. SIAM Journal on Scientific Computing, 1997, 17(1): 227–238
Chan T F, Golub G H, Mulet P. A nonlinear primal-dual method for total variation-based image restoration. Lecture Notes in Control & Information Sciences, 1995, 20(6): 1964–1977
Chambolle A. An algorithm for total variation minimization and applications. Journal of Mathematical Imaging & Vision, 2004, 20(1–2): 89–97
Huang Y M, Ng M K, Wen Y W. A fast total variation minimization method for image restoration. SIAM Journal on Multiscale Modeling & Simulation, 2008, 7(2): 774–795
Bresson X, Chan T F. Fast dual minimization of the vectorial total variation norm and applications to color image processing. Inverse Problems & Imaging, 2008, 2(4): 455–484
Ng M K, Qi L Q, Yang Y F, Huang Y M. On semismooth Newton’s methods for total variation minimization. Journal of Mathematical Imaging & Vision, 2007, 27(3): 265–276
Zhu M Q, Chan T F. An efficient primal-dual hybrid gradient algorithm for total variation image restoration. UCLA CAM Report. 2008, 8–34
Zhu M Q, Wright S J, Chan T F. Duality-based algorithms for total-variation-regularized image restoration. Computational Optimization and Applications, 2010, 47(3): 377–400
Krishnan D, Lin P, Yip A M. A primal-dual active-set method for non-negativity constrained total variation deblurring problems. IEEE Transactions on Image Processing, 2007, 16(11): 2766–2777
Krishnan D, Pham Q V, Yip A M. A primal-dual active-set algorithm for bilaterally constrained total variation deblurring and piecewise constant Mumford-Shah segmentation problems. Advances in Computational Mathematics, 2009, 31(1–3): 237–266
Osher S, Burger M, Goldfarb D, Xu J J, Yin WT. An iterative regularization method for total variation-based image restoration. Multiscale Modeling & Simulation, 2005, 4(2): 460–489
Goldstein T, Osher S. The split Bregman method for l 1-regularized problems. SIAM Journal on Imaging Sciences, 2009, 2(2): 323–343
Glowinski R, Le Tallec P. Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics. Society for Industrial and Applied Mathematics, 1989
Wu C C, Tai X C. Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models. SIAM Journal on Imaging Sciences, 2010, 3(3): 300–339
Darbon J, Sigelle M. Image restoration with discrete constrained total variation part I: fast and exact optimization. Journal of Mathematical Imaging & Vision, 2006, 26(3): 261–276
Duan Y P, Tai X C. Domain decomposition methods with graph cuts algorithms for total variation minimization. Advances in Computational Mathematics, 2012, 36(2): 175–199
Fu H Y, Ng M K, Nikolova M, Barlow J L. Efficient minimization methods of mixed l2-l 1 and l 1-l 1 norms for image restoration. SIAM Journal on Scientific Computing, 2005, 27(6): 1881–1902
Goldfarb D, Yin W T. Second-order cone programming methods for total variation-based image restoration. SIAM Journal on Scientific Computing, 2005, 27(2): 622–645
Oliveira J P, Bioucas-Dias J M, Figueiredo M A T. Adaptive total variation image deblurring: a majorization-minimization approach. Signal Processing, 2009, 89(9): 1683–1693
Bioucas-Dias J M, Figueiredo M A T, Oliveira J P. Total variationbased image deconvolution: a majorization-minimization approach, In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. 2006, 861–864
Chan T F, Esedoglu S. Aspects of total variation regularized l 1 function approximation. SIAM Journal on Applied Mathematics, 2004, 65(5): 1817–1837
He L, Burger M, Osher S. Iterative total variation regularization with non-quadratic fidelity. Journal of Mathematical Imaging & Vision, 2006, 26(1–2): 167–184
Jonsson E, Huang S C, Chan T F. Total variation regularization in positron emission tomography. CAM Report. 1998
Panin V Y, Zeng G L, Gullberg G T. Total variation regulated EM algorithm. IEEE Transactions on Nuclear Science, 1999, 46(6): 2202–2210
Le T, Chartrand R, Asaki T J. A variational approach to reconstructing images corrupted by Poisson noise. Journal of Mathematical Imaging & Vision, 2007, 27(3): 257–263
Rudin L, Lions P L, Osher S. Multiplicative denoising and deblurring: theory and algorithms. In: Osher S, Paragios N, eds. Geometric Level Set Methods in Imaging, Vision, and Graphics. New York: Springer, 2003, 103–119
Huang Y M, Ng M K, Wen Y W. A new total variation method for multiplicative noise removal. SIAM Journal on Imaging Sciences, 2009, 2(1): 20–40
Bonesky T, Kazimierski K S, Maass P, Schöpfer F, Schuster T. Minimization of Tikhonov functionals in Banach spaces. Abstract & Applied Analysis, 2008, 2008(1): 1563–1569
Meyer Y. Oscillating Patterns in Image Processing and Nonlinear Evolution Equations, University Lecture Series. Rhode Island: American Mathematical Society, 2002
Blomgren P, Chen T F. Color TV: total variation methods for restoration of vector valued images. IEEE Transactions on Image Processing, 1970, 7(3): 304–309
Setzer S, Steidl G, Popilka B, Burgeth B. Variational methods for denoising matrix fields. In: Laidlaw D, Weickert J, eds. Visualization and Processing of Tensor Fields. Berlin: Springer Berlin Heidelberg, 2009, 341–360
Esedoglu S, Osher S. Decomposition of images by the anisotropic Rudin-Osher-Fatemi model. Communications on Pure and Applied Mathematics, 2004, 57(12): 1609–1626
Shi Y Y, Chang Q S. Efficient algorithm for isotropic and anisotropic total variation deblurring and denoising. Journal of Applied Mathematics, 2013
Marquina A, Osher S. Explicit algorithms for a new time dependent model based on level set motion for nonlinear deblurring and noise removal. SIAM Journal on Scientific Computing, 2000, 22(2): 387–405
Chan T F, Marquina A, Mulet P. High-order total variation-based image restoration. SIAM Journal on Scientific Computing, 2000, 22(2): 503–516
Gilboa G, Osher S. Nonlocal operators with applications to image processing. SIAM Journal on Multiscale Modeling & Simulation, 2008, 7(3): 1005–1028
Kindermann S, Osher S, Jones P W. Deblurring and denoising of images by nonlocal functionals. SIAM Journal on Multiscale Modeling & Simulation, 2005, 4(4): 1091–1115
Hu Y, Jacob M. Higher degree total variation (HDTV) regularization for image recovery. IEEE Transactions on Image Processing, 2012, 21(5): 2559–2571
Yang J S, Yu H Y, Jiang M, Wang G. High-order total variation minimization for interior SPECT. Inverse Problems, 2012, 28(1): 15001–15024
Liu X W, Huang L H. A new nonlocal total variation regularization algorithm for image denoising. Mathematics and Computers in Simulation, 2014, 97: 224–233
Ren Z M, He C J, Zhang Q F. Fractional order total variation regularization for image super-resolution. Signal Processing, 2013, 93(9): 2408–2421
Oh S, Woo H, Yun S, Kang M. Non-convex hybrid total variation for image denoising. Journal of Visual Communication & Image Representation, 2013, 24(3): 332–344
Donoho D L. Compressed sensing. IEEE Transactions on Information Theory, 2006, 52(4): 1289–1306
Candè E J, Wakin M B. An introduction to compressive sampling. IEEE Signal Processing Magazine, 2008, 25(2): 21–30
Tsaig Y, Donoho D L. Extensions of compressed sensing. Signal Processing, 2006, 86(3): 549–571
Candès E J, Romberg J, Tao T. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on Information Theory, 2006, 52(2): 489–509
Candès E J, Tao T. Near-optimal signal recovery from random projections: Universal encoding strategies?. IEEE Transactions on Information Theory, 2006, 52(12): 5406–5425
Donoho D L, Elad M. Optimally sparse representation in general (nonorthogonal) dictionaries via l 1 minimization. Proceedings of National Academy of Sciences, 2003, 100(5): 2197–2202
Wright J, Ma Y. Dense error correction via l 1-minimization. IEEE Transactions on Information Theory, 2010, 56(7): 3540–3560
Yang J F, Zhang Y. Alternating direction algorithms for ℓ1-problems in compressive sensing. SIAM Journal on Scientific Computing, 2011, 33(1): 250–278.
Natarajan B K. Sparse approximate solutions to linear systems. SIAM Journal on Computing, 1995, 24(2): 227–234
Mallat S G, Zhang Z. Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing, 1993, 41(12): 3397–3415
Tropp J, Gilbert A C. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on Information Theory, 2007, 53(12): 4655–4666
Blumensath T, Davies M E. Iterative thresholding for sparse approximations. Journal of Fourier Analysis and Applications, 2008, 14(5–6): 629–654
Gorodnitsky I F, Rao B D. Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm. IEEE Transactions on Signal Processing, 1997, 45(3): 600–616
Bao C L, Ji H, Quan Y H, Shen Z W. l 0 norm based dictionary learning by proximal methods with global convergence. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 3858–3865
Foucart S, Lai M J. Sparsest solutions of underdetermined linear systems via l q-minimization for 0 < q ≤ 1. Applied and Computational Harmonic Analysis, 2009, 26(3): 395–407
Cai T T, Wang L, Xu G. Shifting inequality and recovery of sparse signals. IEEE Transactions on Signal Processing, 2010, 58(3): 1300–1308
Cai T T, Wang L, Xu G. New bounds for restricted isometry constants. IEEE Transactions on Information Theory, 2010, 56(9): 4388–4394
Chen S S, Donoho D L, Saunders M A. Atomic decomposition by basis pursuit. SIAM Journal on Scientific Computing, 1998, 20(1): 33–61
Efron B, Hastie T, Johnstone I, Tibshirani R. Least angle regression. The Annals of Statistics, 2004, 32(2): 407–499
Figueiredo M A T, Nowak R D. An EM algorithm for wavelet-based image restoration. IEEE Transactions on Image Processing, 2002, 12(8): 906–916
Starck J L, Mai K N, Murtagh F. Wavelets and curvelets for image deconvolution: a combined approach. Signal Processing, 2003, 83(10): 2279–2283
Herrholz E, Teschke G. Compressive sensing principles and iterative sparse recovery for inverse and ill-posed problems. Inverse Problems, 2010, 26(12): 125012–125035
Jin B, Lorenz D, Schiffler S. Elastic-net regularization: error estimates and active set methods. Inverse Problems, 2009, 25(11): 1595–1610
Figueiredo M A T, Nowak R D, Wright S J. Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE Journal of Selected Topics in Signal Processing, 2007, 1(4): 586–597
Kim S J, Koh K, Lustig M, Boyd S, Gorinevsky D. An interior-point method for large-scale l 1-regularized least squares. IEEE Journal of Selected Topics in Signal Processing, 2007, 1(4): 606–617
Donoho D L, Tsaig Y. Fast solution of l 1-norm minimization problems when the solution may be sparse. IEEE Transactions on Information Theory, 2008, 54(11): 4789–4812
Combettes P L, Wajs E R. Signal recovery by proximal forwardbackward splitting. SIAM Journal on Multiscale Modeling & Simulation, 2005, 4(4): 1168–1200
Becker S, Bobin J, Candés E J. NESTA: a fast and accurate firstorder method for sparse recovery. SIAM Journal on Imaging Sciences, 2011, 4(1): 1–39
Osborne M R, Presnell B, Turlach B A. A new approach to variable selection in least squares problems. IMA Journal of Numerical Analysis, 1999, 20(3): 389–403
Li L, Yao X, Stolkin R, Gong M G, He S. An evolutionary multiobjective approach to sparse reconstruction. IEEE Transactions on Evolutionary Computation, 2014, 18(6): 827–845
Chartrand R. Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Processing Letters, 2007, 14(10): 707–710
Candes E J, Tao T. Decoding by linear programming. IEEE Transactions on Information Theory, 2005, 51(12): 4203–4215
Saab R, Chartrand R, Yilmaz Ö. Stable sparse approximations via nonconvex optimization. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. 2008, 3885–3888
Tibshirani R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 1996, 58(1): 267–288
Zhang C H. Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 2010, 38(2): 894–942
Fan J Q, Li R. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 2001, 96(456): 1348–1360
Nikolova M, Ng M K, Zhang S, Ching W K. Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization. SIAM Journal on Imaging Sciences, 2008, 1(1): 2–25
Frank L E, Friedman J H. A statistical view of some chemometrics regression tools. Technometrics, 1993, 35(2): 109–135
Fu W J. Penalized regressions: the bridge versus the lasso. Journal of Computational and Graphical Statistics, 1998, 7(3): 397–416
Lyu Q, Lin Z C, She Y Y, Zhang C. A comparison of typical l p minimization algorithms. Neurocomputing, 2013, 119: 413–424
Candes E J, Wakin M B, Boyd S P. Enhancing sparsity by reweighted l 1 minimization. Journal of Fourier Analysis and Applications, 2008, 14(5–6): 877–905
Rao B D, Kreutz-Delgado K. An affine scaling methodology for best basis selection. IEEE Transactions on Signal Processing, 1999, 47(1): 187–200
She Y Y. Thresholding-based iterative selection procedures for model selection and shrinkage. Electronic Journal of Statistics, 2009, 3: 384–415
Xu Z B, Zhang H, Wang Y, Chang X Y, Liang Y. L 1/2 regularization. Science China Information Sciences, 2010, 53(6): 1159–1169
Xu Z B, Guo H L, Wang Y, Zhang H. Representative of L 1/2 regularization among l q (0 < q ≤ 1) regularizations: an experimental study based on phase diagram. Acta Automatica Sinica, 2012, 38(7): 1225–1228
Candes E J, Plan Y. Matrix completion with noise. Proceedings of the IEEE, 2009, 98(6): 925–936
Cai J F, Candes E J, Shen Z. A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 2010, 20(4): 1956–1982
Boyd S, Parikh N, Chu E, Peleato B, Eckstein J. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations & Trends in Machine Learning, 2011, 3(1): 1–122
Qian J J, Yang J, Zhang F L, Lin Z C. Robust low-rank regularized regression for face recognition with occlusion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2014, 21–26
Liu Y J, Sun D, Toh K C. An implementable proximal point algorithmic framework for nuclear norm minimization. Mathematical Programming, 2012, 133(1–2): 399–436
Yang J F, Yuan X M. Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization. Mathematics of Computation, 2013, 82(281): 301–329
Li T, Wang W W, Xu L, Feng X C. Image denoising using lowrank dictionary and sparse representation. In: Proceedings of the 10th IEEE International Conference on Computational Intelligence and Security. 2014, 228–232
Waters A E, Sankaranarayanan A C, Baraniuk R G. SpaRCS: recovering low-rank and sparse matrices from compressive measurements. In: Proceedings of the Neural Information Processing Systems Conference. 2011, 1089–1097
Li Q, Lu Z B, Lu Q B, Li H Q, Li WP. Noise reduction for hyperspectral images based on structural sparse and low-rank matrix decomposition. In: Proceedings of the IEEE International on Geoscience and Remote Sensing Symposium. 2013, 1075–1078
Zhou T Y, Tao D C. Godec: randomized low-rank & sparse matrix decomposition in noisy case. In: Proceedings of the 28th International Conference on Machine Learning. 2011, 33–40
Zhang H Y, He W, Zhang L P, Shen H F, Yuan Q Q. Hyperspectral image restoration using low-rank matrix recovery. IEEE Transactions on Geoscience & Remote Sensing, 2014, 52(8): 4729–4743
Zhang Z, Xu Y, Yang J, Li X L, Zhang D. A survey of sparse representation: algorithms and applications. IEEE Access, 2015, 3: 490–530
Burger M, Franek M, Schönlieb C B. Regularized regression and density estimation based on optimal transport. Applied Mathematics Research eXpress, 2012, 2012(2): 209–253
Osher S, Solè A, Vese L. Image decomposition and restoration using total variation minimization and the H 1 norm. Multiscale Modeling & Simulation, 2003, 1(3): 349–370
Barbara K. Iterative regularization methods for nonlinear ill-posed problems. Algebraic Curves & Finite Fields Cryptography & Other Applications, 2008, 6
Miettinen K. Nonlinear Multiobjective Optimization. Springer Science & Business Media, 2012
Marler R T, Arora J S. Survey of multi-objective optimization methods for engineering. Structural and Multidisciplinary Optimization, 2004, 26(6): 369–395
Gong M G, Jiao L C, Yang D D, Ma W P. Research on evolutionary multi-objective optimization algorithms. Journal of Software, 2009, 20(20): 271–289
Fonseca C M, Fleming P J. Genetic algorithm for multiobjective optimization: formulation, discussion and generation. In: Proceedings of the International Conference on Genetic Algorithms. 1993, 416–423
Srinivas N, Deb K. Multiobjective optimization using non-dominated sorting in genetic algorithms. Evolutionary Computation, 1994, 2(3): 221–248
Horn J, Nafpliotis N, Goldberg D E. A niched Pareto genetic algorithm for multiobjective optimization. In: Proceedings of the 1st IEEE Conference on Evolutionary Computation. 1994, 1: 82–87
Zitzler E, Thiele L. Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach. IEEE Transactions on Evolutionary Computation, 1999, 3(4): 257–271
Zitzler E, Laumanns M, Thiele L. SPEA2: improving the strength Pareto evolutionary algorithm. Eurogen, 2001, 3242(103): 95–100
Kim M, Hiroyasu T, Miki M, Watanabe S. SPEA2+: improving the performance of the strength Pareto evolutionary algorithm 2. In: Proceedings of the International Conference on Parallel Problem Solving from Nature. 2004, 742–751
Knowles J D, Corne D W. Approximating the non-dominated front using the Pareto archived evolution strategy. Evolutionary Computation, 2000, 8(2): 149–172
Corne D W, Knowles J D, Oates M J. The Pareto-envelope based selection algorithm for multi-objective optimization. In: Proceedings of the Internatioal Conference on Parallel Problem Solving from Nature. 2000, 869–878
Corne D W, Jerram N R, Knowles J D, Oates M J. PESA-II: regionbased selection in evolutionary multi-objective optimization. In: Proceedings of the Genetic and Evolutionary Computation Conference. 2001, 283–290
Deb K, Agrawal S, Pratap A, Meyarivan T. A fast elitist nondominated sorting genetic algorithm for multi-objective optimization: NSGA-II. Lecture Notes in Computer Science, 2000, 1917: 849–858
Zhang Q F, Li H. MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation, 2007, 11(6): 712–731
Ishibuchi H, Sakane Y, Tsukamoto N, Nojima Y. Simultaneous use of different scalarizing functions in MOEA/D. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation. 2010, 519–526
Wang L P, Zhang Q F, Zhou A M, Gong M G, Jiao L C. Constrained subproblems in decomposition based multiobjective evolutionary algorithm. IEEE Transactions on Evolutionary Computation, 2016, 20(3): 475–480
Li K, Fialho A, Kwong S, Zhang Q F. Adaptive operator selection with bandits for a multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation, 2014, 18(1): 114–130
Ke L J, Zhang Q F, Battiti R. Hybridization of decomposition and local search for multiobjective optimization. IEEE Transactions on Cybernetics, 2014, 44(10): 1808–1820
Cai X Y, Wei O. A hybrid of decomposition and domination based evolutionary algorithm for multi-objective software next release problem. In: Proceedings of the 10th IEEE International Conference on Control and Automation. 2013, 412–417
Deb K, Jain H. An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part I: solving problems with box constraints. IEEE Transactions on Evolutionary Computation, 2014, 18(4): 577–601
Yuan Y, Xu H, Wang B. An improved NSGA-III procedure for evolutionary many-objective optimization. In: Proceedings of ACM Annual Conference on Genetic & Evolutionary Computation. 2014, 661–668
Seada H, Deb K. U-NSGA-III: a unified evolutionary optimization procedure for single, multiple, and many objectives: proof-ofprinciple results. In: Proceedings of the International Conference on Evolutionary Multi-Criterion Optimization. 2015, 34–49
Zhu Z X, Xiao J, Li J Q, Zhang Q F. Global path planning of wheeled robots using multi-objective memetic algorithms. Integrated Computer-Aided Engineering, 2015, 22(4): 387–404
Zhu Z X, Jia S, He S, Sun Y W, Ji Z, Shen L L. Three-dimensional Gabor feature extraction for hyperspectral imagery classification using a memetic framework. Information Sciences, 2015, 298: 274–287
Zhu Z X, Xiao J, He S, Ji Z, Sun Y W. A multi-objective memetic algorithm based on locality-sensitive hashing for one-to-many-to-one dynamic pickup-and-delivery problem. Information Sciences, 2015, 329: 73–89
Li H, Gong M G, Wang Q, Liu J, Su L Z. A multiobjective fuzzy clustering method for change detection in synthetic aperture radar images. Applied Soft Computing, 2016, 46: 767–777
Jin Y, Sendhoff B. Pareto based approach to machine learning: an overview and case studies. IEEE Transactions on Systems, Man, and Cybernetics, Part C, 2008, 38(3): 397–415
Plumbley M D. Recovery of sparse representations by polytope faces pursuit. In: Proceedings of the 6th International Conference on Independent Component Analysis and Blind Source Separation. 2006, 206–213
Wright S J, Nowak R D, Figueiredo M A T. Sparse reconstruction by separable approximation. IEEE Transactions on Signal Processing, 2009, 57(7): 2479–2493
Yang Y, Yao X, Zhou Z H. On the approximation ability of evolutionary optimization with application to minimum set cover. Artificial Intelligence, 2012, 180(2): 20–33
Qian C, Yu Y, Zhou Z H. An analysis on recombination in multiobjective evolutionary optimization. Artificial Intelligence, 2013, 204(1): 99–119
Gong M G, Zhang M Y, Yuan Y. Unsupervised band selection based on evolutionary multiobjective optimization for hyperspectral images. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(1): 544–557
Qian C, Yu Y, Zhou Z H. Pareto ensemble pruning. In: Proceedings of AAAI Conference on Artificial Intelligence. 2015, 2935–2941
Qian C, Yu Y, Zhou Z H. On constrained Boolean Pareto optimization. In: Proceedings of the 24th International Joint Conference on Artificial Intelligence. 2015, 389–395
Qian C, Yu Y, Zhou Z H. Subset selection by Pareto optimization. In: Proceedings of the Neural Information Processing Systems Conference. 2015, 1765–1773
Gong M G, Liu J, Li H, Cai Q, Su L Z. A multiobjective sparse feature learning model for deep neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2015, 26(12): 3263–3277
Acknowledgements
This work was supported in part by the National Nature Science Foundation of China (Grant No. 61422209), in part by the National Program for Support of Top-Notch Young Professionals of China.
Author information
Authors and Affiliations
Corresponding author
Additional information
Maoguo Gong received the BS degree in electronic engineering and PhD degree in electronic science and technology from Xidian University, China in 2003 and 2009, respectively. He is currently a full professor with Xidian University. His research interests are in the area of computational intelligence with applications to optimization, learning, data mining and image understanding. He is a senior member of IEEE and Chinese Computer Federation. He is the awardee of the NSFC Excellent Young Scholars Program in 2014.
Xiangming Jiang received the BS degree in mathematics and applied mathematics from Xidian University, China in 2015. He is currently pursuing the MS degree in pattern recognition and intelligent systems at the School of Electronic Engineering, Xidian University. His research interests include image understanding and inverse problem.
Hao Li received the BS degree in electronic engineering from Xidian University, China in 2013. He is currently pursuing the PhD degree in pattern recognition and intelligent systems at the School of Electronic Engineering, Xidian University. His research interests include computational intelligence and image understanding.
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Gong, M., Jiang, X. & Li, H. Optimization methods for regularization-based ill-posed problems: a survey and a multi-objective framework. Front. Comput. Sci. 11, 362–391 (2017). https://doi.org/10.1007/s11704-016-5552-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11704-016-5552-0