Factorial SDE for Multi-Output Gaussian Process Regression

Daniel P. Jeong, Seyoung Kim
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:9755-9772, 2023.

Abstract

Multi-output Gaussian process (GP) regression has been widely used as a flexible nonparametric Bayesian model for predicting multiple correlated outputs given inputs. However, the cubic complexity in the sample size and the output dimensions for inverting the kernel matrix has limited their use in the large-data regime. In this paper, we introduce the factorial stochastic differential equation as a representation of multi-output GP regression, which is a factored state-space representation as in factorial hidden Markov models. We propose a structured mean-field variational inference approach that achieves a time complexity linear in the number of samples, along with its sparse variational inference counterpart with complexity linear in the number of inducing points. On simulated and real-world data, we show that our approach significantly improves upon the scalability of previous methods, while achieving competitive prediction accuracy.

Cite this Paper

BibTeX
@InProceedings{pmlr-v206-jeong23a, title = {Factorial SDE for Multi-Output Gaussian Process Regression}, author = {Jeong, Daniel P. and Kim, Seyoung}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {9755--9772}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/jeong23a/jeong23a.pdf}, url = {https://proceedings.mlr.press/v206/jeong23a.html}, abstract = {Multi-output Gaussian process (GP) regression has been widely used as a flexible nonparametric Bayesian model for predicting multiple correlated outputs given inputs. However, the cubic complexity in the sample size and the output dimensions for inverting the kernel matrix has limited their use in the large-data regime. In this paper, we introduce the factorial stochastic differential equation as a representation of multi-output GP regression, which is a factored state-space representation as in factorial hidden Markov models. We propose a structured mean-field variational inference approach that achieves a time complexity linear in the number of samples, along with its sparse variational inference counterpart with complexity linear in the number of inducing points. On simulated and real-world data, we show that our approach significantly improves upon the scalability of previous methods, while achieving competitive prediction accuracy.} }
Endnote
%0 Conference Paper %T Factorial SDE for Multi-Output Gaussian Process Regression %A Daniel P. Jeong %A Seyoung Kim %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-jeong23a %I PMLR %P 9755--9772 %U https://proceedings.mlr.press/v206/jeong23a.html %V 206 %X Multi-output Gaussian process (GP) regression has been widely used as a flexible nonparametric Bayesian model for predicting multiple correlated outputs given inputs. However, the cubic complexity in the sample size and the output dimensions for inverting the kernel matrix has limited their use in the large-data regime. In this paper, we introduce the factorial stochastic differential equation as a representation of multi-output GP regression, which is a factored state-space representation as in factorial hidden Markov models. We propose a structured mean-field variational inference approach that achieves a time complexity linear in the number of samples, along with its sparse variational inference counterpart with complexity linear in the number of inducing points. On simulated and real-world data, we show that our approach significantly improves upon the scalability of previous methods, while achieving competitive prediction accuracy.
APA
Jeong, D.P. & Kim, S.. (2023). Factorial SDE for Multi-Output Gaussian Process Regression. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:9755-9772 Available from https://proceedings.mlr.press/v206/jeong23a.html.

Related Material