research-article

Operon C++: an efficient genetic programming framework for symbolic regression

Authors:

Bogdan Burlacu,

Gabriel Kronberger,

Michael KommendaAuthors Info & Claims

GECCO '20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion

Pages 1562 - 1570

https://doi.org/10.1145/3377929.3398099

Published: 08 July 2020 Publication History

Abstract

Genetic Programming (GP) is a dynamic field of research where empirical testing plays an important role in validating new ideas and algorithms. The ability to easily prototype new algorithms by reusing key components and quickly obtain results is therefore important for the researcher.

In this paper we introduce Operon, a C++ GP framework focused on performance, modularity and usability, featuring an efficient linear tree encoding and a scalable concurrency model where each logical thread is responsible for generating a new individual. We model the GP evolutionary process around the concept of an offspring generator, a streaming operator that defines how new individuals are obtained. The approach allows different algorithmic variants to be expressed using high-level constructs within the same generational basic loop. The evaluation routine supports both scalar and dual numbers, making it possible to calculate model derivatives via automatic differentiation. This functionality allows seamless integration with gradient-based local search approaches.

We discuss the design choices behind the proposed framework and compare against two other popular GP frameworks, DEAP and HeuristicLab. We empirically show that Operon is competitive in terms of solution quality, while being able to generate results significantly faster.

Supplementary Material

ZIP File (p1562_burlacu_suppl.zip)

Supplemental material.

Download
5.57 MB

References

[1]

Michael Affenzeller and Stefan Wagner. 2005. Offspring Selection: A New Self-Adaptive Selection Scheme for Genetic Algorithms. In Adaptive and Natural Computing Algorithms (Springer Computer Science), B. Ribeiro, R. F. Albrecht, A. Dobnikar, D. W. Pearson, and N. C. Steele (Eds.). Springer, 218--221.

[2]

Sameer Agarwal, Keir Mierle, and Others. 2018. Ceres Solver. http://ceres-solver.org. (2018).

[3]

Lee Altenberg. 1994. The Evolution of Evolvability in Genetic Programming. In Advances in Genetic Programming, Kenneth E. Kinnear, Jr. (Ed.). MIT Press, Chapter 3, 47--74.

Digital Library

[4]

L. Breiman, J. Friedman, C.J. Stone, and R.A. Olshen. 1984. Classification and Regression Trees. Taylor & Francis.

[5]

Thomas F. Brooks, D. Stuart Pope, and Michael A. Marcolini. 1989. Airfoil self-noise and prediction.

[6]

Darren M. Chitty. 2012. Fast parallel genetic programming: multi-core CPU versus many-core GPU. Soft Computing 16, 10 (Oct. 2012), 1795--1814. https://doi.org/

Digital Library

[7]

Vinícius Veloso de Melo, Álvaro Luiz Fazenda, Léo Françoso Dal Piccol Sotto, and Giovanni Iacca. 2020. A MIMD Interpreter for Genetic Programming. In Applications of Evolutionary Computation, Pedro A. Castillo, Juan Luis Jiménez Laredo, and Francisco Fernández de Vega (Eds.). Springer International Publishing, Cham, 645--658.

[8]

R. Duncan. 1990. A survey of parallel computer architectures. Computer 23, 2 (Feb 1990), 5--16.

[9]

Félix-Antoine Fortin, François-Michel De Rainville, Marc-André Gardner, Marc Parizeau, and Christian Gagné. 2012. DEAP: Evolutionary Algorithms Made Easy. Journal of Machine Learning Research 13 (jul 2012), 2171--2175.

Digital Library

[10]

Jerome H. Friedman. 1991. Multivariate Adaptive Regression Splines. Ann. Statist. 19, 1 (03 1991), 1--67.

[11]

Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. 1995. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley Longman Publishing Co., Inc., USA.

Digital Library

[12]

Gaël Guennebaud, Benoît Jacob, et al. 2010. Eigen v3. http://eigen.tuxfamily.org. (2010).

[13]

Maarten Keijzer and James Foster. 2007. Crossover Bias in Genetic Programming. In Proceedings of the 10th European Conference on Genetic Programming (Lecture Notes in Computer Science), Marc Ebner, Michael O'Neill, Anikó Ekárt, Leonardo Vanneschi, and Anna Isabel Esparcia-Alcázar (Eds.), Vol. 4445. Springer, Valencia, Spain, 33--43.

[14]

Donald E. Knuth. 1997. The Art of Computer Programming, Volume 2 (3rd Ed.): Seminumerical Algorithms. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA.

Digital Library

[15]

I. Kolar, P.W. Michor, and J. Slovak. 1993. Natural Operations in Differential Geometry. Springer. https://books.google.at/books?id=4m9x1uL71L4C

[16]

Michael Kommenda, Bogdan Burlacu, Gabriel Kronberger, and Michael Affenzeller. 2019. Parameter identification for symbolic regression using nonlinear least squares. Genetic Programming and Evolvable Machines (2019), 1--31.

[17]

Arthur Kordon. 2008. Evolutionary Computation in the Chemical Industry. Vol. 86. 245--262.

[18]

John R. Koza. 1992. Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA, USA.

Digital Library

[19]

Sean Luke. 1998. ECJ Evolutionary Computation Library. (1998). Available for free at http://cs.gmu.edu/~eclab/projects/ecj/.

[20]

Nicholas Nethercote and Julian Seward. 2007. Valgrind: A Framework for Heavyweight Dynamic Binary Instrumentation. In Proceedings of ACM SIGPLAN 2007 Conference on Programming Language Design and Implementation (PLDI 2007). San Diego, California, USA, 89--100.

Digital Library

[21]

Travis Oliphant. 2006. NumPy: A guide to NumPy. USA: Trelgol Publishing. (2006). http://www.numpy.org/ [Online; accessed <today>].

[22]

Ludo Pagie and Paulien Hogeweg. 1997. Evolutionary Consequences of Coevolving Targets. Evolutionary Computation 5, 4 (1997), 401--418. arXiv:https://doi.org/10.1162/evco.1997.5.4.401

Digital Library

[23]

Riccardo Poli. 2003. A Simple but Theoretically-Motivated Method to Control Bloat in Genetic Programming. In Genetic Programming, Conor Ryan, Terence Soule, Maarten Keijzer, Edward Tsang, Riccardo Poli, and Ernesto Costa (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 204--217.

[24]

W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery. 1992. Numerical Recipes in C. Cambridge University Press, Cambridge.

[25]

Boris Schäling. 2011. The boost C++ libraries. Boris Schäling.

[26]

Erich Schubert and Michael Gertz. 2018. Numerically Stable Parallel Computation of (Co-)Variance. In Proceedings of the 30th International Conference on Scientific and Statistical Database Management (SSDBM '18). ACM, New York, NY, USA, Article 10, 12 pages.

Digital Library

[27]

Erich Schubert and Arthur Zimek. 2019. ELKI: A large open-source library for data analysis - ELKI Release 0.7.5 "Heidelberg". CoRR abs/1902.03616 (2019). arXiv:1902.03616 http://arxiv.org/abs/1902.03616

[28]

Eric O. Scott and Sean Luke. 2019. ECJ at 20: toward a general metaheuristics toolkit. In GECCO '19: Proceedings of the Genetic and Evolutionary Computation Conference Companion. ACM, Prague, Czech Republic, 1391--1398.

Digital Library

[29]

W. Stallings. 2010. Computer Organization and Architecture: Designing for Performance. Prentice Hall. https://books.google.com/books?id=-7nM1DkWb1YC

[30]

S. Wagner, G. Kronberger, A. Beham, M. Kommenda, A. Scheibenpflug, E. Pitzer, S. Vonolfen, M. Kofler, S. Winkler, V. Dorfer, and M. Affenzeller. 2012. Architecture and Design of the HeuristicLab Optimization Environment. In First Australian Conference on the Applications of Systems Engineering, ACASE (Topics in Intelligent Engineering and Informatics), Robin Braun, Zenon Chaczko, and Franz Pichler (Eds.), Vol. 6. Springer International Publishing, Sydney, Australia, 197--261. Selected and updated papers.

[31]

B. P. Welford. 1962. Note on a method for calculating corrected sums of squares and products. Technometrics (1962), 419--420.

[32]

Pawel Widera, Jonathan M. Garibaldi, and Natalio Krasnogor. 2010. GP challenge: evolving energy function for protein structure prediction. Genetic Programming and Evolvable Machines 11, 1 (March 2010), 61--88. https://doi.org/

Digital Library

[33]

I.-C. Yeh. 1998. Modeling of strength of high-performance concrete using artificial neural networks. Cement and Concrete Research 28, 12 (1998), 1797 -- 1808.

Cited By

Zhang YLi XHu WYen G(2025)Multiexpression Symbolic Regression and Its Circuit Design CaseIEEE Transactions on Systems, Man, and Cybernetics: Systems10.1109/TSMC.2024.351967555:3(2250-2263)Online publication date: Mar-2025
https://doi.org/10.1109/TSMC.2024.3519675
Ma TWen YSong XJin PHuang DHan HNan ZYu ZPeng SZhao YChen HDu ZHu XGuo Q(2025) Harmonia : A Unified Architecture for Efficient Deep Symbolic Regression IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2024.344302744:2(737-750)Online publication date: Feb-2025
https://doi.org/10.1109/TCAD.2024.3443027
Wetzel S(2025)Closed-form interpretation of neural network classifiers with symbolic gradientsMachine Learning: Science and Technology10.1088/2632-2153/ad9fd06:1(015035)Online publication date: 11-Feb-2025
https://doi.org/10.1088/2632-2153/ad9fd0
Show More Cited By

Index Terms

Operon C++: an efficient genetic programming framework for symbolic regression
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by regression
    2. Machine learning approaches
      1. Bio-inspired approaches
        Genetic programming
  2. Parallel computing methodologies
    1. Parallel algorithms
2. Software and its engineering
  1. Software organization and properties
    1. Software system structures
      1. Abstraction, modeling and modularity

Recommendations

A Comparison of three evolutionary strategies for multiobjective genetic programming

We report what we believe to be the first comparative study of multi-objective genetic programming (GP) algorithms on benchmark symbolic regression and machine learning problems. We compare the Strength Pareto Evolutionary Algorithm (SPEA2), the Non-...
An Investigation of Geometric Semantic GP with Linear Scaling
GECCO '23: Proceedings of the Genetic and Evolutionary Computation Conference

Geometric semantic genetic programming (GSGP) and linear scaling (LS) have both, independently, shown the ability to outperform standard genetic programming (GP) for symbolic regression. GSGP uses geometric semantic genetic operators, different from ...
Genetic programming performance prediction and its application for symbolic regression problems
Highlights
- Theoretical analysis of GP performance prediction.
- Suggestion of upper bounds ...
Abstract
Predicting the performance of Genetic Programming (GP) helps us identify whether it is an appropriate approach to solve the problem at hand. However, previous studies show that measuring the difficulty of a problem for GP and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion

July 2020

1982 pages

ISBN:9781450371278

DOI:10.1145/3377929

General Chair:
Carlos Artemio Coello Coello
CINVESTAV-IPN

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 July 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GECCO '20

Sponsor:

SIGEVO

GECCO '20: Genetic and Evolutionary Computation Conference

July 8 - 12, 2020

Cancún, Mexico

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

68
Total Citations
View Citations
885
Total Downloads

Downloads (Last 12 months)280
Downloads (Last 6 weeks)13

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang YLi XHu WYen G(2025)Multiexpression Symbolic Regression and Its Circuit Design CaseIEEE Transactions on Systems, Man, and Cybernetics: Systems10.1109/TSMC.2024.351967555:3(2250-2263)Online publication date: Mar-2025
https://doi.org/10.1109/TSMC.2024.3519675
Ma TWen YSong XJin PHuang DHan HNan ZYu ZPeng SZhao YChen HDu ZHu XGuo Q(2025) Harmonia : A Unified Architecture for Efficient Deep Symbolic Regression IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2024.344302744:2(737-750)Online publication date: Feb-2025
https://doi.org/10.1109/TCAD.2024.3443027
Wetzel S(2025)Closed-form interpretation of neural network classifiers with symbolic gradientsMachine Learning: Science and Technology10.1088/2632-2153/ad9fd06:1(015035)Online publication date: 11-Feb-2025
https://doi.org/10.1088/2632-2153/ad9fd0
Muthyala MSorourifar FPeng YPaulson J(2025)SyMANTIC: An Efficient Symbolic Regression Method for Interpretable and Parsimonious Model Discovery in Science and BeyondIndustrial & Engineering Chemistry Research10.1021/acs.iecr.4c0350364:6(3354-3369)Online publication date: 4-Feb-2025
https://doi.org/10.1021/acs.iecr.4c03503
Lu QLuo YLi HLuo JWang Z(2025)Deep Differentiable Symbolic Regression Neural NetworkNeurocomputing10.1016/j.neucom.2025.129671629(129671)Online publication date: May-2025
https://doi.org/10.1016/j.neucom.2025.129671
Kronberger GOlivetti de França F(2025)Effects of reducing redundant parameters in parameter optimization for symbolic regression using genetic programmingJournal of Symbolic Computation10.1016/j.jsc.2024.102413129:COnline publication date: 1-Jul-2025
https://dl.acm.org/doi/10.1016/j.jsc.2024.102413
Crary CPiard WStitt GHicks BBean CBurlacu BBanzhaf W(2025)Using FPGA devices to accelerate the evaluation phase of tree-based genetic programming: an extended analysisGenetic Programming and Evolvable Machines10.1007/s10710-024-09505-226:1Online publication date: 1-Jun-2025
https://dl.acm.org/doi/10.1007/s10710-024-09505-2
Tonda A(2025)Review of PySR: high-performance symbolic regression in Python and JuliaGenetic Programming and Evolvable Machines10.1007/s10710-024-09503-426:1Online publication date: 1-Jun-2025
https://dl.acm.org/doi/10.1007/s10710-024-09503-4
Crary CStitt GBurlacu BBanzhaf W(2025)It’s Time to Revisit the Use of FPGAs for Genetic ProgrammingGenetic Programming Theory and Practice XXI10.1007/978-981-96-0077-9_14(275-295)Online publication date: 28-Feb-2025
https://doi.org/10.1007/978-981-96-0077-9_14
Le BVu TNguyen HVu V(2024)A Genetic Programming-Assisted Analytical Formula for Predicting the Permeability of Pervious ConcreteEngineering, Technology & Applied Science Research10.48084/etasr.761914:3(14775-14780)Online publication date: 1-Jun-2024
https://doi.org/10.48084/etasr.7619
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten