End-to-end symbolic regression with transformers | Proceedings of the 36th International Conference on Neural Information Processing Systems (2024)

Supplemental Material

1. Silviu-Marian Udrescu and Max Tegmark. Ai feynman: a physics-inspired method for symbolic regression, 2020.Google Scholar
2. M. Cranmer, Alvaro Sanchez-Gonzalez, Peter W. Battaglia, Rui Xu, Kyle Cranmer, David N. Spergel, and Shirley Ho. Discovering symbolic models from deep learning with inductive biases. ArXiv, abs/2006.11287, 2020.Google Scholar
3. Jiří Kubalík, Erik Derner, Jan Žegklitz, and Robert Babuška. Symbolic regression methods for reinforcement learning. IEEE Access, 9:139697-139711, 2021.Google ScholarCross Ref
4. Erik Derner, Jirí Kubalik, Nicola Ancona, and Robert Babuška. Symbolic regression for constructing analytic models in reinforcement learning. ArXiv, abs/1903.11483, 2019.Google Scholar
5. Mojtaba Valipour, Bowen You, Maysum Panju, and Ali Ghodsi. Symbolicgpt: A generative transformer model for symbolic regression. arXiv preprint arXiv:2106.14131, 2021.Google Scholar
6. Luca Biggio, Tommaso Bendinelli, Alexander Neitz, Aurelien Lucchi, and Giambattista Paras-candolo. Neural symbolic regression that scales, 2021.Google Scholar
7. William La Cava, Patryk Orzechowski, Bogdan Burlacu, Fabricio Olivetti de Franca, Marco Virgolin, Ying Jin, Michael Kommenda, and Jason H Moore. Contemporary symbolic regression methods and their relative performance. arXiv preprint arXiv:2107.14351, 2021.Google Scholar
8. Nikos Aréchiga, Francine Chen, Yan-Ying Chen, Yanxia Zhang, Rumen Iliev, Heishiro Toyoda, and Kent Lyons. Accelerating understanding of scientific experiments with end to end symbolic regression. ArXiv, abs/2112.04023, 2021.Google Scholar
9. Silviu-Marian Udrescu and Max Tegmark. Symbolic pregression: Discovering physical laws from raw distorted video. Physical review. E, 103 4-1:043307, 2021.Google Scholar
10. Anja Butter, Tilman Plehn, Nathalie Soybelman, and Johann Brehmer. Back to the formula - lhc edition. 2021.Google Scholar
11. Michael Schmidt and Hod Lipson. Age-fitness pareto optimization. In Genetic programming theory and practice VIII, pages 129-146. Springer, 2011.Google ScholarCross Ref
12. Michael Schmidt and Hod Lipson. Distilling free-form natural laws from experimental data. science, 324(5923):81-85, 2009.Google Scholar
13. William La Cava, Tilak Raj Singh, James Taggart, Srinivas Suri, and Jason H Moore. Learning concise representations for regression by evolving networks of trees. arXiv preprint arXiv:1807.00981, 2018.Google Scholar
14. Trent McConaghy. Ffx: Fast, scalable, deterministic symbolic regression technology. In Genetic Programming Theory and Practice IX, pages 235-260. Springer, 2011.Google ScholarCross Ref
15. Marco Virgolin, Tanja Alderliesten, Cees Witteveen, and Peter AN Bosman. Improving model-based genetic programming for symbolic regression of small expressions. Evolutionary computation, 29(2):211-237, 2021.Google ScholarCross Ref
16. Fabricio Olivetti de França and Guilherme Seidyo Imai Aldeia. Interaction-transformation evolutionary algorithm for symbolic regression. Evolutionary computation, 29(3):367-390, 2021.Google ScholarCross Ref
17. Ignacio Arnaldo, Krzysztof Krawiec, and Una-May O'Reilly. Multiple regression genetic programming. In Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, pages 879-886, 2014.Google ScholarDigital Library
18. Marco Virgolin, Tanja Alderliesten, and Peter A. N. Bosman. Linear scaling with and within semantic backpropagation-based genetic programming for symbolic regression. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '19, page 1084-1092, New York, NY, USA, 2019. Association for Computing Machinery.Google ScholarDigital Library
19. Michael Kommenda, Bogdan Burlacu, Gabriel Kronberger, and Michael Affenzeller. Parameter identification for symbolic regression using nonlinear least squares. Genetic Programming and Evolvable Machines, 21(3):471-501, 2020.Google ScholarDigital Library
20. Brenden K Petersen, Mikel Landajuela Larma, T Nathan Mundhenk, Claudio P Santiago, Soo K Kim, and Joanne T Kim. Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients. arXiv preprint arXiv:1912.04871, 2019.Google Scholar
21. Georg Martius and Christoph H Lampert. Extrapolation and learning equations. arXiv preprint arXiv:1610.02995, 2016.Google Scholar
22. Subham Sahoo, Christoph Lampert, and Georg Martius. Learning equations for extrapolation and control. In International Conference on Machine Learning, pages 4442-4450. PMLR, 2018.Google Scholar
23. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998-6008, 2017.Google ScholarDigital Library
24. Guillaume Lample and François Charton. Deep learning for symbolic mathematics. arXiv preprint arXiv:1912.01412, 2019.Google Scholar
25. François Charton, Amaury Hayat, and Guillaume Lample. Learning advanced mathematical computations from examples. arXiv preprint arXiv:2006.06462, 2020.Google Scholar
26. François Charton. Linear algebra with transformers. arXiv preprint arXiv:2112.01898, 2021.Google Scholar
27. Christopher Hahn, Frederik Schmitt, Jens U Kreber, Markus N Rabe, and Bernd Finkbeiner. Teaching temporal logics to neural networks. arXiv preprint arXiv:2003.04218, 2020.Google Scholar
28. Stanislas Polu and Ilya Sutskever. Generative language modeling for automated theorem proving. arXiv preprint arXiv:2009.03393, 2020.Google Scholar
29. Stéphane d'Ascoli, Pierre-Alexandre Kamienny, Guillaume Lample, and François Charton. Deep symbolic regression for recurrent sequences. arXiv preprint arXiv:2201.04600, 2022.Google Scholar
30. Nikos Arechiga, Francine Chen, Yan-Ying Chen, Yanxia Zhang, Rumen Iliev, Heishiro Toyoda, and Kent Lyons. Accelerating understanding of scientific experiments with end to end symbolic regression, 2021.Google Scholar
31. Roger Guimerà, Ignasi Reichardt, Antoni Aguilar-Mogas, Francesco A Massucci, Manuel Miranda, Jordi Pallarès, and Marta Sales-Pardo. A bayesian machine scientist to aid in the solution of challenging scientific problems. Science advances, 6(5):eaav6971, 2020.Google Scholar
32. Patrick Kidger. Sympytorch. https://github.com/patrick-kidger/sympytorch, 2021.Google Scholar
33. Richard Zou Horace He. functorch: Jax-like composable function transforms for pytorch. https://github.com/pytorch/functorch, 2021.Google Scholar
34. Sam Wiseman and Alexander M. Rush. Sequence-to-sequence learning as beam-search optimization, 2016.Google ScholarCross Ref
35. Jerome H Friedman. Greedy function approximation: a gradient boosting machine. Annals of statistics, pages 1189-1232, 2001.Google Scholar
36. Steven H. Strogatz. Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry and Engineering. Westview Press, 2000.Google Scholar
37. Ying Jin, Weilin Fu, Jian Kang, Jiadong Guo, and Jian Guo. Bayesian symbolic regression, 2020.Google Scholar
38. T. Nathan Mundhenk, Mikel Landajuela, Ruben Glatt, Claudio P. Santiago, Daniel M. Faissol, and Brenden K. Petersen. Symbolic regression via neural-guided genetic programming population seeding, 2021.Google Scholar
39. Sean Welleck, Peter West, ji*ze Cao, and Yejin Choi. Symbolic brittleness in sequence models: on systematic generalization in symbolic mathematics. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 8629-8637, 2022.Google ScholarCross Ref
40. Martin Vastl, Jonáš Kulhánek, Jirí Kubalík, Erik Derner, and Robert Babuška. Symformer: End-to-end symbolic regression using transformer-based architecture. arXiv preprint arXiv:2205.15764, 2022.Google Scholar

Index Terms

1. End-to-end symbolic regression with transformers

   1. Computing methodologies

      1. Machine learning

         1. Machine learning approaches

            1. Neural networks

               See Also

               Casper Wilstrup on LinkedIn: Feyn Documentation · Symbolic AI using the QLattice

   2. Theory of computation

• Genetic programming performance prediction and its application for symbolic regression problems

  Highlights

  • Theoretical analysis of GP performance prediction.
  • Suggestion of upper bounds ...

  Abstract

  Predicting the performance of Genetic Programming (GP) helps us identify whether it is an appropriate approach to solve the problem at hand. However, previous studies show that measuring the difficulty of a problem for GP and ...

  Read More

• Developing Postfix-GP Framework for Symbolic Regression Problems

  ACCT '15: Proceedings of the 2015 Fifth International Conference on Advanced Computing & Communication Technologies

  This paper describes Postfix-GP system, postfix notation based Genetic Programming (GP), for solving symbolic regression problems. It presents an object-oriented architecture of Postfix-GP framework. It assists the user in understanding of the ...

  Read More

• A framework for end-to-end video quality prediction of MPEG video

  This paper proposes, describes and evaluates a novel framework for video quality prediction of MPEG-based video services, considering the perceptual degradation that is introduced by the encoding process and the provision of the encoded signal over an ...

  Read More