Stochastic global optimization using random forests

Type of content
Conference Contributions - Published
Publisher's DOI/URI
Thesis discipline
Degree name
Publisher
Journal Title
Journal ISSN
Volume Title
Language
Date
2017
Authors
Robertson BL
Price C
Reale M
Abstract

Global optimization problems occur in many fields i ncluding mathematics, s tatistics, computer science, engineering, and economics. The purpose of a global optimization algorithm is to efficiently find an objective function’s global minimum. In this article we consider bound constrained global optimization, where the search is performed in a box, denoted . The global optimization problem is deceptively simple and it is usually difficult to find the global mi nimum. One of the difficulties is that there is often no way to verify that a local minimum is indeed the global minimum. If the objective function is convex, the local minimum is also the global minimum. However, many optimization problems are not convex. Of particular interest in this article are objective functions that lack any special properties such as continuity, smoothness, or a Lipschitz constant. The CARTopt algorithm is a stochastic algorithm for bound constrained global optimization. This algorithm alternates between partition and sampling phases. At each iteration, points sampled from are classified low or high based on their objective function values. These classified points define training data th at is used to partition in boxes using classification and regression trees (CART). The objective function is then evaluated at a number of points drawn from the boxes classified low and some are drawn from i tself. Drawing points from the low boxes focuses the search in regions where the objective function is known to be low. Sampling

reduces the risk of missing the global minimum and is necessary to establish convergence. The new points are then added to the existing training data and the method repeats. In this article we provide an alternative partitioning strategy for the CARTopt algorithm. Rather than using a CART partition at each iteration, we propose using a random forest partition. Pure CART partitions are known to over-fit t raining d ata a nd h ave l ow b ias a nd h igh v ariance. A r andom f orest p artition h as l ess variance than a CART partition, and hence provides a more stable approximation to where the objective function is expected to be low. We also provide a method for sampling low regions defined by random forest partitions. A preliminary simulation study showed that using random forests in the CARTopt algorithm was an effective strategy for solving a variety of global optimization test problems. The authors are currently refining the method and extending the set of test problems.

Description
Citation
Robertson BL, Price C, Reale M (2017). Stochastic global optimization using random forests. Hobart, Tasmania, Australia: 22nd International Congress on Modelling and Simulation (MODSIM 2017). 03/12/2017-08/12/2017. Proceedings of MODSIM 2017. 784-789.
Keywords
Bound constrained global optimization, CART, CARTopt, random search
Ngā upoko tukutuku/Māori subject headings
ANZSRC fields of research
Fields of Research::49 - Mathematical sciences::4905 - Statistics::490510 - Stochastic analysis and modelling
Rights