The Fokker-Planck machine offers a new method for global optimization. It combines features of continuous simulated annealing and genetic algorithms. The method follows from the fact that a forward Kolmogorov (or Fokker-Planck) equation can be associated with the evolution of a stochastic differential equation. The transition density which satisfies the Fokker-Planck equation is parametrized by means of a Radial Basis Function (RBF) neural network. By sampling the search space, a learning rule for the RBF network is obtained. Points are generated in search space according to the statistics of the RBF network. After sampling the search space at each generation, information of the local geometry at the sampling points is exchanged. This is done by solving a constrained linear least squares problem with as solution the update of the parameter vector of the RBF network.
An on-line learning version of the Fokker-Planck machine is developed which enables to solve larger scale optimization problems, by solving the constrained least squares problem in a recursive way. Furthermore the application of Amari's information geometry (differential geometry for densities) to the Fokker-Planck machine has been investigated, leading to an improved optimization method.
 |