Function Optimization using Connectionist Reinforcement Learning Algorithms
Ronald J. Williams, Jing Peng · Connection Science · 1991
Any non-associative reinforcement learning algorithm can be viewed as a method for performing function optimization through (possibly noise-corrupted) sampling of function values. We describe the results of simulations in which the optima of several deterministic functions studied by Ackley were sought using variants of REINFORCE algorithms. Some of the algorithms used here incorporated additional heuristic features resembling certain aspects of some of the algorithms used in Ackley's studies. Differing levels of performance were achieved by the various algorithms investigated, but a number of them performed at a level comparable to the best found in Ackley's studies on a number of the tasks, in spite of their simplicity. One of these variants, called REINFORCE/MENT, represents a novel but principled approach to reinforcement learning in nontrivial networks which incorporates an entropy maximization strategy. This was found to perform especially well on more hierarchically organized tasks.
Cited by 288 works. doi:10.1080/09540099108946587
Authors
References
- Optimization by Simulated Annealing
- Adaptation in Natural and Artificial Systems
- An Efficient Heuristic Procedure for Partitioning Graphs
- “Neural” computation of decisions in optimization problems
- Learning to Predict by the Methods of Temporal Differences
- Neuronlike adaptive elements that can solve difficult learning control problems
- Connectionist learning procedures
- Temporal credit assignment in reinforcement learning
- Learning Automata: An Introduction
- Learning by statistical cooperation of self-interested neuron-like computing elements.