R-HybrID: Evolution of agent controllers with a hybridisation of indirect and direct encodings

Abstract

Neuroevolution, the optimisation of artificial neural networks (ANNs) through evolutionary computation, is a promising approach to the synthesis of controllers for autonomous agents. Traditional neuroevolution approaches employ direct encodings, which are limited in their ability to evolve complex or large-scale controllers because each ANN parameter is independently optimised. Indirect encodings, on the other hand, facilitate scalability because each gene can be reused multiple times to construct the ANN, but are biased towards regularity and can become ineffective when irregularity is required. To address such limitations, we introduce a novel algorithm called R-HybrID. In R-HybrID, controllers have both indirectly encoded and directly encoded structure. Because the portion of structure following a specific encoding is under evolutionary control, R-HybrID can automatically find an appropriate encoding combination for a given task. We assess the performance of R-HybrID in three tasks: (i) a high-dimensional visual discrimination task that requires geometric principles to be evolved, (ii) a challenging benchmark for modular robotics, and (iii) a memory task that has proven difficult for current algorithms because it requires effectively accumulating neural structure for cognitive behaviour to emerge. Our results show that R-HybrID consistently outperforms three stateof-the-art neuroevolution algorithms, and effectively evolves complex controllers and behaviours.