QDax: Massive Parallelization for Quality-Diversity
Accelerated Quality-Diversity for Robotics through Massive Parallelism
Quality-diversity (qd) algorithms are a well-known approach to generate large-collections of diverse and high-quality policies.However, they are also known to be data-inefficient, requiring large amounts of computational resources and are slow when used in practice for robotics tasks.In this paper, we present a new implementation of map-elites which leverages massive parallelism on accelerators to make qd algorithms more accessible.We first demonstrate the improvements on the number of evaluations per second that parallelism using accelerated simulators can offer and show that qd algorithms are ideal candidates and can scale with massive parallelism to be run at interactive timescales.We then show that the increase in parallelism does not significantly affect the performance of qd algorithms, while reducing experiment runtimes by two factors of magnitudes, turning days of computation into minutes.These results show that qd can now benefit from hardware acceleration, which contributed significantly to the bloom of deeplearning.
