DeepMind, an artificial intelligence subsidiary of Alphabet Inc., is widely known for its contribution to deep reinforcement learning, specifically expertise in complicated games and predicting protein structures. Now, the organization aims to take the next step in robotics research by acquiring MuJoCo.
A recent update on the DeepMinds website describes acquiring the rigid-body physics simulator MuJoCo and has made it freely available for the research community. MuJoCo (Multi-Joint Dynamics with Contact) is a physics engine that aims to facilitate research and development in robotics, biomechanics, graphics and animation, and other areas where quick and accurate simulation is required. As robotics research is freely available, it will have a progressive impact on the work of scientists who are struggling with the costs. At a glance, MuJoCo becomes a vital factor for DeepMinds future. It not only serves as a science or artificial intelligence lab but also serves a business unit for one of the largest tech companies in the world.
Training and testing robots in the practical world is an expensive and slow process. Simulation platforms are a big boon for research in robotics, it allows researchers to train multiple AI agents in parallel at a faster speed than in real life. Today, with advancements in simulated environments, most robotics research teams carry out the bulk of training in their AI models. A model, once trained, enters a testing phase to fine-tune real physical robots further.
In the past few years, several simulation environments for reinforcement learning and robotics have been launched. However, MuJoCo stands out from many physics simulators like Isaac Gym, PyBullet, or Roboschool, because it has fine-grained detail that simulates contact surfaces. It accurately models the physics laws shown in the emergence of physical phenomena such as Newton’s Cradle.
MuJoCo also has an admirable feature that supports simulation of the musculoskeletal model of humans and animals, which is principally crucial in bipedal and quadruped robots. Often, the differences in a model result in performance degradation of AI models. The increased accuracy of the physics environment will ensure minor deviation when comparing results of simulated environments to the real world, also called the ‘sim2real’ gap. A smaller sim2real gap denotes lesser changes when implemented in the physical world.
Before DeepMind open-source MuJuCo, many researchers had to bear the license cost and rely on the PyBullet platform. A recent paper by researchers in Cardiff University states that “the annual cost of MuJoCo institutional license is $3000,” which is an unreasonable deal for many small research teams when a project has a more extended time period. OpenAI released Roboschool in 2017, which was an alternative to MuJoCo (for Gym), its toolkit to train deep reinforcement learning models for robotics and various applications.
DeepMinds also refers to a recent article in PNAS (proceedings of the national academy of the sciences) emphasizing the application of simulation in robotics. The authors have recommended better support for the development of an open-source simulation platform, mentioning “a robust and feature-rich set of four or five simulation tools in the open-source domain is critical to have state-of-the-art solutions in robotics.” To align with the proposed goal, DeepMind has committed to develop and maintain MuJoCo as a free, open-source, community-driven platform. It is also important to remember that MuJoCo is designed to leverage power only from traditional processors (CPUs), not GPUs with many computation cores.
In contrast to the interest, Researchers at the University of Toronto, Nvidia, and other organizations highlighted the limits of MuJoCo simulation platforms that work on CPUs. For instance, OpenAI developed a robotic hand called Dactyl, which is trained on a computational cluster of around 30,000 CPU cores. These costs remain a challenge with CPU-based platforms.
Richard Sutton is one of the pioneers of reinforcement learning and describes it as “the first computational theory of intelligence.” According to DeepMind’s scientists, if one has a complex environment and a sound reinforcement learning algorithm, one can develop AI agents that will acquire the traits of general intelligence. DeepMind envisions developing artificial general intelligence (AGI) that is flexible to understand innate problem-solving capabilities found in humans and animals.
While scientists debate for the ideal paths to AGI, DeepMind has demonstrated its views timely. MuJoCo acquisition can provide DeepMind a powerful tool to test this hypothesis and gradually gain better results. By availing it to small research teams, DeepMind can also help garner talent it would hire in the future.
