Application of deep reinforcement learning for battery design

[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT REQUEST OF AUTHOR.] The conventional material research and development are mainly driven by human intuition, labor, and manual decision. It is ineffective and inefficient. Due to the complexity of material design and the magnitude of experimental and computational work, the discovery of materials with conventional methods usually takes very long development cycles (10-20 years) with enormous labor and costs. To address this challenge, we proposed a machine-learning framework called Material Artificial Intelligence Robotics-driven System (MARS), aiming to reduce the costs with the help of machine learning techniques. We applied advanced deep-learning networks to better predict conductivity. We explored neural network models and tree-based models such as LightGBM. In particular, we made the models more interpretable and identified the relationships between the electrolyte's composition and the ionic conductivity. To search for the optimal conductivity, we developed a sophisticated deep reinforcement learning (RL) model called DDPG (Deep Deterministic Policy Gradient) to explore novel recipes to reach much higher conductivity. DDPG begins the RL process by entering new states through actions, where each action at a specific state (with a one-hot vector, representing selections of electrolyte components) would yield a reward Q, trained by the predictor developed in the previous step. After the optimal compositions have been found for the maximum conductivity, voltage stability and modulus, new measurements would be conducted to confirm these compositions. The new measurement data were then fed back to improve the prediction model. In this way, the prediction model is constantly being updated by each RL prediction. Once a successful update has been made to the prediction model, the whole process iterates. Finally, a well-trained DDPG model combines the benefits of both Q-learning and Policy Gradient method. DDPG is faster, simpler, more robust, and able to achieve much higher conductivity than conventional search methods. Finally, the model could provide compositions that lead to higher conductivities than the highest conductivity in the training data. Then, we generated more training data according to these compositions to retrain the prediction model. The generated recipes have been attested both by machine learning metrics and wet lab experiments. The generated best conductivity (2:51e[superscript -3]) has meet our expectations of battery recipes.