To resemble the body flexibility of biological snakes, snake-like robots are designed as a chain of body modules, which gives them many degrees of freedom (DoF) on the one hand and leads to a challenging task to control them on the other. Compared with conventional model-based control methods, reinforcement learning (RL)-based ones provide promising solutions to design agile and energy-efficient gaits for snake-like robots as RL-based methods can fully exploit the hyperredundant bodies of the robots. However, RL-based methods for snake-like robots have rarely been investigated even in simulations, let alone been deployed on real-world snake-like robots.