In order to solve the problem of agent training in the multi-agent circumstances of robot soccer,a new method of agent learning is put forward,which combines the fuzzy control with the Q-Learning.During the learning process,the reward function is controlled automatically to earn the optimal policy.It is proved that this method is effective on the simulation platform of middle-size league,and the simulation result is good.In addition,it could be adapted to apply in other multi-agent circumstances.