Reinforcement Learning of Deceptive Tactics in Robot Soccer


In this project, we design a framework for soccer playing robots to learn tactics using reinforcement learning. In an adversarial setting, the tactics of interest are non-trivial involving depceptive moves.


In an adversarial scenario, deceptions are powerful tools capable of earning time delayed rewards which an agent can use to circumvent the opponent's counter attack. This paper illustrates deception as a complementary policy to direct objective satisfaction. In this paper, a framework for deceptions is defined to finally determine the number and nature of these actions. A minimal set of these actions ensures fast learning while being robust enough to confront any strong opponent. The focus of this work is primarily on application of Robot soccer. Benchmark examples of tactics involving a subset of players are implemented with deceptions, the results of which are compared to standard hand-coded solutions. This paper focuses on such contests with an aim to generate a library of tactics to deal with them seamlessly without having to customise the function approximator or learning algorithm to a great deal.
The work takes up three vital tactics - retaining possession of a ball in a 3 vs 2 scenario, aquiring the ball in 2 vs 3 scenario and an aggressive defense behaviour in a 1 vs 1 scenario.


  • S. Choudhury "Application of Reinforcement Learning in Robot Soccer", Masters Thesis, IIT Kharagpur (pdf).

  • Details

    Here is a paper with theory and results.

    More details coming soon ...


    Here is Peter Stone's webpage on Keepaway.

    Code will be made available soon.