Reinforcement Learning for Information Retrieval

Authors

Alexander Kuhnle, Miguel Aroca-Ouellette, John Reid, Dell Zhang

{alexander.kuhnle, miguel.aroca, john.reid, dell.zhang}[at]blueprism[dot]com

Description

There is strong interest in leveraging reinforcement learning (RL) for information retrieval (IR) applications including search, recommendation, and advertising. Just in SIGIR-2020, the term “reinforcement learning” is mentioned in 46 different papers. It has also been reported that Internet companies like YouTube and Alibaba have started to gain competitive advantages from their RL-based search and recommendation engines. This full-day tutorial gives IR researchers and practitioners who have little or no experience with RL the opportunity to learn about the fundamentals of modern RL in a practical hands-on setting. Furthermore, some representative applications of RL in IR systems will be introduced and discussed.

This tutorial is part of Search Solutions 2020.

Material

The complete repository for this tutorial can be found at RL Starterpack.

Quick Links:

• RL Slides
• RL For IR Slides
• TQL Exercise
• DQN Exercise
• PG Exercise
• AC Exercise

Schedule

Tuesday 24th November, 2020. All times are UK times (GMT).

• 09:30-10:30 RL Basics and Tabular Q-Learning
• 10:30-10:45 Coffee Break
• 10:45-11:45 Deep Q-Network (DQN)
• 11:45-12:00 Coffee Break
• 12:00-12:30 IR applications using DQN
• 12:30-14:00 Lunch Break
• 14:00-15:00 Policy Gradient (REINFORCE)
• 15:00-15:15 Coffee Break
• 15:15-15:45 Actor Critic
• 15:45-16:00 Coffee Break
• 16:00-17:00 IR applications using REINFORCE
• 17:00-17:15 Outlook

Contact

Questions? Feedback? Please reach out to one of the tutorial organisers using the emails listed at the top of this page.

Additional Resources

Introductory Material

• Spinning Up in Deep RL (OpenAI)
• Deep Reinforcement Learning Course (Thomas Simonini)
• Simple Reinforcement Learning with Tensorflow (Arthur Juliani)
• Deep Reinforcement Learning: Pong from Pixels (Andrej Karpathy)
• An Outsider’s Tour of Reinforcement Learning (Ben Recht)
• Lilian Weng:
  • A (Long) Peek into Reinforcement Learning
  • Policy Gradient Algorithms
  • Implementing Deep Reinforcement Learning Models with Tensorflow + OpenAI Gym
  • and more…

Critical Voices

• Deep Reinforcement Learning Doesn’t Work Yet (Alex Irpan)
• The Policy of Truth (Ben Recht)
• Lessons Learned Reproducing a Deep Reinforcement Learning Paper (Amid Fish)

Books

• Reinforcement Learning - An Introduction (Sutton & Barto, 2018)
• An Introduction to Deep Reinforcement Learning (Francois-Lavet et al., 30 Nov 2018)