Exploration-exploitation dilemma

The exploration-exploitation dilemma refers to the challenge of balancing the desire to explore new options and gather information with the need to exploit known options for maximum gain. This dilemma is particularly relevant in reinforcement learning, where various methods have been proposed to address it, such as initial optimism and ϵ-greedy policies. Multi-armed bandit algorithms are one solution to this dilemma, which is a common issue in adaptive controllers that have conflicting goals of optimal control and fast learning.From: Autonomous Mobile Robots [2018], Contextual Bandit Approach-based Recommendation System for Personalized Web-based Services [2021], The Control Systems Handbook [2019]

Algorithms for Planning under Uncertainty in Prediction and Sensing

View Chapter

Purchase Book

Published in Shuzhi Sam Ge, Frank L. Lewis, Autonomous Mobile Robots, 2018

Jason M. O’Kane, Benjamín Tovar, Peng Cheng, Steven M. LaValle

The exploration–exploitation dilemma is an important enough problem in reinforcement learning that many different methods have been suggested to deal with it, including initial optimism [45], which assigns large initial values to each Q^(x,u), ensuring that each action is tried often enough to “drive down” its Q^ value to near its true value, and ϵ-greedy policies [33] that select the current best action with probability ϵ and choose randomly otherwise. An in-depth study of this topic is given in Reference 46.

Contextual Bandit Approach-based Recommendation System for Personalized Web-based Services

View Article

Journal Information

Published in Applied Artificial Intelligence, 2021

Akshay Pilani, Kritagya Mathur, Himanshu Agrawald, Deeksha Chandola, Vinay Anand Tikkiwal, Arun Kumar

Multi-armed bandit problem is a classical problem in the field of Computer Science. Multi-armed bandit algorithms provide a solution to the exploration-exploitation dilemma. Contextual Bandits are a type of bandit which uses expected payoff to recommend news articles. The expected payoff of a user is calculated using context and unknown bandit parameters. Here, context is a feature vector obtained by using the information of both user and news articles. Some Contextual Bandit algorithms consider the users to be independent of each other, i.e. unknown bandit parameters are calculated for each user independently.

Exploration-exploitation dilemma

Explore chapters and articles related to this topic

Algorithms for Planning under Uncertainty in Prediction and Sensing

Contextual Bandit Approach-based Recommendation System for Personalized Web-based Services