Foraging decisions as multi-armed bandit problems: applying reinforcement learning algorithms to foraging data.

07:00 EST 5th February 2019 | BioPortfolio

Summary of "Foraging decisions as multi-armed bandit problems: applying reinforcement learning algorithms to foraging data."

Finding resources is crucial for animals to survive and reproduce, but the understanding of the decision-making underlying foraging decisions to explore new resources and exploit old resources remains lacking. Theory predicts an 'exploration-exploitation trade-off' where animals must balance their effort into either stay and exploit a seemingly good resource or move and explore the environment. To date, however, it has been challenging to generate flexible yet tractable statistical models that can capture this trade-off, and our understanding of foraging decisions is limited. Here, I suggest that foraging decisions can be seen as multi-armed bandit problems, and apply deterministic (i.e., the Upper-Confidence-Bound or 'UCB') and Bayesian algorithms (i.e., Thompson Sampling or 'TS') to demonstrate how these algorithms generate testable a priori predictions from simulated data. Next, I use UCB and TS to analyse empirical foraging data from the tephritid fruit fly larvae Bactrocera tryoni to provide a qualitative and quantitative framework to analyse animal foraging behaviour. Qualitative analysis revealed that TS display shorter exploration period than UCB, although both converged to similar qualitative results. Quantitative analysis demonstrated that, overall, UCB is more accurate in predicting the observed foraging patterns compared with TS, even though both algorithms failed to quantitatively estimate the empirical foraging patterns in high-density groups (i.e., groups with 50 larvae and, more strikingly, groups with 100 larvae), likely due to the influence of intraspecific competition on animal behaviour. The framework proposed here demonstrates how reinforcement learning algorithms can be used to model animal foraging decisions.


Journal Details

This article was published in the following journal.

Name: Journal of theoretical biology
ISSN: 1095-8541


DeepDyve research library

PubMed Articles [8985 Associated PubMed Articles listed on BioPortfolio]

Guaranteed satisficing and finite regret: Analysis of a cognitive satisficing value function.

As reinforcement learning algorithms are being applied to increasingly complicated and realistic tasks, it is becoming increasingly difficult to solve such problems within a practical time frame. Henc...

How similarity between choice options affects decisions from experience: The accentuation-of-differences model.

Traditional theories of decision making require that humans evaluate choice options independently of each other. The independence principle underlying this notion states that the relative choice proba...

Automatic Configuration of Multi-Objective Local Search Algorithms for Permutation Problems.

Automatic algorithm configuration (AAC) is becoming a key ingredient in the design of high-performance solvers for challenging optimisation problems. However, most existing work on AAC deals with conf...

Search Dynamics on Multimodal Multi-Objective Problems.

We continue recent work on the definition of multimodality in multi-objective optimization (MO) and the introduction of a test-bed for multimodal MO problems. This goes beyond well-known diversity mai...

Psychosocial aspects of participation of the Polish Armed Forces in combat missions.

The military service of Polish soldiers on missions abroad began in 1953. Many years of experience of the Polish army as well as the armed forces of other countries show that being in a mission area h...

Clinical Trials [5632 Associated Clinical Trials listed on BioPortfolio]

Effectiveness of a Community - Based Cross-sector Network for the Management of Mental Problems and Disorders Associated With Forced Displacement Due to Armed Conflict in the Municipality of Soacha - Cundinamarca

It is of great importance to generate interventions that help ensure greater inclusion and social participation of the population that was and is a victim of the armed conflict, especially...

Technological Innovations in Behavioral Treatments for Cigarette Smoking

The purpose of the study is to evaluate a sustainable and broadly accessible treatment delivery model (Motiv8) for smoking cessation based on abstinence-reinforcement.

Improving Outcomes Among Medical/Surgical Inpatients With Alcohol Use Disorders

This project aims to help Veterans who are in the hospital and have untreated alcohol problems. First, the investigators will adapt a Decision Aid that explains alcohol-related treatment o...

Reinforcement-Based Treatment and Abstinence-Contingent Housing for Drug Abusers

Purpose of the project is to examine the effectiveness of Reinforcement-Based Treatment (RBT) on drug abuse and psychosocial outcomes of iner city opiate abusers who have recently complete...

Unified Protocol for Emotional Problems in Victims of the Armed Conflict in Colombia

The present study aims at evaluating the effects of a CBT intervention, a cultural adaptation of the Unified Protocol for the Transdiagnostic Treatment of Emotional Disorders (UP) in victi...

Medical and Biotech [MESH] Definitions

An armed intervention involving multi-national forces in the country of IRAQ.

An armed intervention involving multi-national forces in the country of IRAQ.

Any differences arising between two nations or groups and leading to the intervention of armed forces.

Branch of psychiatry concerned with problems related to the prevention, diagnosis, etiology, and treatment of mental or emotional disorders of Armed Forces personnel.

Extraoral devices for applying force to the dentition in order to avoid some of the problems in anchorage control met with in intermaxillary traction and to apply force in directions not otherwise possible.

Quick Search


DeepDyve research library

Searches Linking to this Article