Pupil correlates of decision variables in mice playing a competitive mixed-strategy game

In a competitive game involving an animal and an opponent, the outcome is contingent on the choices of both players. To succeed, the animal must continually adapt to competitive pressure, or else risk being exploited and lose out on rewards. In this study, we demonstrate that head-fixed mice can be trained to play the iterative competitive game 'matching pennies' against a virtual computer opponent. We find that the animals' performance is well described by a hybrid computational model that includes Q-learning and choice kernels. Comparing between matching pennies and a non-competitive two-armed bandit task, we show that the tasks encourage animals to operate at different regimes of reinforcement learning. To understand the involvement of neuromodulatory mechanisms, we measure fluctuations in pupil size and use multiple linear regression to relate the trial-by-trial transient pupil responses to decision-related variables. The analysis reveals that pupil responses are modulated by observable variables, including choice and outcome, as well as latent variables for value updating, but not action selection. Collectively, these results establish a paradigm for studying competitive decision-making in head-fixed mice and provide insights into the role of arousal-linked neuromodulation in the decision process.

Random Actions in Experimental Zero-Sum Games

A mixed strategy, a strategy of unpredictable actions, is applicable to business, politics, and sports. Playing mixed strategies, however, poses a challenge, as the game theory involves calculating probabilities and executing random actions. I test i.i.d. hypotheses of the mixed strategy Nash equilibrium with the simplest experiments in which student participants play zero-sum games in multiple iterations and possibly figure out the optimal mixed strategy (equilibrium) through the games. My results confirm that most players behave differently from the Nash equilibrium prediction for the simplest 2x2 zero-sum game (matching-pennies) and 3x3 zero-sum game (e.g., the rock-paper-scissors game). The results indicate the need to further develop theoretical models that explain a non-Nash equilibrium behavior.

Event-Related Potentials During Decision-Making in a Mixed-Strategy Game

The decisions we make are sometimes influenced by interactions with other agents. Previous studies have suggested that the prefrontal cortex plays an important role in decision-making and that the dopamine system underlies processes of motivation, motor preparation, and reinforcement learning. However, the physiological mechanisms underlying how the prefrontal cortex and the dopaminergic system are involved in decision-making remain largely unclear. The present study aimed to determine how decision strategies influence event-related potentials (ERPs). We also tested the effect of levodopa, a dopamine precursor, on decision-making and ERPs in a randomized double-blind placebo-controlled investigation. The subjects performed a matching-pennies task against an opposing virtual computer player by choosing between right and left targets while their ERPs were recorded. According to the rules of the matching-pennies task, the subject won the trial when they chose the same side as the opponent, and lost otherwise. We set three different task rules: (1) with the alternation (ALT) rule, the computer opponent made alternating choices of right and left in sequential trials; (2) with the random (RAND) rule, the opponent randomly chose between right and left; and (3) with the GAME rule, the opponent analyzed the subject’s past choices to predict the subject’s next choice, and then chose the opposite side. A sustained medial ERP became more negative toward the time of the subject’s target choice. A biphasic potential appeared when the opponent’s choice was revealed after the subject’s response. The ERPs around the subject’s choice were greater in RAND and GAME than in ALT, and the negative peak was enhanced by levodopa. In addition to these medial ERPs, we observed lateral frontal ERPs tuned to the choice direction. The signals emerged around the choice period selectively in RAND and GAME when levodopa was administered. These results suggest that decision processes are modulated by the dopamine system when a complex and strategic decision is required, which may reflect decision updating with dopaminergic prediction error signals.

On Hagelbarger’s and Shannon’s matching pennies playing machines

In the 1950s, Hagelbarger’s Sequence Extrapolating Robot (SEER) and Shannon’s Mind-Reading Machine (MRM) were the state-of-the-art research results in playing the well-known “matching pennies” game. In our research we perform a software implementation for both machines in order to test the common statement that MRM, even simpler, beats SEER. Also, we propose a simple contextual predictor (SCP) and use it to compete with SEER and MRM. As expected, experimental results proves the claimed MRM superiority over SEER and even the SCP’s superiority over both SEER and MRM. At the end, we draw some conclusions and propose further research ideas, like the use of mixing models methods and the use of Hidden Markov Model for modelling player’s behaviour.

Identification of Non-Equilibrium Beliefs in Games of Incomplete Information Using Experimental Data

This paper studies the identification of players’ preferences and beliefs in discrete choice games using experimental data. The experiment comprises a set of games that differ in their matrices of monetary payoffs. The researcher is interested in the identification of preferences (utility of money) and beliefs on the opponents’ expected behavior, without imposing equilibrium restrictions or parametric assumptions on utility and belief functions. We show that the hypothesis of unbiased/rational beliefs is testable as long as the set of games in the experiment imply variation in monetary payoffs of other players, keeping the own monetary payoff constant. We present conditions for the full identification of utility and belief functions at the individual level – without restrictions on players’ heterogeneity in preferences or beliefs. We apply our method to data from two experiments: a matching pennies game, and a public good game.

An intersubjective model of agency for game theory

This paper proposes a new interpretation of non-cooperative games that shows why the unilateralism of best-reply reasoning fails to capture the mutuality of strategic interdependence. Drawing on an intersubjective approach to theorizing individual agency in shared context, including a non-individualistic model of common belief without infinite regress, the paper develops a general model of a 2 × 2 simultaneous one-shot non-cooperative game and applies it to games including Hi-Lo, Stag Hunt, Prisoners’ Dilemma, Chicken, BoS and Matching Pennies. Results include High as the rational choice in Hi-Lo, and Cooperate as a possible rational choice in the Prisoners’ Dilemma.

Balancing model-based and memory-free action selection under competitive pressure

In competitive situations, winning depends on selecting actions that surprise the opponent. Such unpredictable action can be generated based on representations of the opponent’s strategy and choice history (model-based counter-prediction) or by choosing actions in a memory-free, stochastic manner. Across five different experiments using a variant of a matching-pennies game with simulated and human opponents we found that people toggle between these two strategies, using model-based selection when recent wins signal the appropriateness of the current model, but reverting to stochastic selection following losses. Also, after wins, feedback-related, mid-frontal EEG activity reflected information about the opponent’s global and local strategy, and predicted upcoming choices. After losses, this activity was nearly absent—indicating that the internal model is suppressed after negative feedback. We suggest that the mixed-strategy approach allows negotiating two conflicting goals: 1) exploiting the opponent’s deviations from randomness while 2) remaining unpredictable for the opponent.

How Much Would You Pay to Change a Game before Playing It?

Envelope theorems provide a differential framework for determining how much a rational decision maker (DM) is willing to pay to alter the parameters of a strategic scenario. We generalize this framework to the case of a boundedly rational DM and arbitrary solution concepts. We focus on comparing and contrasting the case where DM’s decision to pay to change the parameters is observed by all other players against the case where DM’s decision is private information. We decompose DM’s willingness to pay a given amount into a sum of three factors: (1) the direct effect a parameter change would have on DM’s payoffs in the future strategic scenario, holding strategies of all players constant; (2) the effect due to DM changing its strategy as they react to a change in the game parameters, with the strategies of the other players in that scenario held constant; and (3) the effect there would be due to other players reacting to a the change in the game parameters (could they observe them), with the strategy of DM held constant. We illustrate these results with the quantal response equilibrium and the matching pennies game and discuss how the willingness to pay captures DM’s anticipation of their future irrationality.