An adaptive deep reinforcement learning framework enables curling robots with human-like performance in real-world conditions

The game of curling can be considered a good test bed for studying the interaction between artificial intelligence systems and the real world. In curling, the environmental characteristics change at every moment, and every throw has an impact on the outcome of the match. Furthermore, there is no time for relearning during a curling match due to the timing rules of the game. Here, we report a curling robot that can achieve human-level performance in the game of curling using an adaptive deep reinforcement learning framework. Our proposed adaptation framework extends standard deep reinforcement learning using temporal features, which learn to compensate for the uncertainties and nonstationarities that are an unavoidable part of curling. Our curling robot, Curly, was able to win three of four official matches against expert human teams [top-ranked women’s curling teams and Korea national wheelchair curling team (reserve team)]. These results indicate that the gap between physics-based simulators and the real world can be narrowed.

Synaptic and intrinsic plasticity coordinate spike output in cerebellar Purkinje cells

SummaryLearning has been thought to be implemented by activity-dependent modifications of synaptic weight and intrinsic excitability. Here, we highlight how long-term depression at parallel fiber to Purkinje cell synapses (PF-PC LTD) and intrinsic plasticity of PCs coordinate the postsynaptic spike discharge from C57BL/6 male mice. Intrinsic plasticity of PCs in the flocculus matched the timing rules and shared intracellular signaling for PF-PC LTD. Notably, the intrinsic plasticity was confined to the dendritic branches where the synaptic plasticity is formed. Besides, when either synaptic or intrinsic plasticity was impaired, the impact of PF inputs was less reflected by the spike output of PCs. In conclusion, synergies between synaptic and intrinsic plasticity may play a role in tuning the PC output, thereby achieving optimal ranges of output.

Endogenous and Exogenous Election Timing

This chapter examines the rules that govern election timing in democracies. It begins by distinguishing between constitutionally fixed (exogenous) and constitutionally flexible (endogenous) election timing, reviews which political actors can call early elections when endogenous election timing is permitted, and notes that early elections are heterogeneous and can be of two distinct types—either triggered by government failure or called for partisan advantage. Next, the chapter summarizes the current understanding of the consequences of election timing rules for four important political outcomes: gridlock resolution, the electoral performance of incumbents, the bargaining power of various political actors in negotiating governments and policy, and the rhythm of policy cycles. Together the findings reviewed in this chapter show that election timing rules are highly consequential: they shape election outcomes, accountability, and policy, with significant implications for governance and voter welfare.

Bimodal stimulus timing-dependent plasticity in primary auditory cortex is altered after noise exposure with and without tinnitus

Central auditory circuits are influenced by the somatosensory system, a relationship that may underlie tinnitus generation. In the guinea pig dorsal cochlear nucleus (DCN), pairing spinal trigeminal nucleus (Sp5) stimulation with tones at specific intervals and orders facilitated or suppressed subsequent tone-evoked neural responses, reflecting spike timing-dependent plasticity (STDP). Furthermore, after noise-induced tinnitus, bimodal responses in DCN were shifted from Hebbian to anti-Hebbian timing rules with less discrete temporal windows, suggesting a role for bimodal plasticity in tinnitus. Here, we aimed to determine if multisensory STDP principles like those in DCN also exist in primary auditory cortex (A1), and whether they change following noise-induced tinnitus. Tone-evoked and spontaneous neural responses were recorded before and 15 min after bimodal stimulation in which the intervals and orders of auditory-somatosensory stimuli were randomized. Tone-evoked and spontaneous firing rates were influenced by the interval and order of the bimodal stimuli, and in sham-controls Hebbian-like timing rules predominated as was seen in DCN. In noise-exposed animals with and without tinnitus, timing rules shifted away from those found in sham-controls to more anti-Hebbian rules. Only those animals with evidence of tinnitus showed increased spontaneous firing rates, a purported neurophysiological correlate of tinnitus in A1. Together, these findings suggest that bimodal plasticity is also evident in A1 following noise damage and may have implications for tinnitus generation and therapeutic intervention across the central auditory circuit.

Role of synaptic dynamics and heterogeneity in neuronal learning of temporal code

Temporal codes are believed to play important roles in neuronal representation of information. Neuronal ability to classify and learn temporal spiking patterns is thus essential for successful extraction and processing of information. Understanding neuronal learning of temporal code has been complicated, however, by the intrinsic stochasticity of synaptic transmission. Using a computational model of a learning neuron, the tempotron, we studied the effects of synaptic unreliability and short-term dynamics on the neuron's ability to learn spike timing rules. Our results suggest that such a model neuron can learn to classify spike timing patterns even with unreliable synapses, albeit with a significantly reduced success rate. We explored strategies to improve correct spike timing classification and found that firing clustered spike bursts significantly improves learning performance. Furthermore, rapid activity-dependent modulation of synaptic unreliability, implemented with realistic models of dynamic synapses, further improved classification of different burst properties and spike timing modalities. Neuronal models with only facilitating or only depressing inputs exhibited preference for specific types of spike timing rules, but a mixture of facilitating and depressing synapses permitted much improved learning of multiple rules. We tested applicability of these findings to real neurons by considering neuronal learning models with the naturally distributed input release probabilities found in excitatory hippocampal synapses. Our results suggest that spike bursts comprise several encoding modalities that can be learned effectively with stochastic dynamic synapses, and that distributed release probabilities significantly improve learning performance. Synaptic unreliability and dynamics may thus play important roles in the neuron's ability to learn spike timing rules during decoding.