Train Scheduling with Deep Q-Network: A Feasibility Test

We consider a train scheduling problem in which both local and express trains are to be scheduled. In this type of train scheduling problem, the key decision is determining the overtaking stations at which express trains overtake their preceding local trains. This problem has been successfully modeled via mixed integer programming (MIP) models. One of the obvious limitation of MIP-based approaches is the lack of freedom to the choices objective and constraint functions. In this paper, as an alternative, we propose an approach based on reinforcement learning. We first decompose the problem into subproblems in which a single express train and its preceding local trains are considered. We, then, formulate the subproblem as a Markov decision process (MDP). Instead of solving each instance of MDP, we train a deep neural network, called deep Q-network (DQN), which approximates Q-value function of any instances of MDP. The learned DQN can be used to make decision by choosing the action which corresponds to the maximum Q-value. The advantage of the proposed method is the ability to incorporate any complex objective and/or constraint functions. We demonstrate the performance of the proposed method by numerical experiments.

Improving Sports Media’s Crystal Ball for National Basketball Association Playoff Elimination

The National Basketball Association (NBA) is divided into two conferences, each of which comprises 15 teams. At the end of the regular season, the top eight teams from each conference, based on winning percentage, compete in the playoffs. Mixed-integer-programming (MIP) models determine when a team has guaranteed its position in the playoffs (clinched) or, conversely, when it has been eliminated before the completion of the regular season. Our models incorporate a series of complex two-way tiebreaking criteria used by the NBA to determine how many more games are needed either to clinch or to avoid elimination. We compare the time at which a given team has clinched or been eliminated, in terms of the number of games played in the season to date, as posted in the NBA official standings, against results from our mixed-integer program. For the 2017–2018 season, when our models outperform those of the NBA, they do so by an average of 4.1 games. We also describe a scenario in which the NBA erroneously reported that the Boston Celtics had clinched a playoff spot and, conversely, show that the Golden State Warriors had clinched a playoff spot before the official announcement by the NBA.

A Multi-Item Replenishment Problem with Carbon Cap-and-Trade under Uncertainty

Recently, as global warming has become a major issue, many companies have increased their efforts to control carbon emissions in green supply chain management (GSCM) activities. This paper deals with the multi-item replenishment problem in GSCM, from both economic and environmental perspectives. A single buyer orders multiple items from a single supplier, and simultaneously considers carbon cap-and-trade under limited storage capacity and limited budget. In this case we can apply a can-order policy, which is a well-known multi-item replenishment policy. Depending on the market characteristics, we develop two mixed-integer programming (MIP) models based on the can-order policy. The deterministic model considers a monopoly market in which a company fully knows the market information, such that both storage capacity and budget are already determined. In contrast, the fuzzy model considers a competitive or a new market, in which case both of those resources are considered as fuzzy numbers. We performed numerical experiments to validate and assess the efficiency of the developed models. The results of the experiments showed that the proposed can-order policy performed far better than the traditional can-order policy in GSCM. In addition, we verified that the fuzzy model can cope with uncertainties better than the deterministic model in terms of total expected costs.

Solving the fair electric load shedding problem in developing countries

Often because of limitations in generation capacity of power stations, many developing countries frequently resort to disconnecting large parts of the power grid from supply, a process termed load shedding. This leaves households in disconnected parts without electricity, causing them inconvenience and discomfort. Without fairness being taken into due consideration during load shedding, some households may suffer more than others. In this paper, we solve the fair load shedding problem (FLSP) by creating solutions which connect households to supply based on some fairness criteria (i.e., to fairly connect homes to supply in terms of duration, their electricity needs, and their demand), which we model as their utilities. First, we briefly describe some state-of-art household-level load shedding heuristics which meet the first criteria. Second, we model the FLSP as a resource allocation problem, which we formulate into two Mixed Integer Programming (MIP) problems based on the Multiple Knapsack Problem. In so doing, we use the utilitarian, egalitarian and envy-freeness social welfare metrics to develop objectives and constraints that ensure our FLSP solutions results in fair allocations that consider the utilities of agents. Then, we solve the FLSP and show that our MIP models maximize the groupwise and individual utilities of agents, and minimize the differences between their pairwise utilities under a number of experiments. When taken together, our endeavour establishes a set of benchmarks for fair load shedding schemes, and provide insights for designing fair allocation solutions for other scarce resources.