Is the Solution More Democracy?

Philosophers often try to “solve” democracy’s problems by arguing we need more and better democracy. They tend to think certain kinds of democratic systems could unleash the hidden “wisdom of the crowds.” Some defenders of democracy propose deliberative democracy and some extol the reliability of large groups. However, both ideas have limitations in the real world. This chapter objects to such arguments as they rely upon mistaken applications of certain mathematical theorems, or they end up retreating toward unrealistic ideals of how people ought to behave. In effect, they say that democracy would be wonderful if only people behaved the right way.

Moderating with the Mob: Evaluating the Efficacy of Real-Time Crowdsourced Fact-Checking

Reducing the spread of false news remains a challenge for social media platforms, as the current strategy of using third-party fact- checkers lacks the capacity to address both the scale and speed of misinformation diffusion. Research on the “wisdom of the crowds” suggests one possible solution: aggregating the evaluations of ordinary users to assess the veracity of information. In this study, we investigate the effectiveness of a scalable model for real-time crowdsourced fact-checking. We select 135 popular news stories and have them evaluated by both ordinary individuals and professional fact-checkers within 72 hours of publication, producing 12,883 individual evaluations. Although we find that machine learning-based models using the crowd perform better at identifying false news than simple aggregation rules, our results suggest that neither approach is able to perform at the level of professional fact-checkers. Additionally, both methods perform best when using evaluations only from survey respondents with high political knowledge, suggesting reason for caution for crowdsourced models that rely on a representative sample of the population. Overall, our analyses reveal that while crowd-based systems provide some information on news quality, they are nonetheless limited—and have significant variation—in their ability to identify false news.

Wisdom of crowds detects COVID-19 severity ahead of officially available data

During the unfolding of a crisis, it is crucial to forecast its severity at an early stage , yet access to reliable data is often challenging early on. The wisdom of crowds has been effective at forecasting in similar scenarios. We investigated whether the initial regional social media reaction to the emerging COVID-19 pandemic in three critically affected countries has significant relations with their observed mortality a month later. We obtained COVID-19 related regionally geolocated tweets from Italian, Spanish, and United States regions. We quantified the predictive power of the wisdom of the crowds using correlations and regressions of geolocated Tweet Intensity (TI) during the initial social media attention peak versus the cumulative number of deaths a month ahead. We found that the intensity of initial COVID-19 related tweet attention at the beginning of the pandemic across Italian, Spanish, and United States regions is significantly related (p < 0.001) to the extent to which these regions had been affected by the pandemic a month later. This association is most striking in Italy as when at its peak of TI in late February 2020 only two of its regions had reported mortality. The collective wisdom of the crowds at early stages of the pandemic, when information on the number of infections was not broadly available, strikingly predicted the extent of mortality reflecting the regional severity of the pandemic almost a month later. Our findings could underpin the creation of real-time novelty detection systems aimed at early reporting of the severity of crises impacting a territory leading to early activation of control measures at a stage when available data is extremely limited.

The Wisdom of the Data: Getting the Most Out of Univariate Time Series Forecasting

Forecasting is a challenging task that typically requires making assumptions about the observed data but also the future conditions. Inevitably, any forecasting process will result in some degree of inaccuracy. The forecasting performance will further deteriorate as the uncertainty increases. In this article, we focus on univariate time series forecasting and we review five approaches that one can use to enhance the performance of standard extrapolation methods. Much has been written about the “wisdom of the crowds” and how collective opinions will outperform individual ones. We present the concept of the “wisdom of the data” and how data manipulation can result in information extraction which, in turn, translates to improved forecast accuracy by aggregating (combining) forecasts computed on different perspectives of the same data. We describe and discuss approaches that are based on the manipulation of local curvatures (theta method), temporal aggregation, bootstrapping, sub-seasonal and incomplete time series. We compare these approaches with regards to how they extract information from the data, their computational cost, and their performance.

Forecasting Political Events

Forecasting political events is a critical activity for social scientists. Forecasting can help test competing theories, let researchers grapple with the true substantive effects of their models, and bridge the gap between academia and the policy world. Forecasting is an academic activity with direct relevance for policymakers. Yet, a variety of cognitive biases can make forecasting challenging, even for experts. Despite these limitations, interest in forecasting is growing. This chapter describes several different approaches to forecasting political events, especially international political events. These methods include game theory, machine learning, statistical analysis, and event data algorithms. Recent research also suggests the way models drawing on the wisdom of the crowds, forecasting teams, and prediction markets can generate large improvements in accuracy when forecasting geopolitical events. All have strengths and weaknesses, given the inherent uncertainty that exists in the political world.

The Wisdom of the Crowds and Cost Overruns in Construction Project Tenders [Abstract]

Aim/Purpose: This study’s objective is to demonstrate the wisdom of the crowds phenomenon in construction project tenders and relate it to cost overruns in these projects. Background: The wisdom of the crowd’s phenomenon is an age-old idea that argues that collective opinion is better than any single (even expert) opinion. The first data-based evidence for it is from the beginning of the twentieth century when statistician Francis Galton attended an exhibition in which attendants were asked to estimate the weight of a large ox. He found that while individual estimates varied considerably, the median estimate was within less than one percent from the true weight. The existence of the wisdom of the crowds has a particularly important implication in tenders. Consider a tender for a contract in which the winner is the bidder that agrees to take the contract for the lowest cost. If the collective bid, i.e., the mean bid, is the most accurate in assessing the true value of the contract, then the winning bid is overestimating the contract and is therefore expected to end up with a loss. Indeed, this winner’s curse, was first observed in tenders in the petroleum industry and has been since found in many other fields. Methodology: All the construction projects that were tendered and completed between January 2017 and July 2020 under the management of the department of engineering and construction, a government agency in Israel, were analyzed. After data cleansing, the data comprised 148 tenders with 1295 bids and total value of 229 million US dollars. For each project we determined the valid bids, average (valid) bid, the winning bid, the original project estimated cost, and the actual payments to the winning contractor (actual project cost). Contribution: Construction projects in the public sector are typically granted through a bidding process in which the lowest bidder is granted the contract. It is therefore of interest to examine whether the wisdom of the crowds and the winner’s curse phenomena are manifested in this type of tenders. The results could help understand the reasons for cost overruns in public construction projects. Findings: 1. Wisdom of the crowds: For each project we computed the ratio of the average bid and the project’s estimated cost. The mean ratio (for the 148 projects) was 1.01 suggesting that, on average, the bids are within 1 percent from the true project value. 2. Winner’s curse: On average the winning bid was 7.9% less than the estimated cost and 8.1% less than the average bid. 3. Cost overruns: On average, the payments to the contractor were 16.3% higher than the estimated cost, and 18.8% higher than the average bid. 4. In total these results demonstrate how contractors are able to overcome the winner’s curse. On average, payments to the contractor were 30.7% higher than their bid. Recommendations for Practitioners: Tender issuing public agencies should take into account that the winning bid is based on unrealistic optimism and when the winning contractor is tested by the real costs, they will be hard pressed to avoid these losses and therefore will drive the project into cost overruns. Recommendations for Researchers: It is important to model the strategic game between contractors and project managers that represent the tender-issuing agency. This may explain why the construction industry is beleaguered by cost overruns. Impact on Society: In the current state, the public is paying more than needed for construction projects since winning contractors are struggling to spin their losses into gains. Future Research: Develop game theory models that are based on our empirical findings and that can help to reduce cost overruns in construction projects.

CrowdDetective: Wisdom of the Crowds for Detecting Abnormalities in Medical Scans

Machine learning (ML) has great potential for early diagnosis of disease from medical scans, and at times, has even been shown to outperform experts. However, ML algorithms need large amounts of annotated data – scans with outlined abnormalities - for good performance. The time-consuming annotation process limits the progress of ML in this field. To address the annotation problem, multiple instance learning (MIL) algorithms were proposed, which learn from scans that have been diagnosed, but not annotated in detail. Unfortunately, these algorithms are not good enough at predicting where the abnormalities are located, which is important for diagnosis and prognosis of disease. This limits the application of these algorithms in research and in clinical practice. I propose to use the “wisdom of the crowds” –internet users without specific expertise – to improve the predictions of the algorithms. While the crowd does not have experience with medical imaging, recent studies and pilot data I collected show they can still provide useful information about the images, for example by saying whether images are visually similar or not. Such information has not been leveraged before in medical imaging applications. I will validate these methods on three challenging detection tasks in chest computed tomography, histopathology images, and endoscopy video. Understanding how the crowd can contribute to applications that typically require expert knowledge will allow harnessing the potential of large unannotated sets of data, training more reliable algorithms, and ultimately paving the way towards using ML algorithms in clinical practice.