Mellers and colleagues found a unique opportunity to investigate different forecasting methods with the occurrence of a two-year forecasting tournament sponsored by IARPA, the Intelligence Advanced Research Projects Activity.

Forecasters from around the world were recruited via professional societies, research centers, alumni associations, science blogs, and word of mouth.

The forecasters were invited to submit estimates for the likelihood that each of 199 geopolitical outcomes would occur on the website www.goodjudgmentproject.com.

The first year began with 2,246 participants, who were randomly assigned to one of the conditions that reflected different types of training (non training, scenario training, or probability training) and different types of group influence (independent, crowd-belief, team, or prediction-market forecasting).

The results showed that probability training was more effective than scenario training, which, in turn, was more effective than receiving no training at all.

And the results also showed that team forecasters were more accurate than crowd-belief forecasters, who were, in turn, more accurate than independent forecasters.

“Greater accuracy in teams was due to members who gathered and shared information, encouraged one another, and discussed issues,” Mellers and colleagues explain.

Intriguingly, the best forecasters at the end of the first year were grouped together in “elite teams” for the second year of the tournament. These “superforecasters” were far more accurate in their forecasts than the other groups in Year 2.

“Results strongly disconfirm the expectations of pro-independence theorists,” the researchers note.

“Our psychological interventions reduced the errors in individual forecasts for events ranging from military conflicts and global leadership changes to international negotiations and economic shifts.”

“To improve geopolitical forecasts, one needs insights from both statistics and psychology.”

(IMage source:Unsplach.com)