Forecasting the evolution of artificial intelligence: meeting with Emile Servan-Schreiber

22 février 2021

On the occasion of the contest to predict the evolution of artificial intelligence launched by Hypermind with the support of Open Philanthropy, we decided to learn more about the initiative from Emile Servan-Schreiber, co-founder of Hypermind and author of Supercollectif (Fayard editions).

Could you introduce us to Hypermind in a few words?

For 20 years Hypermind has been one of the world leaders in mobilizing collective intelligence to make forecasts, generally on geopolitical, economic or technological subjects. Our clients are think tanks, large corporations and governments.

Our “predictive market” platforms and algorithms consolidate the predictions of a multitude of informed people into reliable probabilities, updated in real time and over time, to inform decision-makers in advance of their decisions.

In addition to the platforms, we provide our clients with a panel of several hundred elite forecasters whose skills have been proven over time on a large number of forecasts.

We cultivate this panel via a predictive market open to the public, mainly in France and the United States. Participation is free, but the best predictors are rewarded with prizes from our sponsors and clients. This allows us to constantly identify new talent to enrich the panel of elite predictors.

Over the past decade, Hypermind has been fortunate to be involved in several major research projects conducted by the U.S. government’s IARPA agency to advance the state of the art in collective forecasting. This has allowed us to perfect our technologies and services.

Mass phenomena do not always rhyme with intelligence: Kitty Genovese effect, “sheep” effect, populism... Hypermind, however, bets on the strength of collective intelligence. In your opinion, what are the necessary conditions for the group to pull intelligence upwards?

The received idea is indeed that “crowds accumulate not intelligence but mediocrity,” as sociologist Gustave Le Bon observed in the 19th century in his famous essay on the Psychology of Crowds. He is right – and many of us have had the experience in our personal or professional lives of being caught up in group dynamics or crowd movement towards unworthy behaviour or idiotic thinking.

Yet there can be no doubt that collective intelligence exists. Our civilization is proof of this. So how can we reconcile these two truths?

Over the past twenty years or so, a science of collective intelligence has emerged which has identified a simple but strict recipe for extracting the intelligence of a group: first, it is necessary to cultivate diversity of points of view by encouraging the independence of mind of each individual, then to aggregate opinions using an objective method which takes all opinions into account.

When it comes to choosing a leader, for example, a majority vote can be used. When it comes to making an estimate, one can calculate the average of everyone’s estimates. And when it comes to making a forecast, the most reliable method is to organize bets. That’s what our predictive markets do.

What is the purpose of the AI Progress Prediction Contest that you have launched and who can participate?

Predictions of each other in AI are often fanciful. On the one hand, skeptics who predict that an AI will never be able to do this or that (drive a car, create a work, win a Go championship, etc.) are always wrong. On the other hand, the exalted ones are generally far too optimistic about the chronology of progress.

“To govern is to plan ahead,” advocated the brilliant Émile de Girardin. If we want to give our societies a chance to integrate AI’s progress into their governance – which seems essential in view of its economic and strategic impact – we first need more reliable forecasts.

The aim of this competition is to test the capacity of collective intelligence to make credible medium-term (2023) and long-term (2030) forecasts in this field. The results will be made public in mid-April.

Anyone can participate – just sign up for the Hypermind Prediction Market – but the subject is quite technical. For an amateur, it’s an entertaining way to learn about the state of the art. For a professional, it’s an opportunity to test his ability to predict, which is, according to Yann LeCun, “the essence of intelligence”. Participation is free, but the best predictors will share €6,000 in rewards.

The history of Artificial Intelligence is made up of runaways and winter periods. The most recognized experts have been wrong several times, but one could intuitively think that these experts were the best placed to make such predictions. To what extent do you think that collective intelligence can shed light on such a complex subject and how do you explain it?

On complex and unprecedented subjects, the expertise of each is in fact relative. We were able to observe this during our research in geopolitical forecasting with the intelligence community in the United States. (IARPA), or more recently on epidemics in general and the Covid-19 pandemic in particular.

For example, when we solicited, with the Johns Hopkins University Center for Health Security, the predictions of several hundred public health professionals on the evolution of 19 infectious diseases, including Covid-19, it appeared that individual expert predictions were generally no more reliable than random predictions. In contrast, the consolidated forecasts of all at once were better than those of the best individuals.

This phenomenon known as “crowd wisdom” is based on a mathematical law: Scott Page’s diversity theorem. It proves that collective error is lower as individual errors are lower – that’s obvious – but that it also decreases as the diversity of opinions increases. So the more experts diverge because the subject is complex, the more likely it is that the collective’s consolidated opinion will be the most reliable.

We are living in a period of uncertainty that the world has rarely experienced and that neither human intelligence nor AI have seen coming, as they have in common that they rely primarily on past experiences and a tendency to reproduce the status quo. In what way do you think collective intelligence is superior to AI (computer science) in making predictions? And isn’t this period of doubt likely to put both at risk until we return to a “normal” situation?

Forecasting is the most difficult intellectual task, and no method, no brain, artificial or collective, is perfectly reliable. Success can only be relative, being less wrong, or less often than other methods. Much depends on the quality of the data available.

When there is a lot of relevant historical data, artificial intelligence is the best solution. But when the world changes abruptly, as it is now, and historical patterns and data are suddenly obsolete, collective intelligence is our best omen to try to discern the trends of the new world. CI can guide us in the time it takes to accumulate enough data again to re-train AIs… until the next rupture.

Intelligence in the 21st century will not only be artificial. In terms of FLOPs, no supercomputer yet rivals a human brain. And there are almost 8 billion of us… Hypermind’s mission is to help organize this tremendous potential of collective intelligence towards a better world.

Information and participation in the competition