Data can help policymakers pinpoint a country’s vulnerabilities and provide early warning of conflict and early detection of fragility to inform policy decisions, according to J.J. Messner, executive director at The Fund for Peace (FFP).
At Peacebuilding Deeply’s first Deeply Talks, our editors spoke with Messner about the data behind the 2018 Fragile State Index (FSI), produced yearly by FFP, and discussed how this can impact peacebuilding efforts.
“The data itself comes from roughly 10,000 or more different sources,” Messner said, adding that “the data is a triangulated process between content analysis, quantitative data sets, and qualitative review.”
This analysis is then used to asses the fragility of a country based on 12 social, economic, cohesion and political indicators, with the aim of measuring a state’s vulnerabilities and highlighting elements that could diminish its capacity to manage those pressures.
Here are a few highlights from the conversation:
Alessandria Masi: The Fragile State Index has been around since 2006, but it used to be called the Failed State Index until 2014. What were the reasons behind that name change?
J.J. Messner: The concept of state failure is not really an accurate one, because a state may be fragile for many years, maybe many decades, but there’s never an instance where a country is beyond hope. I think that for some countries, when they ranked poorly on what was the Failed States Index, they would focus more on sort of the terminology of being failed; how dare you call us a failed state, for example. When we changed the name to fragile, I think people really did focus much more on the data and our findings.
Hashem Osseiran: What crucial points about the FSI methodology should our audience know?
Messner: The underlying framework has existed for roughly 25 years, and was put together by a number of institutions, including The Fund for Peace. It has been a framework for conflict early warning and analysis for practitioners in all sorts of different environments around the world for a quarter of a century.
Now, the next question is where does the data come from? And the data is a triangulated process between content analysis, quantitative data sets, and qualitative review. And the most important part of that is the content analysis. We do use content analysis to analyze, generally around about 50 million different documents, reports and other data points, every year, to determine saliency for every sub-indicator and hence every indicator, for every country.
Though I’m not ruling out the possibility that there may be some bias in original data in that dataset, the fact that it comes from such a diverse spectrum of sources means that we’re relatively confident that any such bias will even out.
Masi: Is data collect across languages, or in local languages or just in English?
Messner: The algorithm itself focuses on English only. That’s partly a logistical issue. We have in the past attempted to teach the algorithm other languages. And frankly, it’s not so easy.
We recognize that there is somewhat of an inherent bias in focusing only on English-language data. However, we are confident that we are picking up a large enough diversity of data. And if [a foreign language data point] would be translated into any language, it would most likely be English. So we are relatively confident that we are still picking up a diversity of content, even if it originated in a foreign language.
Osseiran: How do you evaluate whether or not the Fragile State Index or the Conflict Assessment System Tool (CAST) has accurately measured a state’s fragility?
Messner: One metric to look at is comparing across different indices. The preliminary findings are that the Fragile States Index correlates closely with a number of other different indices that measure everything from transparency and corruption through to human rights through to journalistic freedom. And then compare also with economic data and demographic data. There is a high degree of correlation between all of those different indices.
And what makes that important for this discussion is that every one of those indices tend to be measuring somewhat different, but often related, things, but their methodologies are completely different.
If one were to look at simply to the Fragile States Index in isolation, I don’t think anyone would dispute that South Sudan, Somalia, Yemen, Syria, Central African Republic, and Democratic Republic of Congo are in a fragile state that justifies them being categorized as high alert. Likewise I don’t think anyone would dispute that Finland, Norway, Switzerland, and Denmark are very sustainable.
In that sense, we do apply the methodology impartially, and we really do follow what the data is telling us, but at the end of the day, the data is telling us in many ways what we would expect to hear.
Masi: How can the data be used to inform policy decisions for countries that are already in conflict?
Messner: In today’s 24/7 news cycle, there is a tendency for policy to be driven by headlines. Though that may be useful for countries such as Syria that are in the headlines every day, for a country like the Central African Republic that is highly fragile but not anywhere to be seen in American media headlines, the data is very important because it shows policymakers where those vulnerabilities are.
For more information from Deeply Talks: Fragile States and Data for Peacebuilding, listen to the full talk here.