160 Countries in the NCSI: Barriers, Lessons Learnt, and Interesting Facts

27.05.2020 | Radu Antonio Serrano Iova, PhD

| The project is responsive to the following SDG of the United Nations: 17 |

How did we manage to reach 160 countries in the NCSI? This is a blog post about the barriers, lessons learnt and interesting facts of the management of the NCSI.

The NCSI is one of those projects that cannot fully succeed without the participation and collaboration of an international network of volunteers. Our country collaborators have been more than amazing, keeping us up-to-date with the newest cyber security development in their nations, pointing us in the right directions when we needed it the most, and double-checking our evidence entries so that they reflect the realities of their respective countries. Nevertheless, our team only includes more than 110 contributors from 77 countries.

Barrier 1: level of digitalization

The main barriers that we have encountered deal with the levels of digitalization/transparency and the writing systems/language of the country. The former one makes or breaks the possible entry, of a country or a specific piece of evidence, in the NCSI. The NCSI is an evidence-based index. It was designed in such a way that for each indicator that has been attributed to a country, the visitor will be able to see the proof of that indicator, and even access it. That is why we always insist on publicly available official information.

If the country does not have a digital presence on the Internet, it is not possible for us to include it in the Index. On the other hand, if the country does maintain a digital presence, our investigation can start.

“If the country does maintain

a digital presence,

our investigation can start.“

Radu Serrano

Of the 46 NCSI indicators, only 11 of them are able to accept evidence from non-public or non-country-based sources. That leaves us with 76% of the NCSI indicators that must be investigated directly from public sources. If the country does not have an extensive governmental online presence, it makes it harder for us to conduct a complete assessment of the country. However, just because we are not able to conduct a country’s assessment on a particular date, does not mean that we forget about it. We understand that countries have different levels of digital development and priorities, and because of that, we always circle back to see if any advancements have been undertaken.

The indicators of the NCSI

When there is a governmental online presence, we always start our search by identifying the Ministry, Institution or Authority in charge of ICT matters, since cyber security relates to that topic. Afterwards, it is just a measure of how transparent the entity is, regarding legislative documents and other cyber security related records. If we do not find evidence on a specific indicator, we always look deeper, changing our search parameters, filters and location. When even that search does not pan out, we attempt once more at the regional and international levels. Sometimes international organizations (e.g. the ITU, ILO, etc.) maintain legal documents (e.g. criminal codes, national strategies, etc.) of individual countries, and we can present them as evidence when their national version is not publicly accessible. Afterwards we move on to the data protection authority, CERT/CSIRT, police and armed forces websites to search for evidence on their respective indicators (related to capacities 8, 9, 11 and 12).

Even with such a methodical approach, there have been some cases where we have found information in the most unlikely places. In Capacity 6 ‘Protection of essential services’, indicator 6.1 requires a legal act that identifies operators of essential services or critical infrastructure (CI). Some countries, because of their geographical position, experience strong and devastating weather-related events. Because of these continuous crises, they have already identified their CI, in natural crisis management legislation.

Other countries have identified CI in their labour-related laws, where it has been established that CI personnel are not allowed to participate in or conduct strikes. While it might seem like there is no connection to cyber security, these identified CIs can also be threatened digitally. Without specifically thinking on cyber security, these countries have already identified their CIs and can now move forward establishing cyber security requirement and standards for CI operators.

“Without specifically thinking on cyber security,

these countries have already identified their CIs.“

Radu Serrano

Barrier 2: language and document format

We have also encountered technical barriers related to a country’s language and writing system. As explained in our data collection methodology, “All evidence material has to be in English, in order to be verifiable by NCSI experts.”

Our country contributors provide us with translations when the evidence is only available in their language. However, for those countries in which we have not yet obtained on-site volunteers, we work with translation software. It allows us to conduct country assessment and searches, but it affects our research speed.

We have also realized that governmental web sites provide much more information in their national language, than in English (if they have a language change option). Therefore, if we do not find the information in the latter, we switch the page’s language and use the software to search for the relevant evidence. There have been times when we have found what we were looking for, but some of the available evidence are scanned files of official legislative documents. These files in foreign languages are difficult to understand and even more to translate (because their contents cannot be easily selected).

Our team will try their best to figure out the document, but if it is not possible, we will attempt other sources or try to obtain a country collaborator to assist us.

Not all evidence counts

Once we investigate all 46 indicators for a specific country, we submit the found evidence to our experts, who will then check each one and either approve or refuse them. If the link for an indicator is refused, the expert will explain his reasoning behind the refusal. Then our investigative team, or the country collaborator that submitted a data set, will attempt to either find additional information to refute the refusal argument or to submit a new piece of evidence that fits the indicator better.