Official statistics can benefit from new analysis methods arising from its data innovation strategy. The automation of certain tasks, especially data preparation, seems to be particularly promising. Although the first results from the FSO’s pilot projects are encouraging, according to an initial assessment published on the "experimental statistics" microsite, they still need fine-tuning before they can be fully implemented in ongoing production.
On 21 November 2017, the Federal Statistical Office (FSO) published its data innovation strategy. Its main objective was to assess the application of complementary analysis methods from data science to official statistics. Five pilot projects were run, using machine learning, deep learning and small area estimation, enabling improvements in statistical production.
Official statistics has a vast experience and proven expertise dating back decades in the collection, processing, application and dissemination of data and statistical information and it has been able to develop this know-how constantly. Aware of the new challenges arising from the digital transformation, the FSO adopted a strategy for this sector in 2017 in order to tackle the new issues facing our society.
Although they comply with the code of ethics described in the Charter of Swiss Official Statistics, the initial solutions put forward by the FSO in this context are still experimental and provisional in nature. They demonstrate the potential, but also the limits of using complementary analysis methods normally used in data science when applied to official statistics.
Pilot projects for real needs
The topics chosen for the five pilot projects are concrete ones and correspond to the FSO’s real needs. They come from the statistical fields of education, business, land use and social assistance. The data used by the pilot projects are data that are already available at the FSO and no new external data sources have been used.
The provisional results from these pilot projects are available on line in the form of reports on the FSO’s "experimental statistics" website. They are also referenced at international level on the experimental statistics website of the European Union (Eurostat), which also provides references to other countries that are developing experimental statistics.
At international level, these pilot projects have been presented at various European and UN conferences. The five pilot projects feature on the programme of the Swiss Days of Official Statistics 2019.
A laboratory dedicated to data innovation
The FSO has set up a temporary laboratory dedicated to data innovation in order to bring these projects to completion. This cloud platform has been provided and is run by the Federal Office of Information Technology, Systems and Telecommunication (FOITT). In accordance with computer security and data protection directives, this laboratory is separated completely from ongoing statistical production and has absolutely no internet connection. The work carried out has enabled the FSO to gain a wealth of experience in the governance and operational management of this type of platform. Preference was given to open source software.
Data science methods
Supported by external experts from the academic and private sectors as well as by in-house specialists at the FSO, the five pilot project teams first received specialist training in data science methods and in programming. They were then able to put into practice the theoretical knowledge they had acquired. Organised into multidisciplinary teams made up of business experts, IT specialists and methodologists, these teams were able to evolve within an agile structure.
A special logo
The first results show that official statistics can benefit from complementary analysis methods used for new applications and that these methods provide rewarding alternatives when compared with the use of traditional methods in other applications. The automation of certain tasks, such as data preparation, the interpretation of aerial and spatial images or coding operations are areas which appear to be particularly promising for efficiency gains.
However, although these initial results are encouraging, they will require some fine-tuning in order to reach their full potential. This will be necessary before they can be used in ongoing production. For this reason, the FSO intends to continue its work in this area. The first results are, therefore, clearly marked as such and carry a logo that can be easily recognised.