By closely monitoring a system’s data point clouds, scientists can identify the system’s normal state. © DR Scientists, together with local startup L2F, have developed a robust model that can predict when a systemic shift is about to occur, based on methods from a branch of mathematics called topological data analysis.
Topological data analysis (TDA) involves extracting information from clouds of data points and using the information to classify data, recognize patterns or predict trends, for example. A team of scientists from EPFL’s Laboratory for Topology and Neuroscience, L2F (an EPFL spin-off), and HEIG-VD, working on a project funded in part by an Innosuisse grant, used TDA to develop a model that can predict when a system is about to undergo a major shift. Their model, called giotto-tda , is available as an open-source library and can help analysts identify when events like a stock-market crash, earthquake, traffic jam, coup d’etat or train-engine malfunction are about to occur.
Catastrophes and other unexpected events are by definition aberrations - that’s what makes them hard to predict with conventional models. The research team therefore drew on methods from TDA to come up with a novel approach based on the fact that when a system reaches a critical state, such as when water is about to solidify into ice, the data points representing the system begin to form shapes that change its overall structure. By closely monitoring a system’s data point clouds, scientists can identify the system’s normal state and, thus, when an abrupt change is imminent. Another benefit of TDA is that it’s resilient to noise, meaning the signals don’t get distorted by irrelevant information.
Until now, TDA has been used mainly for datasets with a clear topological structure, such as in medical imaging, fluid mechanics, materials science and 3D modeling (e.g., in molecular chemistry and cellular biology). But with giotto-tda, the method can be used to model just about any kind of data set (such as gravitational waves), and the data contained in these sets feed the model’s machine-learning algorithm, improving the accuracy of its predictions and providing warning signs.
Noise and muddled signals
The scientists tested giotto-tda on the stock-market crashes in 2000 and 2008. They looked at daily price data from the S&P 500 - an index commonly used to benchmark the state of the financial market - from 1980 to the present day and compared them with the forecasts generated by their model. The price-based graph showed numerous peaks that exceeded the warning level in the run-up to the two crashes. "Conventional forecasting models contain so much noise and give so many signals that something is about to go awry, that you don’t really know which signals to follow," says Matteo Caorsi, head of the project team at L2F. "If you listen to them all you’ll end up never investing, because there are very few times when the signals are truly clear."
But the signals were very clear with giotto-tda, as the peaks indicating the upcoming crashes were well above the warning level. That means TDA is a more robust method for making sense of volatile movements that may indicate a crash is looming. However, the scientists’ findings concern only one specific market and cover a short period of time, so the team plans to conduct further research with the help of another Innosuisse grant. "The next step will be to apply TDA to deep-learning methods. That will give us valuable information about our model, how interpretable its results are and how robust it is," says Caorsi.
References" giotto-tda: A Topological Data Analysis Toolkit for Machine Learning and Data Exploration ," Guillaume Tauzin, Umberto Lupo, Lewis Tunstall, Julian Burella Pérez, Matteo Caorsi, Anibal M. Medina-Mardones, Alberto Dassatti and Kathryn Hess. Journal of Machine Learning Research, 22(39):1-6, 2021.
