Anomaly detection at CERN websites

The World Wide Web (WWW) was invented by British scientist Tim Berners-Lee in 1989, while working at CERN. The idea was to share information between scientists in universi- ties and institutes around the world. Over time, web services became increasingly important for CERN for transferring large amounts of data to servers over the world. The aim of my summer student project was to detect anomalies in some of CERN's web services. From Prometheus metrics we have some information about the current state of a web service, like response latency and error rate. We want to avoid a high error rate over a long time period. Moreover the error rate is strongly connected to the response latency of the website: as the latency explodes we might run into errors. The end goal is to predict upcoming errors or upcoming rapid changes to latency from our metrics. We do this by forecasting the error rate or response latency given the other It is also interesting to try to classify different states of the data into stable or anomalous states. In this case we have to choose some levels for the different metrics where some levels are treated as anomalous in contrast to stable. We explore different approaches to this problem including time series forecasting, t-SNE visualisation, recurrent LSTM networks and lastly Transformers.