Carlos Rodrigues

In 2017, an average of 200,000 new malware samples have been captured, each day. This value increased by 328%. Cybercriminals have stepped up their game, and they already use advanced techniques to penetrate organisation defences and steal critical data causing millions in losses. Organisations' cyber defence departments have to reinvent their defence mechanisms to keep up with the new threats whereas regular SIEM are not sufficient. Adding more bodies to defensive efforts no longer improves defence due to a diminished returns from increased human labour and manual defensive tactics. This evolving landscape of threats demands innovation. A plethora of new defensive tactics are mandatory to advance a defensive posture in a challenging and impactful cyber warzone. It is time to bring AI to the fight. During this talk we will explain the journey to expand and enhance our current defences. We have designed and implemented a next-generation data analytics platform which leverages the current SIEM by extending its limited storage with a serverless data lake and by strength the alarm system with AI capabilities. On this talk, we will explore the solution high-level architecture and how we leverage AWS serverless components to build a lake that collects per day 6 TB of new log data from 3 continents with a rate of 60000 events per second. Hence, decoupling storage (S3) from processing (Spark) allows to scale each component independently. With this architecture, security analysts can perform their forensic analysis over years of historical data without the system compromises query performance or availability. On the other hand, AWS SageMaker cloud instance armed with GPUs empower our Data Scientists to train models on years of historical data but taking only a few training hours. As outcome, models deployed to production not only reduced the vast amount of false positives compared to the rule-based SIEM alerting system but also managed to identify new threats.