How we run Flink on Kubernetes at Telefónica NEXT
Telefónica NEXT’s Data Anonymization Platform relies on Apache Flink to process billions of events near real-time every day. The first introduction of Apache Flink into our tech stack was at the end of 2015 but a lot has changed since then. By now, our deployment method of choice is Flink on Kubernetes and this talk aims to shed light on some of the lessons we have learned on our path towards production-readiness. In particular, the audience will learn how we solve declarative job submission and about cloud-native monitoring using Prometheus as well as minor problems we had to solve or work around.