Kafka streams testing: A deep dive

Day 4 /  / Track 2  /  EN

Even though Kafka streams "come bundled" with TopologyTestDriver, which makes streaming applications testing quite enjoyable, it's not so simple: there are limitations and certain classes of defects that this technology doesn't match. For integration tests, you can check the behavior of a real cluster but it's not simple: we're entering a slippery road of asynchronous tests with timeouts, flakiness, and unsolved questions. Conclusion: you need to use both approaches + head.

Download presentation


Ivan Ponomarev

Tech lead in KURS company, tutor at MIPT. Has more than 15 years of experience in IT. Writes code and manages projects ranging from ERP systems customization to building a real-time webscraping system. Sometimes shares his experience as a speaker at conferences and meetups.

John Roesler
John Roesler
Confluent, Inc.

John Roesler is an Apache Kafka committer and PMC member and a software engineer at Confluent. He focuses on Kafka Streams. Before that, he spent eight years at Bazaarvoice, building their massive-scale, high-throughput stream processing platform.

Invited Experts

Victor Gamov

Viktor Gamov is a Developer Advocate at Confluent, the company that makes an event streaming platform based on Apache Kafka. Back in his consultancy days, Viktor developed comprehensive expertise in building enterprise application architectures using open source technologies. He enjoys helping architects and developers to design and develop low latency, scalable and highly available distributed systems. He is a professional conference speaker on distributed systems, streaming data, JVM and DevOps topics, and is regular on events including JavaOne, Devoxx, OSCON, QCon, and others. He co-authored O'Reilly's "Enterprise Web Development". He blogs at http://gamov.io and co-hosts DevRelRad.io. Follow Viktor on Twitter @gamussa, where he posts there about gym life, food, open source, and, of course, Kafka and Confluent!