Scaling Netflix's threat detection pipelines without streaming
Netflix’s “Psycho Pattern” stitched togetherSpark, Kafka, and Airflowinto a relentless micro-batch pipeline. It tracked high watermarks for near-real-time threat detection—fast enough, sharp enough. Then came the Flink switch. Lower latency? Sure. But it missed the mark. Signal quality stayed flat...