Source: medium.com
Adobe Experience Platform’s Streaming Sources and Destinations Overview andCategory: Business, Data, Kubernetes, Infrastructure, containerization, firewall
Adobe recently announced two new Streaming Sources and Destinations as part of its real-time customer data platform offering: Azure Event Hub and AWS Kinesis. This blog will detail the use cases, how we started, what technical decisions we took, and the challenges we faced while executing this release. Adobe Experience Platform has a large Kafka infrastructure with multiple environments and data centers.
The options available from Kafka Connect were: “Failing fast to learn fast” is a great engineering principle and we can leverage some of Kafka Connect’s constructs to make sure that we fail fast if we are not able to instantiate any new connector.
You might need to spit some metrics out if you have done special handling for batching and error handling to track your latency and performance numbers as we did.
The options available from Kafka Connect were: “Failing fast to learn fast” is a great engineering principle and we can leverage some of Kafka Connect’s constructs to make sure that we fail fast if we are not able to instantiate any new connector.
You might need to spit some metrics out if you have done special handling for batching and error handling to track your latency and performance numbers as we did.
Related Articles
Community Partners
DevOps Careers