Source: Amazon Web Services - Bay Area

Democratize Lineage Tracking & Monitoring

Note 1: This meetup is in the tall building (Building 20) next to our usual location.
Note 2: The tentative schedule for the AWS CD is out. https://awscommunityday.com/schedule/. Scroll to the bottom there and hit Register before the tickets run out.

We are introducing a new meetup series on Data democratization. The first meetup in this series is this one featuring speakers from Intuit and Linked In.

Details:
Democratize Data is a meetup series where we explore capabilities for making data platforms self-serve for data analysts, scientists, and everyone within the organization! Data Platforms today introduce significant accidental complexity in the journey of data users as they discover, collect, prep, build, and deploy dashboards/models. In this meetup series, we explore capabilities in the form of tools and frameworks that simplify the journey of data users and reduce the time to insights.

PROGRAM:
6:00 - 6:30 Refreshment reception
6:30 - 7:30 Superglue: Intuit’s Data Lineage Tracking framework
7:30 - 8:00 Third Eye: LinkedIn’s Business-Wide Monitoring Platform

Superglue: Intuit’s Data Lineage Tracking framework
Consider the scenario of a business metric value that appears incorrect -- can a data user today trace and analyze all the tables and jobs involved in generating the metric? Data pipelines in production are complex involving hundreds of tables, jobs, and scripts. At Intuit, we have developed SuperGlue — a tool that seamlessly tracks lineage of complex production pipelines making it self-serve for Data Users to interpret, debug, and iterate on data pipelines. Users start-off by logging into the SuperGlue portal and search for lineage on any job, table, report or model. SuperGlue is open-sourced at: https://github.com/intuit/superglue

Third Eye: LinkedIn’s Business-Wide Monitoring Platform
ThirdEye is a comprehensive platform for real-time monitoring and root cause analysis that covers a wide variety of use-cases. LinkedIn relies on ThirdEye to monitor site performance, track member growth, understand adoption of new features, flag sustained attempts to circumvent system security, and many other areas. ThirdEye provides a shared infrastructure for AI based anomaly detection and interactive data analysis of various systems. It connects to a large number of data sources to gather information and learns over time to generate more relevant detection and analysis results through user interaction. ThirdEye is open-sourced at: https://github.com/apache/incubator-pinot/tree/master/thirdeye

Newsletter
  • Get the latest DevOps jobs, events and curated articles straight to your inbox, once a week

  • Community Partners