Look at GDPR from a Data Engineering Perspective
GDPR has been being discussed quite frequently in the past year, many businesses need to invest more into R&D to become GDPR compliant. This talk discusses about the concepts, challenges, analytical needs, ideal system setup and a possible roadmap for existing projects from a data engineering perspective. It is meant to be a starter of discussion around how we address the need of customer service, application debug, event sourcing and analytics within the GDPR context, rather than a step-by-step guide.
Outline/Structure of the Talk
- Data related regulations
- OLTP vs OLAP
- Different categories of data
- Ownership of data
- Filtering of information
- Challenges
- Immutable events
- Logs (local + cloud)
- Extraction of PII related meta data
- Name, Email stats
- Age stats
- Roadmap
- Green field projects
- Existing projects
Learning Outcome
- Basic understanding of GDPR and who needs to be compliant
- Different requirements for OLTP and OLAP systems and data ownership
- Common challenges brought by GDPR and some possible solutions
- Metadata management
- The roadmap to GDPR compliance
Target Audience
Data Engineers, Data Analysts, Software Developers
Prerequisites for Attendees
Basic knowledge of application design and data warehouse design
Basic knowledge of micro-services architecture
Basic knowledge of cloud computing
Links
GDPR Explained: https://www.youtube.com/watch?v=acijNEErf-c