Security & Chaos Engineering: A Novel Approach to Crafting Secure and Resilient Distributed Systems
Modern systems pose a number of thorny challenges and securing the transformation from legacy monolithic systems to distributed systems demands a change in mindset and engineering toolkit. The security engineering toolkit is unfortunately out-of-style and outdated with today's approach to building, security and operating distributed systems. The speed, scale, and complex operations within microservice architectures make them tremendously difficult for humans to mentally model their behavior. If the latter is even remotely true how is it possible to adequately secure services that are not even fully comprehended by the engineering teams that built them. Security Chaos Engineering helps teams realign the actual state of operational security as well as build confidence that their security actually works the way the think it does. Chaos Engineering allows for security teams to proactively experiment on recurring incident patterns to derive new information about underlying factors that were previously unknown by reversing the postmortem and preparation phases. This is done by developing live fire exercises that can be measured, managed, and automated. It develops teams by building a learning culture around system failure to challenge engineering teams to proactively, safely discover system weakness before they disrupt business outcomes. In this session we will introduce a new concept known as Security Chaos Engineering and how it can be applied to create highly secure, performant, and resilient distributed systems.
Outline/Structure of the Talk
- Title Slide
- About the Speaker & Speaker Contact Info
- Visual Overview of Example Modern Distributed Systems - Our systems have evolved beyond human ability to mentally model their behavior.
- Review of the complexity of current software engineering models, techniques, and methods.
- Highlight the ever-widening gap between those models and security as an engineering discipline.
- Complexity vs. Simplicity
- Software only increases in Complexity it never decreases.
- Accidental vs. Essential Complexity
- Complex Adaptive Systems – Dr. David Woods
- Woods Theorem
- The difficulty in understanding our own systems
- How these concepts are impacting security
- Incident, Outages and Breaches are happening more often and getting worse.
- Teams spend too much time reacting to outages instead of building more resilient systems.
- Chaos Engineering Defined
- Chaos Engineering Origin Story – What, Why and How of Chaos Monkey and ChAP at Netflix
- Chaos Engineering is not just Netflix – 1200+ companies now are adopting chaos engineering
- Why is it so important to do Chaos Engineering?
- The Normal Condition of a Human & Systems they Build is to fail
- Bring Order to Chaos with Chaos Engineering
- Ways it is used in Security – Validate runbooks, determine control effectiveness, learn new insights into system behavior, validate architectural patterns, and proactively identify system gaps and problems in system security before they impact customers.
- How Security Chaos Engineering is different that Penetration Testing, Adversarial Testing, Red/Purple Teaming, etc.
- A Shift in Mindset: Stop looking for better answers and start asking better questions.
- What is the system actually doing?
- Has it done this before?
- Why is it behaving that way?
- What is it supposed to do next?
- How did it get into this state?
- The 1st Open Source Security Chaos Engineering Tool – ChaoSlingr
- Function review
- How experiments are constructed, deployed, and executed.
- How it works
- Where to get started
- Speaker’s Reflection: The value learned by the speaker in applying these tools and techniques at the largest healthcare company in the world.
- Review of more Example Security Chaos Experiments and some using Kubernetes
1: Learn a new technique for uncovering system weaknesses in systems security.
2: Change incident response and security engineering team thinking.
3: Identify the hidden costs of security Incidents.
4: Discover new ways to proactively expose gaps in how we think our systems security works vs. the operational reality.
5: Learn about the importance of recalibration and understanding failure in distributed systems security
6: Learn about a new open source tool that they can use to do Security Chaos experiments
7: How to apply Chaos Engineering with Security to create a DevSecOps culture
8: The business value of Chaos Engineering with Security and how to get started
software engineers, software security engineers, security engineers, technology executives
Prerequisites for Attendees
Foundational knowledge of information security practices,
Basic knowledge of distributed systems
General knowledge of build systems at large scale