Moving Your Pipeline to Kubernetes, Things I Wish People Had Told Me

Kubernetes married with a cloud provider elastic, highly available infrastructure. Many CI engines today (Jenkins, Bamboo, Gitlab, CircleCI), provide native integration with kubernetes so that your build and deploy workload can be elastically executed. This allows your pipeline to meet the needs of your schedule, be it the 4pm pile on to commit code before going home, the mad rush to get a hot fix to production, or the surge of an unexpected customer ask. Gone are the days of the build queue growing and you CI engine collapsing under the weight of a hundred build requests. In order for a pipeline to leverage this capacity changes must be made to the pipeline architecture. Tools must be dockerized, the ephemeral nature of running docker must be considered, kubernetes specifications or helm charts must be generated for the application, automated testing must be adapted to work in the new architecture, and then there is the database. Each one of these issues, plus many others I’ve missed contained unfortunate, unforeseen pitfalls that translated in schedule delays. Join Glenn as he helps you short circuit the pitfalls of migrating to kubernetes off of your static in-elastic virtual infrastructure.

 
 

Outline/Structure of the Case Study

The presentation will be divided into several sections:

  • Quick overview of docker
  • Quick overview of kubernetes
  • Pros and cons of containerizing your build tools.
    • Cacheing solutions for maven/gradle/et al.
  • Approaches for executing your tests inside kubernetes
  • Discussion of ephemeral test environments
  • Persistance
  • Databases
  • Environment management inside kubernetes

Learning Outcome

For this session at the end you should have an approach to solve the following issues:

  • How to approach migrating your pipeline to kubernetes
  • A taxonimy of high level design mistakes to avoid
  • A laundry list of technical mistakes to avoid
    • What to do about persistence at the file system and database level
    • When to have databases inside and outside the cluster and what are the design considerations
    • An approach to managing your static environment into kubernetes and what it means for CD
    • How to avoid letting your build tools run afoul of dockers ephemeral nature.
    • How to partition build workload for faster build times and avoid build taking longer in kubernetes
    • How to ensure deployments to kubernetes are as efficient as deployments to the current virtual infrastructure.
  • How to migrate your infrastructure as code deployments to kubernetes

Target Audience

People who are looking to move their DevOps infrastructure to kubernetes or are looking for solutions to issues in their new kubernetes DevOps implementation.

Prerequisites for Attendees

Participants should know:

  • High level information about containers (docker)
  • What the purpose of a DevOps pipeline is and the typical inputs
  • What problems kubernetes is supposed to solve for containers
schedule Submitted 5 months ago

Public Feedback

comment Suggest improvements to the Speaker
  • George Dinwiddie
    By George Dinwiddie  ~  5 months ago
    reply Reply

    I like these aspects of the submission, and they should be retained:

    • Experienced advice on the details

    I think the submission could be improved by:

    • Writing the abstract to attract the right attendees. Right now, it's very hard to read. The first sentence isn't even a sentence.
    • Appeal to those who don't yet know they're going to use Kubernetes. What will it do for them?
    • For all attendees, why should they attend this talk? You want the abstract to help the right people self-select into the session.

  • Liked Max Saperstone
    keyboard_arrow_down

    Max Saperstone - Building Confidence In Your Automated Tests

    45 Mins
    Keynote
    Beginner

    The growth of automation testing in today’s software development organizations is changing the the way we test applications. Software development practices have matured over the last 30 years, to include all forms of testing to verify software quality. In the last ten years, there has been a huge spike in the adoption of automated tests, effectively replacing some of these manual testing practices, and supplementing many traditional testing activities. Many parts of the software development industry, however, are wary of replacing manual testing with automated testing. Not only is there often a lack of confidence in the automation tests, many see automated testing as fragile, unmaintainable, and ultimately, something delivering a low return on investment. Max believes that by employing mature software development techniques, we can achieve robust, maintainable, tests, that deliver confidence of the application under test. In addition to discussing how to structure automated tests that are cleaner, more maintainable and efficient, developer testing, and deployment techniques can be used to programmatically verify test correctness. Drawing on his experiences building test automation, test frameworks and advising organizations to adopt test automation, Max will walk us through how to mature your test automation practices.

  • Liked Jeff Dalton
    keyboard_arrow_down

    Jeff Dalton - Big Agile is Coming: Will Technology Leaders Blow it?

    Jeff Dalton
    Jeff Dalton
    Chief Evangelist
    Agile CxO
    schedule 4 months ago
    Sold Out!
    45 Mins
    Keynote
    Executive

    2019 will be the year of Big Agile, where large adopters like General Motors, the Department of Defense, the State of Michigan, Lockheed Martin, and others, who have combined IT budgets exceeding 100 Billion dollars, have all announced their desire to “go agile” at a scale not yet seen in our community. Are technology leaders who cut their teeth in a low-trust, command-and-control, high-documentation environment prepared to make a successful transformation? What will Big Agile look like, and how will it affect the rest of the community?

    This will be our industry’s biggest challenge, and I've been studying it for years. As the large adopters in the federal government and corporate sector begin to adopt agile, they’ll bring their habits, culture, and bureaucracies with them, and they need to get in front of the wave.

  • Liked Gene Gotimer
    keyboard_arrow_down

    Gene Gotimer - Get to Green: How to safely refactor legacy code

    Gene Gotimer
    Gene Gotimer
    Principal Consultant
    Coveros, Inc.
    schedule 3 months ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    For many of us, legacy code is a fact of life. Code without tests -- no safe way to make changes, no safety net, no hope of untangling the web of accumulated ugliness, an incomplete understanding (or less) of how it really behaves. And your next set of changes is just going to add to the garbage pile and make it worse. You need tests so you can safely make changes, but you can't add tests without changing the code. It is a chicken-and-egg problem.

    So how do you turn legacy code into code you can change confidently? Slowly, one step at a time. Join Gene as he shares his experiences working with a monolithic codebase that was so bad it made national news. He'll go over the steps he and his team used to refactor the code safely by using mocking frameworks, mutation testing, and patience to build an understanding of how the code worked so that they could change it confidently.

    This talk is for anyone that has inherited legacy code that they aren't confident in and wants to make it something they can work on and improve. You'll leave with some tools and techniques that will help you change your legacy code into something maintainable.

  • Liked David W Kane
    keyboard_arrow_down

    David W Kane - Hang Out with the DevOps Folks!

    10 Mins
    Lightning Talk
    Intermediate

    One of the things I like about AgileDC is that I see a lot of familiar faces. Not just familiar from previous AgileDC events, but from other Agile events in town, other conferences, Meetups and such. I also go to local DevOps events, and I see familiar faces there too, but I don't observe much overlap between the two. In this talk I will discuss whether this division is real, or perhaps just a figment of my imagination, whether we as an Agile community should care, and what we should do about it.

  • Liked Craeg K Strong
    keyboard_arrow_down

    Craeg K Strong - Kanban Antipatterns: What You Don’t Know Can Hurt You

    45 Mins
    Workshop/Game
    Beginner

    In this interactive workshop we will examine multiple examples of Antipatterns observed in real-world Kanban boards. In each case we will identify the issues and discuss ways to improve the situation. We will review a number of better alternatives and see how the improvements map to the core principles of Kanban such as visualization, managing flow, and making policies explicit. Brand new to Kanban? Learning by example is a great way to get started! A long-time Kanban veteran? Come to see how many antipatterns you recognize and help firm up our Kanban Antipattern taxonomy and nomenclature!

    Kanban is an extremely versatile and effective Agile method that has seen significant growth in popularity over recent years. Kanban’s flexibility has led to widespread adoption to manage business processes in disparate contexts such as HR, loan processing, drug discovery, and insurance underwriting, in addition to Information Technology. Like snowflakes, no two Kanban boards are alike. The downside to this flexibility is there is no well-known and easily accessible library of patterns for designing effective Kanban boards. Like Apollo engineers, teams are expected to design their board starting from first principles. Unfortunately, sometimes teams get stuck with board designs that may not provide the visibility and insight into their workflow they hope to see. Worse, some designs actually may serve only to obscure the situation. Working within the limitations of an electronic board can exacerbate the problem even further. Is all hope lost? Certainly not!

    Let’s learn more about effective Kanban system design by examining what to avoid and why. Learning by example is effective and fun!

  • Liked Thomas Stiehm
    keyboard_arrow_down

    Thomas Stiehm - Shifting Security Left - The Innovation of DevSecOps

    Thomas Stiehm
    Thomas Stiehm
    CTO
    Coveros, Inc.
    schedule 5 months ago
    Sold Out!
    45 Mins
    Talk
    Beginner

    DevSecOps uses application security practices that have been around for a while. The innovation of DevSecOps is incorporating security into the daily workflow of the team rather than leaving them to the end of a release like many legacy processes do. Shifting security left is made possible by the ability to automate many aspects of security testing and verification. DevSecOps leverages DevOps practices to make application security a first-class citizen in the practices of modern software product development. DevSecOps starts with a culture change mindset of cross-functional teams creating software through collaboration and fast feedback cycles.

    The security in DevSecOps starts before the code is written by using techniques like threat modeling and risk analysis to help figure out who might want to attack you and how they might do that. This often ignored security practice can be enabled by following the DevSecOps practices of having a cross-functional team involved in the process from the beginning, including security professionals.

    Next, DevSecOps maps application security practices into the build pipeline for a project in order to provide quick feedback about the security posture for any change made to the software. By using automation to allow the team to move quickly while maintaining confidence in the health of the code base, DevSecOps extends that health check to include application security checks. While automation can be used to make security data collection easier it is important to understand what security practices still require a human being.

    This talk focuses on how, when, and where practices should be incorporated into a build pipeline to get the most value out of your security practices through automation. It explores what manual security work still needs to be done by a person and how to maximize value while minimizing the effort of human beings.

  • Liked Cherie Silas
    keyboard_arrow_down

    Cherie Silas - Forget Climbing the Ladder - Take the Escalator to Success!

    45 Mins
    Talk
    Beginner

    Ever wonder if climbing the ladder of success is really worth it? So much competition. So many people to out-perform. So many people to compete for to get the best jobs, the highest raise, the biggest bonus. And that frustrating bell curve! There's a better way. Forget the ladder! Take the Escalator instead.

    In this talk I will share my personal experience with the uselessness of climbing the ladder just to find out that you have reached the top only to be miserable and alone. What I have learned through countless experiences in my career is that ladder climbing is a futile way of progressing. There is a more powerful, effective, and satisfying way to achieve success. Take the escalator instead.

    The escalator turns what doesn't seem to make sense -- helping others succeed, helping your competitors win, giving away what others are selling - into your golden ticket to success. Join this talk to understand more about why the ladder method does not work long term and how the escalator model can bring more fulfillment, greater rewards, and financial security without the stress.

  • 45 Mins
    Workshop/Game
    Beginner

    How do you get people to agree on priorities when they may have different objectives? You may be facing an issue now where stakeholders are pushing against each other in order to get their work done first. What would happen if we could create an open dialog among stakeholders and have them understand different perspectives and focus on the goals of the greater good instead of just their own? Let’s face it, proper prioritization is the difference between writing code and developing valuable solutions.

    In this simulation style workshop, you’ll learn practical methods for bringing stakeholders together and openly discuss their different priorities to agree on what’s most important overall. You will see first hand how a combining group discussion with proven prioritization methods such as Weighted-Shortest Job First and (WSJF) and Must Have, Should Have, Could Have and Won’t Have (MoSCoW) work.

  • Liked Cherie Silas
    keyboard_arrow_down

    Cherie Silas / Chester Jackson - Coaching Change with Moving Motivators

    45 Mins
    Workshop/Game
    Beginner

    Presentation Overview: (What is the “message”? What key points will you make?)

    Change is hard – Staying motivated during change is even harder! But you can help your teams identify what things about the change are working for them and what they need to do to make small shifts that can keep them motivated.

    Is your team or team member facing a big change decision? Moving Motivators can help you identify the best choice by looking at how the factors in the change impact your long term motivation.

    Learn to discover and prioritize your motivators so you will understand how change today might impact you in the long run. See which choice is better when there are multiple options. Make conscious trade off decisions in a logical way with long term thinking in mind.

  • Liked Max Saperstone
    keyboard_arrow_down

    Max Saperstone - Getting to Continuous Testing

    45 Mins
    Case Study
    Beginner

    Max will tell the story of how a healthcare company striving to get to continuous releases built up their automation to secure confidence in regular releases. Initially, as no test automation existed, Max was able to take a greenfield test automation opportunity, and in the span of 12 months, develop over 2000 test cases. A testing pipeline was created to verify the integrity of the automated test cases, and to build docker containers for simple execution of the tests. These containers could then be simply re-used by developers and the DevOps team to verify the application. Max will walk through the feedback loop created, which allowed verification of the application go from hours to minutes.

    Max will discuss what processes and paths were taken to achieve continuous testing on this project. While he will cover the tools used and why they were chosen, the main focus will be on the HOW and WHY certain patterns and activities were performed. These choices were critical to achieving continuous testing, rather than just good testing coverage in CI or CD, even allowing a push left for performance and security. Additionally, some time will be spent on the organizational and culture changes that occured, and how he was able to accomplish this push for adoption in an organization that resisted automation, and had major quality problems.

  • Liked Richard Mills
    keyboard_arrow_down

    Richard Mills - DevOps Patterns to Enable Success with Microservices

    Richard Mills
    Richard Mills
    DevOps Solution Lead
    Coveros, Inc.
    schedule 6 months ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    DevOps can help you dig out of the problem you created for yourself: you spent your lunch period reading the interwebs, drank the kool-aid, and decided to embrace the utopia of microservices to solve all your fragile legacy monolithic code issues and allow you to release small independent changes into production. What you didn't realize is that you've translated an early-lifecycle code architecture problem into a late-lifecycle release management and quality assessment nightmare.

    This microservice thing has not provided the nirvana you expected. You ended up with:

    • a set of federated services that have hidden dependencies
    • independent applications maintained by teams that don't talk to each other
    • inability figure out which versions work together in your test environments, much less production
    • the need to test that your still-monolithic system works in pieces and as a whole

    You discover that this looks suspiciously like a DevOps problem and your pipeline is critical to your success.

    Someone once said to me "if you are building microservices without DevOps, you've already failed." I've learned that the integration problems created by independent microservices require a high level of automation with a pipeline that works independently of each service and can detect changes that break other services. The pipeline needs to facilitate communication between teams and assess which changes and versions work with each other.

    In this talk, I highlight the important things you need to succeed with microservices and avoid some of the common problems. Participants will leave with some new ideas on what they might be doing wrong in their current microservice-based project and/or anticipate what's going to go wrong if they are just getting started.

  • Liked David Fogel
    keyboard_arrow_down

    David Fogel / David Bujard - Nine levels of Agile Hell... and how to get out!

    45 Mins
    Talk
    Beginner

    Government Agile transformations can feel like overwhelming efforts – but do not abandon hope! This interactive, audience-driven presentation reviews how government and large organizations ESCAPE common Agile adoption challenges.
    You - the audience - will prioritize your pain points; we’ll focus on the five Agile hells most highly prioritized. We will discuss real examples of “escaping” out of each Agile hell, with pro tips and success patterns you can apply.
    The Agile hells we've escaped include:

    • No Transformation hell - A federal program or department wants to change but can’t start or can’t finish
    • Too Fast hell - Newly Agile federal programs sometimes respond TOO rapidly, too often changing priorities.
    • Technical hell - Programs can become bogged down in technical debt and manual processes.
    • No Trust hell - Government delivery can be slowed by lack of trust between contractors and feds, between business and IT, or between compliance and delivery groups.
    • Product Owners hell - Government Product Owners can be unavailable, think they are managers, aren’t empowered to provide vision, or struggle with prioritization
    • Too Big hell- A frequent pattern in federal Agile! Large batches produce slow progress, low visibility and high complexity, seen in big programs, big deployments, and big contracts.
    • Collaboration hell - Government teams can struggle with collaboration within the same organization across roles and across the fed-contractor divide.
    • Stove-piped hell - Government organizations can struggle to collaborate across contractual or organizational boundaries within the same enterprise
    • Leadership hell - An organization can only be as agile as its leadership. In the government, how can you work with leaders who aren't ready to be agile?

    For each Agile hell, we focus on successful techniques to escape from these common dynamics. Unlike other presentations, we won't be doing a deep dive, but we will cover the most important challenges our audience face.

  • Liked Ben Morris
    keyboard_arrow_down

    Ben Morris - Agile the Hard Way - Lessons from a Government Project

    Ben Morris
    Ben Morris
    Consultant
    STSI
    schedule 4 months ago
    Sold Out!
    45 Mins
    Case Study
    Intermediate

    "Agile OSS" jet on the ground, with an arrow leading to "the cloud"

    The talk walks through lessons learned from a specific government project, jumping head-first into agile, open source, and (oh dear) the cloud. In the jet plane metaphor, it's not about the theory of why a plane should fly (as a physicist), but about the big and little forces that will act on it in practice (as a test pilot).

  • Liked Dave Witkin
    keyboard_arrow_down

    Dave Witkin - Go Big Without Blowing It: How Does Scrum@Scale Help?

    Dave Witkin
    Dave Witkin
    Principal
    Packaged Agile
    schedule 3 months ago
    Sold Out!
    45 Mins
    Talk
    Beginner

    Do you work on programs requiring collaboration of 30 people or more? Have you heard of Scrum@Scale? While many people have heard of SAFe, far fewer are familiar with Scrum@Scale. Did you know Scrum@Scale was built by Jeff Sutherland, the co-creator of Scrum? And that the very first scaled Scrum project began in 1983?

    Come learn about Scrum@Scale, where it is being used, and why it may be a better fit for your organization than other scaling methodologies. I am both a SAFe and Scrum@Scale practitioner and trainer, and will provide a balanced comparison with pros / cons.

  • Liked Thomas Stiehm
    keyboard_arrow_down

    Thomas Stiehm - Continuous Build and other DevOps anti-patterns, and how to overcome them

    Thomas Stiehm
    Thomas Stiehm
    CTO
    Coveros, Inc.
    schedule 5 months ago
    Sold Out!
    45 Mins
    Talk
    Beginner

    Continuous Build is an anti-pattern that I have often seen where a team will have what they call Continuous Integration (CI) in place but it only builds the code, there are no tests or static analysis run. Certainly, this is better not building but it leaves a lot of health check information on the table that is considered part of CI. Without this information, you can never really gain the confidence that your build is healthy. The whole goal of CI is to feel that your build is healthy so not tests and analysis means you aren’t going CI.

    Just like CI, other DevOps practices can be hard to understand, implement, and get right. Even with the best of intentions, we make mistakes or misinterpret the implementation of a technique. Learn how to spot common DevOps anti-patterns and how to correct them. These patterns include

    1. Continuous Build - CI without tests isn’t CI
    2. Turn the unit tests off to build the release
    3. Don’t automate that, it is my job
    4. Different build process for developers and high environments
    5. Different deployment process for developers, test environments and/or production
    6. Not having a production-like environment to test in before production
    7. Saving performance testing for the end of the release
    8. Saving security testing for the end of the release
    9. Never asking the users about the software
    10. Only automating build and deployment, not testing
    11. Not having retrospective
    12. Restricting retrospectives to only the development part of the process
    13. Running analysis and never looking at or acting on the findings
    14. Reduce coverage or static analysis gates to get a build to pass

    We have all experienced a time where we wanted to believe we could make an anti-pattern work but it never does. It is better to learn how to spot these and how to correct them than it is to try to keep tweaking a broken process hoping this time it will be better.

  • Liked Thomas Stiehm
    keyboard_arrow_down

    Thomas Stiehm - Nobody Cares about Security and What DevSecOps is doing about it

    Thomas Stiehm
    Thomas Stiehm
    CTO
    Coveros, Inc.
    schedule 5 months ago
    Sold Out!
    45 Mins
    Talk
    Beginner

    Application security is the poster child for third-class citizens in the software development world (behind Quality Assurance). DevSecOps is trying to turn that around and get more people, teams, and companies to care about security as our online and real-world lives become more intertwined.

    Application security has a bad reputation with many people. It comes into the development process late and demands a lot. Who wants to deal with that? We have actual business value to get out the door. “Those things” won’t happen anyway. And when “those things” do happen, it will just become an exercise in finger-pointing and blame. Security is an ugly affair that no one wants a part of.

    DevSecOps is a movement within the DevOps and Security worlds to reverse this decades-long drama by getting the people creating and updating software to build security practices into their process from the beginning, even before the code is written. This allows security professionals to become the evangelist of security practices where they can help the teams adopt practices and teach them how to use the tools to resolve issues themselves. No longer dependent on the specialists the teams can address security findings as they are found and make the workload manageable by spreading it across their implementation cycles-- proactively, not reactively.

    Attendees will leave with an understanding of how to map security concepts onto a delivery pipeline, how to “sell” security concepts to stakeholders, and how automation makes it easier to gather security data and act upon it. Learn what is needed to get started with DevSecOps so that you can start creating secure software today.

  • Liked Gene Gotimer
    keyboard_arrow_down

    Gene Gotimer - DevOps for Leadership

    Gene Gotimer
    Gene Gotimer
    Principal Consultant
    Coveros, Inc.
    schedule 3 months ago
    Sold Out!
    45 Mins
    Case Study
    Executive

    Organizations and leaders are often supportive of DevOps, but they don't always understand what DevOps is and what it will change. It isn't a one-size-fits-all issue; different environments need different benefits from a DevOps transformation. Join Gene Gotimer as he shows how to determine what parts of DevOps your organization needs to concentrate on first and how you should define success. This isn’t all strategic, high-level thinking- there is plenty of tactical, “what is the next step?” planning on how to shape your delivery pipeline to work for your situation and organization.

  • Liked Katrina L. C. Tanner
    keyboard_arrow_down

    Katrina L. C. Tanner - Playdoh...play on!

    Katrina L. C. Tanner
    Katrina L. C. Tanner
    Scrum Master
    Cvent
    schedule 4 months ago
    Sold Out!
    10 Mins
    Lightning Talk
    Beginner

    What happens when 25 product owners, developers, and quality engineers reflect on their previous sprint through the eyes of a child?

    Playdoh took my teams, remote and collocated, from showing up to an obligatory meeting on their calendar to intentional interpersonal and intrapersonal dialogue.

    Between the carpet and the sticky fingers, adults are usually cleaning up more Playdoh than playing with it themselves. Even to my own surprise, in this “Play Retro,” our team members connected, not only on a professional, but also on a personal level in ways they had not done before.

    This session will give an overview of our experience using a “Play Retro” and leave you with tips and things to consider planning a retrospective.

  • Liked Gene Gotimer
    keyboard_arrow_down

    Gene Gotimer - A practical approach to building security in

    Gene Gotimer
    Gene Gotimer
    Principal Consultant
    Coveros, Inc.
    schedule 3 months ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    The release date is a week away. Development is complete. The code works, and everything looks good. Marketing is ready with the media blitz. Our customers are waiting to get their hands on the new features and are sure to give us good feedback. The only step left is to get the security group to scan the application and give us the approval to release. Cross your fingers- let’s hope we get the green light! Otherwise, I don’t know what we are going to do.

    DevOps, and more importantly, DevSecOps, promises to do away with rolling the dice at the end and hoping we are allowed to release what we built. But how do we get to DevSecOps when we have a separate security sign-off, governance, regulations, and even corporate policies that say security gets the final word? How do we get from a classic security model to a DevOps-friendly process?

    Join Gene as he discusses practical steps for adding security to your existing development process. Attendees will learn tools and types of testing they can introduce to build security in from the beginning, eliminating the late surprises that the security team might find right before release.

  • Liked Claire Atwell
    keyboard_arrow_down

    Claire Atwell / Kerri Sutey - Lost in Transformation? Using the Integral Agile Model to Find Your Way out of the Maze

    45 Mins
    Workshop/Game
    Intermediate

    You think you have your transformation figured out, but then your efforts to address one issue, you create unintended consequences that become the next hurdle. The only way to find your way out of the transformation maze is to understand your complex ecosystem and plan for change. In this workshop, we will explore how leveraging both the Integral Agile Model and a framework, such as SAFe can lead to executing successful transformation strategies. Join us as we discuss how to pair the Integral Agile Model with your framework to create a holistic view of your entire eco-system of leaders, vendors, compliance needs, and policy. We will also explore how to bring organizational goals to life by creating a priorities-based roadmap. If you're feeling lost in transformation, this workshop is for you!