Banking on DevOps & Continuous Delivery
"Mess" - it all started with this word and resulting in lots of "Waste"; We group of people put together a marathon effort to do production deployment for an application that had our 2/3 of our bank staff members as users. We did this way for 10 years, we managed to do the same way irrespective of how much time, money, repetition in our process etc., 7th June 2014 was an eye opener for me and my team, the particular application deployment made us to realize how we have complicated things (inherited) and costing so many - $$$, Time, ; IMHO this is also start of our Bank's DevOps journey.
In this presentation, I would like to share as an experience report my personal findings during my journey in transforming our manual application deployment process to automated deployment. Started with hate and soon became the most loved process/tool - saving us not only time and money and also position our team internally to be superior and inspire others to work towards this.
I am BMK, from New Zealand sharing my DevOps story and our organization's journey
Outline/Structure of the Experience Report
Background and Context -(3 minutes)
About our bank, which is highly regulated like other financial institutions; In the opening I talk about the 70+ applications that we maintain/support as a team, love with operations team, change management, documentation for deployment; setting the scene of our complex environment (you can align now with your organization).
3 Hrs - 3 Days Deployment process ( 5 minutes)
In this I would like to setup the scene around as to how, what we thought, we could do in 3 hours took us 3 days for a particular application deployment and share the experience pre-during-post deployment process.
- Operations Team - Silo and very limited application knowledge; missing server and no alerts, mis-configured load-balancer, locked service accounts
- Hefty word documenting deployment steps for all the components and manual Post Implementation Verification (PIV) steps
- Development->Production Server; Configuration issues, forgotten DNS entries, Central landing page throwing "Object Reference Exception"
- Business pressure on "Go-Live" date and training plan impacting 2/3 of the bank's retail users
How did we transform - Our Journey - 10 minutes
How did we manage to Go-Live with an application that we had all sorts of troubles to start with; share our journey in recognizing the problems and importantly getting this visible to our team/people leaders, technical owners and making them believe in simplifying deployment process/possibly automating that and urging Dev-Ops to work together; Our DevOps Journey started here, but wait!!! it is not over yet; I am sharing all the pain points that we need to go thru to get there. Understand this journey still in-progress.
- Culture, Process, Governance and Red-Tape procurement process
- Socializing with various stakeholders - Service Management, Security, Architecture, Governance, Operational Support teams, Sourcing and Procurement team
- Making the team believe in "Need for DevOps" - is it threat for overtime work? Who will pay for the tool? Who owns the tool? and Who owns the process?
How did we win:
- POC - like any other POC, we wanted people to see, to believe the benefits; I had PowerPoint, and more importantly I had a working demo
- Slow and Steady - One baby step at a time; We introduced automated deployment tool, we identified the pattern, documented the process within the tool
- Believe that for some complex problems, the solutions are simple;
- Get the people on-board - show them you could integrate their Governance requirements and satisfy them
- Motivate individuals to ask right questions and understand the whole pipeline, value stream, identify the waste
Conclusion (2 minutes hopefully)
Summary of our journey, next steps
This session is about a journey which is still in progress, now it is enterprise wide; We run now our own internal DevOpsDays, We talk about PaaS, uplifting the team maturity - CI, CD;
1. Passion for DevOps - How to build interest, helping, coaching and make the team to believe what you believe-in; Make them feel they belong here - High performing IT teams
2. Realizing the "need for speed", Leadership team's buy-in; Getting them on-board; Showing them examples
3. Build community of practices: Motivate individuals to come, participate and contribute to enterprise DevOps: Example: Running DevOpsDays, CoffeeOps, Intranet Community; after all this is Community Owned - not individuals
4. Responsibility is - not just talk about "problems" but also offer solutions; But be careful you need to keep the enterprise requirements around compliance, security, regulations and governance in mind.
5. You might not be able to do all the changes by yourself, overnight; But you will be able to lead the change! believe in it!
Finally - anybody could add value to any team from any where - What I learned from attending DevOps Enterprise Summit 2016
Software Engineers, DevOps enthusiasts, practice leads, Operations engineer