Techniques to Speed Up your Build Pipeline for Faster Feedback.

We would like to share our experience and journey on how we brought down our Jenkins build pipeline time down from over 90 minutes to under 12 minutes. In the process, we would share specific techniques which helped and also some, which logically made sense, but actually did not help. If your team is trying to optimize their build times, then this session might give you some ideas on how to approach the problem.

Development Impact - For one of our build job, below graph shows how the number of builds in a day have increased over a period of time as the build time has reduced. Frequency of code check-in has increased; Wait time has reduced; failed test case faster to isolate and fix.

Business Impact - More builds leading to quicker feedback and faster story acceptance and less story spill over.

 
4 favorite thumb_down thumb_up 3 comments visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/structure of the Session

  • My introduction (1 min).
  • Why long running pipeline was hurting (2 min)
  • Demonstrate example of Disk IO intensive operations ( 2 min)
  • How the JUnit ant task options affect the time required to run tests (5 minutes)
  • In-memory/In-Process db (5 min)
  • SSD, Hybrid Disks, RAM Drives, tmpfs (5 min)
  • How CPU profiling gives further insights into our code (10 min)
    • CPU intensive operations – e.g. addDays.
    • Disk IO intensive operations
      • Scanning of resource bundle files.
      • Loading Spring context from tests all tests.
    • How profiling can bring down not only the time of JUnit tests, but can also have a big impact on your production code efficiency.
  • Questions ( 5 min )

Learning Outcome

  • Importance of getting fast feedback.
  • How to investigate long running tests.How to profile long running tests using Eclipse.
  • How to run Tests concurrently.
  • RAM Disks, SSD, Hybrid disks.
  • Don’t assume; validate your hypothesis.

Target Audience

Software Developers, Testers, DevOps

Requirements

Projector, Laptop, Mic, Speakers, Pointing device.

schedule Submitted 2 years ago

Comments Subscribe to Comments

comment Comment on this Proposal
  • Joel Tosi
    By Joel Tosi  ~  2 years ago
    reply Reply

    Hi Ashish,

        Very interesting submission.  I know you have marked this as intermediate - how would some groups new to CD benefit from the session?  Or would you recommend they not attend?  Would there be any pre-existing knowledge that would be needed to get the most out of your session, and if so could we call that out?

     

    Best,

    Joel

    • Joel Tosi
      By Joel Tosi  ~  2 years ago
      reply Reply

      Hi Ashish,

         Friendly reminder we are looking for some feedback on your session to help us make a decision.

      Best,

      Joel

      • Ashish Parkhi
        By Ashish Parkhi  ~  2 years ago
        reply Reply

        Hi Joel,

        I am sorry for the delay in response, was caught up with further upgrades to our CI infrastructure.

        This session, as it stands today, does not cover the CD aspect. The session does not explicitly talk about how to setup CI, though it would give a good idea of what a good CI should have. Users who are not familiar with CI, or very new to CI would get a good idea for why they should have CI in place. So I would urge them to attend this session too.

        The users who are already having a CI in place would benefit the most by this session as they could experiment with the techniques covered and implement them in their CI.

         

        Thanks,

         

        Ashish


  • Naresh Jain
    Naresh Jain
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    As more and more companies are moving to the Cloud, they want their latest, greatest software features to be available to their users as quickly as they are built. However there are several issues blocking them from moving ahead.

    One key issue is the massive amount of time it takes for someone to certify that the new feature is indeed working as expected and also to assure that the rest of the features will continuing to work. In spite of this long waiting cycle, we still cannot assure that our software will not have any issues. In fact, many times our assumptions about the user's needs or behavior might itself be wrong. But this long testing cycle only helps us validate that our assumptions works as assumed.

    How can we break out of this rut & get thin slices of our features in front of our users to validate our assumptions early?

    Most software organizations today suffer from what I call, the "Inverted Testing Pyramid" problem. They spend maximum time and effort manually checking software. Some invest in automation, but mostly building slow, complex, fragile end-to-end GUI test. Very little effort is spent on building a solid foundation of unit & acceptance tests.

    This over-investment in end-to-end tests is a slippery slope. Once you start on this path, you end up investing even more time & effort on testing which gives you diminishing returns.

    In this session Naresh Jain will explain the key misconceptions that has lead to the inverted testing pyramid approach being massively adopted, main drawbacks of this approach and how to turn your organization around to get the right testing pyramid.

  • Liked Naresh Jain
    keyboard_arrow_down

    IAmA (I Am A ... Ask Me Anything)

    Naresh Jain
    Naresh Jain
    schedule 2 years ago
    Sold Out!
    60 mins
    Keynote
    Beginner

    On Reddit, IAmA stands for "I am a" and AMA stands for "ask me anything".

    In an IAmA post, a person will post what they are, and other people will ask the original poster some questions to gain insights about the experience the person has had.
     
    Ex: I'm Jeff Patton, creator of Story Mapping, Ask Me Anything... OR I'm Diana Larsen, co-creator of the Fluency Model and co-author of Agile Retrospectives, Ask Me Anything...

    We plan to take this concept and apply it to the Agile context. We've few luminaries at the conference and we plan to do an live interview with them using this format.

  • Dhaval Dalal
    Dhaval Dalal
    Naresh Jain
    Naresh Jain
    schedule 2 years ago
    Sold Out!
    90 mins
    Workshop
    Beginner

    "To know, is good. To live, is better. To be, that is perfect." - The Mother

    During the Agile adoption, its a common complain that many team in many organizations get caught up in the ceremonies or mechanics of Agile and fail to understand/appreciate the true value and spirit of Agile. And because of this, the original intent of the Agile movement itself is lost. This is a serious issue!

    This workshop will highlight, a well-proven approach to transformation (not adoption) and show the distinct steps in this journey that an individual or a collective goes through when learning anything new. Activities, serving as examples, in the workshop, will focus to show the journey - that is, how to begin with rituals, then gradually move to practices, arriving at principles and eventually internalizing the values. Witnessing this gradual process of transformation will help participants discover for themselves their current progression. We hope this will serve as a guiding light during their Agile journey.

    Finally, we will leave the participants to ponder upon and discover for themselves their ideals in life and work as this is not only applicable to software development, but also to any discipline where humans are involved, including life itself.

  • Liked Naresh Jain
    keyboard_arrow_down

    Sell Before you Build (MVP Hacks)

    Naresh Jain
    Naresh Jain
    schedule 2 years ago
    Sold Out!
    60 mins
    Case Study
    Intermediate

    Before you write any code, make sure you have a failing test." This was a revolutionary idea, when it was first pitched in the late 90’s. Many successful entrepreneurs have been practicing a similar approach - "Before you build a product/service, make sure you have paying customers." In this talk, Naresh Jain shares his approach of finding effective MVPs to validate his Educational Product and why Agile Methods simply fail to do so. If you are interested in finding out how to maximise your validated learning for minimum investment, then this session is for you. Recently Naresh's article on this topic was published by InfoQ.

  • Liked Prasad Kunte
    keyboard_arrow_down

    Implementing Agile Engineering Practices in Legacy Codebases

    Prasad Kunte
    Prasad Kunte
    Naresh Jain
    Naresh Jain
    schedule 2 years ago
    Sold Out!
    45 mins
    Case Study
    Intermediate

    Afraid of legacy code? Don't be!!!

    Most successful product companies are confronted with the problem of legacy code.

    What is a legacy code?

    • A code which is in production for several years.
    • A super-complex, hard to understand code base, written by different set of developers. 
    • Outdated Technology stack.

    But the most hurting reality is:

    Lack of confidence in the code due to zero or poor test coverage.

    Due to this reality, developers are often scared to touch it. They have very little confidence that "their code change wouldn't break the existing application in production."

    Recently at IDeaS, we came across such situation, where we needed to enhance one of our products containing legacy code. We started looking into the code and soon figured out that it was developed in 2007, hardly ever touched (& still working in production :)). The original team, which has worked on this product, could not be traced anymore.

    As this product has expanded to attract new customers, we had to change it significantly in order to support new customer's specifications. We had to make sure that the product was backward compatible and supported the earlier specifications, while we enhance the new specification.

    One simple option was to COPY PASTE every single method which needs to be modified and use an if-statement to decide which method to call. This certainly seems like an easy method, since the chances of breaking existing code is very little. 

    Today we all know this is a BAD option!!!

    Instead, our team decided to refactor the existing code to support plug-and-play approach for different specification. But before we started refactoring code, we had to build a safety net of tests around the existing code.

    How do we put the safety net? Ideal way would be to implement the Test Pyramid first. But, that would have taken significant time to be ready with the pyramid before we start touching the legacy code. And obvious, we would have missed the business goals.

    What do we do?

    Instead of building the entire test pyramid, we decided to attack different layers of the test pyramid, one at a time. Along the way, we followed the following approach:

    1. Re-structuring the Project code-base
    2. Establishing a baseline database: After taking a dump from the production database, we cleared out surplus data from the DB and setup a seed database with automed scripts
    3. Creating/fixing the build script 
      1. Setting up an auto DB deploy tool and integrating it with build scripts
    4. Set up basic CI pipeline
    5. Write a few work-flow tests to capture the system's flow from user's point of view
      1. Find the inception point in the code from where we can exercise the code
      2. Restify the application at the inception point (one service at a time)
      3. Setup authorization for production and test environment
      4. Build minimal test-data set for different environment 
      5. Create a few work-flow tests via the inception point (Test itself should not be coupled with the underlying database or implementation level components)
    6. Write business logic acceptance test to capture various complicated business rules
    7. Test drive the new enhancement or bug fixes
    8. Every time we touch legacy code, refactor the code and improve test coverage at unit level

    This really helped us test driven the new code and implement all the layers of the test pyramid.

    If you've a similar situation, join us, as we share our experience on how to confront legacy code.

  • Sachin Natu
    Sachin Natu
    Naresh Jain
    Naresh Jain
    schedule 2 years ago
    Sold Out!
    60 mins
    Case Study
    Intermediate

    Adopting agile development practices and continuous delivery is becoming a norm in the software industry. Time to market and frequent releases have drastically reduced time available for regression testing. Inspection is considered wasteful. Faster feedback cycles during development is crucial. These have created lot of challenges for testing community, which traditionally relies on manual testing assisted by UI based test automation.

    This is an experience report of transforming testing practices across organization, which decided to embrace Agile. Today our testers are not trying to find defects, instead they collaborate with product management and developers to prevent them in the first place. In fact, during the appraisal process, the defects found by them is ignored, instead we focus on how much time they are able to dedicate to collaboration and exploratory testing. The boundaries between developers and testers have faded away and today quality is whole team's responsibility.

    We started with less than 20% of our testers with automation skills (mostly UI automation) and rest of them relying on manual testing. However, today, all our testers practice BDD. They have picked up Java & Groovy programming skills. They are able to contribute Workflow tests, Integration tests and Business Logic Acceptance Tests. Early collaboration and pairing is the norm. By the time developers are done with their tasks, all checks are already automated and hence we are able to deploy software every fortnight to production.

    Are your testers finding it hard to make this transition to an Agile mindset? This session will give you some concrete ideas based on our transition at IDeaS.  

  • Naresh Jain
    Naresh Jain
    schedule 2 years ago
    Sold Out!
    90 mins
    Tutorial
    Intermediate

    In order to achieve my goals, as a buyer of your product, I want awesome feature.

    AT: make sure your users stories don't get in the way.

    Users Stories, the tool teams use to break big ideas into small demonstrable deliverable, are easy to describe and challenging to write effectively. In this hands-on workshop you'll learn how to write great user stories that adhere to the INVEST principle. We'll learn various techniques to slice your stories using the vertical-slicing approach. We will discuss what elements should be included in the stories, what criteria you should keep in mind while slicing stories; why the size of your user story is important and how to make them smaller and efficient.

  • Liked Ashish Parkhi
    keyboard_arrow_down

    Gamifying Agile Adoption - An Experiment

    Ashish Parkhi
    Ashish Parkhi
    Naresh Jain
    Naresh Jain
    schedule 2 years ago
    Sold Out!
    45 mins
    Case Study
    Intermediate

    While having a chat with Naresh Jain, he suggested me to go through the Ted Talk – “Gaming can make a better world” by Jane McGonigal. I found the title very weird and was wondering how is that possible? After going through the talk though, I was amazed. I started wondering if I can use the gamification technique in Agile Adoption, in our Products, in Performance Management Systems, in Employee Engagement Programs?

    Dhaval Dalal introduced me to Prof. Kevin Werbach’s definition of Gamification – “The use of game elements and game design techniques in non-game contexts.

    For our 4th ShipIt Day, organized on 25th/26th Sept 2014 at IDeaS, I decided to explore the idea of using game elements and game design techniques in the context of Agile Adoption. The idea was to create a gaming system which will automatically collect data, i.e. without explicit user intervention,  from multiple sources like Jenkins, Rally and manually from individuals and offer Star’s for positive behavior and deduct Star’s otherwise.

    The aim was to help the team get continuous visual feedback on how they are doing, adopt agile practices, visualize sense of accountability, visualize sense of achievement, drive positive behavior, create healthy competition, create a culture of appreciation, help performance tracking and create transparency.

     

    Landing Page

    User Profile

     

     

     Update - 

    1. Deducting points seems to be bothering the individuals. Now we are experimenting with getting rid of negative points and introducing short lived badeges instead e.g. "Build Breaker". 
    2. We have now added more badges to recognize individual efforts in various categories.
    3. Working on open sourcing the core app at https://github.com/IDeaSCo/rockstar
  • Aslak Hellesøy
    Aslak Hellesøy
    schedule 2 years ago
    Sold Out!
    60 mins
    Keynote
    Intermediate

    As lead developer of Cucumber and author of The Cucumber Book, Aslak gets asked to consult with organisations who want to introduce Behaviour-Driven Development (BDD). Time after time, he meets teams who are trapped doing half-arsed agile. They do the easy, obvious, visible agile practices, and none of the powerful, hard-to-master, hard-to-see ones.

    When these teams ask for help learning BDD, we get a chance to remind them how important conversations and collaboration are in software development. We teach them to write tests before they write code, as a way to explore and discover the hidden details of a requirement just before they dive in and start building it. This talk will make you wince with recognition, laugh with despair, and finally inspire you with stories of teams that have finally, after years of flaccid scrumming, discovered the true collaborative heart of agile software development. You’ll see patterns you recognise from your own teams, and gain insights about how to fix them.

  • Naresh Jain
    Naresh Jain
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    Why do we see new process or methodology or movement every 10 years or so in the software industry? And why don't successful companies ride on their success forever? Naresh Jain uses the Adaptive Change Cycle (ACC) to explain the rationale behind it. Once you understand ACC, this talk will help you understand how to prepare yourself and your organisation to quickly move through the 4 stages of ACC: reorganise, exploit, conserve and release to constantly keep innovating. If you feel agile methods are stagnating and looking for what to expect next, this talk might give you some ideas.

  • Anand Murthy Raj
    Anand Murthy Raj
    schedule 2 years ago
    Sold Out!
    60 mins
    Talk
    Advanced

    There is a world of difference between the management principles in the era of the Industrial revolution to the current world of information revolution. We need to acknowledge the increasing gap between the way employees are being managed at work, and the way they want to be managed. Many surveys have been done in this area, ending in headlines like “6 out of 10 employees are miserable” and “74 percent of staff not engaged at work.” Dig into these surveys and you’ll see the quality of leadership on top of the list of complaints. The poor state of leadership and management skills in organizations is being driven by a broad range of factors, including but not limited to

    The changed nature of work - More of skill and innovation

    The increasing education of employees - Continuous improvement and learning needs

    The needs of later generations - The more sofisticated needs of the system

    The pace of change - The quick pace of changes

    In the current “rat race”, the management focusses on higher productivity and higher ROI, but they lack the capacity to “motivate” their employees that can take them to the next level of thinking. The "leaders" do not understand the subtle difference between teaching, mentoring and coaching. They use these words interchangeably without realizing that coaching teams can help them to increase the productivity, morale and create great employees resulting in more successful products.

    A new coaching model is required to enable continous improvement with quick feedback cycles that fosters adaptability and innovation.

    Two relevant quotes which I can relate to this issue are

    "It is people who build organizations and not organizations who make people" - unknown

    "Beware of little expenses; a small leak will sink a great ship" - Benjamin Franklin