The key objectives of Organizations is to provide / derive value from the products / services they offer. To achieve this, they need to be able to deliver their offerings in the quickest time possible, and of good quality!
In such a fast moving environment, CI (Continuous Integration) and CD (Continuous Delivery) are now a necessity and not a luxury!

There are various practices that Organizations and Enterprises need to implement to enable CD. Testing (automation) is one of the important practices that needs to be setup correctly for CD to be successful.

Testing in Organizations on the CD journey is tricky and requires a lot of discipline, rigor and hard work. In Enterprises, the Testing complexity and challenges increase exponentially.

In this session, I am sharing my vision of the Test Strategy required to make successful the journey of an Enterprise on the path of implementing CD.

 
3 favorite thumb_down thumb_up 0 comments visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/structure of the Session

The session will be organized as follows:

  • What does CD really mean?
  • Describe the case-study of an Enterprise on the path to implement CD
  • Importance of the Test Pyramid
  • What are the different types of testing required? (Starting from Unit, and all the way up to UI, NFRs, batches, migrations, etc.)
  • What are the different types of Environments needed?
  • Test data challenges and requirements
  • Importance of selecting the right tools and technology for Testing
  • How does Manual testing fit in this?
  • Ways to show real-time assessment of state of the product-under-test in order to take meaningful decisions for next deployment stage

Learning Outcome

We will learn the following:

  • What does CD really mean?
  • Testing is a Team Responsibility
  • How does Testing fit into CD?
  • A sample Test Vision and Strategy required for Enterprises on a journey to implement CD

Target Audience

Managers, Leads, Hand-on team members (devs / qas / product owners)

schedule Submitted 3 years ago

Comments Subscribe to Comments

comment Comment on this Proposal

  • Anand Bagmar
    Anand Bagmar
    Director - Quality
    Vuclip Inc.
    schedule 3 years ago
    Sold Out!
    45 mins
    Demonstration
    Intermediate

    The key objectives of organizations is to provide / derive value from the products / services they offer. To achieve this, they need to be able to deliver their offerings in the quickest time possible, and of good quality!
    In order for these organizations to to understand the quality / health of their products at a quick glance, typically a team of people scramble to collate and collect the information manually needed to get a sense of quality about the products they support. All this is done manually.


    So in the fast moving environment, where CI (Continuous Integration) and CD (Continuous Delivery) are now a necessity and not a luxury, how can teams take decisions if the product is ready to be deployed to the next environment or not?

    Test Automation across all layers of the Test Pyramid is one of the first building blocks to ensure the team gets quick feedback into the health of the product-under-test.

    The next set of questions are:
        •    How can you collate this information in a meaningful fashion to determine - yes, my code is ready to be promoted from one environment to the next?
        •    How can you know if the product is ready to go 'live'?
        •    What is the health of you product portfolio at any point in time?
        •    Can you identify patterns and do quick analysis of the test results to help in root-cause-analysis for issues that have happened over a period of time in making better decisions to better the quality of your product(s)?

    The current set of tools are limited and fail to give the holistic picture of quality and health, across the life-cycle of the products.
    The solution - TTA - Test Trend Analyzer

    TTA is an open source product that becomes the source of information to give you real-time and visual insights into the health of the product portfolio using the Test Automation results, in form of Trends, Comparative Analysis, Failure Analysis and Functional Performance Benchmarking. This allows teams to take decisions on the product deployment to the next level using actual data points, instead of 'gut-feel' based decisions.

    There are 2 sets of audience who will benefit from TTA:
    1. Management - who want to know in real time what is the latest state of test execution trends across their product portfolios / projects. Also, they can use the data represented in the trend analysis views to make more informed decisions on which products / projects they need to focus more or less. Views like Test Pyramid View, Comparative Analysis help looking at results over a period of time, and using that as a data point to identify trends.

    2. Team Members (developers / testers) - who want to do quick test failure analysis to get to the root cause analysis as quickly as possible. Some of the views - like Compare Runs, Failure Analysis, Test Execution Trend help the team on a day-to-day basis.

    NOTE: TTA does not claim to give answers to the potential problems. It gives a visual representation of test execution results in different formats which allow team members / management to have more focussed conversations based on data points.

  • Liked Sachin Natu
    keyboard_arrow_down

    Sachin Natu / Naresh Jain - Inverting Test Pyramid - A First Hand Experience Report

    45 mins
    Experience Report
    Intermediate

    Test automation is extremely crucial in adoption of an agile delivery. However, it can take one for a ride, if the approach is not correct. In this sensational, heart throbbing, experience report, we'll share our story of how we turned around an inefficient, expensive automation style to lean, efficient style. In addition to sharing a real-world example, we'll also share some of the key challenges we faced and how we solved them. If you are convinced about the Testing Pyramid, but are struggling to invert it, then this session is for you.

    Business Impact:

      Earlier Defect Detection - Higher test coverage at Unit/Intermediate layers lead to earlier defect detection. Reduced number of issues found on higher test environments/Production. Reduced cost of defect fixing.

      Reduced maintenance cost - UI tests are fragile and costlier to maintain Vs backend tests. No of changes in services layer are comparatively less.

      Reduced test execution time - Backend tests are much faster. Almost 7-10  times faster than UI Tests - improved build certification time.

      Test feedbacks are naturally distributed across layers of application. Test feedbacks are more pin pointed/ granular.

  • Liked Ashish Parkhi
    keyboard_arrow_down

    Ashish Parkhi / Naresh Jain - Techniques to Speed Up your Build Pipeline for Faster Feedback.

    45 mins
    Experience Report
    Intermediate

    I would like to share my experience and journey on how we brought down our Jenkins build pipeline time down from over 90 minutes to under 12 minutes. In the process, I would share specific techniques which helped and also some, which logically made sense, but actually did not help. If your team is trying to optimize their build times, then this session might give you some ideas on how to approach the problem.

    Development Impact - For one of our build job, below graph shows how the number of builds in a day have increased over a period of time as the build time has reduced. Frequency of code check-in has increased; Wait time has reduced; failed test case faster to isolate and fix.

    Business Impact - More builds leading to quicker feedback and faster story acceptance and less story spill over.

  • Liked Sunil Mundra
    keyboard_arrow_down

    Sunil Mundra - Getting A Partner To Adopt Agile

    Sunil Mundra
    Sunil Mundra
    Principal Consultant
    ThoughtWorks
    schedule 3 years ago
    Sold Out!
    20 mins
    Case Study
    Intermediate

    Due to the business benefits which accrue from Agile, clients are demanding their IT Departments/Partners to adopt Agile. It is quite common to find a situation where the client has adopted Agile, but its Partner/Vendor has not.


    This talk is based on my consulting engagement with a client who had adopted Agile and their partner had not, and the client wanted the partner to Adopt Agile.


    The talk will cover the critical challenges encountered in getting the partner to adopt Agile, especially given the wide difference in cultures of both organizations and also the organizations being located in different continents. The talk will also cover the key learnings from this journey.

  • Liked Shirish Padalkar
    keyboard_arrow_down

    Shirish Padalkar - Application Security - The Agile Way

    Shirish Padalkar
    Shirish Padalkar
    Senior Consultant
    ThoughtWorks
    schedule 3 years ago
    Sold Out!
    45 mins
    Talk
    Beginner

    Traditionally application security has involved upfront design and a big bang penetration test after development. This leads to the phenomenon of “bolt-on” security that translates into increased cost and complexity.

    Drawing on our experience on real-world projects, we show how security can be baked-in on an agile project. Using case studies we demonstrate how security concerns are captured during project inceptions, how developers write secure code, security testing is automated and how configuration management can help achieve secure deployments. This talk introduces several new concepts like secure by design, secure design patterns and lightweight code reviews.

  • Liked Anand Bagmar
    keyboard_arrow_down

    Anand Bagmar - The Agile “Chalta-Hai (It’s OK)” Manifesto

    Anand Bagmar
    Anand Bagmar
    Director - Quality
    Vuclip Inc.
    schedule 3 years ago
    Sold Out!
    45 mins
    Experience Report
    Intermediate

    The Agile Manifesto was formulated by 17 people in 2001. We know the principles of the Agile Manifesto … but do we really understand it?

    Depending on the organisation culture, the team culture and various other factors, they reach varying levels of Agile adoption. Martin Fowler talks about the levels of adoption and the path to get better via his post on “Your Path through Agile Fluency”.

    Not surprisingly, not all Agile project implementations are successful.

    This session is going to take you through a journey to talk about some of the Myths of Agile and also behaviors that inhibit organisations and teams to reach great(er) heights in Agile Fluency to achieve Agile’s benefits.  As a result, the Agile Manifesto has remained on paper, but teams have come up with their own ‘workarounds’ - which are not truly solutions to solve a complex problem well.

    We accept it because of our “chalta-hai (it’s ok)" attitude. At the end, what are we then left with? The Agile “Chalta-Hai (It's OK)” Manifesto.

  • Liked Yilmaz Guleryuz
    keyboard_arrow_down

    Yilmaz Guleryuz - Fibonacci Pairing: Learning through Collaboration

    Yilmaz Guleryuz
    Yilmaz Guleryuz
    incognito
    WareNinja
    schedule 3 years ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    According to many researches, we learn the best when we experience it personally, and we comprehend when we teach to someone else.

    So if you want to achieve ample understanding of a topic, you’ll need to follow these basic steps:

    "Listen -> Observe -> Do -> Share"

     

    Pair-programming is a well-known practice, and it has been introduced by one of the thought leaders of Agile world, Kent Beck. I respect any technique that encourages collaboration and interaction among teams, and gets teams to communicate better.

    However, if you want to establish understanding of mastery or a skill-set, and increase the shared knowledge across your team members, then you need to re-think & apply new techniques.

    In this talk, I’ll share about What-Why-How of "Fibonacci Pairing" technique, which is yet another discovery from practice.

    The main goal of that technique is to establish understanding and improve collaboration, without overcrowding by artificial ways like lecturing.

    Fibonacci Pairing is simple yet very effective technique that can improve team work. This technique should be regarded as a complementary way to improve team culture effectively.

    We will elaborate with practical examples, real life experiences, and applicable ways onto different contexts.

  • Liked Frans Overbeek
    keyboard_arrow_down

    Frans Overbeek - Spotless Clean Frontend Development with BDD, DSL and Storyboards

    Frans Overbeek
    Frans Overbeek
    Architect
    Capgemini
    schedule 3 years ago
    Sold Out!
    45 mins
    Demonstration
    Beginner

    After 3 years of improving our frontend development process we ended up with mixing BDD, DSL and Storyboards for developing spotless clean frontend code. Each approach is already powerful on his own but combining these will make them unbeatable. The characteristics are collaborate, communicate and visualize, BDD is the collaborating process between disciplines, DSL for enabling the communication and Storyboards for visualization.

    Let me show you how this works by live coding a frontend application using this mixture of approaches. The process is supported by a homemade tool based on CasperJS and PhantomCSS. And all scripts are written in JavaScript. At the end of the session we can discuss if the created code is spotless clean.

  • Liked Unnat Gupta
    keyboard_arrow_down

    Unnat Gupta / Shree Damani - Calculating RoI on Agile Enablement

    45 mins
    Talk
    Advanced

    "We want to be Agile!!...

    Why?

    Because its cool, and its becoming a norm, it will help us to cope with changing requirements, help us deliver faster etc etc."

    Isn't this a common sentiment in organizations struggling with the ever changing user/customer taste?

     

    With Agile going main-stream with most organizations looking to have at least a few business critical projects run in an Agile way, the question of ROI comes up. Shifting from a traditional way of building software to an Agile way, requires change and as any good business leader would know; change is not free. Business leaders would like to understand and justify the return on Investment to make this shift. In our talk, we will be talking about how to look at the Agile process holistically and how this process affects budgeting and how early value realization can help offset the cost of change. We will also discuss stories of other in house IT shops and product houses who have made this shift and the journey they have undertaken

    From our experience of working with such organizations, we have found that for these process-focused Agile adopters, much of their measurements include:

     

    - how long is our stand-up?

    - how long is our build?

    - how many stories do we have?

    - how many points can we fit into a sprint? etc.

     

    From their perspective, they already have plenty of metrics. Often it's also the case that they're getting benefit, just because common sense does kick in behind the scenes, and because they're delivering more frequently as a result in the reduction of documentation, so they don't always run out of money either. That leads to bad habits, possibly, rewarding wrong practices. In this talk we want to discuss metrics we have used on the projects and have found useful. Metrics like: Cycle Time, Time to market (also called Lead Time), Collaboration, Quality (in terms of code complexity , code coverage, test pyramid) and bus factor. One thing to note is that any of these metrics alone would not provide holistic way of measuring benefit, and hence a combination of them is required.

     

  • Liked Unnat Gupta
    keyboard_arrow_down

    Unnat Gupta / Shree Damani - Prioritization Techniques: Lets move beyond MoSCoW!!

    90 mins
    Workshop
    Intermediate
    • Have you been in a situation where everything gets prioritized as MUST HAVE?
    • Have you been in a situation where you have find it difficult to get different stake holders to agree on relative priority of different features?
    • Most of the time is spent in discussiing low value features?
    • Whoever screams the loudest, gets their pet features prioritixed high?
    • Do you want to learn some more prioritization games/techniques that can be used to start prioritizing at Feature level and subsequently refine it to story level?
    • You feel the current technique(s) you use for prioritization are time consuming and ineffective?

    If answer to any of the above questions is yes, this is the workshop for you to attend

    Why Prioritization?

    Customers are never thrilled to find out they can’t get all the features they want in release 1.0 of a new software product. In reality, customer expectations are high, timelines are short, and resources are limited. Any project with resource limitations has to establish the relative priorities of the requested features, use cases, or functional requirements. Prioritization helps the project manager resolve conflicts, plan for staged deliveries, and make the necessary trade-off decisions. Thus, requirement prioritization is used in Software development for determining which requirements of the software product/application should be included in a certain release. Requirements are also prioritized to minimize risk during development so that the most important or high risk requirements are implemented first.

    Several methods for assessing a prioritization of software requirements exist. In this workshop we are going show some of techniques/games we have used for feature prioritization.

     

  • Liked Unnat Gupta
    keyboard_arrow_down

    Unnat Gupta - Agile Business Analysis Anti-Patterns

    45 mins
    Talk
    Intermediate

    In this talk we will pick up various (5-7) business analysis anti-patterns, specially for Agile projects, that either we ourself have practiced at some point of our BA carrier or have seen other BA's doing. We will talk about the symptoms which act as a sign of presence of these anti patterns, why are the problems associated with them and what are the ways to get rid of them.

    These anti-patterns may range from behavior with customers to behavior with the own team.

    Some of the anti-patterns we are planning to discuss:

    1) BA aka The Order takers
    2) Task (UI/backend) based stories
    3) Engrossed in too much detail to miss the view of bigger picture - story verus feature 
    4) As a “user”...., where the user is either "system" or "product owner"
    5) Leave the NFRs / CFRs to the Developers/Tech lead
    6) Detail the hell out of stories
    7) Focus on Happy Paths only
    8) Focus on building a software over solving the real problem
    9) Resist change in requirements

     

     

  • Liked Pankaj Kanchankar
    keyboard_arrow_down

    Pankaj Kanchankar - Line Managers - an Endangered Species in Agile

    Pankaj Kanchankar
    Pankaj Kanchankar
    Agile Coach
    ThoughtWorks
    schedule 3 years ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    The matrix organization of yore relied on maximizing returns on each skillset. This lead to having line managers and practice horizontals.
    Engineering managers looking after developers and practice managers looking after the respective practices of BA, QA and PMO. This lead to having multiple lines of reporting for team member whilst on the project.
    In Agile teams, focus is on the self organising teams of empowered employees working towards common success criteria (project success is team success). Not everyone can be a PO or a Scrum Master. So is the role of so called line managers or practice managers become redundant?
    Whats their role in the agile teams?
    How their role needs to transform

    In this talk I will be addressing these questions. Bring out how some of their responsibilities are now taken up by the team or Product Owner or Scrum Master. I will also be suggesting how line managers can take this as an opportunity to morph into more meaningful roles that help the organization and teams. 


  • Liked Prafulla Girgaonkar
    keyboard_arrow_down

    Prafulla Girgaonkar / Naresh Jain - The Art of SQL Database Refactoring

    45 mins
    Experience Report
    Intermediate

    "We've tested this feature thoroughly and it worked really well. But for some weird reason, it's really slow in production today...must be a network issue...or may be the server is having a bad day..."

    Do you often hear these kinds of comments in your development team? Let us guess, your application is very data-centric and churns big blocks of data on every user request. And under the hood, your application is most probably heavily dependent on long/complex queries with joins, temp-tables, case-statements, nested queries, etc.

    These SQL queries probably started-out very simple. But as your requirements evolved, iteration after iterations, the queries also grew in complexity. And most often, even if you test-drove your newer stories, the performance of these complex queries is not evident until you run them in production. 

    Given that our requirements will evolve and so will our database, how do you deal with the above problems?

    There are TWO essential parts to evolutionary database design:

    1. The art of refactoring your SQL queries.
    2. Figuring out the right balance of what processing is done in SQL on the DB sides and what is done on your service side in your App/Web Server.

    Join us as we take a tour of how we refactored our complex, non-performant queries and overall DB without hurting our time-to-market.

  • Liked Sridharan Vembu
    keyboard_arrow_down

    Sridharan Vembu - Relevance of the '12 principles' through project lifecycle - A Practitioner's View

    45 mins
    Case Study
    Intermediate

    This talk is about taking a closer look at how one or more of the 12 principles behind Agile Manifesto are closely connected to the different stages of the project lifecycle and how they impact the right choice of practices and tools at each stage. 

    Few sample scenarios: 

    1. Major change in the way iteration planning was done - common backlog for the platform (comprising of different application teams), think each 'iteration' as a 'release' - deployment of business features to production end of each iteration - resulted in greater collaboration, no separate integration/stabilization phase towards major commercial launch

    Relevant Principle: Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale

    2. Reflecting on team organization - one large team (or) multiple smaller teams and / or feature teams, concepts like Mountaineers-Divers, Navigators-Drivers -> effective and easy context sharing, no stepping-into-each-others-shoes, efficient balance between big picture view and attention to details and such.       

    Relevant Principle: At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly 

    3. Feature kick-offs, analysis volleyballs, need basis Dev/QA/BA huddles, vide calls with distributed teams, subject-specific-google-hangouts -> Effective communication and fewer email conversations  

    Relevant Principle: The most efficient and effective method of conveying information to and within a development team is face-to-face conversation

    Experience clearly suggests that following the right principle at the right time for a specific situation ensures successful outcome, while ignoring one or many of these principles often results in failure.

    4. Adverse effects of measuring the delivery team's efficiency of the team one-dimensionally based on the Story Points delivered

    Relevant Principle: Working software is the primary measure of progress.

     

    In this talk, through specific practical examples, I would be explaining

     - identifying the right principles for each life cycle stage of the project/program

     - deriving the right practices based on the principles and following them effectively to deliver value to customer 

     - business and delivery constraints that prevented us from adhering to some of these principles, resulting in not-so-desired outcomes

    In summary, I would like to emphasis the importance and relevance of the 12 Agile Software Principles behind Agile Manifesto in everyday life of a Agile Practitioner.

  • Liked Sridharan Vembu
    keyboard_arrow_down

    Sridharan Vembu - Turning around a twice-failed distributed enterprise program into success

    45 mins
    Experience Report
    Intermediate

    The common myth about agile methodology is, it is suited for smaller, co-located teams, would not scale up for big enterprises and is best suited for smaller, less complex programs.

    In this talk, I intend to share how we went about setting up, executing and successfully delivering probably one of the most complex and strategic programs for one of our customers. This program was the first ever successful adoption of the fully distributed agile implementations for the customer.

    Context: The client is the leading Telecom Operator in the UK, having their captive and other strategic partners based out of India. The program was highly strategic for the client and the predicted ROI was high.

     - The implementation was tried twice by different vendors for more than an year, but failed to deliver; root causes were not analyzed

     - The Program Sponsor had one last chance to try and deliver the platform successfully, against a very tight schedule

     - Continued Operational risk with the legacy system in place 

    Outcome of our engagement:

     - Core functional application ready in pre-production by the end of first release cycle (4 months from engagement start); fully ready to functionally scale easily and quickly

     - Adoption of the technical and execution approach to other related programs within the portfolio 

    Our Approach:

     - Outcome of initial assessment of the existing codebase was to go with re-write from scratch; was a really hard sell, but was the RIGHT thing to do

     - Re-define the release cycle: extend development period by embedding integration testing as part of development cycle and cut down on the low level design phase

     - Need-basis colocation of functional SMEs with development team 

     - Direct access to Product Owners: weekly video calls, must-attend iteration show-cases, etc

     - Remove unnecessary operational overheads, in terms of people as well as organizational processes

     - Well-defined, pragmatic strategy for Integration testing (major constraints - lead time for test data preparation, limitation in re-usability of test data, lack of e2e functional understanding within team)

      - Smart seeding of other vendor team members (with good functional/domain understanding) into the core team

      - Zero compromise on basic hygienic practices: IPMs, Showcases, communicate negative-news-first with alternate solutions/workarounds, strict removal of wastes, inclusive decision making, highest degree of code coverage, sanity test suite, e2e basic automation suite   

      - Building trust between distributed teams: cross-pairing, align on core work hours across time zones, joint showcases and retrospectives (shared responsibility)

    Challenges faced:

      - Big push to release the core functional platform into production in 3 months (immediate next release)

      - Working out of other vendor premises: seen as threat to their business, lack of cooperation and collaboration   

      - Product Owners based out of UK, no easy / frequent access

      - Functional SMEs/designers based out of different location

      - Release cycle that was in place: 8 weeks of design, 4 weeks of development and 8 weeks of testing!

      - Distributed and isolated testing teams

      - Highly manual and time-consuming E2E testing processes

      - Multiple interfacing systems, both upstream and downstream 

      - Client development team based out of UK, different execution approach, lack of trust between the teams

    In summary, I would like to share the unique aspects of the execution approach that made this program a real success for the customer, though some of the approaches might be tried out in different environments and project situations.

  • Liked Khaarthigha S
    keyboard_arrow_down

    Khaarthigha S - Scaling Agile For Enterprises with Distributed Engagement Models

    45 mins
    Case Study
    Advanced

    I would like to share my experience in consulting and managing a distributed team - Identification of strategies for a transformation of "a lifeless program to a Successful Program " and journey from "Collective Inception to Collective delivery" 

    This becomes challenging especially with a complicated -distributed engagement model for our client which is a reputed and huge enterprise with presence in every corner of the world.

    In a complete globalized world, the major bottleneck for a huge enterprise is the effective functioning of globally distributed teams despite using Agile,lean.

    In my presentation, I am going to share the approaches that we tried to address the pain points including the following:

    1. Not even able to plan the Iteration planning meeting - Iteration planning not producing the outcome despite hours of planning meeting
    2. Manage dependencies between teams for a collective delivery
    3. Communication channel between teams  (Change how you communicate/coordinate)
    4. To bring the organic coherence between teams despite the cultural difference
    5. To also worry about the unknown interfaces & disastrous scenarios
    6. Different team communities with different process and practices impacting the other team’s delivery
    7. To sustain the work ecosystem for all the teams
    8. Inoffensive collective Retrospective for a constructive learning
    9. Major Natural pain point – “its not the distance, it’s the time zones”
    10. Above all, Conflict Resolution

    Eg: one part of approach which we tried was "Mountaineer-Diver Model". 

     

    Impacts of above are listed below:

    • Dynamic Dependecy resolution between teams ( instead of long hours of call for each dependency)
    • Collective , Objective planning for all the teams by matching the dependencies so that the delivery is not affected and also "All teams walking in same speed"
    • More common understanding and project focus in all teams (Frustration with the team members reduced)
    • All members from different teams directly interact and work even they are distributed ( as they spend some time physically working together as "integration teams")
    • As a result of above -> 2 key metrics improved :
      • Velocity of all teams improved
      • Development and Testing complete even before the deadline -> Delivery before the scheduled date
      • Very less time spent in meetings for conflict & dependency resolutions, planning , etc.. 

     "Project execution was the key success". 

    This will help in approaching the issues pragmatically , dynamically and also help understand how its better to make a hybrid out of multiple tools rather than using only one single process tool.