"We've tested this feature thoroughly and it worked really well. But for some weird reason, it's really slow in production today...must be a network issue...or may be the server is having a bad day..."

Do you often hear these kinds of comments in your development team? Let us guess, your application is very data-centric and churns big blocks of data on every user request. And under the hood, your application is most probably heavily dependent on long/complex queries with joins, temp-tables, case-statements, nested queries, etc.

These SQL queries probably started-out very simple. But as your requirements evolved, iteration after iterations, the queries also grew in complexity. And most often, even if you test-drove your newer stories, the performance of these complex queries is not evident until you run them in production. 

Given that our requirements will evolve and so will our database, how do you deal with the above problems?

There are TWO essential parts to evolutionary database design:

  1. The art of refactoring your SQL queries.
  2. Figuring out the right balance of what processing is done in SQL on the DB sides and what is done on your service side in your App/Web Server.

Join us as we take a tour of how we refactored our complex, non-performant queries and overall DB without hurting our time-to-market.

 
11 favorite thumb_down thumb_up 1 comment visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/structure of the Session

1. Introduction to common scenarios leading to long/complex queries ( 5 min)

2. Problems faced by developers and typicall approaches to fix them (5 min)

3. Finding and dealing with non-performant queries courageously. (5 min)

4. Technique to refactor them and ensure the overall quality (15 min)

5. Pragmatic guidelines for maintaining the right balance between processing on DB-side vs. App/Web server side (10 min)

6. Questions (5 min)

Learning Outcome

1. way to work with long queries

2. technique to refactor them

3. guidelines to restricing dependency on SQL

4. Pros and cons of dependency on SQL.

Target Audience

Tech Leads, Developers, Testers, DBAs,

schedule Submitted 3 years ago

Comments Subscribe to Comments

comment Comment on this Proposal
  • Rahul Saraf
    By Rahul Saraf  ~  3 years ago
    reply Reply

    Very useful  


  • 90 mins
    Workshop
    Beginner

    "To know, is good. To live, is better. To be, that is perfect." - The Mother

    During the Agile adoption, its a common complain that many team in many organizations get caught up in the ceremonies or mechanics of Agile and fail to understand/appreciate the true value and spirit of Agile. And because of this, the original intent of the Agile movement itself is lost. This is a serious issue!

    This workshop will highlight, a well-proven approach to transformation (not adoption) and show the distinct steps in this journey that an individual or a collective goes through when learning anything new. Activities, serving as examples, in the workshop, will focus to show the journey - that is, how to begin with rituals, then gradually move to practices, arriving at principles and eventually internalizing the values. Witnessing this gradual process of transformation will help participants discover for themselves their current progression. We hope this will serve as a guiding light during their Agile journey.

    Finally, we will leave the participants to ponder upon and discover for themselves their ideals in life and work as this is not only applicable to software development, but also to any discipline where humans are involved, including life itself.

  • Liked Sachin Natu
    keyboard_arrow_down

    Sachin Natu - Inverting Test Pyramid - A First Hand Experience Report

    45 mins
    Experience Report
    Intermediate

    Test automation is extremely crucial in adoption of an agile delivery. However, it can take one for a ride, if the approach is not correct. In this sensational, heart throbbing, experience report, we'll share our story of how we turned around an inefficient, expensive automation style to lean, efficient style. In addition to sharing a real-world example, we'll also share some of the key challenges we faced and how we solved them. If you are convinced about the Testing Pyramid, but are struggling to invert it, then this session is for you.

    Business Impact:

      Earlier Defect Detection - Higher test coverage at Unit/Intermediate layers lead to earlier defect detection. Reduced number of issues found on higher test environments/Production. Reduced cost of defect fixing.

      Reduced maintenance cost - UI tests are fragile and costlier to maintain Vs backend tests. No of changes in services layer are comparatively less.

      Reduced test execution time - Backend tests are much faster. Almost 7-10  times faster than UI Tests - improved build certification time.

      Test feedbacks are naturally distributed across layers of application. Test feedbacks are more pin pointed/ granular.

  • Liked Ashish Parkhi
    keyboard_arrow_down

    Ashish Parkhi - Techniques to Speed Up your Build Pipeline for Faster Feedback.

    45 mins
    Experience Report
    Intermediate

    I would like to share my experience and journey on how we brought down our Jenkins build pipeline time down from over 90 minutes to under 12 minutes. In the process, I would share specific techniques which helped and also some, which logically made sense, but actually did not help. If your team is trying to optimize their build times, then this session might give you some ideas on how to approach the problem.

    Development Impact - For one of our build job, below graph shows how the number of builds in a day have increased over a period of time as the build time has reduced. Frequency of code check-in has increased; Wait time has reduced; failed test case faster to isolate and fix.

    Business Impact - More builds leading to quicker feedback and faster story acceptance and less story spill over.

  • Liked Archana Joshi
    keyboard_arrow_down

    Archana Joshi - Applying Gamification Techniques in Agile

    45 mins
    Case Study
    Intermediate

    Background: 

    • One of the world’s largest legal product & services provider.
    • Development and global rollout of online legal research product for law firms to efficiently provide online research and analytical capabilities.
    • Several hundred million dollar product revenue
    • 400 Team size.
    • Product Development and subsequent enhancement and support.
    • Stringent Timelines
    • Most of the team new to agile
    • 4 Timezones

    Key Challenges on "people" front:

    1. Scale up the team to use agile in short time frame
    2. Engage and motivate the team 
    3. Be a true vendor partner with the customer

    Industry analysts predict rapid adoption of gamification. By 2014, more than 70% of Global 2000 organizations will have at least one gamified application, according to Gartner, Inc. In this presentation, we would be focusing on how gamification helped us adddress the above challenges.  Concept of “gamification” was used to drive the team to embrace agile and improve employee engagement and motivation. This helped further boost the quality and increase the overall customer satisfaction across sprints and releases.

  • 45 mins
    Experience Report
    Intermediate

    Over the last decade, eXtreme Programming practices like User Stories, Evolutionary Design, Test-Driven Development (TDD), Behavior Driven Developer (BDD), Refactoring, Continuous Integration and Automation have fundamentally changed software development processes and inherently how engineers work.

    Having experienced various benefits from XP practices on our J2EE stack, our team started to apply these practices to extract, transform, and load (ETL) and Data Analytics side of our product. Unfortunately, there is very little guidance available in this context, esp. for the SAS Platform. Right from finding the unit testing framework to structuring the code to designing our modules and setting up a Continuous Integration builds, our team had to figure out everything, the hard way.

    Join us to understand the challenges we faced during this process and how we resolved these challenges.

  • Liked Alexey Ilyichev
    keyboard_arrow_down

    Alexey Ilyichev - Skype goes agile: don't repeat our mistakes

    Alexey Ilyichev
    Alexey Ilyichev
    Agile Coach
    ScrumTrek
    schedule 3 years ago
    Sold Out!
    45 mins
    Case Study
    Intermediate

    In 2011, I worked for Qik, a startup that got aqcuired by Skype. At that time Skype was in the middle of an agile transition. Аfter aquisition, Qik team was told to adopt the Agile process used by Skype. I worked with the team as an agile coach. After adopting Skype's "agile" process, our ability to deliver was brought down to almost zero. In this talk, I'll tell you the story as it happened, analyze the key problems that we faced and describe how we finally solved them. Come to this talk, if you want to avoid similar mistakes. If you are already through with your transition, I would be interested to know if you see any patterns. 

  • Liked Shirish Padalkar
    keyboard_arrow_down

    Shirish Padalkar - Application Security - The Agile Way

    Shirish Padalkar
    Shirish Padalkar
    Senior Consultant
    ThoughtWorks
    schedule 3 years ago
    Sold Out!
    45 mins
    Talk
    Beginner

    Traditionally application security has involved upfront design and a big bang penetration test after development. This leads to the phenomenon of “bolt-on” security that translates into increased cost and complexity.

    Drawing on our experience on real-world projects, we show how security can be baked-in on an agile project. Using case studies we demonstrate how security concerns are captured during project inceptions, how developers write secure code, security testing is automated and how configuration management can help achieve secure deployments. This talk introduces several new concepts like secure by design, secure design patterns and lightweight code reviews.

  • Anand Bagmar
    Anand Bagmar
    Director - Quality
    Vuclip Inc.
    schedule 3 years ago
    Sold Out!
    90 mins
    Case Study
    Intermediate

    The key objectives of Organizations is to provide / derive value from the products / services they offer. To achieve this, they need to be able to deliver their offerings in the quickest time possible, and of good quality!
    In such a fast moving environment, CI (Continuous Integration) and CD (Continuous Delivery) are now a necessity and not a luxury!

    There are various practices that Organizations and Enterprises need to implement to enable CD. Testing (automation) is one of the important practices that needs to be setup correctly for CD to be successful.

    Testing in Organizations on the CD journey is tricky and requires a lot of discipline, rigor and hard work. In Enterprises, the Testing complexity and challenges increase exponentially.

    In this session, I am sharing my vision of the Test Strategy required to make successful the journey of an Enterprise on the path of implementing CD.

  • Liked Yilmaz Guleryuz
    keyboard_arrow_down

    Yilmaz Guleryuz - Fibonacci Pairing: Learning through Collaboration

    Yilmaz Guleryuz
    Yilmaz Guleryuz
    incognito
    WareNinja
    schedule 3 years ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    According to many researches, we learn the best when we experience it personally, and we comprehend when we teach to someone else.

    So if you want to achieve ample understanding of a topic, you’ll need to follow these basic steps:

    "Listen -> Observe -> Do -> Share"

     

    Pair-programming is a well-known practice, and it has been introduced by one of the thought leaders of Agile world, Kent Beck. I respect any technique that encourages collaboration and interaction among teams, and gets teams to communicate better.

    However, if you want to establish understanding of mastery or a skill-set, and increase the shared knowledge across your team members, then you need to re-think & apply new techniques.

    In this talk, I’ll share about What-Why-How of "Fibonacci Pairing" technique, which is yet another discovery from practice.

    The main goal of that technique is to establish understanding and improve collaboration, without overcrowding by artificial ways like lecturing.

    Fibonacci Pairing is simple yet very effective technique that can improve team work. This technique should be regarded as a complementary way to improve team culture effectively.

    We will elaborate with practical examples, real life experiences, and applicable ways onto different contexts.

  • Liked Frans Overbeek
    keyboard_arrow_down

    Frans Overbeek - Spotless Clean Frontend Development with BDD, DSL and Storyboards

    Frans Overbeek
    Frans Overbeek
    Architect
    Capgemini
    schedule 3 years ago
    Sold Out!
    45 mins
    Demonstration
    Beginner

    After 3 years of improving our frontend development process we ended up with mixing BDD, DSL and Storyboards for developing spotless clean frontend code. Each approach is already powerful on his own but combining these will make them unbeatable. The characteristics are collaborate, communicate and visualize, BDD is the collaborating process between disciplines, DSL for enabling the communication and Storyboards for visualization.

    Let me show you how this works by live coding a frontend application using this mixture of approaches. The process is supported by a homemade tool based on CasperJS and PhantomCSS. And all scripts are written in JavaScript. At the end of the session we can discuss if the created code is spotless clean.