What's the big deal? Dealing with Big Data challenges

Big data is huge! with billions and billions of data sets and a need to analyze and apply that to real-life problem-solving is a challenge. Are traditional methods successful in solving big data problems? Traditional methods of applying water-fall has not been very successful, does Scrum work well? Let's take a look at what some industries have done to deal with big data challenges.

Let’s take a look at the current state of big data, if traditional methodologies are providing the necessary answers quick enough. Is Scrum a good framework to alleviate some of the challenges to deal with big data?

We will discuss the challenges of using traditional methodologies for dealing with big data and evaluate a framework like Scrum to deal with these challenges.

 
 

Outline/Structure of the Case Study

1. Introduction - Big Data in all industries - 5 min.

  • Google HiN1 Virus data
  • Government data - how big is it?
  • Healthcare
  • Financial
  • E-Commerce
  • Other

2. Four V's of Big Data. - 5 min.

  • Volume - Scale
  • Velocity - Streaming data
  • Variety - Different forms
  • Veracity - Uncertainty
  • Apply a 5th V - Values of Scrum - which value is important in this situation?

3. Big data complexity and how empirical process control theory can be applied? 15 min

  • We will look at the Four V's we discussed and how the empirical process control theory can be applied to alleviate these challenges.
  • How will chunking in terms of context of data helps?

4. Walkthrough my experiences in transforming big data projects using Scrum 20 min

  • Management buy-in
  • Experimenting with small teams
  • Transforming data teams using empiricism (transparency, inspection and adaptation)
  • Learning from industry trends
  • Metrics

5. Exercise/activity: 15 min

Scrum assessment canvas for big data. Participants will fill their current assessment on the canvas to assess current state and steps to take to deal with their big data projects using Scrum.

Canvas has the following sections to be filled:

  • Current challenges with big data in our organization
  • Top 3 things that worked in the past
  • Identify steps to improve current situation
  • Prioritize top 3 steps to try at work with my team
  • One take away from this talk
  • What I learnt from others (fill in during discussion & debrief)

Debrief - 10 min.

  • This will be an interactive discussion about the data the participants gathered on their canvas and share with others
  • Fill in what I learnt from others

Summary - 5 min.

Learning Outcome

Participants will have an understanding of dealing with the challenges of the big data industry and how some organizations have been using a framework like Scrum to alleviate those challenges. They will have a chance to take some information back after I share my experiences in working with several big data industries.

Participants will be able to do an assessment of their current state of big data projects and steps to take to deal with their big data projects using Scrum.

Target Audience

New to Big Data, Experienced Big Data Practitioners, ScrumMasters, Product Owners, Business Stakeholders, Managers, Developers, DBAs, Architects

Prerequisites for Attendees

Data knowledge any industry. Anyone who deals with data.

schedule Submitted 1 year ago

Public Feedback

comment Suggest improvements to the Speaker
  • George Dinwiddie
    By George Dinwiddie  ~  1 year ago
    reply Reply

    Raj,

    Your outline/structure is too generic to get a sense of your talk. Please provide the reviewers with more details.

    Also, the abstract seems to be pitched toward people who are already looking for a way to apply Scrum to Big Data. Might there be other people who could benefit from this talk? How could you interest them?

    The abstract is the place where you sell your session to the right prospective attendees. Help them recognize themselves and their situation, and tell them what benefits they'll get by attending. Give them enough information about the content to convince them that they'll get that benefit.

    - George, AgileDC Program Chair

    • Raj Kasturi
      By Raj Kasturi  ~  1 year ago
      reply Reply

      George, Thanks so much for the feedback.  I have reviewed your feedback and made updates in the abstract directly. Thanks.

      --Raj

    • Raj Kasturi
      By Raj Kasturi  ~  1 year ago
      reply Reply
       
      I have updated my content directly on the topic in the case study details. Thanks.

      Raj Kasturi
      ------=_Part_815749_846549542.1534898881064--