schedule Dec 15th 01:15 PM - 02:00 PM place Crystal 2 people 11 Interested

Tests are good. Generative tests are better. But we don't write them often enough. Why not?

Generative testing isn't new; John Hughes' QuickCheck is now implemented in 30 languages. Yet, it is still not an every day tool for most software teams. Why not?

I have seen that maintaining a generative, or simulation testing suite can require the full-time attention of a few senior engineers. And over time, the tests become complex, brittle, ineffective, and hard to diagnose.

But isn't generative testing supposed to make testing simpler, and more effective?

I believe that it requires a rather different paradigm of thought from what we're used to with traditional testing. And recognising this paradigm shift can help us leverage it effectively.

I'll discuss a holistic perspective of generative testing to elucidate this thought paradigm.

Using a practical scenario, I'll discuss the patterns that emerge from this thinking, and how they address the concerns of making generative tests more effective, and maintainable.

 
1 favorite thumb_down thumb_up 0 comments visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/structure of the Session

  1. Basis of the talk [2m]
  2. Outline of a practical system, based on which the patterns can be understood [2m]
  3. Benefits and problems of the current state of generative tests [5m]
  4. Setting up the right terminology for the remainder of the talk [5m]
  5. Detailed patterns in [~25m]
    1. generating input
    2. executing the tests
    3. assertions, properties
    4. diagnosis
  6. Summary [2m]

Learning Outcome

I'll try to answer the following questions through the talk:

  • How to minimise the regression in our tests as the source changes?
  • How to model dependencies between actions, and generate meaningful flows to test?
  • How to model state in tests, while testing complex stateful systems?
  • What are some generic system invariants we can use broadly?
  • How do we trawl through thousands of test results and diagnose them?
  • How do we keep tests deterministic, and repeatable?

Target Audience

People who are interested in testing, generative, or property testing, etc.

Prerequisite

  • Know about quick check, and property based tests
  • Have some experience in writing and running generative tests
schedule Submitted 3 months ago

Comments Subscribe to Comments

comment Comment on this Submission

  • Liked Abhinav Sarkar
    keyboard_arrow_down

    Abhinav Sarkar - Many Ways to Concur

    Abhinav Sarkar
    Abhinav Sarkar
    Principal Engineer
    Flipkart
    schedule 3 months ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    Easy concurrency is one of the main prophesied benefits of the modern functional programming (FP) languages. But the implementation of concurrency differs widely between different FP languages. In this talk, we shall explore the methods and primitives of concurrency across three FP languages: Haskell, Erlang, and Clojure (with core.async).

    We shall learn about and compare the trade-offs between

    • the green threads and STM channels oriented concurrency of Haskell
    • everything-is-a-process and message-passing actor pattern of Erlang
    • macro-based state machine code transformation of Clojure/core.async