Migrating the herd - How Salesforce is migrating 35,000 Selenium RC tests

location_city Bengaluru schedule Sep 5th 11:45 - Jan 1st 12:00 AM place Grand Ball Room 1

The Quality Engineering team at Salesforce fell into the same pit that most other technology companies have: Selenium tests were so easy to create that we created thousands upon thousands of them. The difference is our Selenium RC pit is over 35,000 tests deep. This talk is about how we are indentifying what each of our tests are doing, how we can transition tests to a different and faster layer and how we are migrating from Selenium RC to WebDriver across a massive scale.


Outline/Structure of the Case Study

1. Introduction to engineering scale at Salesforce: 5 minutes

2. Explore the CI process at Salesforce: 5 minutes

3. Learn the history of Selenium at Salesforce, and how we wound up with so many Selenium tests: 5 minutes

4. Discuss why we could no longer support so many Selenium tests and what drove us to consider refactoring: 5 minutes

5. Learn how we managed the migration starting with our slowest and most complex Selenium tests. 10 minutes

5. Learn the criteria Salesforce developed in regards to what should be a WebDriver tests versus what should be re-written as a JS unit test. 5 minutes

6. Learn how to propose your own migration efforts to management in order to convince them the effort is worth the cost. 5 minutes

7. Questions: 5 minutes

Learning Outcome

1. First and foremost I want everyone to understand why an effort at this scale is worth the effort.

2. Learn how we tackled the migration starting with our longest running and most fragile tests.

3. Learn how we discovered speedups beyond just switching from RC to WebDriver such as different ways in which we created data to test against.

2. Learn how we determined what should be migrated to WebDriver, and what should be tested at a lower and faster level.

3. Learn how to make a business case for a migration such as this at your own company.

Target Audience

Developers, DevOps, Release, Quality Engineers, Product Owners, Product Managers

schedule Submitted 5 years ago

Public Feedback

comment Suggest improvements to the Speaker
  • Dave Haeffner
    By Dave Haeffner  ~  5 years ago
    reply Reply

    I'm glad to see another talk around the arduous task of migrating a massive set of tests from Selenium RC to WebDriver. Especially after Jason Leyba's closing keynote last year about how they did it at Google (you can see a write-up of it at http://elementalselenium.com/tips/28-upgrading-from-rc).

    It sounds like you have a nice story with some good lessons learned framed for beginners.

    Your #1 Learning Outcome is "Learn what first drove us to study our existing 35,000 Selenium RC tests." but it looks like you tackle this as #4 in your outline and spend a good deal of time with it there (15 minutes). Are the pieces that come before it relevant to tell your story and make this compelling?

    • David Tolley
      By David Tolley  ~  5 years ago
      reply Reply

      I agree with your comments, originally I didn't have the outcomes in the priority that they should have been. I refactored the outcomes and the outline a little to reflect more emphasis on the how instead of the why.

  • Liked dima kovalenko

    dima kovalenko - Managing Builds with DotCi

    30 Mins

    DotCi is an Open Sourse Project which integrates GitHub, Jenkins, Docker and allows a one click setup of new Jenkins builds with a simple YAML file. DotCi is platform independent and can support Ruby, Java, Python, Node and many other builds. It sets up a brand new prestine build environment for every build, which allows you to have consistent testing results, where build failure due to bad node environment is a thing of the past!

  • Liked Jim Evans

    Jim Evans - Introducing Windows Phone Driver

    Jim Evans
    Jim Evans
    Selenium Project
    schedule 5 years ago
    Sold Out!
    60 Mins

    In the last few years, use of the Selenium WebDriver API to automate web, hybrid, and native applications on mobile platforms has exploded. The creation and proliferation of such projects as Selendroid, ios-driver, and Appium have given WebDriver users the ability to leverage their exisiting knowledge to the Android and iOS platforms. Windows Phone has not had a similar automation story, until now. This presentation introduces a new, actively-maintained open-source implementation of the WebDriver API designed to be used on the Windows Phone platform.