Build your own Internet of Continuously Delivered Things
It is predicted that till 2025 there will be over 75 billion devices spewing 180 zettabytes of data and generating up to 6 trillion dollars. That enormous increase force companies to introduce a continuous approach to deliver the product as fast as possible and be able to compete on the market.
The main question is how to test application for end user among so much hardware equipment and ecosystems combining HW, FW, mobile devices and complex backend architecture? Considering all factors and possible obstacles is it for companies a real “A New Hope” for companies or just simply “Mission Impossible”?
I will take the participants on a journey to the IoT world. It will be a talk about the challenges that any tester will face at some point. I will present the dangers, risks and snares but also good practices and practical approach to mobile E2E test automation for the IoT solutions in CI approach.
Technical examples will be presented using Python languages and supported by physical devices (mobile phones and IoT equipment).
Outline/Structure of the Case Study
1) Introduction to IoT
2) Basic problems in IoT testing
3) Case study presentation (how to control and read IoT devices via browser - Selenium - and mobile phone using Appium)
4) Step by step solution found
5) Components needed to implement before
6) "Glueing" components do deliver ready test environment
7) Integration of devices side (IoT) and customer side (mobile)
7) Final results, numbers, view for product people
8) Takeaways
Learning Outcome
- Understanding of HW/SW dependencies for mobile testing
- Integration of Selenium & Appium for browsers and real devices with IoT equipment
- Possible problems with wireless protocols testing on mobile (Wi-Fi, BLE)
- Vision system usage in testing (controlling IoT devices via Appium)
- CI flow for IoT SDLC
- Example of usage in Python and/or Bash languages
Target Audience
Automation QAs, DevOps, Test architects, TestOps
Prerequisites for Attendees
The concept of Continuous Integration
General experience/knowledge about SDLC
Video
Links
schedule Submitted 3 years ago
People who liked this proposal, also liked:
-
keyboard_arrow_down
Virender Singh - Rise of Shadow DOM - Lets solve it through WebDriver
45 Mins
Demonstration
Beginner
Shadow DOM is kind of web standard that developers use to encapsulate their custom HTML code and style components so that no other global style components can override their code. It ensures that a component will work in any environment, even if other CSS or JavaScript is run on the page i.e. Shadow DOM hides away the complexity of the components from the rest of the page because it renders separately from main DOM.
Selenium legacy selectors are not able to interact these custom elements under the shadow DOM so we need java script to interact with shadow DOM elements and we also require JavascriptExecutor interface for their execution.
We will also talk about the WebdriverIO tool v5.5.0, which implicitly support Shadow DOM elements.
-
keyboard_arrow_down
Jesus Sanchez Martinez - Test and monitor one website is not that hard, but what if you need to do it to over 40 websites?
45 Mins
Talk
Advanced
Onestic QA department made and maintained test suites, which was a huge bottleneck in our development process.
In order to solve it, management bought us an idea: our developers must be able to build their tests using a DSL framework without friction. QA maintain a big library that provides resources to developers. From there, they were free to extend this library in each project. With all of this we have a CI process, a 1 hour execution to monitor our results and of course our bot, SpongeBot.
SpongeBot can check 40+ e-commerce sites, with 4+ environments, for desktop and mobile platforms always available for developers.
With this solution we achieve to decentralize the work, add value testing production and the confidence in SpongeBot notifications if something goes wrong.
-
keyboard_arrow_down
Zach Attas - Selenium For All - Setting Your Team Up For Success So Anyone Can Understand and Write Tests
45 Mins
Talk
Intermediate
Are you the only one contributing to your test suite, because others feel too intimidated to contribute to it?
Have you ever looked at a test fail and not known where to investigate first?
This talk will cover techniques for uncomplicating your test suites, so they can clearly test exactly what's needed to be tested, allowing others to contribute tests. When the process of writing tests is shared, everyone has an impact on quality, and that can be quite contagious for a company's culture.
In addition, this talk will cover techniques for failing transparently, so you will no longer feel stressed triaging test fails. Quality is something shared by the entire team, so anyone should be able to triage test fails. When tests fail for a specific, clear reason, you'll be thanking your past self for writing logic to fail transparently, saving you half a day going down a rabbit hole.
-
keyboard_arrow_down
Moshe Milman - Fixing Your Automation Challenges in the Era of CI/CD
45 Mins
Talk
Intermediate
Continuous Delivery is now the holy grail of IT organizations, but most companies are still struggling with the transition into shorter release cycles and faster, more frequent deployments. A key challenge that companies are facing in that transition lies around test automation, and in this talk we will cover these challenges in details and demonstrate how successful companies are addressing it.
The key takeaways from this session include:
1) Testability hacks + best practices2) Tips for reducing your testing matrix to support the automation speed required for frequent builds/releases
3) Examples of team structure/architecture from leading companies to support better alignment around automation4) Guidelines for picking the right open source tools / frameworks
5) Learnings/Issues to avoid when designing your automation architecture -
keyboard_arrow_down
Diego Molina - Test Automation is not just coding
45 Mins
Talk
Intermediate
We learn more from our failures than our successes. I want to share one of my failure stories, where I learned that coding is not the most important task in Test Automation.
A failure taught me that coding is not the most important task in Test Automation. I fixed a bug, added tests, and shipped it to production. After that, a bug report came in showing that the fix created a new bug. I felt bad, I did not understand how that happened. A hotfix solved the issue, but the root cause was not addressed: what did I miss?
By taking a step back, I understood the situation, it all happened by overseeing basic concepts in testing (like understanding how the system works). The problem was that coding had more priority than creating a test plan. After this, I prioritized tasks better to avoid this situation to happen again.
This talk shows that testing concepts are more important than ever, in a time where tools promise to do everything, we focus less on what and how to test, and more in using tools to test. It outlines how a test strategy can leverage a continuous testing setup. Finally, it shows that failing is ok, but failing again for the same reasons is not.
-
keyboard_arrow_down
Christian Bromann - The Nuts and Bolts of WebdriverIO
90 Mins
Workshop
Beginner
There are thousands ways if not more to setup your automation testing environment. It is often crucial when it comes to stability and maintainability. While you can build a custom framework that fits your needs there are often already sophisticated setups or frameworks available that get you up and running very quickly.
WebdriverIO is one of these frameworks that is written in Node.js. It gets you up and running within minutes and allows you to scale up your test suite while maintaining your execution time by running tests in parallel. With its huge project community it is an ideal choice for small as well as big projects that use modern frameworks such as React, Angular or Vue.js.
In this workshop you will learn everything you need to know to run a successful, stable and maintainable WebdriverIO framework. It starts with an introduction to the project and the basic concepts and will end with a sophisticated framework that includes testing strategies like Frontend Performance Testing as well as complex browser interaction with Puppeteer.
-
keyboard_arrow_down
Srinivasan Sekar / Sai Krishna - Testing And Observability in an Integrated Microservices environment
Srinivasan SekarLead ConsultantThoughtWorksSai KrishnaLead ConsultantThoughtworksschedule 3 years ago
45 Mins
Case Study
Intermediate
Leading-edge applications are dynamic and adaptive in capabilities that require people to use increasingly dexterous tools and supporting infrastructure, including microservices. All of these applications leverage data in new ways. Decoration and tagging of data with intelligent meta-data have become more important than data itself. To keep up with evolving needs, enterprise devs across industries are shifting from traditional app architectures in favor of more fluid architecture for building data-centric applications.
Microservices break traditionally structured applications into manageable pieces that can be developed and maintained independently. microservices are often decoupled, allowing for updates with little to no downtime, as the other components can continue running.
Moving to distributed Microservices ecosystem brings its own challenges; Among them is the loss of visibility into the system, and the complex interactions now occurring between services. Monitoring these applications only reports the health of it but Observability provides granular insights about the behavior of the system along with rich content. In this talk, we will cover the difference of Monitoring and Observability, data path engineering challenges, pillars of observability, distributed tracing of various microservices, testing in distributed microservices ecosystem, automated observability, etc.
-
keyboard_arrow_down
Dawid Pacia - Internet of Things - risks, challenges, and flying snacks
45 Mins
Case Study
Beginner
IoT world grows by 30% every year (up to 25 billion(!) devices till 2025) and that’s just a matter of time when some of us will have to…get back to the past. Many of the current testers very soon will start their adventure with real hardware devices. It also means more and more complicated integration with modern services and technologies (Amazon DRS, Alexa, Behavioral Recognition, vision processing, AI, ML and many more).
What are the limitations encountered in such conditions? What are the main challenges and what a modern tester should know before she or he starts their testing game? Why we should focus on safety aspects? And finally, how to find yourself in a testing environment when Time to Market statement has paramount importance in terms of building business models?
During my presentation, I’ll show how current challenges and risks will change in the near future and how to deal with them to avoid a safety headache. I’ll also present the way to easier identify and mitigate them. All of that in the context of pet cameras, snack launchers, and lasers…in other words - IoP (Internet of Pets) world. All dogs and cats are warmly welcomed!
The content of this presentation consists of 3 major parts:
- Risks, challenges and safety aspect the modern tester should be aware of
- The real-life examples of safety vulnerabilities and its impact on customers (and the scale of it!)
- What makes testing so exciting despite so much responsibility and so many challenges
Risks, challenges and safety aspect:
1) Be aware of flaky tests and its reasons
Now we live in the era of continuously delivered things. That causes even more and more test automation that we rely on. However, can we still treat every environment as safe enough? We should be still aware of false negatives (the test is marked as Passed when the functionality is not working) and false positives (the test is marked as Failed when the functionality is working). There is also something even worse - flaky tests (tests failing intermittently). And there are a lot of reasons behind that… (both, human and environmental ones)
2) Updates (OTA) as the most risky area
One of the most used “functionalities” by (aware or not) customers. When you think about SW update, usually that’s not something to be considered and safety critical. For web applications, every blocker can be fixed by fast deployment or bugfix. For mobile apps, it’s a matter of releasing a new version to store. Even for a desktop, you can force your user to reinstall it once again. But what if something goes wrong for IoT/HW devices? Is there any safety “belt” or procedure to prevent your firmware from turning into a brick?
3) Scale matters
New smart devices are very often responsible for public and private safety. To cover dedicated area (or just get your customers involved) there are a lot of them needed - sometimes thousands of them! Do you think that scale can be skipped during testing? How to ensure safe infrastructure when your network is growing up? Which parameters should be considered as important (e.g connectivity issues, configuration, geolocation, external influences, wireless signals multiplication)?
4) Testing real user behavior
Is your customer always following feature workflow intended by you? In most cases, your predictions are not fully consistent with the user’s real behavior. In that case, to achieve the high level of safety, you must rethink and redesign your test strategy (and implementation) approach. Let’s look at the example…
5) Focus on CXA/CEA (customer experience assurance)
Keep your customer safe! That’s the main principle for modern, smart devices world. Sometimes functionality working perfectly according to business requirements, but not exactly covering users’ expectations, can be much more harmful than functionality not working at all! What if light or laser strength suits all regulations but is not appropriate for young children? What if pets’ cookies are flying too fast and can frighten your cat? What if your house locks automatically and doesn’t want to let you in?
6) Limited trust in open source
We all know that statement. It relates not only to safety aspects but it also relates to security issues even more! Using open source for testing in the worst case can destroy your infrastructure or make it unmaintainable… Is there any chance to avoid such destructive impact?
7) Keeping safety during working with IoT & HW
Last but not least! Working with HW and equipment means embracing all physics rules. Rules that are sometimes ruthless… It is more than recommended to get familiar with some major principles while working with current, volts, and temperature - to minimize the number of explosions in your test lab :)
-
keyboard_arrow_down
Dawid Pacia - Put your TestOps shoes on! Improving Quality by Process Automation
45 Mins
Case Study
Intermediate
Automate everything! That’s the most suitable description of DevOps culture. The culture that quickly created job position with the same name. Position, mostly focused on broadly defined automation, leading to fast product delivery. And the division was pretty simple: DevOps = Process automation, QA = Test automation. But is it the right approach? What about (still more and more) popular “(Dev)TestOps” term?
Classical testers are now also very often responsible for set-up and maintenance of the major part of Continuous Integration or Continuous Delivery environment (especially test automation part). The main problem from the business perspective is, like always, time! E.g. many start-up and companies in a phase of early, dynamic growth cannot afford to waste too much time on test automation. How to speed up the delivery process in that cas? How to quickly generate a valuable increment?
I’ll show you how to improve and speed up testing and delivery process by clever automation in 3 steps:
- Automation supporting manual regression testing activities
- One Click test environment setup
- Preparing fully readable and executable test cycles and test cases
- Ensuring that unchanged component is not double-checked or unnecessarily tested
- Automation maintaining project workflow, transitions, and statuses
- Recognizing when an issue is QA ready vs. when it is dev ready
- Avoiding (very!) common misunderstanding regarding testable issues
- Assuring that functionalities have been released without reading documentation and changelogs
- Automation enhancing bug catching and reporting during testing or normal application usage
- Comprehensive incorporation of all crashes with bug reporting system
- Immediate notification based on priority and severity threshold level
- Automatic preparation and update of reported bugs, including necessary statistics
All of that in the context of 5 good (TestOps’) friends: Test Management Tool (Zephyr), Project Management Tool (Jira), Crash & Log Reporting System (Crashlytics), Communication & Notifications Channel (Slack) and Continuous Integration Tool (Jenkins).
- Automation supporting manual regression testing activities
-
keyboard_arrow_down
Anton Angelov - Combining Load and Functional Testing- Reusing WebDriver Tests for Load Testing
45 Mins
Demonstration
Advanced
Typically, industry software testing practice is to separate load testing from functional testing. Different teams with different skills and expertise do their testing at different times, and each evaluates the results against its own criteria.
Anton Angelov will present to you how you can reuse your functional tests for load testing. We will review the design of a load testing engine that will reuse the same functional tests and at the end of the execution, generate a comprehensive report. We will also talk about how to utilize the new features in Selenium 4.0 in this load testing solution. As part of the reviewed tooling, there will be a discussion about the best possible checks you can do in your coded load testing library and the essential load testing metrics you need to collect during the run.
-
keyboard_arrow_down
Wim Selles - Do you know the dependency of your dependencies dependency?
45 Mins
Case Study
Beginner
Nowadays a lot of people are using JavaScript based automation frameworks, and why not? They are the closest language to what your developers are using and it's easy to use. You just
- Install NodeJS on your local machine
-
npm install this-module
-
npm install that-module,
- script a test-case
-
npm run test
And you're done, easy peasy, isn’t it? But when you look at your project you see something called
-
node_modules
-
package.json
-
package-lock.json
And you hear your developers talk about
- ES6, ES7
- TypeScript
- Polyfills
- NodeJS LTS, 8, 9, 10, 11, 12, 13 (and counting)
- Promises, asynchronous behavior
- ...
And you are like……………...
If you recognize yourself in this then this presentation might be useful for you, because during this presentation I will cover NodeJS, the biggest dependency of your dependencies dependency.
-
keyboard_arrow_down
Arjan Blok - TypeScript: your gateway drug to the JavaScript world
45 Mins
Talk
Intermediate
A big part of the testing community is using Java based tooling to do automation. Selenium Java bindings and RestAssured are goto testing libraries for a lot of us. Yet most of the applications we are testing nowadays are written in JavaScript; the language of the web. I speak from experience when I say that writing your automation code in the same language as your application under test is written in will bring you a lot of benefits. It creates a common ground for knowledge sharing, problem solving and ultimately: developers willing to write and debug integration tests.
However, as a tester, programming might not be second nature to you and learning a new untyped language like JavaScript, with all its quirks, can look like a daunting task. Enter TypeScript: a typed superset of JavaScript which will instantly make you feel at home!
-
keyboard_arrow_down
Krishnan Mahadevan - My experiments with Grid
45 Mins
Tutorial
Intermediate
Everyone starts off with a simple grid setup which involves a hub and one or more nodes.
This traditional setup is a good start but the moment one starts to get serious with the selenium grid and decide to house their own selenium grid for their local executions, that is when issues start.
My experiences with the Selenium grid in the past couple of years has led me to get introduced some of the most prevalent problems with maintaining an in-house selenium grid.
- Nodes get unhooked randomly due to network glitches.
- Nodes introduce false failures due to memory leaks.
- Selenium Grid running out of capacity.
- Nodes require OS upgrades/patches etc.
- Needing to deal with auto upgrades by browsers (especially chrome and firefox)
Some of these issues I managed to fix by building a "Self Healing" Grid wherein the nodes automatically get restarted after they have serviced "n" tests. But that still didn’t solve many of these other problems.
That was when I felt, what if there was an on-demand selenium grid.
What if the Grid could do the following ?
- The Grid auto scales itself in terms of the nodes based on the current load.
- The Grid does not require a lot of infrastructure to support it.
- The Grid can plug itself into some of the cloud providers or leverage a solution such as Docker so that the nodes can be spun and shutdown at will.
That was how the idea of "Just Ask" an on-demand grid was born.
Just-Ask is an on-demand grid. It has no nodes attached to it.
It’s designed to spin off nodes on demand, run test against the newly spun off node and after test runs to completion, clean-up the node as well. The node can be backed by anything. It could be Docker (or) it could be a VM running on any of the popular clouds.
The session aspires to walk the audience through with my experiments with the selenium grid, my learnings on the selenium grid internals and how I used all of that knowledge to build my own On Demand Selenium Grid. What better avenue to share these learnings than a Selenium Conference.
The session will introduce the audience to the grid internals and their concepts such as
- What is a Selenium Remote Proxy ? What is it used for? What can you do with it?
- What is a Hub (or) Node level Servlet ? When would you need one ?
- All of this followed by a quick demo on "Just Ask", the on-demand grid that I have built and open sourced here: https://github.com/rationaleEmotions/just-ask
-
keyboard_arrow_down
adam goucher - Running Selenium Grid in AWS
480 Mins
Workshop
Intermediate
The workshop is the full day version of my Selenium Conf London talk where you will securely and efficiently build a modern Selenium Grid in AWS. Topics will include;
- Infrastructure as Code using Terraform
- Designing your Selenium Grid for the AWS Cloud
- Security Fundamentals
- Maximum result for minimum costs
- And of course, run a script to prove it all works
This is a hands-on workshop where each participant will create their own grid in the AWS cloud using starter scripts provided. As a result, all participants will need to have created a new AWS account prior to this workshop which they have full control over and a laptop.
Note: Participants will accrue a small amount of AWS charges for the infrastructure they build during the day, but it will be minor.