Dear Testers,

Desktop app automation skill is often overshadowed by the buzz surrounding mobile and web-based testing, but it remains an essential skill for automation testers in 2023 and beyond. Despite the proliferation of mobile and web-based applications in recent years, desktop apps remain a critical aspect of the software landscape and cannot be ignored by testers and developers.

There are several industries that continue to heavily rely on desktop apps, including finance, healthcare, Space, Aeronautics,Digital Ocean and government. These industries often have strict regulatory requirements and data security concerns, making web-based solutions inadequate. In addition, many desktop apps are used for resource-intensive tasks, such as data analysis and visualization, which may require the processing power and offline capabilities of a desktop environment.

Some Popular Desktop Apps that we use everyday :

1. Adobe Softwares

2. Microsoft Windows Default Desktop Apps.

3. Microsoft Teams/Slack/Etc.

4. PC Games

5. And Many more...

Furthermore, many desktop apps generate significant revenue for their developers. According to a study by the Software and Information Industry Association, the global market for desktop apps was worth $76 billion in 2020 and is expected to continue growing in the coming years. As such, ensuring the quality and reliability of these apps through automation testing is crucial for the success of these businesses.

In summary, desktop app automation is still highly relevant and cannot be ignored in 2023 due to the continued reliance on desktop apps by certain industries and the significant revenue they generate. We believe that a session on this topic would be of great value to attendees of the testing conference.

We are aware of the continuous monitoring of various data intensive systems and services across cloud platforms and on-premise settings. However, when it comes to continuous monitoring in alignment with continuous testing of visually heavy live streaming applications, we are left with the fewer options especially in the open source space.

In an attempt to address the challenges around these areas, we've had to build ImageVision for detecting various types of visual anomalies for the sensor and time-series based live streaming applications. In the context of Agile and DevOps centric delivery models, the purpose and premise of ImageVision gains significance.

Code based conventional open-source automation tools are not sufficient to address the deeper 
challenges of modern hi-tech digital transformation systems that include, but not limited to Edge, cloud and container computing and plenty of visual plotting of sensor data streaming in real-time among others.

Based on my research and exploration, I observed that it's unlikely to find even off-the-shelf commercial tools to address the complex automation requirements of detection of visual anomalies in real-time plotting of sensor data streaming that I've addressed in my projects with ImageVision.

Also, the open source automation tools can be better complemented and extended with the integration of visual automation tool like ImageVision to add more flexibility and versatility.

ImageVision is a multi-purpose in-house visual automation tool. It's not meant for replacement of code based automation tools, but it's definitely extending and complementing greatly the traditional code based automation approaches. Even it has proved to be an absolute mandatory in fulfilling the needs of extremely complex visual anomaly detection scenarios and also in many of the complex and tricky UI interactions where the code based automation approaches have been failing to meet the expectations disgracefully.

Listed below various purposes and the features of the tool.

1. Detect various types of anomalies in visual plotting of stream of  sensor  data points in real time in a fast paced agile delivery model, which otherwise would be impossible to detect.
2. Continuous testing combined with continuous monitoring for tracking anomalies in dev/test/staging envs.
3. Perform any type of complex automation interactions as if a real human user would do within the visual plotting regions/widgets or on any part of the target region of the screen/image
4. Capture any web element as an image for further automation actions
5. Compare sets of baseline and runtime images with each other using sophisticated computer vision algorithms
6. Plenty of configurations that offer degrees of flexibility to apply the algorithms as a group or individually and in any order of applying them
7. Potentially offering UI and platform agnostic E2E visual automation capabilities
8. Apply image processing techniques on the basis of configurations including ignoring certain regions of images during grab and compare operations
9.  Comprising three core modules : Image Grab, Image Compare, and Image Interact. Image Interact module is internally      named Actionize
10. Currently, addressing three major anomalous conditions in real time mode: I. data gap in the plotting of data or no plotting for certain amount of time or no plotting continuously or intermittently, II. occurrence of hang issue in the plotting - again continuously or intermittently, III. occurrence of visually flickering effect in the plotting
11. Pretty easy to integrate and use within automation test flows - just one single API call with configurations would carry out all the required tasks and actions
12. Modules can either be run independently or as a sequence of dependent chain based on the tasks or workflows
13. Json format reports each for image grab, compare, interactions, and each anomalous condition
14. The user API set or TDK(Test Development Kit) that invokes the backend engine has been built with TypeScript and JavaScript. A group of backend engines each for unique purpose that carry out the actual tasks and actions has been built with python and computer vision libraries

Microsoft's Adaptive Cards are platform-agnostic snippets of UI, authored in JSON, that apps and services can openly exchange. When delivered to a specific app, the JSON is transformed into native UI that automatically adapts to its surroundings. It helps design and integrate light-weight UI for all major platforms and frameworks.

How to design your Adaptivecard to send your testResults to Microsoft Teams.

By Using Adaptivecards we can create a card as per requirements and share the results to Microsoft teams once it is executed.

this can be extened for sending alerts for sensitive testcases execution status without waiting for the execution to complete.

for more info on adaptive cards refer to https://adaptivecards.io/