Feature Matching-based Approaches to Improve the Robustness of Android Visual GUI Testing

In automated Visual GUI Testing (VGT) for Android devices, the available tools often suffer from low robustness to mobile fragmentation, leading to incorrect results when running the same tests on different devices. To soften these issues, we evaluate two feature matching-based approaches for widget detection in VGT scripts, which use, respectively, the complete full-screen snapshot of the application ( Fullscreen ) and the cropped images of its widgets ( Cropped ) as visual locators to match on emulated devices. Our analysis includes validating the portability of different feature-based visual locators over various apps and devices and evaluating their robustness in terms of cross-device portability and correctly executed interactions. We assessed our results through a comparison with two state-of-the-art tools, EyeAutomate and Sikuli. Despite a limited increase in the computational burden, our Fullscreen approach outperformed state-of-the-art tools in terms of correctly identified locators across a wide range of devices and led to a 30% increase in passing tests. Our work shows that VGT tools’ dependability can be improved by bridging the testing and computer vision communities. This connection enables the design of algorithms targeted to domain-specific needs and thus inherently more usable and robust.

Comparison of Graphical User Interface Testing Tools

Graphical User Interface or better known as the user interface is the liaison of users with electronic devices such as computers. The Graphical User Interface uses icons, menus, and some other visual indicators to represent the information contained in the interface of the application being used. The Graphical User Interface I must pass the Graphical User Interface Testing stage to ensure that every element in the Graphical User Interface is not an error and by the specified one. Also, we know that Graphical User Interface Testing is a set of activities that aim to test the Graphical User Interface I of the test object to ensure that the Graphical User Interface complies with the specifications specified in the software design document. In this research, we try to compare four Graphical User Interface testing tools which ae: Robotium, Espresso, UI Automator, and Pix2Code. By exploring these 4 testing tools we find out that Pix2code can only identify objects, especially label objects. Pix2code can only meet 3 out of 7 predefined criteria. This indicates that there are still many objects of the android application that Pix2code has not been able to identify. In other words in the Graphical User Interface testing section, pix2code can play a role in identifying each object contained in the application and can be done at the design stage. The result that we get from this research is that the GUI testing tools could identify many parts and almost every object in the application except the Pix2code. For future development, Pix2code as a testing tool requires development in the form of a desktop display such as the UI Automatorviewer so that it can display every detail of the object including the attributes of the object.