Selenium has long been THE tool for our end-to-end monitoring. Selenium provided us with a solid basis for automated tests and enabled us to monitor our customers' applications in detail. The broad browser support and the large developer community were definitely advantages that helped us in our work. However, we encountered difficulties in recording videos during the test runs. While Selenium can capture screenshots by default, capturing videos requires additional effort and complex configurations.
In other projects, we started using Playwright and found that it could hold a candle to Selenium in many ways. In particular, we were impressed by the speed at which Playwright runs tests. In addition, Playwright offers a built-in video recording function that could be seamlessly integrated into our test runs.
Given these benefits, we decided to rethink our approach to end-to-end monitoring. Below we will detail four reasons why we made this change and how this decision has impacted us.
Speed
A significant factor that influenced our switch to Playwright is the speed of test execution. In fact, some of our tests were twice as fast with Playwright than with Selenium. This increased speed allows us to perform more test runs per day, resulting in an increased number of results that we can provide to our customers. This helps to improve responsiveness to potential problems.
Video recordings
Another compelling reason for our switch to Playwright was the integrated support for video recording during test runs. In contrast to Selenium, where recording videos requires additional tools and complicated configurations, Playwright offers this function out-of-the-box. All you have to do is give the browser an option with the desired path to the video file:
BWith Selenium, it is recommended to map video recordings via a Selenium Grid. Selenium Grid is a tool that is part of the Selenium test framework and was specially developed to run automated tests in parallel on multiple machines and in different browser environments. There is a central server that receives test requests from clients and forwards them to the corresponding nodes. A node is an execution unit on which the tests run in the desired browsers. The sessions can be recorded with the help of Chrome instances, which are created as docker containers if required. Detailed instructions can be found here: https://medium.com/@trivedevops/dynamic-selenium-grid-cluster-deployment-and-video-recording-integration-c8f9c0695055
An alternative option to the Selenium Grid is to take many screenshots during the test and then compile them into a video.
To record videos using screenshots, FFMPEG must be installed on the computer.
For Mac/Linux with the help of brew:
brew update
brew upgrade
brew install ffmpeg
For Windows: https://phoenixnap.com/kb/ffmpeg-windows
To ensure that screenshots are taken continuously during the test, a corresponding function (here ‘ffmpegOutPrintScreenshots’) could be added to the logging. The individual images are then combined into a video using ffmpeg (here ‘stopRecording’):
Auto-waits & -checks
EA decisive advantage of Playwright are the integrated auto-waits and checks. These features make it easier to handle asynchrony and make our tests more robust and reliable. Playwright performs relevant checks automatically to ensure that elements are available and ready for interaction. In comparison, Selenium often requires the explicit specification of ‘ExpectedConditions’ to wait for elements before interactions are possible. Although this approach can offer advantages in certain situations, such as complex applications with dynamic content, Playwright's automatic handling of wait times and checks makes it much easier to implement and maintain our tests.
In Selenium, the ExpectedConditions ‘visibilityOf’ and ‘elementToBeClickable’ must be called explicitly:
Automatic scrolling to elements
EAnother compelling reason for our switch to Playwright is the auto-scroll feature that Playwright offers. Unlike Selenium, Playwright automatically scrolls to the items you are looking for, resulting in more robust testing. Auto-scrolling ensures that elements that are outside the visible area are automatically moved into the visible area of the browser window before interactions take place. This significantly reduces the likelihood of errors and flakiness in the tests. Although it is possible to implement scrolling manually in Selenium, this is often error-prone and leads to more unstable tests.
In Selenium, this behaviour can be implemented by first checking whether a particular element is in the current viewport (isElementInViewport ) of the browser and, if not, scrolling to the corresponding position (scrollIntoViewAndClick):
Conclusion
Overall, it can be said that both Selenium and Playwright have their own strengths and use cases. While Selenium has long been an established tool for browser automation testing and remains relevant for certain scenarios, our switch to Playwright for end-to-end monitoring has proven to be extremely worthwhile. End-to-end monitoring is fundamentally different from traditional functional testing, as it is less about testing individual functions and more about monitoring the performance of business-critical services from the user's perspective. In this context, Playwright proves to be superior, as its speed, in-built video recording capability and other advanced features are particularly relevant. Playwright's ability to implement automatic wait times, checks and auto-scrolling results in more robust and reliable tests that are better suited to monitoring services.
