At Accedo, we provide Quality Assurance (QA), User Experience Design (UX) and strategic consulting services throughout the entire video experience lifecycle of each of our customers. In a series of posts, we will be taking a deep dive into each service offering, starting with Xuan Xu, Director Quality Assurance, who explains why test automation has become the bedrock for ensuring a consistent and trusted experience across multi-screen video applications.
We all know video experience has been changing rapidly and the video industry is projected to be one of the fastest-growing industries of the next decade.
Video was once confined to a single screen – the linear TV, but now people are using more and more screens . It happens in our own lives, either personally or professionally. So today it is so fragmented across multiple screens. Users expect information on all screens and also across multiple channels. They also expect everything should be working across devices. Apart from device, there are also content, screen, technology, and data fragmentations.
When we are talking about multi-screen application testing, it means massive platforms to test, i.e. mobile, web, tablet, desktop, smart TV, game console, STB (set top box), etc., not to mention the various screen resolutions, chipsets (leading to different performance), the operating systems and versions, and even models in different regions.
How do you ensure that users can experience the same video experience in this fragmental world? As an operator, do you expect your service can be consumed by end users via all kind of devices? To the testing world, this just adds another dimension or complexity of software testing in this industry, there are still extensive testing scopes such as performance testing, security testing, integration testing, compliance testing, and so forth.
Manual testing is performed by a human sitting in front of a couple of devices, following test cases, carefully navigating the application UI, trying various possibilities and input combinations, comparing the results to the expected behaviour and reporting their observations.
Manual tests are still essential during application development cycles, or agile sprint cycle as testers should verify the functionality and also understand how it has been implemented.
Obviously there are the limitations of manual testing for multi-screen applications:
- Manual testing cannot achieve good coverage of the fragmentations, i.e. devices, OS versions, as a human has only two hands.
- Manual testing can not scale, for example, full regression testing for one medium size video application might take a couple of days, and that is not counting full device coverage.
- Manual testing relies heavily on tester behavior or experience, even the same test case execution might lead to different results
Test automation has been getting more and more popular for software development and testing. Automated testing is able to compare the results to the expected behaviour and report the results directly. Additionally, it can run through same test cases crossing multiple platforms and devices, again and again.
Let’s take a look what test automation can bring to multi-screen application development and testing. And also what kind of automated testing is required by this industry.
Cost saving and time to market
Test automation has been recognised as critical for big software development companies in the past, but is often thought to be expensive or difficult for smaller companies to implement. Most of the video application development has been updated with new features in a very fast pace. It is time sensitive but also expect quality, which means many of them require regression testing.
Automated testing can reduce the time to spend on regression testing and significant time-saving that translates directly to cost saving from a long term perspective. However, cost is sensitive to many smaller companies from a short term view, efficiency in automation development will be always a consideration.
Test framework, industry standard, to afford the technology evolving
The wide variety of technologies and styles of testing make creating a single, universal test tool impossible. It makes more sense to have a test automation framework which can unify test cases, test reports and also can be extended with bundled and self-made test libraries.
Testers are usually less-technical at coding compared to developers. The way to motivate them to contribute with test automation is to find the less-coding and more efficient tools or approaches, to speed up the entire automation effort.
The modular test automation framework is a wrapper between test cases development and the low level industry libraries, which is extendible such as Selenium, Appium, REST API, STB-testers.
Moving to the cloud
Moving test environments to the cloud provides additional benefits and agility to test automation.
- Traditionally, manual testers must work with physical devices in the local office. However, if the automation environment is in cloud, testers can trigger the test job anywhere, anytime and get the result available and visible by everyone.
- Deploying automation environments to the cloud means you can build up the environment from templates, bring up the environment, execute automation tests and tear down the environment once done.
- Cloud-based workflow also helps to make updates in real time and gives testers full visibility of their collaborations, especially to a global team.
Once automated tests are created, they can easily be repeated and also extended to perform some tasks which are impossible with manual testing. Thanks to a good understanding and sense of quality, Accedo has spent more than 5 years investing in an in-house multi-screen application test automation solution, Accedo Qbots™, which we believe fulfills the requirement of multi-screen application automation, and can benefit our customers as well as consumers.
What is Accedo Qbots™?
Accedo Qbots™ can black-box test mobile, web, smart TV/STB app as well as backend API, completely unmodified, integrated with industry-standard libraries, and Accedo AppGird™.
- Adapt for multi-screen video applications, real devices and complete end-to-end testing.
- Integrated with Accedo AppGrid™, complete cloud solution.
- Business driven framework, and quick to develop, deploy and extend.
Currently supported platforms(multi-screen) of Accedo Qbots™ include
- Roku Box
- Apple TV
Certainly there are many backend technologies are supported for all multi-screen application end-to-end testing, such api, database, remote, screen capturing, etc. And in the Qbots roadmap, it is going to support even more platforms, including Samsung Smart TV, Playstation, Xbox and so on.
To see Qbots in action, stop by our booth (14:E14) at IBC 2016.