Challenges and considerations of IoT testing
Internet of things (IoT) vulnerabilities and their potentially catastrophic results make it imperative that IoT devices go through rigorous testing before seeing the light of day. But testing the security, performance, connectivity, and a number of things for millions of devices is no easy task. Not to mention the IOT in IoT, that is, the internet of things (IoT) problems that arise with interoperability testing (IOT).
Chris Riley from DevOps.com sites several challenges that arise when testing IoT devices:
- Replicating a fully-connected environment requires too much money and effort—and sometimes it’s simply inconceivable.
- The subsystems, subcomponents, and services that are interconnected are owned by different groups and third party vendors. Inability to access just one dependent subcomponent could impact the testing of the whole application.
- Considerable effort and coordination among different teams is needed to get the right test data populated across all the systems.
- Availability of the device only with limited capacity and at inconvenient times
- The problem of testing for the IoT raises issues of responsibility. Who is at fault if the device in your vehicle starts emitting erroneous data that causes On-Board diagnostic connector to report non-existent events that destabilize the control loops used by different parts of the vehicle, causing a failure that leads to an accident? Is it the vendor of the device or the controller for being unable to cope with extraneous events? Your customers will hate if you use classic test approach.
Riley’s first point is particularly intriguing. Thousands of devices need to be tested at one time, often in adverse environments: underground, underwater, in freezing temperatures, etc. The ability to simulate those situations will give IoT developers a leg up when it comes time for implementation. But even once it is determined a device is functioning, testers must make sure a number of different long-term elements are in place. With several million sensors and different types of devices connected to intelligent software, functionality validation is not enough in an environment as complex as IoT.
Sunil Sehgal, Managing Partner at TechArcis Solution, gave a list on LinkedIn of quality assurance (QA) tests done to IoT implementations:
- Usability / User Experience
- Connectivity
- Security
- Identity and Authentication
- Data Encryption
- Data Protection
- Data Security storage in cloud environment
- Interoperability
- Compatibility
- Performance
- Automation
All the different aspects that make up an IoT system, from Big data to long term sustainability, make testing that much more complex. But there are tools and strategies to help combat these complexities.
Approaches to solving problems
One way to do this is by using a devops and agile approach, which can help accelerate the testing process. Virtualization allows testers to be less dependent on test infrastructure, offering the all-important interconnectivity.
Additionally, using a platform-based approach allows companies to construct a system that requires interoperability between platform elements and places a high value on software—the ultimate source of “smartness” in test system design, according to National Instruments. A platform-based approach is a single, flexible hardware architecture deployed across many applications, which removes hardware complexity and makes each new problem primarily a software challenge.
National Instrument’s LabView RIO devices combine a microprocessor and a field-programmable gate array (FPGA), and the end user uses this to operate off-the-shelf measurement or control hardware.
“For example, a company like Airbus, that is controlling a motor or control system – that system would be running on dedicated hardware,” said Jamie Smith, NI director of embedded systems product marketing to IoT Journal. But [in the past] if the customer wanted to use the same control system in a brand-new style of actuator [for a different factory application], they would need to redesign the hardware. [With LabVIEW] they can use [our] same piece of electronics and reconfigure it without having to redesign the hardware.”
Another important aspect of IoT testing is the use of human test subjects. There is simply no better way to test a device than putting it in the same scenarios it will face when officially deployed. Test subject could be patients to monitor or test a health device. It could be an animal to test a bio-chip, or a driver to test a particular automobile.
Hardware problems with software solutions
There is almost always a software solution to help expedite the process of creating and releasing a product, and that is no exception to IoT solutions. Where National Instruments both hardware in the form of sensors and actuators as well as its LabView software, companies like Applause, SmartBear and TechArcis have testing and quality assurance software designed specifically for the internet of things.