{"id":3040,"date":"2020-06-16T13:27:38","date_gmt":"2020-06-16T12:27:38","guid":{"rendered":"https:\/\/www.brainbox.consulting\/?p=3040"},"modified":"2020-08-27T09:32:07","modified_gmt":"2020-08-27T08:32:07","slug":"the-value-of-testing-in-internet-of-things-iot","status":"publish","type":"post","link":"https:\/\/brainbox.consulting\/talent\/the-value-of-testing-in-internet-of-things-iot\/?lang=nl","title":{"rendered":"The value of Testing in Internet of Things (IoT)"},"content":{"rendered":"<h1><strong>What is IoT?<\/strong><\/h1>\n<p><span data-preserver-spaces=\"true\">The Internet of Things, known as IoT, refers to the collection of physical devices that are connected to the internet, gathering and sharing data.<\/span><\/p>\n<p><span data-preserver-spaces=\"true\">These devices can sometimes talk to other related devices and act on the information shared, a process called machine-to-machine (M2M) communication. Humans can also interact with these devices to set them up, to give them instructions, to gather information or access data.<\/span><\/p>\n<p><span data-preserver-spaces=\"true\">The IoT involves extending Internet connectivity to beyond standard devices; like desktops, laptops, smartphones or tablets. This includes everything from cellphones, coffee makers, washing machines, headphones, lamps, wearable devices and almost anything else you can think of.<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-2461 alignleft\" src=\"https:\/\/www.brainbox.consulting\/talent\/wp-content\/uploads\/2020\/04\/The-IoT-image-300x199.png\" alt=\"\" width=\"537\" height=\"356\" srcset=\"https:\/\/brainbox.consulting\/talent\/wp-content\/uploads\/2020\/04\/The-IoT-image-300x199.png 300w, https:\/\/brainbox.consulting\/talent\/wp-content\/uploads\/2020\/04\/The-IoT-image-768x510.png 768w, https:\/\/brainbox.consulting\/talent\/wp-content\/uploads\/2020\/04\/The-IoT-image.png 1024w, https:\/\/brainbox.consulting\/talent\/wp-content\/uploads\/2020\/04\/The-IoT-image-391x260.png 391w\" sizes=\"(max-width: 537px) 100vw, 537px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: left;\">Source:\u00a0\u00a0<a href=\"https:\/\/www.iso.org\/files\/live\/sites\/isoorg\/files\/archive\/Ref2112\/ref2112_infography_iot.pdf\"><span data-preserver-spaces=\"true\">iso<\/span><\/a><span data-preserver-spaces=\"true\">\u00a0<\/span><\/p>\n<p>&nbsp;<\/p>\n<h2 style=\"text-align: left;\"><strong>Building Blocks of IoT\u00a0<\/strong><\/h2>\n<ul>\n<li style=\"text-align: left;\"><span data-preserver-spaces=\"true\">Physical devices\/Sensors\u00a0<\/span><\/li>\n<li style=\"text-align: left;\"><span data-preserver-spaces=\"true\">Gateway\u00a0<\/span><\/li>\n<li style=\"text-align: left;\"><span data-preserver-spaces=\"true\">Cloud\u00a0<\/span><\/li>\n<li style=\"text-align: left;\"><span data-preserver-spaces=\"true\">Analytics (optional)\u00a0<\/span><\/li>\n<li style=\"text-align: left;\"><span data-preserver-spaces=\"true\">Use Interface (Mobile\/Web Applications<\/span><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-2455 alignnone\" src=\"https:\/\/www.brainbox.consulting\/talent\/wp-content\/uploads\/2020\/04\/Building-Blocks-of-IoT-300x73.png\" alt=\"\" width=\"691\" height=\"168\" srcset=\"https:\/\/brainbox.consulting\/talent\/wp-content\/uploads\/2020\/04\/Building-Blocks-of-IoT-300x73.png 300w, https:\/\/brainbox.consulting\/talent\/wp-content\/uploads\/2020\/04\/Building-Blocks-of-IoT-768x187.png 768w, https:\/\/brainbox.consulting\/talent\/wp-content\/uploads\/2020\/04\/Building-Blocks-of-IoT-1024x249.png 1024w, https:\/\/brainbox.consulting\/talent\/wp-content\/uploads\/2020\/04\/Building-Blocks-of-IoT.png 1590w\" sizes=\"(max-width: 691px) 100vw, 691px\" \/><\/p>\n<h2><strong>How IoT works?<\/strong><\/h2>\n<p><span data-preserver-spaces=\"true\">An IoT system consists of smart devices that are web-enabled and use embedded systems, such as sensors, processors, hardware for communication and software applications, to collect, send and act on the data they gather from their environment. IoT sensors share the data they gathered by connecting to an IoT gateway from which the data is sent to the cloud to store or to be analyzed or to take certain action depending on the situation.<\/span><\/p>\n<p><span data-preserver-spaces=\"true\">IoT devices can communicate with other connected\/related devices, share the information among them and react to it as necessary. Users can also interact with these smart devices to access data or give them a command to perform certain actions.<\/span><\/p>\n<p><span data-preserver-spaces=\"true\">Artificial Intelligence (AI) and Machine Learning are widely being used in IoT today to make the data available to users according to their personal preferences, thus improving the efficiency of IoT devices and enhancing user experience.<\/span><\/p>\n<p>&nbsp;<\/p>\n<h2><strong><span data-preserver-spaces=\"true\">What is IOT Testing?<\/span><\/strong><\/h2>\n<p><span data-preserver-spaces=\"true\">IoT Testing means checking\/testing of IoT devices. There is an increasing need to deliver better and faster services and a huge demand to access, create, use and share data from any device. The thrust is to provide greater insight and control, over various interconnected IoT devices.\u00a0<\/span><\/p>\n<p><span data-preserver-spaces=\"true\">IoT brings many new things to the table in terms of testing requirement, including quite a few unusual device types like home appliances, watch sensors etc.\u00a0<\/span><\/p>\n<p><span data-preserver-spaces=\"true\">With such an eclectic mix of products in the IoT ecosystem, it becomes imperative for the QA team to come with an effective test strategy to meet the quality of these diverse products. Hence, the IoT testing framework is important.<\/span><\/p>\n<p>&nbsp;<\/p>\n<h2><strong><span data-preserver-spaces=\"true\">Why testing is important in IoT?<\/span><\/strong><\/h2>\n<p><span data-preserver-spaces=\"true\">IoT is a rapidly developing market. Count of devices and software in IoT is increasing day-by-day. According to Gartner, approximately 20.4 billion devices get connected by the end of 2020. Delivering robust, high-quality IoT solutions quickly to the market is a key requirement for businesses.<\/span><\/p>\n<p><span data-preserver-spaces=\"true\">IoT system involves different devices, applications and data systems which makes it very complex and each of these components has different requirements for testing. The lack of interoperability between these components can cause issues. The technology companies develop solutions independent of each other, using different platforms and this may result in issues in the integration of components with one another.<\/span><\/p>\n<p><span data-preserver-spaces=\"true\">End-users benefit from increasing connectivity of devices, but this, in turn, adds to the complexity in development and testing of IoT solutions.\u00a0<\/span><\/p>\n<p><span data-preserver-spaces=\"true\">Hence it is essential to implement a robust testing process that covers not only the device but also the interaction of the device with the cloud and mobile applications (including all the integrations), to ensure the product is functioning as intended.\u00a0<\/span><\/p>\n<h2><strong>Iot Testing Challenges:<\/strong><\/h2>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-2457 alignleft\" src=\"https:\/\/www.brainbox.consulting\/talent\/wp-content\/uploads\/2020\/04\/Challenges-in-IoT-300x244.png\" alt=\"\" width=\"432\" height=\"351\" srcset=\"https:\/\/brainbox.consulting\/talent\/wp-content\/uploads\/2020\/04\/Challenges-in-IoT-300x244.png 300w, https:\/\/brainbox.consulting\/talent\/wp-content\/uploads\/2020\/04\/Challenges-in-IoT.png 693w\" sizes=\"(max-width: 432px) 100vw, 432px\" \/><\/p>\n<h3><strong>Platform Diversity\u00a0<\/strong><\/h3>\n<p>IoT devices use different protocols such as as Message Queuing Telemetry Transport (MQTT), Extensible Messaging and Presence Protocol (XMPP), Constrained Application Protocol (CoAP) among others to communicate with controllers and each other. Hence it is required to test them across various protocols to ensure the devices work seamlessly.<\/p>\n<h3><strong>Cloud Diversity<\/strong><strong>\u00a0<\/strong><\/h3>\n<p>IoT ecosystem can use different cloud platforms to send and collect data and to connect different components of the system. Some of them are Azure IoT, IBM Watson and AWS. So it is imperative to test IoT devices across these cloud platforms to guarantee their usability.<\/p>\n<h3><strong>Security and Privacy<\/strong><\/h3>\n<p>The amount of data collected and shared by the connected devices in an IoT system is huge. The concern of data leak and unauthorized access rise due to this high volume of data sharing. So it becomes critical and of high priority to identify the vulnaribilities and address them.<\/p>\n<p>&nbsp;<\/p>\n<h3><strong>Real time data velocity<\/strong><strong>\u00a0<\/strong><\/h3>\n<p>Connected IoT devices need fast communication channel and this is ensured by good network status. The devices can face problems due to slow internet connection, bad network infrastructure\u00a0or problems with network hardware, affecting their performance. So IoT devices and applications must be tested under these different conditions to avoid any loss of data if in case any of the above issue appear.<\/p>\n<h3><strong>Interoperability\u00a0<\/strong><\/h3>\n<p>Interoperability between components of an IoT system like physical device, gateway, cloud and application is a key challenge. Each IoT device has its own hardware and application software. Th software interacts with hardware to issue commands or analyze the data collected by the device. Since there are so many variants of hardware and software, as well as different versions of firmware and operation systems, it is a real challenge to test all the combinations and interoperability.<\/p>\n<h3><strong>Scalability<\/strong><\/h3>\n<p>IoT solution deployment entails thousands of interconnected devices, which connect to servers over network. Server infrastructre is built from applications from different vendors and on multiple interconnected services. Testing such complex environment and simulating real time situations is a challenge.<\/p>\n<h3><strong>User experience\u00a0<\/strong><\/h3>\n<p>A consistent and hassle free user experience is most important for any IoT solution. Hence to test the user friendliness of the IoT devices and applications across all platforms of mobile, desktop or any other smart device is very critical and also challenging.<\/p>\n<p>&nbsp;<\/p>\n<h2><strong>IoT testing strategy<\/strong><\/h2>\n<h3><strong><img decoding=\"async\" loading=\"lazy\" class=\"wp-image-2459 alignleft\" src=\"https:\/\/www.brainbox.consulting\/talent\/wp-content\/uploads\/2020\/04\/IoT-testing-strategies-300x244.png\" alt=\"\" width=\"434\" height=\"353\" srcset=\"https:\/\/brainbox.consulting\/talent\/wp-content\/uploads\/2020\/04\/IoT-testing-strategies-300x244.png 300w, https:\/\/brainbox.consulting\/talent\/wp-content\/uploads\/2020\/04\/IoT-testing-strategies.png 624w\" sizes=\"(max-width: 434px) 100vw, 434px\" \/><\/strong><\/h3>\n<h3><strong>Device-level testing\u00a0<\/strong><\/h3>\n<ul>\n<li>Device Communication:<\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">Testing should be done to verify that the data transmitted between the device and gateway is linear and properly acknowledged. If the communication between them doesn\u2019t work properly, there may be a chance of data loss.<\/span><\/p>\n<ul>\n<li>Connectivity:<\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">Connectivity test validates both wired and wireless network configuration of the smart device and it can switch between different networks and connect to different security settings.<\/span><\/p>\n<ul>\n<li>OTA\/Upgrade:<\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">Tests should be done to ensure the device performs as expected without any data loss when a new software or hardware upgrade occurs. Also, it should be tested to make sure that they can be downgraded to previous versions if necessary.<\/span><\/p>\n<h3><\/h3>\n<h3><strong><span data-preserver-spaces=\"true\">Cloud level testing\u00a0<\/span><\/strong><\/h3>\n<p><span data-preserver-spaces=\"true\">Most of the test at cloud level (Unit\/Integration\/System and Performance ) is API based and are easily automated. These test should be part of continues integration system to provide early feedback.<\/span><\/p>\n<p>Performance\/Scalability\/Reliability:\u00a0\u00a0The two most important aspect of the cloud are:<\/p>\n<ul>\n<li><span data-preserver-spaces=\"true\">How quickly it responds to the user actions\u00a0<\/span><\/li>\n<li><span data-preserver-spaces=\"true\">How well it scales to accommodate user demand\u00a0<\/span><\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">So testing should include these aspects. Example of load test would be to simulate a million users accessing the service at once and observing how the service responds.\u00a0\u00a0<\/span><\/p>\n<p><span data-preserver-spaces=\"true\">o What\u2019s the maximum level of latency that occurs if you scale from a thousand to a million units?\u00a0\u00a0<\/span><\/p>\n<p><span data-preserver-spaces=\"true\">o If the servers are resilient enough, the customer should experience at most a \u201cglitch,\u201d but should still be able to continue using the connected device.\u00a0<\/span><\/p>\n<ul>\n<li>Security<\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">The purpose of the security test is to make sure the data flowing through the cloud to all its endpoints is protected. This is particularly important for the devices that collect and store sensitive information. One way to test cloud security is through penetration testing. This involves locating vulnerabilities at data paths to ensure that the firewalls, and other security measures, are set up correctly and doing their job.\u00a0<\/span><\/p>\n<ul>\n<li><span data-preserver-spaces=\"true\"> \u00a0 \u00a0Data Integrity:\u00a0<\/span><\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">IoT is all about data sharing, hence test should be designed to assure that the data remains consistent through data creation to data consumption\/aggregation.<\/span><\/p>\n<h3><strong><span data-preserver-spaces=\"true\">Application-level testing<\/span><\/strong><\/h3>\n<ul>\n<li>\u00a0\u00a0Performance Testing:<\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">Performance testing is necessary to determine the scalability, reliability, stability and responsiveness of the application. Without this test, the software may suffer from issues such as slowness, crashing or inconsistent behaviour across different platforms or when under load.<\/span><\/p>\n<ul>\n<li>\u00a0Compatibility Testing<\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">Due to diversity in mobile devices, browsers, operating systems and platforms, a compatible test should be designed to check if the application behaves as expected in all the conditions.\u00a0<\/span><\/p>\n<ul>\n<li>Usability<\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">The Usability testing is done to measure the user\u2019s ease of handling and experience of the application.<\/span><\/p>\n<ul>\n<li>Connectivity<\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">Since the application will be using the internet, it is necessary to test its behaviour when it changes between the network, when there is a loss of connectivity or when there is a slow\/intermittent network. Also, it should be tested on different network speeds\/types like 2G, 3G, LTE or WiFi etc.<\/span><\/p>\n<ul>\n<li>False tolerance tests:<\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">These are negative tests that check abnormality of the application when subjected to undesirable conditions.<\/span><\/p>\n<p><span data-preserver-spaces=\"true\">Some of the common false tolerance tests are when the application is left idle for a prolonged period, kill and relaunch the application etc.<\/span><\/p>\n<h3>Value-added testing<\/h3>\n<ul>\n<li><span data-preserver-spaces=\"true\"> Exploratory Test\u00a0<\/span><\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">Exploratory testing is an unscripted testing technique used to discover unknown issues. The learning from exploratory tests provides good feedback in designing the future regression test suite.\u00a0<\/span><\/p>\n<ul>\n<li>Field testing:<\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">Generally, the final step is field testing. When you have a product you believe is fully functioning and ready for customers, test it among a small group of 10-20 users in each region you plan to deploy a product.<\/span><\/p>\n<ul>\n<li>Regulatory and conformance:<\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">A well tested IoT product may also require the right certifications to enter the market. IoT devices generally have to meet various certification requirements for the network, protocol compliance, device drivers, app store submissions, etc.<\/span><\/p>\n<h3><strong><span data-preserver-spaces=\"true\">Test tools and Frameworks\u00a0<\/span><\/strong><\/h3>\n<p><span data-preserver-spaces=\"true\">To test complex IoT solutions effectively and execute a wide range of tests mentioned above, the right set of automation, simulation, virtualization, monitoring tools\/frameworks are vital.\u00a0<\/span><\/p>\n<p><span data-preserver-spaces=\"true\">Some of the supported tools\/frameworks are listed below:<\/span><\/p>\n<ul>\n<li>Device\/Protocol Simulators:<\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">IoT simulators allow you to design, create and test IoT applications and devices without actually using real IoT boards.\u00a0<\/span><\/p>\n<ul>\n<li>Mobile\/Web application Testing tools:<\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">They provide automated functional mobile testing that replicates end-user experience and ensures that the application works as expected.\u00a0<\/span><\/p>\n<p><span data-preserver-spaces=\"true\">Commonly used tools are Appium, Selenium, SeeTest, Test Complete, Kobition, Appuim Studio.\u00a0<\/span><\/p>\n<ul>\n<li>Security testing tools:<\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">They can be classified into threat modelling, static code analysis and runtime threat inducing tools.\u00a0\u00a0<\/span><\/p>\n<p><span data-preserver-spaces=\"true\">Tools such as Micro Focus\u00ae Fortify on Demand, OWASP ZAP, VGC and Microsoft\u00ae. Threat Modelling Tool identifies threats, prioritize them and also provide recommendations on how to fix them.\u00a0<\/span><\/p>\n<ul>\n<li>API testing tools:<\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">Most cloud solutions are now built using REST APIs and Web services. Tools such as Postman, SoapUI, Progress\u00ae Telerik\u00ae Fiddler\u2122, etc. test their connectivity, response, and performance.\u00a0<\/span><\/p>\n<ul>\n<li>Performance testing tools:<\/li>\n<\/ul>\n<p><span data-preserver-spaces=\"true\">Some of the commonly used tools for application performance testing are Jmeter, Blazemeter, Neoload, Loadrunner, etc.\u00a0<\/span><\/p>\n<p>&nbsp;<\/p>\n<h2><strong><span data-preserver-spaces=\"true\">Conclusion<\/span><\/strong><\/h2>\n<p><span data-preserver-spaces=\"true\"><strong>IoT<\/strong> involves the interconnection of multiple components. This makes the interaction and compatibility between the components highly complex. So a good testing strategy with\u00a0<\/span><\/p>\n<p><span data-preserver-spaces=\"true\">emphasis on Test-As-A-User (TAAS) approach and use of appropriate tools, can ensure quality and robust IoT solution to be delivered.<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><span data-preserver-spaces=\"true\">Contact us at\u00a0<a href=\"mailto:sales@brainbox.consulting\">sales@brainbox.consulting<\/a>\u00a0for more information and we would be happy to assist you with IoT testing.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Codeless test automation is creating automated tests which allows you to test an application without writing a single piece code\/script. This approach has been steadily evolving with constant innovation on this technology.<\/p>\n","protected":false},"author":8,"featured_media":2473,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_mi_skip_tracking":false},"categories":[23],"tags":[],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/brainbox.consulting\/talent\/wp-json\/wp\/v2\/posts\/3040"}],"collection":[{"href":"https:\/\/brainbox.consulting\/talent\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/brainbox.consulting\/talent\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/brainbox.consulting\/talent\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/brainbox.consulting\/talent\/wp-json\/wp\/v2\/comments?post=3040"}],"version-history":[{"count":3,"href":"https:\/\/brainbox.consulting\/talent\/wp-json\/wp\/v2\/posts\/3040\/revisions"}],"predecessor-version":[{"id":3043,"href":"https:\/\/brainbox.consulting\/talent\/wp-json\/wp\/v2\/posts\/3040\/revisions\/3043"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/brainbox.consulting\/talent\/wp-json\/wp\/v2\/media\/2473"}],"wp:attachment":[{"href":"https:\/\/brainbox.consulting\/talent\/wp-json\/wp\/v2\/media?parent=3040"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/brainbox.consulting\/talent\/wp-json\/wp\/v2\/categories?post=3040"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/brainbox.consulting\/talent\/wp-json\/wp\/v2\/tags?post=3040"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}