Start of a New Year
We thought we would do something a little different for our newsletter this quarter. We thought you might be interested in hearing from a vendor that has been through the Certification process. Learning from what they experienced, the benefits they received from the certification, and general feedback. With that stated I would like to introduce you to Thomas, an employee of Schneider Electric. I’ll let him describe himself and his experiences and thoughts.
“I work for Schneider Electric in a test lab dedicated to pre-certification and interoperability testing of industrial communication protocols. As part of my everyday job, I test Schneider Electric products. The lab offers testing for all fieldbus protocols used transversally in the Schneider Electric offer (Ethernet/IP, PROFINET, and Modbus). We started looking at testing our OPC UA-based product interfaces a number of years ago, as the demand for these interfaces increased.
When we started the process of having one of our products certified, we first reviewed what functionality the product was expected to support and made sure we understood what OPC UA-defined profiles the functionality is mapped to. For some features it was clear, but for some specific ones it required more effort, since the product was built using an SDK and some of the functionality was just inherent in the SDK.
Once we had a clean list of Profiles and optional Conformance Units, we downloaded the test tools provided by the OPC Foundation and began the process of validating the functionality. The CTT was not the easiest to configure, but once we figured out all of the configuration details, the tools were very helpful. We had to perform some manual testing, since not all tests could be automated. We sent the report of the CTT and manually executed test cases to the product development team who then identified areas of the product where we had to make corrections / updates. After the reported compliance issues were resolved by the developers, we tested our product against other Schneider Electric and third party products to ensure interoperability. The last steps of the internal pre-certification testing was validation of the behavior according to the test cases defined by the OPC Foundation and a 72-hrs efficiency test were the product is connected and exercised by a number of other OPC UA products.
We were still checking our product, but knew that the OPC Foundation Test Lab can be booked especially around shows and we wanted the certification available for a show. So as soon as we had identified all Profiles/Facets, we submitted an application with a “ready for testing date” a couple of months before the show. From our own experience we knew that testing would discover some issue(s) and we wanted to give R&D a chance to fix issues and update the version of the product in the lab and still have it certified for the show. The application form included information about the product, the supported functionality, contact information for testing and the company information for billing. The OPC Foundation returned with an estimate of the costs to test the product and when they could be able to start the testing. The estimate included all interfaces supported by the product, in this case both OPC UA Client and OPC UA Server.
Since I run our test lab, I wanted to see how the OPC Foundation Certification Test Lab operates. So even though it was not required, I travelled with my product to the OPC Foundation Certification Test Lab to observe the testing. The test lab was very friendly and relatively easy to get to. I was given an office I could work in when I had other calls or work that I needed to attend to. I found the tester very professional and a high-quality testing process. I answered some questions about the product, but in general I was not needed for the testing. I did spend time asking questions and talking about testing with the lab director. It is always good to learn more about how other labs operate. The lab had a large array of possible server products that my client could be exercised against, as well as a large array of the typical client products that my product would encounter in the field. A selection of these products was used in testing the interoperability of my product and also, they were used as part of the 72-hour efficiency test.
For the final result, the OPC test lab generates a product signature (hash of the software or firmware) that is included as part of the final report. This requires the final released software. So, after a brief suspension of testing (to allow for the final build to be generated), the test lab ran the final verification testing and extended-time tests. When testing was done and the product had passed, I received an email with detailed reports indicating the results of the testing. These reports include both Server and Client detailed test record results, a performance report, and a short summary report indicating that we had passed testing. The reports can include warnings or recommendations, (i.e. when something is ok to do, but may not be the best way to do something in OPC UA). The email also included the details of the test cost (in days of testing) and a logo license agreement that needed to be signed. Once the bill was paid and logo license was signed, I received an official certificate with a set of logos. The product listing on the OPC Foundation website was updated to indicate that it is a certified product.
The overall experience was positive. The test lab exposed some minor issue in our product, that we did not find. The level of expertise in OPC was very high and this high level of expertise is especially important when testing clients which have less automated testing. The only somewhat negative item, was that if an issue is raised about a test (not product), it can take a while. The issue might have to be discussed in the Compliance Working Group steering committee and/or scheduled for a discussion in the UA Working Group. Which might take several weeks, because it is a mostly volunteer-based standards group. Nevertheless, in the end, the issues were resolved. All other aspects of the testing went well, and we would bring the product back for the next cycle. The actual cost of the testing was also very reasonable and we feel it did improve our product.
I hope this write up of my experience with getting a product certified is useful.
–Thomas
(& Paul & Alex)