Technical Committee (TC) Methods for Testing and Specification (MTS) Activity Report 2021

Chair: Dirk Tepelmann, Spirent Communications

Responsible for the identification and definition of advanced specification and testing methods, taking advantage of formal approaches and innovative techniques to improve the efficiency and economics of both description of the standard and the development of associated test specifications.

Our Methods for Testing and Specification committee (TC MTS) creates standards and guides for testing and specification languages. Providing frameworks and methodologies that enable other ETSI committees to produce documents that are easy to understand and use, our work is critical to the market success of many technologies.

TC MTS is the steering committee for the definition of the methodologies for ETSI’s CTI (Centre for Testing and Interoperability). Together with CTI, TC MTS develops background material for the use within the CTI group. The committee also conducts field trials and pilot applications of new test methods to ensure they are ready for daily use at ETSI.

During 2021 TC MTS continued to evolve and maintain ETSI’s enormously successful testing language, TTCN-3, along with its tool conformance test suites. This work was complemented by further updates to our Test Description Language (TDL) that fills the gap between the simple expression of ‘what needs to be tested’ and the concrete coding of executable tests with existing test specification languages such as TTCN-3. This work notably saw finalization of groups of new conformance tests, addressing Core Standard test cases, XML test cases, Object Oriented extension test cases and JSON test cases.

Within TC MTS, the MTS Testing (TST) working group develops studies, guidelines, test catalogues and test specifications for specific ICT technologies that are not already covered by other ETSI groups. Testing can include – but is not limited to – conformance, interoperability, security, and performance testing.

In June TST announced completion of an initial set of standards addressing the testing of the IoT MQTT and CoAP protocols, and the foundational security IoT-Profile. These standards fill the gaps for the quality assessment of some of the most relevant communication protocols and system requirements of today’s industrial IoT systems using standardized testing techniques from ETSI.

The TS 103 596 series provides an overall test suite structure and catalogue of test purposes for the Constrained Application Protocol (CoAP). It can serve as a reference for both client-side test campaigns and server-side test campaigns. Part 1 specifies conformance issues, part 2 security issues and part 3 performance issues.

The TS 103 597 series provides an overall test suite structure and catalogue of test purposes for the MQ Telemetry Transport (MQTT). It is also divided into three different parts for conformancesecurity and performance testing.

TS 103 646 specifies testing for selected security requirements as known in the IEC 6244-4-2 standard. The chosen requirements have been collected by defining a dedicated IoT profile. The resulting IoT profile represents a generic minimum level of security for IoT devices.

All Test purpose catalogues were produced using the Test Description Language (TDL-TO) specified in ETSI Standard ES 203 119-4 (previously published in 2020). The TDL-TO library modules corresponding to the Test purpose catalogue are contained in an archive which accompanies the standards. The TDL-TO developments were achieved using the TDL Open Source Project (TOP), a free tool supported by ETSI members to write and validate TDL code.

See the full list of TC MTS publications in 2021 here.

See the full list of TC MTS Work Items currently in development in 2022 here.

Organized by TC MTS as a virtual event from 19-21 October, the 8th User Conference on Advanced Automated Testing (UCAAT) provided a platform for the testing community – including users, vendors, service providers and researchers from many different application domains – to share experiences and learn about the latest advances in the industrial use of test automation.

Titled ‘Testing to the Edge’, the 2021 event featured two and half days of tutorials, keynotes, and presentations. UCAAT is dedicated to the practical engineering and application aspects of automated testing, including experiences from the use of new techniques for test automation, advanced test automation processes, and standardized testing languages and methodologies for test automation in different application domains, such as IT services, telecommunications, transportation, healthcare, finance, and smart cities.