Technical Committee (TC) Methods for Testing and Specification (MTS) Activity Report 2023

Chair: Philip Makedonski, Institut für Informatik – University of Göttingen

Identifying and defining advanced specification and testing methods, taking advantage of innovative techniques to improve the efficiency and economics of both the standard description and associated conformance and interoperability testing processes.

 Rapid advances in areas such as ICT, industrial automation, transportation, healthcare and biotechnology are placing ever increasing demands on testing methodologies and techniques.

 ETSI’s Methods for Testing and Specification committee (TC MTS) has primary responsibility for identifying and defining advanced specification and testing methods, taking advantage of innovative techniques to improve the efficiency and economics of both the standard description as well as associated conformance and interoperability testing processes.

 Providing frameworks and methodologies that enable other ETSI Technical Bodies to produce documents that are easy to understand and use, the committee’s activity is critical to the market success of many technologies.

 The work of TC MTS defines methodologies to develop standards and testing specifications in co-operation with other relevant ETSI Technical Committees. The applicability of existing international standards to the requirements of ETSI standardization is also considered in the definition of specification methods and languages.

 TC MTS is the steering committee for the definition of the methodologies for ETSI’s Centre for Testing and Interoperability (CTI). Together with CTI, TC MTS develops background material for the use within the CTI group. The committee also conducts field trials and pilot applications of new test methods to ensure they are ready for daily use at ETSI.

 This work sees TC MTS maintain active liaison exchange with other ETSI Technical Bodies and groups, as well as expert communities from ETSI Partners.

During 2023 TC MTS continued to evolve and maintain ETSI’s highly successful testing language, TTCN-3 (Testing and Test Control Notation version 3), along with its tool conformance test suites. This work has been complemented by further updates to our Test Description Language (TDL) that fills the gap between the simple expression of ‘what needs to be tested’ and the concrete coding of executable tests with existing test specification languages such as TTCN-3.

TTCN-3 Maintenance and Evolution

During 2023 TTFs T023 and T032 continued maintenance of the TTCN-3 language and corresponding test specifications. TC MTS has meanwhile been preparing a significant revision of the TTCN-3 language, and has submitted a proposal for a Testing Task Force to undertake this work, scheduled to start in mid-2024.

 Publications during the year included updates to various parts of the ETSI Standard ES 201 873 on TTCN-3:

These were complemented by updates to test specifications described in the multi-part TTCN-3 Conformance Test Suite TS 102 950:

TDL and TOP enhancement

During 2023 work focused on the TDL Open Source Project (TOP), with significant enhancements developed by TTF T022 to improve usability and enable compilation and execution of TDL test cases.

This resulted in the publication of updated versions of two parts of the ETSI Standard:

TTF T034 was established to continue this activity and define a runtime interface for TDL.

MTS TST WG

Within TC MTS, the Testing (TST) working group develops studies, guidelines, test catalogues and test specifications for specific ICT technologies that are not already covered by other ETSI groups. Testing can include – but is not limited to – conformance, interoperability, security and performance testing. In 2023, the TST WG completed work on two items relating to IoT security, resulting in the publication of:

  • TR 103 946 V1.1.1 Security validation of IoT architecture application and conformity - case study experiences
  • TS 103 942 V1.1.1 Security Testing; IoT Security Functional Modules

AI Testing

Reflecting the growing importance of Artificial Intelligence in the AI-related work items, work was launched during the year on a new Technical Specification – to be published as TS 104 008 – on the Continuous Auditing Based Conformity Assessment (CABCA) process for AI-enabled systems, including its definition, principles, tasks and benefits compared to traditional ‘point-in-time’ certifications. Further progress was also made on a Technical Report – to be published as TR 103 910 – on test methodology and test specification for AI-enabled systems.

In addition, 2023 saw the preparation of a TTF on harmonized documentation scheme for trustworthy AI. Such documentation will be necessary for AI systems undergoing upcoming conformity assessment schemes in the context of the EU AI Act and would in general increase confidence in the quality and capabilities of AI systems.

10th UCAAT Conference

Organized by TC MTS and the ETSI Events team, the 10th User Conference on Advanced Automated Testing (UCAAT) took place on 14-16 November 2023, hosted by Nokia in the Polytechnic Timisoara University, Romania. Featuring tutorials, keynotes and presentations, this well-established annual event addresses the practical challenges of testing and test automation faced by industry today.

See the full list of TC MTS Work Items currently in development here.