Technical Committee (TC) Methods for Testing and Specification (MTS) Activity Report 2022
Chair: Philip Makedonski, Institut für Informatik
Identifying and defining advanced specification and testing methods, taking advantage of innovative techniques to improve the efficiency and economics of both the standard description and associated conformance and interoperability testing processes.
Our Methods for Testing and Specification committee (TC MTS) has primary responsibility for identifying and defining advanced specification and testing methods, taking advantage of innovative techniques to improve the efficiency and economics of both the standard description and associated conformance and interoperability testing processes.
Providing frameworks and methodologies that enable other ETSI Technical Bodies to produce documents that are easy to understand and use, the committee’s activity is critical to the market success of many technologies.
The work of TC MTS to develop standards and testing specifications uses methodologies that are established in co-operation with other relevant ETSI Technical Committees. The applicability of existing international standards to the requirements of ETSI standardization is also considered in the definition of specification methods and languages.
TC MTS is the steering committee for the definition of the methodologies for ETSI’s CTI (Centre for Testing and Interoperability). Together with CTI, TC MTS develops background material for the use within the CTI group. The committee also conducts field trials and pilot applications of new test methods to ensure they are ready for daily use at ETSI.
During 2022 TC MTS continued to evolve and maintain ETSI’s enormously successful testing language, TTCN-3, along with its tool conformance test suites. This work has been complemented by further updates to our Test Description Language (TDL) that fills the gap between the simple expression of ‘what needs to be tested’ and the concrete coding of executable tests with existing test specification languages such as TTCN-3. This activity has expanded in recent years, and now embraces new conformance tests, addressing Core Standard test cases, XML test cases, Object Oriented extension test cases and JSON test cases.
Much of the committee’s work in 2022 was focused on a significant programme of publications, updating existing standards and specifications within the scope of Task 1 ‘TTCN-3 maintenance and further development’ and Task 2 ‘Conformance tests for TTCN-3 tools’ under TTF 014:
- ES 201 873-1 V4.14.1 - Part 1: TTCN-3 Core Language
- ES 201 873-5 V4.9.1 - Part 5: TTCN-3 Runtime Interface (TRI)
- ES 201 873-6 V4.13.1 - Part 6: TTCN-3 Control Interface (TCI)
- ES 201 873-7 V4.10.1 - Part 7: Using ASN.1 with TTCN-3
- ES 202 781 V1.9.1 - TTCN-3 Language Extensions: Configuration and Deployment Support
- ES 202 784 V1.9.1 - TTCN-3 Language Extensions: Advanced Parameterization
- ES 202 785 V1.9.1 - TTCN-3 Language Extensions: Behaviour Types
- ES 203 790 V1.4.1 - TTCN-3 Language Extensions: Object Oriented features
- TS 102 950-1 V1.10.1 - TTCN-3 Conformance Test Suite; Part 1: Implementation Conformance Statement (ICS)
- TS 102 950-2 V1.10.1 - TTCN-3 Conformance Test Suite; Part 2: Test Suite Structure and Test Purposes (TSS&TP)
- TS 102 950-3 V1.10.1 - TTCN-3 Conformance Test Suite; Part 3: Abstract Test Suite (ATS) and Implementation eXtra Information for Testing (IXIT)
- TS 103 253 V1.7.1 - Conformance test suite for using XML schema with TTCN-3; for use of XML and JSON schema: Implementation Conformance Statement (ICS)
- TS 103 254 V1.7.1 - Conformance test suite for using XML schema with TTCN-3 for use of XML and JSON schema; Test Suite Structure and Test Purposes (TSS&TP)
- TS 103 255 V1.7.1 - TTCN-3 Conformance Test Suite for use of XML and JSON schema: Abstract Test Suite (ATS) and Implementation eXtra Information for Testing (IXIT)
- TS 103 663-1 V1.3.1 - TTCN-3 Object Oriented extensions Conformance Test Suite - Part 1 : Implementation Conformance Statement
- TS 103 663-2 V1.3.1 - TTCN-3 Object Oriented extensions Conformance Test Suite - Part 2: Test Suite Structure & Test Purpose
- TS 103 663-3 V1.3.1 - TTCN-3 Object Oriented extensions Conformance Test Suite - Part 3: Abstract Test Suite & IXIT
TDL and TOP enhancement
Meanwhile TTF T013 has finalized the work of TDL and TOP, adding new features and maintaining existing TDL standards and their implementation within the TOP project. This resulted in the publication of 7 ETSI Standards together with an accompanying Technical Report:
- ES 203 119-1 V1.6.1 - Test Description Language; Part 1: Abstract Syntax and Associated Semantics
- ES 203 119-2 V1.5.1 - Test Description Language; Part 2: Graphical Syntax
- ES 203 119-3 V1.5.1 - Test Description Language; Part 3: Exchange Format
- ES 203 119-4 V1.5.1 - Test Description Language; Part 4: Structured Test Objective Specification
- ES 203 119-6 V1.3.1 - Test Description Language; Part 6: Mapping of TDL to TTCN-3
- ES 203 119-7 V1.3.1 - Test Description Language; Part 7: Extended Test Configurations
- ES 203 119-8 V1.4.1 - Test Description Language; Part 8: Textual Syntax
- TR 103 119 V1.3.1 - Test Description Language; Reference Implementation
MTS TST WG
Within TC MTS, the MTS Testing (TST) working group develops studies, guidelines, test catalogues and test specifications for specific ICT technologies that are not already covered by other ETSI groups. Testing can include – but is not limited to – conformance, interoperability, security and performance testing. Work was launched in May 2022 on two new Work Items covering:
- IoT Module Security Testing, Security Testing; IoT Security Functional Modules.
- Security validation of IoT architecture application and conformity Case Study Experiences.
During the year TC MTS finalized jointly with TC INT two Technical Reports on Artificial Intelligence (AI) in Test Systems and Testing of AI Models. The committee has also continued its participation in the OCG AI group.
Organized by TC MTS with the ETSI Events team, the UCAAT conference – titled ‘Testing of Trustworthy Systems’ – was hosted by Siemens in Munich from 13-15 September 2022. Featuring tutorials, keynotes and presentations, this well-established annual event addresses the practical challenges of testing and test automation faced by industry today. UCAAT is dedicated to all aspects of automated testing and is not limited in scope. Rapid technical advances in application areas such as information and communication technology, industrial automation, transportation (automotive, avionics, rail), enterprise information technology, healthcare, and biotechnology, as well as shifting operational landscapes such as DevOps practices and machine learning techniques applied to software development are ever increasing the demands on testing.
See the full list of TC MTS Work Items currently in development in 2022 here.