Speech and multimedia Transmission Quality (STQ) Testing

Introduction

Since years, several test campaigns have been designed to measure, analyze and compare speech quality parameters for speech terminal equipment, in narrowband (up to 3.4 kHz) and more recently wideband (up to 7 kHz).

For IP terminals and gateways all conversational aspects, including speech sound quality, echo measurements, double talk performance and the transmission quality in the presence of background noise, are considered.

An important characteristic of these Plugtests™ events or test events is that quality is considered from the user's point of view, including the acoustic interfaces for the speech terminal. As traditional speech communication evolves into innovative services such as multimedia conferencing, and as new means of delivery are introduced (e.g. VoIP), verification of acceptable levels of quality is vital. The distinctive Plugtests™ approach implies specific environments and processes; through a special analysis of test logs and diagnosis of the equipment behaviour, it also generates information that is useful for system optimization. The approach significantly reduces the time needed for manufacturers to develop and test their products, thus both speeding up their availability and improving their quality. Each participating manufacturer receives all his individual results, including detailed information about potential improvements, and is able to compare these results with those of all the other participants which are published in an anonymous test report at the end of the event.

For the test events implemented for multiplay services, the most of the tests are done without the acoustical interface for speech quality and without the audio/video terminal for IPTV. However these tests are done using actual offerings from ISPs.

Our Role & Activities

Our role is both to define the test methodologies and the relevant test set-ups to be implemented for the test events.

The standards produced by TC STQ are implemented for these plugtests/ events, giving first the opportunity to manufacturers not equipped for these kinds of innovative test equipments to become acquainted with, and trained in, these new processes and new standards.

The results of the tests provide, through anonymous reports, a feedback on the real performance of equipments or services. This may consolidate the requirements available in the standards or may provide change requests for revised versions of the standards.
It should also be needed in some cases to integrate quality tests with interoperability events, because from the user's point of view the expectation is to have not only interoperable services but also offering a good QoS/QoE. 

Several documents from STQ are available:

For Plugtests / Speech Quality Test Events

TR 102 648-Series: Test methodologies for ETSI test events and results
Part 1: VoIP Speech Quality Testing
Parts 2, 3,…: ETSI Plugtests Speech Quality Test Event Reports

For Plugtests/events for multiplay services

TR 102 716 Series: Guidelines, objectives and results of speech quality analysis in the context of interworking plugtests for multiplay services
Part 1: Guidelines and objectives
Part 2: Results

Standards

The following is a list of the 20 latest published ETSI standards on STQ testing.

A full list of related standards in the public domain is accessible via the ETSI standards search. Via this interface you can also subscribe for alerts on updates of ETSI standards.

For work in progress see the ETSI Work Programme on the Portal

Standard No. Standard title.
TR 103 138 Speech and multimedia Transmission Quality (STQ); Speech samples and their use for QoS testing
TS 103 296 Speech and Multimedia Transmission Quality (STQ); Requirements for Emotion Detectors used for Telecommunication Measurement Applications; Detectors for written text and spoken speech
TS 102 250-2 Speech and multimedia Transmission Quality (STQ); QoS aspects for popular services in mobile networks; Part 2: Definition of Quality of Service parameters and their computation
TS 103 222-3 Speech and multimedia Transmission Quality (STQ); Reference benchmarking, background traffic profiles and KPIs; Part 3: Reference benchmarking, background traffic profiles and KPIs for UMTS and VoLTE
TS 103 222-4 Speech and multimedia Transmission Quality (STQ); Reference benchmarking, background traffic profiles and KPIs; Part 4: Reference benchmarking for IPTV, Web TV and RCS-e Video Share
TS 103 222-1 Speech and multimedia Transmission Quality (STQ); Reference benchmarking, background traffic profiles and KPIs; Part 1: Reference benchmarking, background traffic profiles and KPIs for VoIP and FoIP in fixed networks
TR 102 581 Speech Processing, Transmission and Quality Aspects (STQ); A Study on the Minimum Additional Required Attenuation on the Antenna Path of the Field Test Equipment
TS 102 929 Speech and multimedia Transmission Quality (STQ); Procedures for the identification and selection of common modes of de-jitter buffers and echo cancellers
TS 101 563 Speech and multimedia Transmission Quality (STQ); IMS/PES/VoLTE exchange performance requirements
EG 202 396-3 Speech and multimedia Transmission Quality (STQ); Speech Quality performance in the presence of background noise; Part 3: Background noise transmission - Objective test methods
TS 102 250-5 Speech and multimedia Transmission Quality (STQ); QoS aspects for popular services in mobile networks; Part 5: Definition of typical measurement profiles
TS 103 224 Speech and multimedia Transmission Quality (STQ); A sound field reproduction method for terminal testing including a background noise database
TS 102 250-3 Speech and multimedia Transmission Quality (STQ); QoS aspects for popular services in mobile networks; Part 3: Typical procedures for Quality of Service measurement equipment
TR 101 578 Speech and multimedia Transmission Quality (STQ); QoS aspects of TCP-based video services like YouTubeTM
ES 202 396-1 Speech and multimedia Transmission Quality (STQ); Speech quality performance in the presence of background noise; Part 1: Background noise simulation technique and background noise database
TS 102 250-2 Speech and multimedia Transmission Quality (STQ); QoS aspects for popular services in mobile networks; Part 2: Definition of Quality of Service parameters and their computation
TR 103 219 Speech and multimedia Transmission Quality (STQ); Quality of Service aspects of voice communication in an LTE environment
TR 103 138 Speech and multimedia Transmission Quality (STQ); Speech samples and their use for QoS testing
ES 202 737 Speech and multimedia Transmission Quality (STQ); Transmission requirements for narrowband VoIP terminals (handset and headset) from a QoS perspective as perceived by the user
ES 202 738 Speech and multimedia Transmission Quality (STQ); Transmission requirements for narrowband VoIP loudspeaking and handsfree terminals from a QoS perspective as perceived by the user