9th UCAAT - User Conference on Advanced Automated Testing

UCAAT 2022  AGENDA (13-15 September)

• 13 September: Pre-Conference Tutorials and Conference Day 1
•  

   

  PRE-CONFERENCE TUTORIALS

  09.00 - 12.30: TUTORIAL 1: Bughunt! An interactive workshop in Model-Based Testing with the Axini Modeling Platform
  Machiel Van der Bijl and Ulyana Tikhonova, Axini

  • The problem we are addressing: How do you know that your system conforms to its requirements and specifications in an automated fashion? The state of practice is to automate test-cases. However, how does one create and maintain a high coverage (regression) test-set in an economically feasible fashion? Model-based testing is an interesting theory that brings specification and testing together. It makes it possible to test your systems with unprecedented precision in a maintainable way.

    In this interactive workshop you get to know the Axini Modeling Platform. It is a platform to model and test systems. Tests are model-based, which means that the platform fully automates the testing process: test-case generation, execution and evaluation. Participants will take part in the bug hunt. We present 10 different implementations of the Smart Door IoT system, with several bugs in them. The person that finds the most bugs wins. In order to find the bugs, participants will model the specification of the Smart Door and then execute automatically generated tests against the implementations – all in the Axini Modeling Platform.

  • Key Learning points:
    • How to model a system to model-based test it. In this workshop we use the Axini Modeling Language, a rich modelling language with actions (inputs and outputs), data, time, non-deterministic choices and loops, etc.
    • How to debug models
    • How to execute tests against implementations
    • How to analyze and diagnose the executed tests
      
      

  10.30 - 11.00: Networking Coffee Break

  11.00 - 12.30: TUTORIAL 2: Test Intelligence
  Jakob Rott, CQSE

  • Many Teams must test more and more functionality in less and less time. Historically grown test suites are often not up to this challenge since they test too much and too little at the same time. Too much, since they contain redundant tests that cause execution and maintenance costs but provide little value over similar tests. Too little since important functionality remains untested.
    
    In this talk, I present analysis techniques to uncover these problems in your own system: History analyses of the version control system show, where in the past most bugs occurred. Test gap analysis reveals, which code changes have not yet been tested. And pareto optimization of test suites and test impact analysis identifies tests that right now have the best cost-benefit ratio. For each analysis, I outline the research foundation and give examples and experiences from application in practice.
    
    
  • Key Learnings:
    • Which data sources are available and which data they contain
    • Which analyses exist that make use of the data
    • How to detect test gaps (=untested changes)
    • Where in the system most bugs remained undetected in the past and how this can help improve their discovery in the future
    • How to find redundant tests in historically grown test suites using pareto-optimization
    • Which test impact analysis approaches work and how they discover those tests, that find new bugs most quickly.
      
      

  12.30 - 14.00: Networking Lunch

•  

   

  CONFERENCE DAY 1

  14.00 - 14.30: Conference Opening

  Andreas Ulrich, UCAAT PC Chair
  Philip Makedonski, ETSI TC MTS Chair
  Anthony Wiles

  14.30 - 15.45: SESSION 1: INDUSTRIAL SYSTEMS

  • 14.30 - 14.55: A New Way for Testing in the Autonomous Factory – Simulation Based Testing
    Matthias Eichenseher, Siemens AG
  • 14.55 - 15.20: Keyword-Driven Simulation Testing for Open-Source Robotics
    Niels Hoppe and Steffen Lüdtke, Fraunhofer FOKUS
  • 15.20 - 15.45: Back-to-Back Testing Applied on Distributed PLC
    Anais Guignard, Systerel

  15.45 - 16.15: Networking Coffee Break

  16.15 - 17.30: SESSION 2: FAILURE ANALYSIS

  • 16.15 - 16.40: Automatic Log Analysis: Expert Knowledge for Everyone!
    Graham Rawlings and Stephan Schulz, Giesecke+Devrient
    
  • 16.40 - 17.05: AI-Based Defect Analytics and Assignment
    Rajesh Kumar, Gabor Pal Csaki and Michael Schmidts, Siemens AG
  • 17.05 - 17.30: ML-Based Test Prioritization “To Fail First” as a Service
    Alexandre Vernotte and Aymeric Cretin, Smartesting, Femto-ST Institute
• 14 September: Conference Day 2
•  

   

  CONFERENCE DAY 2

  09.00 - 10.00: KEYNOTE SPEECH - Testing AI: A New Testing Specialism
  Stuart Reid, CTO, STA Consulting, Korea
  

  • Artificial intelligence (AI) is now a mainstream technology that touches the lives of most of us, with a global market predicted to reach US$500 billion within two years.
    AI-based systems, and machine-learning (ML) systems in particular, are different from traditional IT systems, as the underlying code is not written by a programmer but is inferred from patterns in data. This major change to classic software development has two significant knock-on effects. It has created the attractive new role of AI specialist. However, it has also created several new sources of specialised defects, such as the training data, the deliverable ML model and the ML development framework.
    
    This talk proposes a new software testing specialism that is focused on addressing the new challenges introduced by AI and ML. To be able to test a ML system, testers need to understand the underlying technology and how it can go wrong. They also need to be able to address those system attributes, such as ethics and trust, which are so important with AI-based systems.
    
    From now on, as AI-based systems grow in complexity and criticality, achieving acceptable levels of quality and trust will only be possible by employing independent, AI test specialists. This talk will explain why they are needed, what specialist tasks they will do, and suggest how we should handle the transition to a future where the AI test specialist is a recognized role.
    
    

  10.00 - 10.25: Posters Introduction
  5' per poster

  10.25 - 11.00: Networking Coffee Break

  11.00 - 12.15: SESSION 3 - AI SYSTEMS

  • 11.00 - 11.25: JanIA: Intelligent Practices for Automated Testing
    Pablo Manso García-Mauriño, MTP
  • 11.25 - 11.50: Automating Adversarial Robustness Testing of DNN Models
    Albert Negura and Kobus Grobler, NavInfo Europe B.V.
  • 11.50 - 12.15: Debugging Machine Learning Models
    Emmanuel Charleson Dapaah and Jens Grabowski, University of Göttingen

  12.15 - 13.45: Networking Lunch

  13.45 - 15.00: SESSION 4 - REAL-TIME SYSTEMS

  • 13.45 - 14.10: Identifying and Classifying Uncertainties to Support Testing of Industrial Elevators
    Tao Yue, Simula Research Laboratory
  • 14.10 - 14.35: Benefits of Test Automation in Time Travel Testing
    Aritra Majumdar, EPAM Systems
  • 14.35 - 15.00: Increase Robustness with Time Travel – Is It Possible and Reasonable?
    Rosalinde Schuster, Victor Navratil and Stephan Schulz, Giesecke+Devrient

  15.00 - 15.30: Networking Coffee Break

  15.30 - 17.10: SESSION 5 - DEPENDABLE SYSTEMS

  • 15.30 - 15.55: Overview of Work in Progress to Update IEEE 982, Standard for Measures of the Software Aspects of Dependability
    Robert Binder, IEEE
  • 15.55 - 16.20: Assessment of GNSS-Based Equipment in the Context of AEBS-UNECE R152
    Ernst Phillip Mrohs and Marcel Visser, NavCert GmbH
  • 16.20 - 16.45: Overview of the ETSI Electronic Signatures Testing Activities
    Laurent Velez, ETSI
  • 16.45 - 17.10: Continuous Quality Assurance Strategy of an NFVI
    Szilard Szell, Eficode Oy
• 15 September: Conference Day 3
•  

   

   CONFERENCE DAY 3

  09.00 - 10.00: KEYNOTE SPEECH - Testing Challenges for Cyber Physical Systems
  Jan Tretmans, Research fellow at ESI (TNO), Eindhoven

  Complex high-tech cyber physical systems increasingly depend on software: software controls, connects, and monitors almost every aspect of system operation, be it a car, a 'smart' factory, a printer, an airplane, or a wafer lithography system. Consequently, ever more system failures are due to software bugs. Moreover, we build ever larger, more complex, and increasingly connected software systems, with more and more options and features, and evolving ever faster. There is a serious threat that testing methods cannot keep pace with these developments, and will endanger the fast and thorough testing of future generations of complex systems, thus hampering innovation. In this presentation we discuss trends, challenges, and research directions for testing future generations of high-tech cyber physical systems.

  10.00 - 12.15: SESSION 6 - METHODOLOGY

  • 10.00 - 10.25: Holistic Model-Based Approach for Test Generation from Safety Models
    Philipp Dormeier, Siemens AG

   

  10.25 - 11.00: Networking Coffee Break
  

  • 11.00 - 11.25: Towards the Absence of Bugs: An Intelligent Combination of Static and Dynamic Analysis to Identify Vulnerabilities in the Development Process
    Ramon Barakat, Fraunhofer FOKUS
  • 11.25 - 11.50: Systematic Selection of Testing Methodology for Low-Code Development
    Shreyasi Warunkar, S&N CQM GmbH
  • 11.50 - 12.15: How to Make your Automation Last Forever
    Martin Gijsen, Sogeti

  12.15 - 13.30: Networking Lunch

  13.30 - 15.55: SESSION 7 - TEST AUTOMATION

  • 13.30 - 13.55: Isn’t Test Automation a Silver Bullet?
    Mesut Durukal, 
  • 13.55 - 14.20: Best Practices of Agile Testing in Cloud Software Products
    Gemmilyn Chu and Daniel Ardelean, Nokia
  • 14.20 - 14.45: A Spring in Tester’s Life Using SpringBoot
    Ashokkumar Rajendiran, PagoNxt

  14.45 - 15.05: Networking Coffee Break

  • 15.05 - 15.30: A Beginners Guide for Monkey Test Automation on Android & iOS Native Apps
    Lakshminarayanan Krishnamurthy, EPAM Systems
  • 15.30 - 15.55: Complementing GUI Testing Scripts with Smart Monkey Testing
    Pekka Aho, Tanja E. J. Vos, Fernando Pastor Ricos and Olivia Rodriguez Valdes
    Universitat Politecnica de Valencia, Open Universiteit

  15.55 - 16.15: BEST PAPER AWARD / CONFERENCE CLOSING
  Andreas Ulrich, UCAAT PC Chair
  Philip Makedonski, ETSI TC MTS Chair

The independent Programme Committee is composed mostly of industry stakeholders nominated by the appointed Programme Committee Chair.
The Programme Committee determines the call for presentations and evaluates the received presentation, poster, and tutorial proposals to select the best ones for the conference programme.

Programme Committee Chair:

• Andreas Ulrich, Siemens AG

Programme Committee Members: 

• Jorge Asensio, ARHS Developments
• Sergio Borghese, NetResults Srl
• Carlos Cárdenas, Dekra
• Rémi Caudwell, Sogeti
• Tibor Csöndes, Ericsson Hungary
• Zhen Ru Dai, Hamburg University of Applied Sciences (HAW)
• Alexis Despeyroux, Testing Solutions and Services
• Muslim Elkotob, Vodafone
• Baris Güldali, S&N Invent
• Raquel Jimenez, MTP
• Martti Käärik Elvior, LLC
• Philip Makedonski, University of Göttingen
• Ultan Mulligan, ETSI
• Dejan Ničković , AIT
• Jorge Pascal, Audi
• Axel Rennoch, Fraunhofer FOKUS
• Stephan Schulz, Giesecke + Devrient
• Matthias Simon, Nokia
• Szilard Szell, Eficode Oy
• Dirk Tepelmann, Spirent
• Peter Zimmerer, Siemens AG

The following companies will exhibit at UCAAT 2022:

Axini, the next step in test automation
At Axini we improve the way software is made, in particular the quality of software. We offer a platform in which your domain experts specify the requirements and functionality of your system and our platform fully automatically takes care of the testing. This approach fits perfect with Agile development. Don’t wait until the system integration for errors to manifest – detect problems as early as possible in the development process.
Powered by formal techniques developed over 35 years of academic research, Axini is the leading company on model-based engineering and model-based testing in The Netherlands. We have over 10 years of experience applying these formal techniques with great success in high-tech, finance and rail.

Kalifast is a software solution whose development began in 2010, sponsor of UCAAT since 2016, visitors will discover the advances of this solution always at the forefront of innovation.
Kalifast aims to be an all-integrated solution that goes from requirements management, to test automation, to project management.
On the automation side, Kalifast has a powerful scheduler allowing it to perform in particular:
- Scenarios requiring several devices, which makes it possible to test, for example: chat conversations, user / administrator interactions, etc.
- The use of robots of different technologies in the same scenario
- Adaptable scenarios according to the results
- Variabilizations that can use Excel files of thousands lines

MTP is a Spanish company founded in 1997 and it is offering Quality Assurance and Quality Control Professional Services for Digital Business. In MTP, there are more than 600 employees distributed across Spain, Mexico, Brazil, and the United States. MTP clients are National and International organizations from all industries. MTP is totally independent of development manufacturers and most of the employees are ISTQB/SCRUM Certified.
MTP Digital Business Assurance business comprehends the following services:
• Quality Assurance (QA): Functional and performance testing and test automation/regression
• Cybersecurity (CS): Audits for SW applications, mobile applications, and IoT
• User Experience (UX): usability and accessibility testing and user interface re-engineering
• DevOps and Agile Development (DA): support for agile methodologies deployment

SMARTESTING develops cognitive test solutions that boost the efficiency of IT Quality Assurance teams.
Our products combine visual representations and AI technologies to facilitate the agile design of business relevant tests and accelerate their automation.
SMARTESTING enjoys a worldwide reputation of expert in Model-Based Testing. In 2017, SMARTESTING launched YEST®, a visual and Agile test design tool. YEST® is the fruit of our team’s unique know-how, experience and unrelenting determination to offer a simple MBT tool to the IT testing community.
SMARTESTING also develops GRAVITY, a usage-driven automation testing solution. GRAVITY automatically creates automated regression tests from the identification of key customer journeys on the application under test.
SMARTESTING is present in France and Germany. www.smartesting.com - contact@smartesting.com

Should you also wish to exhibit at UCAAT 2022, and introduce your organization, company, product or service to people interested in advanced automated testing

• complete and return the application form below for submission to events@etsi.org.
  Exhibitor application form

Exhibition Stands are free of charge and consist of:

• A 4sqm stand with 1 table, 2 chairs, electricity and internet connection
• Possibility to display company’s own exhibition material on the stand (pop up, banner...)

Exhibitors Visibility

• UCAAT exhibitors will be listed on the website
• The exhibition area will be announced at the opening of conference
• The Conference programme will promote the exhibition area
  
  • Encourage participants to visit the stands during coffee/lunch breaks
  • Exhibition stands plan available in printed programme
  • 10 lines company profile and logo in the printed programme

Should you need any further information, please contact events@etsi.org

German Testing Day Digital 2022
Be a part of the company-independent conference on software quality on 03 + 04 May 2022! For everyone involved in quality assurance of business IT systems as well as embedded systems, the German Testing Day Digital offers the perfect platform! The main focus is on the sharing of experiences & new trends as well as methods. Look forward to 25 cutting-edge sessions on testing including inside talks by Vodafone, Raiffeisen Bank, Siemens Healthineers and many more!
» More info

HUSTEF is one of Europe’s premier Software Testing & Quality conferences with 730+ attendees from 30+ countries preserving its very friendly and familiar atmosphere.This year we are coming back as a LIVE event on 4-6 October!
Would you like to join us as a Speaker? Submit your proposal and see you in Budapest, Hungary!

The ASQF is the network of experts for quality. With 1.200 members we have been developing and securing software and system quality for 25 years.
Upcoming event: The ASQF Safety & Security Day on September 7, 2022 invites you to discuss the topic “Your assets are at risk: processes, methods and tools to protect them” (from the perspective of functional safety, cybersecurity and safety at work).
» View all ASQF events

UCAAT 2022 is hosted by Siemens AG in Munich, Germany on 13 – 15 September 2022.

The conference will take place in the Siemens AG Conference Center 
Otto-Hahn-Ring 6, 81739 München
Location: Central site
Building: 12

U/S-Bahn station: Neuperlach Süd - Line U5

It is planned that the event takes place as a physical event in a large auditorium and will also include exhibitions.

UCAAT is an established series of annual events. Previous editions of the UCAAT have focused on different aspects of test automation following emerging trends through the years. Information about previous editions of the UCAAT, including the programmes and presentation slides, are available from the archived websites linked below. Browse through the archives and discover past experiences shared at UCAAT!

• UCAAT 2021
• UCAAT 2020 (cancelled)
• UCAAT 2019
  
• UCAAT 2018
  
• UCAAT 2017
• UCAAT 2016
• UCAAT 2015
• UCAAT 2014
• UCAAT 2013