際際滷

際際滷Share a Scribd company logo

USING TDL FOR STANDARDISED TEST PURPOSE DEFINITIONS

Axel Rennoch
Axel Rennoch

1) The document discusses ETSI's use of the Test Description Language (TDL) and TTCN-3 for standardizing test purpose definitions. 2) TDL is used at the test design level to define test objectives and expected behavior, while TTCN-3 is used at the test implementation level. 3) ETSI has applied TDL and TTCN-3 across several technical committees and projects, developing over 300 test purposes for domains like mobile networks and IoT.

1 of 16
Download to read offline
USING TDL FOR STANDARDISED TEST PURPOSE DEFINITIONS Philip Makedonski, Ilie-Daniel Gheorghe-Pop, Axel Rennoch, Finn Kristoffersen, Bo邸tjan Pintar, Andreas Ulrich IEEE QRS/STV, 13th December 2020, Macau (China)
2 Our Context ETSI Testing approach TDL TTCN-3 Application samples within ETSI TC INT projects TC MTS standardisation OUTLINE
3 ETSI is a leading standardisation organisation for Information and Communication Technology (ICT) standards fulfilling European and global market needs. ETSI test specifications are developed according to the well-proven methodology defined in ISO/IEC 9646. This framework recommends that the test specifications include: Test Purposes, Test Descriptions and Test Cases. OUR CONTEXT
4 Technical Committee Methods for Testing and Specification Working Group TDL provides and oversees roadmap for further development of TDL and the TDL open source project Working Group TST develops IoT test catalogues and specifications (not covered elsewhere) The types of testing include conformance, interoperability, security and performance testing The initial technical focus is: IoT network layer (communication protocols, node connectivity, edge computing etc.), Basic security of IoT devices ETSI TC MTS
5 ETSI TESTING APPROACH Source: https://tdl.etsi.org/index.php/introduction Test Execution Level Test Implementation Level Test Design Level Requirements Level
6 QUALITY ASPECTS 1. Conformance 2. Robustness / Security 3. Performance Black-Box perspective
7 Test Description Language Design, documentation, representation of formalised test descriptions Scenario-based approach Testing and Test Control Notation Specification and implementation of all kinds of black-box tests Component-based approach ETSI TEST LANGUAGE STANDARDS
8 1) Test configurations 2) Test Suite Structure 3) Test purpose (catalogue) 4) Test implementation (TTCN-3) TEST PURPOSE SPECIFICATION, SAMPLE MQTT
9 TDL-TO TEST OBJECTIVE SPECIFICATION Informal text specification (semi-structured) Simple description structure (event occurrence sequences) Global keyword definitions (domain specific) Single test observation (for pass/fail verdict criteria) Test Purpose { TP Id TP_HELLO_MSG_SERVER /* Summary */ Test objective "Establishing Connection with EndpointUrl" Reference "OPC-UA,Part-6- Mappings#section-7.1.3" Expected behaviour ensure that { when {} then {} } }
10 TTCN-3 THE TEST EXECUTION LANGUAGE 1) Test configurations 2) Test Suite Structure 3) Test purpose (catalogue) 4) Test implementation (TTCN-3)
11 Technical Committee Core Network and Interoperability Testing (INT) Specialist/Testing Task Forces on Voice and video services over LTE For multiple interfaces of the identified test configurations Packages with common domain definitions (library concept) More than 300 TDL-TO test purposes TDL Open Source Project (TOP) tools APPLICATION: INTEROPERABILITY TESTS
12 APPLICATION: IOT-TESTWARE THE TWO PILLARS ETS Reporting Logging Legend: TP: Test Purpose TSS: Test Suite Structure ATS: Abstract Test Suite ETS: Executable Test Suite SUT: System Under Test TSS TP ATS
13 AVAILABLE TEST SPECS MTS TST WORK PROGRAMME IEC 62443-4-2 CoAP MQTT https://portal.etsi.org/tb.aspx?tbid=860&SubTB=860
14 Abstract Keeps you focused on what to test Standardised Helps you to produce repeatable results independently from a chosen certain tool or tool provider Application focus Wide range of features for todays interconnected, concurrent, embedded, real-time systems Semi-formal Helps you to keep test specifications consistent over evolving systems (meta-modelling supported static code analysis) Multiple syntaxes Provide test specifications in a language that different stakeholders understand best (graphical, textual, other) Tool support The TDL Open source Project (TOP) offers tool support for the use of TDL BENEFITS OF TDL AS A TEST SPECIFICATION LANGUAGE
15 Standardised test purposes Used in multiple domains: e.g. mobile, access/core networks, ITS Test types, e.g. conformance, interop, security Advanced testing technology: Used for certification: e.g. UMTS, LTE, 5G, oneM2M ETSI continues maintenance and evolution TDL: https://tdl.etsi.org/ TTCN-3: http://www.ttcn-3.org/ SUMMARY & OUTLOOK
16 CONTACT Partially funded by ETSI in the context of the STF projects 454, 476, 492, 522, 574 and 577 German Federal Ministry for Economic Affairs and Energy (IoT-T project) Thank you for your attention! makedonski@cs.uni-goettingen.de ilie-daniel.gheorghe-pop@fokus.fraunhofer.de axel.rennoch@fokus.fraunhofer.de finn@cinderella.dk pintar@sintesio.org andreas.ulrich@siemens.com

More Related Content

USING TDL FOR STANDARDISED TEST PURPOSE DEFINITIONS

  • 1. USING TDL FOR STANDARDISED TEST PURPOSE DEFINITIONS Philip Makedonski, Ilie-Daniel Gheorghe-Pop, Axel Rennoch, Finn Kristoffersen, Bo邸tjan Pintar, Andreas Ulrich IEEE QRS/STV, 13th December 2020, Macau (China)
  • 2. 2 Our Context ETSI Testing approach TDL TTCN-3 Application samples within ETSI TC INT projects TC MTS standardisation OUTLINE
  • 3. 3 ETSI is a leading standardisation organisation for Information and Communication Technology (ICT) standards fulfilling European and global market needs. ETSI test specifications are developed according to the well-proven methodology defined in ISO/IEC 9646. This framework recommends that the test specifications include: Test Purposes, Test Descriptions and Test Cases. OUR CONTEXT
  • 4. 4 Technical Committee Methods for Testing and Specification Working Group TDL provides and oversees roadmap for further development of TDL and the TDL open source project Working Group TST develops IoT test catalogues and specifications (not covered elsewhere) The types of testing include conformance, interoperability, security and performance testing The initial technical focus is: IoT network layer (communication protocols, node connectivity, edge computing etc.), Basic security of IoT devices ETSI TC MTS
  • 5. 5 ETSI TESTING APPROACH Source: https://tdl.etsi.org/index.php/introduction Test Execution Level Test Implementation Level Test Design Level Requirements Level
  • 7. 7 Test Description Language Design, documentation, representation of formalised test descriptions Scenario-based approach Testing and Test Control Notation Specification and implementation of all kinds of black-box tests Component-based approach ETSI TEST LANGUAGE STANDARDS
  • 8. 8 1) Test configurations 2) Test Suite Structure 3) Test purpose (catalogue) 4) Test implementation (TTCN-3) TEST PURPOSE SPECIFICATION, SAMPLE MQTT
  • 9. 9 TDL-TO TEST OBJECTIVE SPECIFICATION Informal text specification (semi-structured) Simple description structure (event occurrence sequences) Global keyword definitions (domain specific) Single test observation (for pass/fail verdict criteria) Test Purpose { TP Id TP_HELLO_MSG_SERVER /* Summary */ Test objective "Establishing Connection with EndpointUrl" Reference "OPC-UA,Part-6- Mappings#section-7.1.3" Expected behaviour ensure that { when {} then {} } }
  • 10. 10 TTCN-3 THE TEST EXECUTION LANGUAGE 1) Test configurations 2) Test Suite Structure 3) Test purpose (catalogue) 4) Test implementation (TTCN-3)
  • 11. 11 Technical Committee Core Network and Interoperability Testing (INT) Specialist/Testing Task Forces on Voice and video services over LTE For multiple interfaces of the identified test configurations Packages with common domain definitions (library concept) More than 300 TDL-TO test purposes TDL Open Source Project (TOP) tools APPLICATION: INTEROPERABILITY TESTS
  • 12. 12 APPLICATION: IOT-TESTWARE THE TWO PILLARS ETS Reporting Logging Legend: TP: Test Purpose TSS: Test Suite Structure ATS: Abstract Test Suite ETS: Executable Test Suite SUT: System Under Test TSS TP ATS
  • 13. 13 AVAILABLE TEST SPECS MTS TST WORK PROGRAMME IEC 62443-4-2 CoAP MQTT https://portal.etsi.org/tb.aspx?tbid=860&SubTB=860
  • 14. 14 Abstract Keeps you focused on what to test Standardised Helps you to produce repeatable results independently from a chosen certain tool or tool provider Application focus Wide range of features for todays interconnected, concurrent, embedded, real-time systems Semi-formal Helps you to keep test specifications consistent over evolving systems (meta-modelling supported static code analysis) Multiple syntaxes Provide test specifications in a language that different stakeholders understand best (graphical, textual, other) Tool support The TDL Open source Project (TOP) offers tool support for the use of TDL BENEFITS OF TDL AS A TEST SPECIFICATION LANGUAGE
  • 15. 15 Standardised test purposes Used in multiple domains: e.g. mobile, access/core networks, ITS Test types, e.g. conformance, interop, security Advanced testing technology: Used for certification: e.g. UMTS, LTE, 5G, oneM2M ETSI continues maintenance and evolution TDL: https://tdl.etsi.org/ TTCN-3: http://www.ttcn-3.org/ SUMMARY & OUTLOOK
  • 16. 16 CONTACT Partially funded by ETSI in the context of the STF projects 454, 476, 492, 522, 574 and 577 German Federal Ministry for Economic Affairs and Energy (IoT-T project) Thank you for your attention! makedonski@cs.uni-goettingen.de ilie-daniel.gheorghe-pop@fokus.fraunhofer.de axel.rennoch@fokus.fraunhofer.de finn@cinderella.dk pintar@sintesio.org andreas.ulrich@siemens.com