SUT Operator

Shares the details about the network interfaces of his system under test.

Operates the system under test during the testing session.

Logs evidences in the tool.

As a SUT Operator, I want to verify the conformance of the message captured by the tool

As a SUT Operator, I shall be able to ask the test bed to verify the conformance of a given message that has been captured.

When the message is captured in the context of a test step, the issuing actor and performed transactions are known. As a consequence, the test tool shall be able to determine by itself which conformance checker tool and validation profile to use, or, at least, ask the SUT Operator to select from a sub set of possibilities.

As a SUT Operator, I want to select the message to use as evidence for a given test step

As a SUT Operator, when I execute a test and the exchanged messages are captured by an intermediary tool, I want to be able to select the ones that shall be used as evidences for a given test step.

When I select a message, I shall be able to tell the tool to attached it as an evidence to the test step and there shall be a placeholder for me to enter a comment.

When the message is linked to the test step, the following details shall be displayed along with a link to see the full content of the message: 

As a SUT Operator, I want to see the messages exchanged between two sytems under test

As a SUT operator, when I execute a test run through an intermediary tool that captures the messages exchanged between the two systems under test, I shall be able to see the messages that have been captured to select the ones that are relevant.

When I am executing a test case, for each test step, I shall see the list of messages that have been captured and that are relevant for the test step. The messages that are displayed shall be selected based on 

As a SUT Operator, I want to configure my system under test to interact with my partner during the test execution

Prerequisite

As a SUT Operator I have selected a test case to execute and the test partners for this test case.

Description

As a SUT operator, before I can execute a test case against my test partner's system under test, I shall see the network details to contact it. It will allow me to configure my system under test to direct the messages to the correct network interface or to configure my application and/or firewall.

Test session participants execute a peer-to-peer test

Submitted by abe on Thu, 12/08/2022 - 12:12
Priority
Essential - Must have, it is essential for the use cases
Status
Draft
Customer
Innovation
Business owner
Anne-Gaëlle BERGE
Description

This is a user need defines in the context of the new Gazelle product. 

As a participant (SUT operator) to an interoperability testing event, I want to execute a peer-to-peer test with one or more other systems under test and report the result of the execution to the event management team, for review and grading. To prove the correct execution of the test case, I might be asked to provide evidences of the following types:

  • Screenshots;
  • Messages exchanged between the systems;
  • Log files.
What is/are the targeted context(s)?
Who is the targeted audience?
Targetted goal(s)

As a SUT Operator, I want to be able to test my SUT against other SUTs, by following a defined test case, and demonstrate to a monitor that the execution went as expected.

When I am asked to report evidences of the correct execution of the test case, in particular to report the messages exchanged between the SUTs, the tool shall help me in identifying which messages are relevant for the current test case. Ideally, to save time and execute as many tests as possibile during the event, I execute the test and search for the messages in a single place.

Expected benefit

The user experience is improved

The number of tools to gain experience with is reduced

Globally, the time to execute and report about a test execution is shorten and the evidences are relevant.

Opportunities

Reduce the number of "screens" to be developed and make use of existing Gazelle components to automate part of the process.

As a SUT Operator, I want to be able to check the conformance of the IUA Extended Token produced by my SUT

As a SUT Operator, I want to be able to check the conformance of the IUA Extended Token produced by my SUT so that I'll make sure that :

  • The mandatory attributes are present with their right values (error message if failed or info if passed).
  • If the optional attributes are present, their values are conform to the specifications (error message if failed or info if passed).
  • No other irrelevant attributes are present (Warning message if present, no check of the value).

Below are the attributes to test :

As a SUT Operator, I want to be able to check the conformance of the IUA Basic Token produced by my SUT

As a SUT Operator, I want to be able to check the conformance of the IUA Basic Token produced by my SUT so that he'll make sure that :

  • The mandatories attributes are present with their right values (error message if failed or info if passed).
  • If the optional attributes are present, their values are conform to the specifications (error message if failed or info if passed).
  • No other irrelevant attributes are present (Warning message if present, no check of the value).

Below are the attributes to test :

A SUT operator validates an ITI-71 transaction to make sure that his messages comply with the specifications.

Submitted by vhn on Tue, 11/29/2022 - 10:48
Priority
High - Major feature, it achieves a lot of work
Status
Draft
Customer
Canada Health Infoway
Business owner
Vincent HOFMAN
Description

The workflow involves 2 main actors (we don't check the IDP part) :

  • The Authorization Client : A client that retrieves access tokens.
  • The Authorization Server : A server that issues access tokens to requesting clients.

In order to check the compliance a complete ITI-71 transaction, I need to validate :

  1. The Get Authorization Code request (Http GET message).
  2. The Get Authorization Code response (with the code).
  3. The Get Access Token Request (Http POST message, code included).
  4. The Get Access Token response (with also a Token validation).

The steps of this workflow SHALL be tested either for a Basic Token or an Extended Token.

 

What is/are the targeted context(s)?
Targetted goal(s)

The main objective is to offer to SUT operators a way to verify the correctness of the HTTP requests and responses issued by their systems under test, against the ITI-71 requirements. In case of failure of the SUT to demonstrate that it complies with the specifications, the SUT operator shall understand what and where the errors are.

When the HTTP message contains a body (Steps 2 to 4), the SUT operator shall be able to assess as well the conformance of its content against specifications.

Expected benefit

Short term benefit (< 1 year):

Canadian second Projectathon's participants will be able to test IUA ITI-71 messages before and during the event. It will also permit to increase consequently the validation time for monitors.

Middle and long term benefit (>= 1 year):

Since Canada Health Infoway choose to use the unmodified IHE specifications for IUA, this Validation Profile will be used in all the other IHE platforms.

Opportunities

This User Need is almost the same as SUT Operator verifies the conformance of the RAD-107 request and response messages.

A common validation service should be developped.

 

Documentation

Complete and up to date IUA specifications : https://profiles.ihe.net/ITI/IUA/index.html

SUT Operator verifies the conformance of the RAD-107 request and response messages

Submitted by abe on Fri, 11/18/2022 - 16:08
Priority
High - Major feature, it achieves a lot of work
Status
Under development
Customer
ANS
Business owner
Marie Bruillard
Description

ANS has defined a new business use case that allows the sharing of imaging among practioners that are not on the same network. The DRIM-M project relies on the fact that a DICOM KOS is published in the patien health record and contains pointers to the images that are stored in the PACS of the facility where the exam took place. When the practitioner wants to access images that are referenced in an imaging report, he first retrieves the DICOM KOS and then use the RAD-107 transaction to retrieve the images in the PACS.

Two actors are involved in this process:

  • the DRIMBox source is the system (or component of a system) that acts as a gateway to the PACS. It behaves as the Imaging Document Source actor of the IHE WIA profile
  • the DRIMBox consommatrice is the system (or component of a system) that initiates the RAD-107 transactions to retrieves the images. It behaves as the Imaging Document Consumer actor of the IHE WIA profile.

In the context of the Segur program, ANS wants to verify the ability of

  • the DRIMBox consommatrice to send out an HTTP request to the DRIMBox source that complies with the DRIM-M requirements;
  • the DRIMBox source to build an HTTP response that complies the the DRIM-M requirements.

 

The SUT operator shall be able to verify the conformance of the message by either:

  • Uploading the message content in the EVSClient
  • Selected the message to validate in the Gazelle Proxy
What is/are the targeted context(s)?
Who is the targeted audience?
Targetted goal(s)

The main objective is to offer to SUT operators a way to verify the correctness of the HTTP requests and responses issued by their systems under test, against the RAD-107 requirements. In case of failure of the SUT to demonstrate that it complies with the specifications, the SUT operator shall understand what and where the errors are.

When the HTTP message contains a body (RAD-107 response), the SUT operator shall be able to assess as well the conformance of its content against specifications.

Expected benefit

Short-term: offering an automated way to verify the conformance of the messages issued in the context of the RAD-107 transaction (DICOMWeb / WADO-RS). It will benefit as well to the IHE Connectathon participants who have currently no mean of non-manual verification.

Medium-term: offering an HTTP validator (HTTP headers for requests and responses) that can be customised based on the requirements to be tested (DICOMWeb QIDO-RS, FHIR, IUA in particular).

Long-term: coupling this HTTP validator with the Message Content Analyzer to be able to provide to the SUT operators a complete analysis of the HTTP messages his SUT produces.

Opportunities

Enlarge the conformance testing capabilities of the test bed by offering a new feature: validation of HTTP messages, including validation of the HTTP headers.

Introduce this new conformance checking tool to the IHE Connectathon participants. It will increase the level of testing for an important set of HTTP-based profiles and thus improve the level of confidence in the systems under test which implement such specifications. Forseen profiles to be tested first: WIA, WIC, AIR, and IUA.

Leverage this new conformance checking tool for testing the IUA profile in the context of the PAN Canadian project.

Documentation
  • RFC 2616: Hypertext Transfer Protocol -- HTTP/1.1
  • RFC9110: HTTP Semantics
  • WIA: IHE Profile Web-based Access to Images 
  • CI-SIS (ANS): Acces aux Documents de Santé en Imagerie (currently in public comment status, link to the final version to be added later on)

 

The user shall have access to an overview of the test runs for a given testing session whatever the current status of the related test cases

Submitted by abe on Wed, 10/26/2022 - 15:20
Priority
High - Major feature, it achieves a lot of work
Status
Draft
Customer
DG Santé
Business owner
Malo Toudic
Description

In Gazelle Test Management, the tests are executed from the Test execution page. This page also logs all the runs executed by the systems. The test runs are linked to the test case they relate to.

The test execution page shows only the test cases with a status set to "ready". As a consequence, if a test case is deprecated at some points, the SUT operator cannot access the test runs that were executed when the test case was "ready". 

In the same way, the "All test runs" page displays only the test runs that relates to the "ready" test cases.

As a consequence, we experience a loss of information and that is not accessible neither from the SUT operator's or Testing Session Manager's point of view.

  • As a SUT Operator, I want to see all the test runs that have been executed by a system under test owned by my organisation, during a given testing session, at any time.
  • As a TSM, I want to see all the test runs that have been executed during a given testing session, at any time.
  • As a user, for each test run, I want to see the following details:
    • Test run identifier
    • Test case keyword (and a link access its details)
    • Whether the test case is currently deprecated
    • Last update date/time
    • Test run status
    • The role assigned to my SUT in this particular test run
    • The other partners
    • The status of the test run
    • If a monitor is assigned, his first and last names
  • As a user, I want to be able to access the test run page for any test run (whatever the status of the test case is)
  • As a user, I want to be able to access the description of the test case.
  • As a user, I want to be able to filter on
    • System (name/keyword)
    • Test case keyword
    • Monitor
    • Type of test
    • Domain
    • Tested profile
    • Tested actor
    • Tested profile option
    • Involved transactions
    • Test run status
  • As a testing session manager or admin user, I want to be able to filter by Organisation (name/keyword)
  • As a user, I want the test runs to be grouped by tested SUT capabilities
  • As a user, for each SUT capabilities, I want to see the evaluation.
  • As a user, I want to be able to sort on test case and test status within each SUT capability.
What is/are the targeted context(s)?
Who is the targeted audience?
Targetted goal(s)

Allow the users to see all the test runs from a test session, regardless of the current status of the executed test cases.

Expected benefit

User will have access to a page that summarizes the outcome of their test campaign

Opportunities

We might also want to experience new widget/features on that page like displaying statistics.