Testing Session Manager

Responsible for the organisation of a testing session.

Assigns the monitors to the test session and assign test cases to monitors.

Accepts or not the registration of systems under test

 

User creates as many presets as he needs

In Gazelle Test Management, on several occasions, the user is invited to create preset. He creates a cookie that works like a bookmark. Presets are available:

  • On the Test execution and logs page;
  • On the test repository page.

In both cases, the user is allowed to create 4 presets at a maximum. Except there is a technical reason, there is no other reason why the users could no create as many presets as he needs.

Communication between management, participants, and monitors

Submitted by abe on Mon, 12/20/2021 - 11:28
Priority
Medium - Feature to be considered
Status
Draft
Customer
Renovation
Business owner
Anne-Gaëlle BERGE
Description

In the last years, the Projectathons have been held online as well as the two last European Connectathons. In that context, Rocket.chat has been used as a communication tool between the participants, with the test session managers, and with the monitors. 

The SHARAZONE participants also use this tool for one-to-one communications. It is also the place where the SHARAZONE administrator broadcast information to the participants: new object available, tool maintenance, and so on.

All the involved parties agree that this tool is really useful and might also have an interest during onsite testing events.

What is/are the targeted context(s)?
Targetted goal(s)

Main goal: allow participants to online events to communicate in a timely manner: find test partners, execute test cases, review test cases, share screen.

Secondary goals:

  • Support participants during the preparatory phase
  • facilitate the exchange of information (patient identifier, logs) during on-site events
  • be able to leave a message to an on-site participant (or monitor or TSM) when it is busy on another subject.

 

Expected benefit

Preparatory phase:

  • better support to the participants
  • create a place for exchange

Online:

  • make the test event happens

Onsite:

  • fasten the test execution and debugging

 

Documentation

As a Test Session Manager, I want to broadcast information to all the participants (text message / attachments).

As a Test Session Manager, I want to broadcast information to all the monitors (text message / attachments).

As a Test Session Manager, I want to hold calls with all the participants to the online event (video is optionnal).

As a Test Session Manager, I want to share my screen with all the participants when the event is online.

As a monitor, I want the participant to share his screen with me so that I can verify the outcome of the test even when working remotely.

As a SUT operator, I want to share information with my partners of the test instance.

As a monitor, I want to contact all the SUT operators of a particular test instance when we are working online.

As a SUT operator, I want to contact the management team when I have a question during an online event.

As a participant (SUT operator, monitor, test session manager), I want to receive all the notifications in one place.

As a test session manager, I want to enrol all the participants to the event in the communication tool.

As a participant, I want to use my Gazelle login to access the communication space.

As a test session manager, I want to offer one public channel per system or at least one public channel per organisation.

As a test session manager, I want to reuse the same "default" channels from an event to the other.

As a test session manager, I want to pin a message at the top of a given channel as a reminder to readers.

Split of responsibilities for SUT

Submitted by abe on Fri, 12/17/2021 - 13:48
Status
Draft
Customer
Renovation
Business owner
Anne-Gaëlle BERGE
Description

Gazelle Test Management requires an owner to be set for each SUT. However, the owner might be a person who does not attend the event. In that case, it is problematic for the other participants and the test session manager because there is no onsite contact point.

In addition, some companies regiter their SUT for a large amount of capabilities and they attend with more than 3/4 persons. In that case, it is also hard for the participants to know who to address testing inquiries.

This new need tries to solve (at least partially) this issue.

What is/are the targeted context(s)?
Who is the targeted audience?
Targetted goal(s)

Give an accurate single point of contact to test partners

Expected benefit

Ease the communication between participants

Target the correct person when looking for test partners

Documentation

As a SUT Operator, I want to know who to contact to test a particular profile with one of my possible test partners.

If a communication tool is set up for the event, I want to have a link to send a direct message to my contact.

Manage the test session scope within the tool

Submitted by abe on Thu, 12/16/2021 - 15:30
Status
Draft
Customer
Renovation
Business owner
Anne-Gaëlle BERGE
Description

When the registration to a test event is open, a list of profiles is offered to the participants. In most of the cases, not all the actors of all the profiles will be selected by the registrants. That means that the test session manager needs to assess whether it is possible to test each individual profile or not. 

Currently, Gazelle Test Management allows the test session manager to download a PDF file which gives, for each actor of each profile the names of the systems that claim to support it. Then, the test session manager has to maintain, outside of Gazelle, the list of testable profiles.

When time permits, the test session manager marks the SUT AIPO for non-testable profiles as "no peer". It is a manual action that should be avoided.

 

What is/are the targeted context(s)?
Who is the targeted audience?
Targetted goal(s)

Support better the test session manager and domain managers in decided whether a profile can be tested or not.

Expected benefit

Ease the registration process for the test session manager.

Give a better overview of the scope of the test session to the participants.

Documentation

As a test session manager, I want to see the list of profiles selected for the current test session and, for each profile, I need the following information:

  • What are the actors involved in the profile;
  • What are the (non-dropped) SUT registered for each actor of this profile;
  • Which ones of the SUT are already accepted to the session.

As a test session manager, I want to be able to give a status to each profile among: not testable, testable, at risk, not assessed yet.

As a test session manager, I want to decide when to make the list of profiles (along with their status) public to the test session participants.

As a test session manager, I want to be able to ask the tool to mark as "no peer" all the AIPOs when the profile's status is set to "not testable".

As a SUT operator, I shall not be able to add a "not testable" profile to my SUT. However, I want to see it in the list of profiles so that I understand why I cannot select it.

As a SUT operator, I shall not be able to start a test for a not testable profile also the actors appear in the test plan (with status = "no peer").

 

SUT management page for test session manager

Submitted by abe on Thu, 12/16/2021 - 12:52
Priority
Medium - Feature to be considered
Status
Draft
Customer
Renovation
Business owner
Anne-Gaëlle BERGE
Description

Especially during the registration period, the test session manager has to regularly access the list of registered SUT to perform several operations or simply to follow-up on the registration status.

Currently there are too many pages targetted to the test session manager that are redundant and none of them show a complete view. This need is aimed to simplify the SUT management pages for the test session manager.

What is/are the targeted context(s)?
Who is the targeted audience?
Targetted goal(s)

Offer to the test session manager a single page where to manage the SUT for the current test session.

Expected benefit

Ease the registration process for the test session manager and save him time.

Opportunities

Reduces the number of pages to be maintained in the future.

Documentation

As a test session manager, I want to access the list of registered systems and see the following information for each of them:

  • SUT name
  • Organisation's name (+ ability to link it to more organisations)
  • Main point of contact for the SUT (full name + link to send an email)
  • Number of AIPO the SUT is registered for (allow to quickly see the dummy systems)
  • Registration status (+ ability to change it)
  • Whether the SUT is accepted or not to the session (and have the ability to change it)
  • Whether a contract has been received for the organisation (when a contract is needed)
  • The table where the system sits (+ ability to update it)

Use a standardized language

Submitted by abe on Thu, 12/16/2021 - 11:25
Status
Draft
Customer
Renovation
Business owner
Anne-Gaëlle BERGE
Description

In Gazelle, the wording is not always consistent with ISTQB and it is really Connectathon oriented. It leads to confusion to Gazelle users. The wording shall be adapted in several places to improve the user experience.

What is/are the targeted context(s)?
Targetted goal(s)

Adapt the wording to the event supported by Gazelle TM.

Use a common language.

Expected benefit

Improve the user experience.

Opportunities

Sharazone, Inter-AMC and EPD projects will directly take advantage of this update.

Documentation

As a first action, we should make the current Connectathon menu agnostic of the type of test session:

  • Use the "type" (from the testing session configuration) as display name in the menu bar where "Connectathon" is currently used.
  • In case we do not merge the preparatory tests and test session tests:
    • The "Pre-Connectathon" sub menu might be renamed into "Preparatory tests"
    • The "Pre-Connectathon Tests" sub menu might be renamed into "List of preparatory tests"
    • The "Pre-connectathon results" sub menu shall be renamed into "Preparatory test results"
  • The "Connectathon" sub menu might be renamed into "Testing"
  • The "Connectathon" menu entry (lower level) could be renamed "Execute tests" (as well as the short cut in the top menu bar)
  • The "Connectathon result" menu entry should be renamed "Test session results"
  • The "Download connectathon report" menu entry should be renamed "Download test session report"

Then, everywhere in the tool, we should use the following vocabulary:

Current term New (English) New (French)
TF Interoperability model  
Tests list Tests Tests
Test definition Test cases Cas de tests
MetaTest List    
Configuration SUT Configurations Configurations des SUT
Network Network details Informations réseau
All configurations SUT network configurations Configurations réseau des SUT
Connectathon    
PreConnectathon Preparatory tests  
Test instance Test run  
Testing session Test session  
system SUT  

 

Add dashboard for the TSM to follow-up with the registration and test execution

Submitted by abe on Thu, 12/16/2021 - 11:00
Status
Draft
Customer
Renovation
Business owner
Anne-Gaëlle BERGE
Description

During the registration and the execution of the event, the test session manager has to look into different places to find useful figures to manage the event. To ease the follow-up of the session, creating dashboard to quickly have a look to various indicators is necessary.

What is/are the targeted context(s)?
Who is the targeted audience?
Targetted goal(s)

Give an overview of the registration and the test execution to the test session manager.

Ease the report to the IHE T&T committee by provided the expected figures to the test session manager.

Expected benefit

Ease the work of the test session manager and save him time to support better the participants.

Documentation

For a specific testing session:

  • As a test session, I need to know how many profiles are offered for registration (overall, per domain)
  • As a test session manager, I need to know how many organisations have registered SUTs and how many of them have downloaded their contract, and how many have sent it back;
  • As a test session manager, I need to have an overview of the statuses of the SUTs (total + in progress, completed, dropped, accepted);
  • As a test session manager, I need to have an overview of the number of participants (total + monitor vs vendor) and when (which days) they intend to attend to the event;
  • As a test session manager, I need to have an overview of the status of the SUT network configurations (percentage of approved configurations, list of accepted systems with no approved configuration)
  • Before the event starts, as a test session manager, I need to have an overview of the status of the preparatory tests.
  • During the event, I need to have an overview of
    • the status of the test runs (running, waiting for validation, verified, passed etc)
    • list of organisations with no test
    • list of SUTs with no test
  • After the event, I need to have an overview of the test session results:
    • How many AIPOs have been marked as Passed?
    • How many different profiles have been marked as Passed for at least one system?
    • Top 5 profiles in terms of Passed AIPO
    • How many different actor/profile pairs have been marked as Passed for at least one system?
    • How many different tests have been executed (overall and per test types: no-peer, peer-to-peer, group)
    • Average of test runs (verified, partially verified, failed) per monitors
    • Average of test runs (verified, partially verified, failed) per SUT
    • Percentage of not verified tests (to be verified/critical) on the overall number of completed tests (excl. running, aborted, paused)
    • List of domains where tests have been executed
  • For each profile, the list of systems for each actor
    • Hide dropped systems
    • Highlight accepted systems
  • Indicators when
    • Each actor is covered by at least 3 differents systems
    • A system acts as all the actors
    • Only systems from the same organisation are registered for the profile
    • No registration at all