Conformity Assessment

In Gazelle Test Management, the tests are executed from the Test execution page. This page also logs all the runs executed by the systems. The test runs are linked to the test case they relate to.

The test execution page shows only the test cases with a status set to "ready". As a consequence, if a test case is deprecated at some points, the SUT operator cannot access the test runs that were executed when the test case was "ready". 

In the same way, the "All test runs" page displays only the test runs that relates to the "ready" test cases.

As a consequence, we experience a loss of information and that is not accessible neither from the SUT operator's or Testing Session Manager's point of view.

• As a SUT Operator, I want to see all the test runs that have been executed by a system under test owned by my organisation, during a given testing session, at any time.
• As a TSM, I want to see all the test runs that have been executed during a given testing session, at any time.
• As a user, for each test run, I want to see the following details:
  • Test run identifier
  • Test case keyword (and a link access its details)
  • Whether the test case is currently deprecated
  • Last update date/time
  • Test run status
  • The role assigned to my SUT in this particular test run
  • The other partners
  • The status of the test run
  • If a monitor is assigned, his first and last names
• As a user, I want to be able to access the test run page for any test run (whatever the status of the test case is)
• As a user, I want to be able to access the description of the test case.
• As a user, I want to be able to filter on
  • System (name/keyword)
  • Test case keyword
  • Monitor
  • Type of test
  • Domain
  • Tested profile
  • Tested actor
  • Tested profile option
  • Involved transactions
  • Test run status
• As a testing session manager or admin user, I want to be able to filter by Organisation (name/keyword)
• As a user, I want the test runs to be grouped by tested SUT capabilities
• As a user, for each SUT capabilities, I want to see the evaluation.
• As a user, I want to be able to sort on test case and test status within each SUT capability.

When navigating and working in Gazelle Test Management, the users need to access other test tools in order to execute the test cases.

As a tool administrator, I want to provide the links to the tools of interest for the users to access them from whatever page in Gazelle Test Management. This list shall be organised according to the project's context. 

As a SUT operator or a monitor, I want to see the links to the test tool I need to use when I execute/grade tests. The links shall be available on all the pages so that it is very quick to find the pointers.

In the context of the Segur labelling, the operator uses a tool named Convergence to submit his evidences.

Part of the testing is conducted in Gazelle Test Management. Currently, the workflow is as follows:

• The operator logs into the relevant test session in Gazelle Test Management;
• The operator executes the test cases for his system under test;
• The operator load the evidences in Gazelle Test Management;
• The operator marks the test run as "ready to verify";
• A monitor grades the test run;
• If the result is set to "verified", the operator copies the link to the test run and paste it in Convergence.

To avoid manual copy/paste and the potential errors that might cause, the operator wants to be able to pull the test runs from Convergence. The result shall be a JSON object that lists the test runs with, at least, the following informations:

• Test session where the test has been executed;
• System under test which executed the test;
• Name of the organisations involved in the test;
• Test case name;
• Test run status;
• URL to the test run;
• Last update date and time.

There should not be a need for the user to authenticate himself when Convergence calls Gazelle Test Management. The Convergence tool shall be authorized to query data on behalf of the logged in user. Input parameter: organisation's name or keyword.

Assumption: The organisation's name or keyword in Gazelle and Convergence are equal.

The SUT edition page shall be reworked in order to make it clearer, and more complete, so that the participants to not have to jump in multiple locations to manage their systems.

Avoid using tabs and use a navigation bar instead.

Especially during the registration period, the test session manager has to regularly access the list of registered SUT to perform several operations or simply to follow-up on the registration status.

Currently there are too many pages targetted to the test session manager that are redundant and none of them show a complete view. This need is aimed to simplify the SUT management pages for the test session manager.

In Gazelle, the wording is not always consistent with ISTQB and it is really Connectathon oriented. It leads to confusion to Gazelle users. The wording shall be adapted in several places to improve the user experience.

In order to follow-up with their progresses during the event, the SUT operators need to access some graphics to show them their progresses in terms of:

• SUT registration status (dropped, in progress, completed, accepted)
• Approved vs unapproved network configuration (overall, per SUT)
• Connectathon/Projectathon test statuses (overall, per SUT, per SUT per type: preparatory tests, group tests, no-peer, etc)
• "AIPO" Results (overall, per SUT)