![]() |
![]() |
![]() |
Test books are a collection of test sheets representing the test scenarios a tester should pass in order to validate the software.
The test book is divided into Modules which are again divided into Steps. You can consider these as chapters and sub-chapters. Of course all the tests can be put into the same module and step, but experience shows that it is better to find some other way of classification.
The access to the book is done through two main entry points:"Passing a test" means that the scenario that was written in the test book has been run. The result of the run may be OK, or NOT-OK. The result is considered OK if the goal of the test has been reached.
Sometimes, the purpose of the test is to obtain an error message. If the error message has been reached, the result of the test can be considered OK, even though the scenario has lead to a failure.
When writing the contents of the test book, you need to describe the contents of the modules, the steps, and the test sheets.
When clicking on the button "Test Book", you see the following table :
Identification |
|||||||||||
Project : |
MyProj Ñ |
SubProject: |
SubProject Ñ |
Phase : |
Integ Ñ |
||||||
Contents |
|||||||||||
|
MyMod Ñ |
New |
Edit |
View |
|
Delete |
|||||
|
MyStep Ñ |
New |
Edit |
View |
|
Delete |
|||||
|
042 Ñ |
New |
Edit |
View |
|
Delete |
|||||
Book |
|||||||||||
|
|||||||||||
|
|||||||||||
First you need to identify the project, the subproject and the phase you are working on. Each project, project and each phase can have it's own test book. This is because you may have two different test books during in-house and on-site validation phases.
Then you can create new modules, steps and test sheets by pressing the test button.
You can edit these by pressing the edit button.
You can view them by pressing the view button. If you view a step, it will also show the test sheets that are included. If you view the module, it will also show the steps. If you click on the view button of the book, you will see the entire book.
The print button allows you to print a text file to the printer instead of the view you have on the screen. Remember that it can be possible to import the HTML source into your favourite test editor, and so you can format your output any way you like.
The delete button lets you to delete the selected item if there are no contents. For example, you can delete a module if there are no steps inside. You can only delete a test if there are no results recorded.
If you really want to delete some items, you can remove the corresponding files with extreme caution. Their location is explained in chapter 9.3 Test book files.
When you create a new test sheet, you have to fill out the following form:
Title |
|||||||
Author: |
Ñ |
Impact : |
Ñ |
Reference |
|||
Objectives: |
|||||||
Environnement: |
|||||||
Pre-requisite: |
|||||||
Steps to perform: |
|||||||
Expected results: |
|||||||
Post Steps: |
|||||||
Notes: |
|||||||
The title is the title of the test.
The author is the person who writes the test.
The impact is the type of the test (functional, robustness, etc.) as configured in the setup.
The reference is the file or directory or script that is used to run the test. No verification is done on this field.
The rest of the fields are free, and pretty much self explanatory. The user has to describe:
These explanations should allow anyone to reproduce the same test in the same conditions.
Each test of each test book can be passed several times. One of those times is called a "test campaign". During a campaign, parts or the entire test book is passed on one or several versions of the software.
If you wand to pass tests, you have to click on the "Test Sheet" buttun in the main menu, and you see the following table :
When clicking on the button "Test Book", you see the following table :
Identification |
||||||||||
Project : |
MyProj Ñ |
SubProject: |
SubProject Ñ |
Phase : |
Integ Ñ |
|||||
Campaign : |
Ñ |
New |
||||||||
Contents |
||||||||||
|
View |
|
Stats |
|||||||
|
MyMod Ñ |
View |
|
|||||||
|
MyStep Ñ |
View |
|
|||||||
To be able to pass tests, you need to choose a test campaign. You can also create a new test campaign by giving it a name. By default, the starting date is proposed, but you are free to choose any other. Make sure that each member of the testing team knows in which campaign to record his results.
The view button lets you see a summary of the test book. A view of a step will look like this :
Tst# |
Title |
Result |
Tester |
Date |
Problems |
Status |
000 |
First test |
NOK |
bkiss |
14/10 |
T.i20.00045 |
A |
001 |
Second test |
OK |
bkiss |
14/10 |
||
002 |
Third test |
NT |
||||
003 |
Fourth test |
NA |
The test number is a link that leads you to the description of the test as it was written in the test book.
The test result is a link that leads to a form where you can record the result of the tests. The values of this filed are explained later (9.2.1 Test Results ).
The tester is the person who passed the test.
Only the day and the month of the date are shown in this table to save some space. The exact date is available in the history part of the results.
The problem field is a link that leads directly to the bug report which has been raised when the test failed. Several bug reports can be created for the same test if needed.
The status field gives the status of the bug report. This field is updated automatically when the page is displayed.
Be careful: the fact that the problem has been corrected (C state) does not necessarily mean that the correction has been delivered.
When you click on the test result link you arrive on the following table :
001 |
Title of the test |
|||||||||||||
Tested by : |
myself Ñ |
Version: |
1.2.3.4 Ñ |
Result : |
NOK Ñ |
|||||||||
Date : 2001/06/21:14.08.52 |
OK |
|||||||||||||
Obtained results: |
||||||||||||||
Related Fact Sheets: |
||||||||||||||
New |
||||||||||||||
2001/06/17:22.13.16 Test Created by bkiss (OK) |
||||||||||||||
2001/06/17:22.22.41 Test Changed by bkiss (NOK) |
||||||||||||||
2001/06/21:14.07.43 Test Changed by bkiss (NOK) |
||||||||||||||
2001/06/21:14.08.52 Test Changed by bkiss (NOK) |
||||||||||||||
To fill in the form, you have to :
The history will show all the evolutions.
The test book files are placed in the same directory as your data:
<data directory>/Tests/<Proj>/<SubProj>/<Phase>/Book/
Where :
In this directory you will find the module, the step, and the test files.
The test result files are placed in their own directories named by campaign:
Book/test/C.<campaign>/<mod>.<step>.<###>
When you click on the [Stats] button in the statistics window, the statistics are computed for the current campaign:
You can see the header with
Each module is in a separate table. The total concerning each module is shown at the bottom of each table.
The total concerning the whole test book is shown in a separate table at the bottom of the page.
In each module, you can see one row per step.
Each step contains a certain number of tests. This number is shown in the number of tests column.
Out of these tests, some do not need to be passed. The number of tests to pass is shown in the next column. (not in NA state)
The next column shows the number of tests passed. This does not mean that the result is OK, it only means that the scenario has been run, with or without success.
The number of tests that succeeded is shown in the next column marked "OK".
The last three columns show a percentage. These are:
This last percentage (OK/P) is the indicator of the quality of the software. The higher the number of tests run, the more accurate this figure is.
![]() |
![]() |
![]() |