Going to the extreme - xBTM

Now that the project has finished it is time to sum up my experiences of adapting Thread-Based Test Management (TBTM). Since I generally do not believe in rigorously adhering to a protocol, I ended up not using TBTM strictly, but instead embraced a hybrid of Session-Based Test Management (SBTM) and TBTM. Naturally this hybrid will be denoted xBTM. 

The project

In order for this text to make sense, a few words on the project are needed. The customer runs an application that generates output which is stored daily in a single XML file, and later used by the customer’s own applications to derive vital business statistics. There were defects that needed correction, and the customer also asked for new features regarding how log on/log off was recorded. The corrections as well as the new requirements should be implemented in a new file that was supposed to be generated in parallel with the old file for a transition period. 

The team

It was a small project. I acted as project manager, test manager and tester. Later in the project I was joined by a second tester. There was one single developer. 

TBTM

Most of the time my working environment in general is simply too hectic and borderline chaotic for SBTM to be suitable. There tends to be a lot on interruptions and distractions, and it is rarely possible to sit down and focus on a single test task for a given time period. Therefore I decided to give TBTM a try when a new project started.

My first step was to make a mind map containing all my test ideas as threads. Since I am very fond of open source products I used the software FreeMind. In the mind map below, e.g. File Name is a thread. XML file is the actual product. ID denotes a defect, and CR denotes a new change request. The test threads are grouped in two ways. Most groups represent key areas, e.g. Generation which means file generation. Other groups are formed based on the type of testing, e.g. Stress testing. In some cases I felt short notes were needed to explain the thread, and in those cases I attached text files – knots – to the thread. These files are shown as red arrows in the mind map. You click on the arrow to open the text file.

Test plan.

I tried using colours and icons to make the mind map easier to read. The warning signs mark key areas that I judged to be high risk and especially important. The stop light marks a thread that was tied off. That particular change request was retracted by the customer just before the project started. The initial mind map as it was before I started testing made up my test plan and was sent to the customer.

During the test period I would constantly be updating the mind map and it would always give me an accurate picture of the current status of the testing. As soon as I started working on a thread I would mark it with a smiley, see image below. Threads where I found defects were marked with a red cross, and threads where I felt sufficient testing for delivery had been done (not the same thing as claiming to be done testing!) were checked off in green.

Snapshot of mind map in the middle of the test period.

 

Initially my goal was to write daily status reports containing a few short notes on what I had done. In reality I only kept this up for four days. With my constantly updated mind map and the occasional session report (see below) I really did not feel a need for it.

When there was no more time for testing, I took the current status of my mind map and used it as my test report, see below. This test report was sent to the customer.

Test report.

 

SBTM

I do still like SBTM and I find the session reports as well as the time-boxing very useful, so when circumstances would allow, I created test charters and ran time-boxed test sessions. Typically a couple of threads would make one session, but in come cases one thread deserved a session of its own. For example, looking at the test plan the two threads File name and File header belonging to the key area File generation would be tested in the same session, whereas Install from scratch and Install upgrade belonging to the key area Installation were tested in separate sessions.

In most cases I would draw a simple activity diagram – pattern – for each test charter, see example below. I prefer using yEd for my diagrams.

Test charter.

 I wrote short session reports according to a new AddQ session report template, see image below.

SBTM session report template.

Using the AddQ tool SBTExecute I could then derive metrics such as Total session time (for all sessions), Number of test charters, Number of test charters per key area, and so on from my session reports. However, since I was still experimenting in this project, as well as working on the template and the tool as I went along, I could not obtain any useful statistics. Also note than any metrics derived would only be valid for my SBTM sessions – and a considerable part of the testing was done according to TBTM.

Summary report generated by SBTExecute.

Summary and Conclusions

What is the main difference to previous projects I have worked on? Well...the first thing that comes to mind is that this time I actually used the test plan! And I have read and come back to the test report. Keeping the mind map up to date has been much easier than updating any other kind of status document, which means that it actually has been updated. As it turned out, even the developers liked it – they preferred looking at the mind map over using our bug tracking tool!

I immediately liked the combination of TBTM and SBTM – it is a simple matter of applying the methodology that is best suited for the task at hand. Some threads were fairly isolated and straight forward to test, making them suitable for SBTM. Other threads were very convoluted and had to be explored together with the developer, making them better suited for TBTM.