I attended a HYSEA Workshop on S/W Products’ Testing, organized at Microsoft Campus. Software testing workshops/talks are not as frequent as those for technology or technology trends and it was heartening to see a step towards correcting this anomaly. In this post I will try to outline the topics that were discussed and give opinions wherever I have one. The focus, as the title suggests, was on testing software products.

The first talk was a keynote address by Amit Chatterjee, General Manager, Visual Studio Team Test at Microsoft India Development Center. He lamented the near absence of innovation in the field of software testing. Most product companies, Microsoft included, have focussed creating tools for enabling the development process and the developer. Software testing as a component of product development lfecyle has been largely neglected. There are very few cases where software testers get involved in the product life cycle at an early stage mainly due to i) Lack of the required skills ii) Lack of adequate tools. This is the reason why we do not have software testers who vet early stage artifacts like the design/architecture of the product. Product testers are a handicapped lot today.

Amit made a prediction that software testers would be obsolete (not required) 10-20 years from now and the role would evolve into that of software designers. Such would be the level of software test automation that quality would be taken for granted. Amit also put his fingers on three trends that would help bring about the change in the approach to software testing and quality

1. Virtualization: Not just virtualization(software/hardware) as we know it but also the use of crowd-sourcing to test products. Software produts are designed to run in gamut of environments but there is a lmit to which this product can be subjected to all those environments keeping the depth of testing uncompromised. Entrusting the end user to test the product on his environment ensures that the product runs on the environment it is intended to run on + its functionalities are tested by actual users.

2. Information: Testers today are handicapped when it comes to INFORMATION. Information will not just be restricted to ‘test execution’ and ‘test case’ reports’ but data on “what code is running”, “what is different from the last build” etc., Again the tester himself would be technically sound to use this information.

3. Visualization: Basically how software testing is viewed today. The software is considered done once the deadlines are reached not when we reach a conclusion that software is sufficiently bug-free. This needs to change. Just the way when a car is is assembled completely we know it is ready to be shipped, we should have something similar in software to ensure we do not have to depend on ‘experience’ (which is always a subjective thing).

Therafter, Madhu Murty Ronanki of Primora talked about ‘Challenges in Software Product Testing’. His discussion was more general in nature and touched upon how IT Services companies have achieved a level of maturity when it comes to testing software applications but have quite some ground to cover when it comes to software products. To summarise his points:

  • It is important the tester comes out of the ‘I am not concerned with the code. I am only going to do black box testing’ mentality. He should make an effort to understand the opportunity or requirement that the product is intended to address and also the technology behind the product.
  • It is important to gain domain knowledge
  • It is important to recognize that the a different set of testing aspects gain precedence for products. Interopability and Usability testing hold a lot of importance in this context.

The third speaker, Ashwin Palaparthi of Valueminds, talked on “Ten Current Noteworthy Trends in the Application of Software Testing’. His talk focussed on the need to bring the punch into the activity of software testing by marrying the existing testing processes with the output from mathematical and statistical research. He seconded Amit’s point of adopting a new approach to software testing where tools play a more important role. The actual topics are worthy of a separate blog post, which I will be writing soon.

The fourth session was by Hema Venktaraman who discussed “Usability Testing” (UT). The talk was very detailed on the “how” of UT but missed out on the more important “what” and “why”. She touched upon the various roles during UT, who should do UT, why should a developer stay away during UT and other such details. But it failed to drive the point of how should usability be kept in mind by a tester. How usability issues vary with the audience. A pertinent topic but could have been wider in its scope.

The last session was ours where Sai and I talked about “Testing Enterprise Mashups” – the challenges, the consideration and the approach. We will be running a series on this topic on this blog. Make sure you catch those.