The impact of detecting a bug late in the delivery cycle

The impact of detecting a bug late in the delivery cycle can be best illustrated with this short joke:

Once upon a time, in a jungle there lived a strong and cruel lion. He would hunt and kill animals at random to satisfy his own sadistic pleasure rather than his hunger. All the forest animals lived in perpetual fear of the lion. This cruel and reckless killing went on and on until one day, when the Lion was sleeping all the other animals mustered courage and gathered for a meeting.

“This simply cannot go on!”, the wise old Elephant trumpeted. “We cannot continually live in fear of of the lion. We have to find a solution.”

All the animals in the gathering nodded in assent.

“The Elephant speaks the truth.”, the Giraffe said. “Let’s get our heads together!”

“Let’s set a trap for the Lion in the forest and kill him”, the monkey said.

“Let’s seal off the entrance to his cave. He’ll remain trapped inside and die of hunger and thirst.”, the deer suggested.

The Elephant shook his head. “All that you have suggested has been tried before. The Brute is too strong to be trapped inside. He will simply dig his way out and continue his evil deeds. He is also too cunning and shrewd and can smell any trap.”, he replied sadly.

“What do you suggest then?”, the Wild Buffalo asked.

The Elephant replied, “I have thought about it long and hard. There is only one way. It’s not an easy solution, but it will guarantee that all in the forest will live in peace at least.”

With bated breath the animals waited for the pachyderm to announce his solution.

“I propose that we request the lion to kill one chosen animal each day instead of him killing all of us randomly and indiscriminately. That way only one animal has to die and the rest of us can continue living in peace. We can draw lots and choose the animal from amongst us.”, the Elephant said.

With no better solution in sight, the other animals agreed to his plan. Trembling with fear, they went to the Lion’s cave to put forth their request.

The Lion, fresh after a good sleep was surprised to see the entire forest gather in front of him when he stepped outside.

“Aha!”, he roared. “Today my food has found me, saving me the trouble to search.”

The Elephant mustered courage and addressed the Lion thus, “O Mighty King of Beasts! In the past few years, none of us have been able to escape your cruelty. We live in constant fear of being hunted by you. Our lives and daily routines are in shambles. We cannot continue living like this. Therefore, for the good of the jungle and all those who live in it, we have a proposition for you. Please hear us out. We beg you agree to it.”

The Lion stifled a yawn and said, “Ok, go ahead!”

The Elephant outlined his proposition while all the other animals listened on eagerly.

Finally, when the Elephant was done explaining his solution, the Lion thought for a while.

Then, after what seemed like an eternity for the other animals he said, “I agree!”

The announcement brought loud cheer from the gathered crowd.

The Lion waited for the cheers to subside and then said, “But, I have one condition.”

Silence fell across the crowd, each animal wondering what condition the Lion wanted, worried whether they would be able to fulfill it or not.

“Name it!”, the Elephant said.

The Lion said, “I will tell you a joke. My condition is that all the animals HAVE to laugh. If I see anyone not laughing, the agreement becomes null and void and I will resume my old ways.”

Relieved, all the animals nodded in assent. The Lion started his joke. It was a PJ and probably one of the lousiest jokes they had ever heard. But when he was done, all the animals broke out into hilarious laughter. Even the crocodiles suppressed their tears and were rolling on the ground laughing their head off.

The Lion surveyed the crowd to check if any animal was not laughing, but could find none. But just as he was about to turn away, in the corner he noticed a tortoise peering outside it’s shell. The tortoise looked around grim and serious, probably wondering what the commotion was all about.    

“Aha! I see one animal not laughing to my joke.”, he said pointing to the tortoise. “The agreement is now null and void. I shall resume my old ways again”, the Lion roared.

The animals angrily glared at the tortoise which innocently went inside its’ shell, totally oblivious to the situation.

Once again, the Lion resumed it’s old cruel ways. But the next day, the animals gathered outside the Lion’s cave again and one more put forth their proposition. The Lion agreed but with the same condition as before. This second joke was even more stale and lousy than the first one, but all the animals put up a brave front and laughed their head off. All, except the tortoise which once again grimly stared outside his shell, a dead serious expression on his face. Once again, the Lion noticed the Tortoise and pointed it out with devilish glee. All the animals looked daggers at the amphibian which, oblivious to the hostility directed at it slunk back inside its’ shell.

And yet again, the lion resumed it’s cruel ways. But after a few days, the animals gathered again outside the Lions’ den to make the same request. This time they did not tell the tortoise as they feared it would spoil the show again. But the tortoise got wind of the plan and followed them at a distance.

But this time, the Lion had a different condition.

“I will tell you a story. All of you will have to CRY at this one. This is your last and final chance. If you screw up I will get back to my old ways and this time, no one will stop me!”, the Lion roared.

Terrified, the animals waited till the Lion completed his story. When he was done, they started bawling shamelessly. The Hyena came out of his comfort zone and really cried his heart out. Loud sounds of tears and crying could be heard from all sides. The Lion did a careful survey to check for any defaulters. And to his surprise, this time he saw the tortoise rolling over in his shell, LAUGHING away to glory.

Enraged, he roared, “THIS IS AN INSULT! THE DEAL’S OFF!” pointing towards the amphibian.

Dismayed, the animals looked at the direction the Lion was motioning and noticed the tortoise laughing.

Once again, oblivious to the situation around him, the tortoise innocently said between laughs, “That first PJ was hilarious!”  

--------------------------------------------------------

Here are some lessons from this joke:


Hope you enjoyed the joke and more so, the meaning I tried to convey through it. Detecting a major bug very late in the delivery cycle can cost a lot to fix in terms of effort, quality and not to mention frayed tempers and overall project health. So as a good tester, it's of paramount importance to prioritize your tests and make sure the critical scenarios are covered first to reduce risks.

Happy testing !!

Cheers,

Srinivas Addanki

 

The current state of Software QA in the Indian IT industry

An ex-military man applied for the post of Software QA Engineer in a reputed Indian IT firm. Being technically competent, he easily cleared all the technical and HR rounds. While the offer letter was being prepared, the HR executive and the candidate were having a conversation.

The exec noticed that the candidate had ticked the physical disability check-box.But on the surface, the candidate appeared physically fit, with no sign of any injury whatsoever.

Curious to know, the HR asked the ex-military guy, “Er.. Sir, I’ve noticed that you’ve ticked the physical disability checkbox in the form. May I ask about the nature of your disability?”

With a deep sigh, the Military man responded, “If you must know, my testicles were blown off by a landmine in a military operation.”

“Ouch.”, the HR executive responded.

“Now that you know, I would appreciate if you keep this fact private. I would also like to make it clear that I do not wish to be shown any preferential treatment or discrimination owing to my disability.", the Military man stated clearly.

The exec assured him that he would keep the fact private and that there would be no preferential treatment or discrimination shown to him in any form whatsoever.

After the papers were signed, the exec warmly shook hands with the candidate and said, “Welcome to the company. You can join the coming Monday. Your daily reporting time is 11 AM.”

The Military guy responded, “Thanks a lot. By the way, I noticed that your company working hours are from 9 AM. Why should I report at 11? Am I being placed in a shift?”

The HR replied, “Err…. No. That’s keeping you ‘unique disability’ in mind.”

The Military guy flared up; anger bristling in his voice. “Did I not just tell you that I don’t want to be shown any preferential treatment for my disability?”

The exec replied in a calm and soothing tone, “I am extremely sorry to have offended you sir. I did not mean to show you any preferential treatment. Allow me to explain. Our company QA team reports at 9 AM each day. But since the build arrives at 11 AM, for two hours our team does nothing except scratch their balls. Since this does not apply to you, you might as well report at 11 AM.”

And that, ladies and gentlemen is the current state of Software testing in the Indian IT industry.

  

Murphy's Laws of Software testing

Taken from my facebook wall. Here goes:

1) Bugs can easily be created. They cannot however, be easily destroyed. But they can easily get transferred from one platform to another.

2) There is no such thing as a bug free application. But yes, bugs do come free with every application. :)

3) Bugs do not get reproduced when you demo them to the developer / project manager.

4) But will magically reappear from nowhere when you demo your product to the client / customer.

5) Your most bug-free product version will reside in your developer’s local machine / session. Bugs never happen there.

6) The more the number of developers working on the product, the more bugs there will be to find.

7) Bugs which have been deemed fixed and closed always reoccur in Production.

8) No matter how extensive your test coverage is, you will always miss / overlook one aspect of the product.

9) And that’s where the bug will usually occur in production.

10) No matter how many critical bugs you find, you will be pilloried for the one bug you failed to detect.

11) The PM and delivery manager will always consider the testing team as dead weight.

12) The bug which takes the maximum amount of time to reproduce will always take the least amount of time to fix.

13) The bug which causes the maximum amount of damage always requires the smallest of fixes.

14) Customers / End users usually find more bugs than your testers’ do.

15) Bugs will never be apparent if you keep looking for them, but when you are not, they will pop up from every nook and corner of the application and will hit you in the face.

16) Automation might get things done quicker, but the amount of time it takes to script the tests and run them will make you think of running the tests manually.

17) The most critical and time consuming bug for the developers to fix will always occur at the fag end of the testing cycle or when the product is ready to ship.

18) You will spend a huge amount of time and energy in detecting a bug only to find that it now not reproduced.

19) Even if you do reproduce the goddamn bug and file it, it inadvertently turns up not to be in scope.

20) Even if it is in scope, someone else would have already filed it a long time back.

21) The product / feature for which you have detected the maximum number of bugs will most certainly be de-scoped.

Developers’ ultimate justification for bugs:

“But it does not happen on my local machine!!” :)

"This issue is not in scope."

Murphy’s Ultimate Law of all products:

“It just won’t work!”

The death of the Testing specialist

In many QA Job opportunities and postings these days, I have observed that almost 90% of the job descriptions specify coding and automation knowledge as mandatory. This does seem natural, going by the logic that IT and software companies do require a certain amount of computing cognizance. But as a hard-core QA professional who’s spent more than 12 years trying to break code, it really doesn’t make sense.

And why you ask? Surely a good tester should know how to code? I beg to differ. While I agree that knowledge of coding is definitely an advantage for a tester; a feather in one’s cap, it is no substitute for hard-core testing and QA skills. It takes a fair bit of creativity, instinct and effort to find critical bugs in software. In today’s highly complex software scenario, testing has evolved into a highly specialized and viable career option rather than an offshoot of development and coding which many sadly perceive it to be even today. It is a mistake to ask your tester for coding skills. If the testers start to code, then what will the coders do?

In the good old days of yore, software project teams were split into specialized teams based on their job functions: BA teams for requirement gathering and liaising with the client, A technical architect for designing the software architecture, DBA teams for setting up and maintaining the Database,  Development teams for coding and unit testing, a QA team for preparing Test cases, RTM and testing the application and an Application Maintenance team for providing support after product Go-live. Each team had a manager for guidance. Each team had their defined boundaries and everyone had a clear cut task and skill. The work was well-defined and everyone was happy.

But the scenario is changing fast. Software is growing in complexity day by day. The landscape has become very competitive, especially for IT service providing companies based in India who look to add value to their clients by delivering high quality software in a reasonably quick time. Newer and more dynamic methods of Software delivery such as Agile have replaced traditional software models. In a bid to maximize revenue, many companies actually take testing very lightly and look to reduce the head count of the testing team. In many teams, the developer to Tester ratio is highly skewed in favour of the dev team. Many project and delivery managers sadly have a very biased opinion about testing. “Testing?!! Mehhh ….. anybody can do that!” they say. Very little priority and thought is given to testing these days and this is the prime cause of software failures. When software fails, the credibility of the organization and the team which built the software takes a hit and this results in numerous other complications.

Which is why you need specialized testing teams for detecting bugs before deploying the application and going live or before UAT. The art and science of detecting defects in code is a very specialized skill which requires a lot of patience, perseverance, skill, creativity, instinct and experience. No amount of coding experience can substitute it.

Let’s take a simple example. Lets’ say you’re a newly appointed chef in a reputed hotel and you have just rolled out a new signature dish. You will of course, taste a bit of it to check if all the ingredients have the right consistency. Too much sugar or too little salt, a dash of chilly and pepper to boost the taste. If you’re satisfied, you can send the dish out to the customer. But you would be taking a great risk in doing so. A wise and experienced cook will always have the maturity to let a specialist taster and sampler to sample the dish before sending it out. Let’s face it, we’re human. If we were in the cook's place, given the time and effort we have spent in making the dish we will tend to feel a bit biased towards it. Deep in our heart, we would like to believe that our dish is perfect. This unfortunately, is an ideal scenario. Something which is non-existent in the real world. Therefore, we would need someone experienced, neutral and unbiased for tasting our dish. Someone who can rightly detect any inherent flaws we might have inadvertently made in preparing the dish. Someone who understands what cooking and good taste is all about and also understands the customers’ needs. This would go a long way in ensuring that your dish is well received. The flip side is: if your dish isn’t well received, you can always blame the taster ... :)

The same is the case with Software Development. Software Developers and Managers must realize that no matter how expert they are in interpreting the requirements and coding and despite their best efforts, bugs and critical errors inadvertently creep in. Some might be minor or cosmetic issues, but some might be show stoppers and might ruin the software’s performance, costing the company a huge amount of money to make a fix. Like I said earlier, we’re only human. And developers do have the tendency to think that their code is infallible. Every Software team needs a good amount of specialized and experienced testers who can detect an inherent amount of bugs in the application. Like the specialized tasters, testers (pardon the synonym) help in reducing errors and improving software quality by making sure your software product performs as per specifications and meets the end users’ requirements. As I mentioned earlier this kind of testing is a full time job which requires loads of patience, perseverance, meticulousness, creativity, a sharp eye for defects and various other positive adjectives. Having a good knowledge of coding makes you a good programmer but does not necessarily make you a good tester. And of course; if things do go wrong in production … you always have the QA team to blame …. :).

There are of course some areas where coding expertise is required, such as Automation testing. But there again, a good record and play tool eliminates the need for complex scripting. And as I mentioned in an earlier blog post (Manual V/s Automation), a good round of manual testing is required before you can proceed for further testing such as Automation and performance.

So whether you like it or not, Dev guys; wake up and smell the coffee. You will need testers to examine your code.

Asking your tester to code is like asking your number 11 batsman to score a century each time he comes out to bat. When there are runs required to be scored and wickets are few in hand, your batting skills will come in handy. But that's not always applicable. You have specialist batsman to do the job and if they have let you down then you cannot blame the tail-enders for not scoring runs always. As in software, each person in a team should have a core skill (batting, bowing, wicket keeping) and he should stick to it and develop it well.  

This is an open request to all IT companies, big or small to let go of their obsession for testers with coding knowledge. This attitude has made testing an endangered profession as it is. If this attitude persists the day is not far off when testing and QA will be added to the extinct professions’ list. You ignore specialized testers at your own peril. In the long run, the effects on Software Quality will be there for all to see.