QA Guru

On the Noble Art of Quality Assurance

2008-02-21T09:57:00.000-08:00

So you want to be a Software Tester?

Do you have a knack for figuring things out? A knack for figuring things out that goes beyond the obvious, even to the point of figuring things out that the developer didn't figure out when he was writing the code? And yet, do you feel it is time to elevate that 'knack' to a whole new level? Do you want to take charge of the test process and elevate it to an art form?

Then Welcome to the Noble Art of Quality Assurance!

The goal of Quality Assurance in a software organization is simple: perfecting the software? That means: perfecting the developers, perfecting the processes by which software is developed, and ultimately perfecting the relationship that the software (and the company behind it) has with the end user: the customer.

There are many today that do not comprehend the difference between Software Quality Assurance and Software Test. They may deem that 'Quality Assurance Engineer' is simply a more elaborate title placed upon the Software Tester, or they may consider that the Quality Assurance Engineer is simply a Software Tester who is also schooled in the use of various tools for reporting of test metrics.

Both of these concepts touch upon the truth, but they also both entirely miss it.

The use of metrics with respect to testing, development, test coverage, bug finding, and so on, are only a tool that the QA Engineer uses on his path to helping mature his company's development processes. The end goal must always remain in sight: help mold this company's products into something the customers can depend on. Help elevate the quality of workmanship that goes into the product. Quality of workmanship is the path to a quality product, a dependable product that the customer will thank you for, both by continued patronage and word-of-mouth promotion.

In a company that makes proper use of the Quality Assurance role, the Quality Assurance Engineer is able to use both title and tools to elevate the quality of workmanship. The company gives the QA Engineer the authority to hold up a release, if needed, in order to ensure the highest quality product is released. The company gives the QA Engineer the authority to hold developers accountable to fulfillment of requirements, and overall meeting of the project vision.

This is not an 'authority' that the QA Engineer abuses, or uses lightly. It is a serious thing to hold up a software release. It can ultimately result in lost revenue. But just as serious is the authority to release a product 'on time' even if it is not ready. Not only will such a release result in lost revenue, but what's more, lost credibility.

It is thus through the use of various tools in reporting test coverage and other quality metrics that the QA Engineer provides 'Assurance' to the project management, and various other project stakeholders, on the release-ability of the product. The QA Engineer must become 'master' of both the company development processes, and the tools which enable him to provide some level of assurance in a product.

To speak of the role of Quality Assurance like this is to underscore that it is one of the most important roles in a Software Organization. The QA Engineer must not merely elevate the process of software testing to an art form, but must elevate the entire process of software development to an art form. In the end it is the artwork itself (a finely crafted piece of software) that provides the entire team that worked on it with the satisfaction of having perfected something and brought it out into popular usage. And the QA Engineer can sleep at night, knowing that he has in no small way helped accomplish this.

Test?

2007-09-27T14:24:00.000-07:00

Has The Seattle School District ( http://seattletimes.nwsource.com/html/education/2003906580_bus27m.html ) ever heard the word "Test" before?

How much you want to bet this software project was something that they insisted on being released on time (when school started) even if it wasn't ready yet?

Compromise leads to chaos!

~ QA Guru

You haven't got anything to sell but quality

2006-05-26T08:26:00.000-07:00

I'm reading numerous books on quality as I prepare for my ASQ Certification and I really liked the quote I found below:

Your customers are in a perfect position to tell you about quality, because that's all they're really buying. They're not buying a product. They're buying your assurances that their expectations for that product will be met.

And you haven't really got anything else to sell them but those assurances. You haven't really got anything else to sell but quality.

(Guaspari, in Kan)

Regards!

Basil the Bugsmasher

Model-Based Testing

2006-05-18T16:03:00.000-07:00

Some great articles by Harry Robinson:

http://www.geocities.com/model_based_testing/shoestring.htm

http://www.geocities.com/model_based_testing/intelligent.pdf

and the model-based testing web site in general:

http://www.geocities.com/model_based_testing/

Model-based testing is nothing new, and was basically first eluciated by Chin-Kuei Cho back in 1980 (An Introduction to Software Quality Control) and elaborated on in 1987 (Quality Programming: Developing and Testing Software with Statistical Quality Control).

The basic idea is fairly simple. You cannot test every path through a program, nor all inputs in all configurations for all possible outputs, because it is often (usually) infinite. You therefore have to model the processes of the software, and choose what specific inputs and configurations you want to use.

There are four basic methods of selecting sample test data:

1) Regular Sampling (each sample has an equal chance of being selected)
2) Weighted sampling (some regular samples mixed with some specially chosen samples that have more interesting characteristics)
3) Boundary Sampling (selecting boundaries for all samples)
4) Invalid Sampling (selecting some out-of-range samples to make sure the program correctly handles them.

I usually use all four. But I would usually run two separate tests: one with Invalid Sampling (a stress test) and one with Weighted Sampling that includes (Regular Samples, Boundary Samples, plus some other specially chosen samples.)

2006-03-23T09:52:00.000-08:00

An interesting article by IEEE on "Why Software Fails"
http://www.spectrum.ieee.org/sep05/1685

IT pros say they cannot stop every threat

2006-02-22T09:52:00.000-08:00

I ran across this excellent article at "SearchSecurity.com" today:

http://searchsecurity.techtarget.com/originalContent/0,289142,sid14_gci1168007,00.html?track=NL-102&ad=543470

They say:

Security incidents can slip right past an IT shop amid a merger, tight staffing or when technology deployments outpace an enterprise's ability to keep up. In a recent survey, some IT professionals admitted this is exactly the scenario they're dealing with.

And this is precisely the scenario that the world of computer users is faced with constantly in the software they use. Companies are always in a rush to adapt new technologies with little or no thought toward Quality Assurance (which can and should include security vulnerability assessment).

In all, 28% of respondents said they have "little or no confidence" that they've detected all significant security breaches in the past year. Meanwhile, 26% rated their current IT environment as more vulnerable than it was the year before.

That doesn't surprise me either. Even on a small scale I've experienced lately a management imperative of "just get it out there" - without planning or scheduling any quality control or test. Being the evil QA guy I am, I've recently even poked my nose into things that were not in mo domain and pointed out to management that certain vulnerabilities were blantantly obvious. But I still wasn't asked to fully QA the thing, even after spending 10 minutes and pointing out dramatic flaws.

I've also spent a lot of time lately testing on the fringes of various open source projects, and it is frankly appaling how many bugs and potential security vulnerabilities are present.

One wonders what we are setting ourselves up for in this computer crazed society where corporations have such lack of discretion.

Regards,
Basil

Good Reference Articles

2005-12-16T15:44:00.000-08:00

The following are good reference articles:

"Doing it right the first time" doesn't solve the problems inherent in Quality:
Write First Time is Wrong

missionaries of sloppiness

2005-12-07T13:41:00.000-08:00

There's a good article at Salon.com on the very topic I blogged about when I started this QA Guru page...

Cheers!
Goo

A Good QA Interview Question

2005-11-18T10:24:00.000-08:00

"What is 1 + 2 + 3 + 4 + 5, etc all the way to 100?"

My daughter ( 9 years old ) asked me that the other day. I started toying with it my head, and after a few minutes I realized I could figure it out quite easily without a calculator or a scrap of paper.

Why is that a good interview question?
a) It presses the potential QA candidate to think.

Can they work this problem out without relying on a piece of paper or a calculator? Can they see the patterns and then utilize them to acheive the goal?

b) It presses for a sort of abstraction that is really very valuable in a QA candidate.

In my opionion a QA Engineer has to be a level above a development engineer. We have to understand the developer's tools and issues, constraints and frame of reference, and then we have to OUTSMART him!

Would you make a good QA Engineer?

Can you figure out the answer without using a scrap of paper or a calculator?

~ Basil ~

Software and the Downfall of Western Civilization

2005-05-10T20:53:00.000-07:00

Software and the Downfall of Western Civilization

Some friends and I were discussing the other night: what will finally end up bringing about the downfall of western civilization?

One friend said it would be the Fascist Republicans, another said it would be the Liberal Democrats. I cleared my throat and spoke up loudly: "Poorly written software."

My friends chuckled and went on.

But I was serious!

Now, I've been mulling over the idea for some time. I think actually I should write a book on how poorly written software will bring about the downfall of western civilization. I'm the perfect person to write such a book: smack-dab in the middle of the software industry, seeing first-hand what it is like. I also happen to have been interviewing candidates for positions at our company lately: quite a few candidates, actually. The more people I interview, the more I realize how pervasive the problem really is. Companies are stretching themselves as hard as they can for the ultimate goal of ROI. ROI (Return on Investment) is such a pervasive buzz-word that lately it was even slapped back in my face recently when I asked my boss about the possibility of attending the World Conference on Quality and Improvement which will be in Seattle this year (next week, in fact). "Not unless there will be some ROI, Buddy."

I believe it is precisely this attitude that underscores my belief that poorly written software will ultimately bring about the collapse of Western Civilization. Everything in the Software Industry is centered on the concept of ROI, so much so, that quality takes a secondary place in the mentality of leadership. How can leaders of a software company question the role of software quality? What has lead up to this?

The First-To-Market syndrome is at the root of it all. Companies are stressed to be First-To-Market with every new product they offer. The idea is that if you can get it out there first, you will have a market advantage over your competition. Never mind whether or not it works. Of course, the former (getting it out there first) makes perfect sense. But what companies fail to realize, over and over again, is that it doesn't make any sense without the later (making it work right.) It is my opinion that consumers are fed up with faulty software and service patches. I, as a consumer, am certainly fed up with it.

But what can I do?

Probably nothing. The statistics show that consumers will buy the product that hits the market first. I guess I go against the tide. I always wait until something has hard the bugs worked out of it in the market before I make my purchase. Usually by then the price has also gone down.

So, it begs the question: will the real downfall of Western Civilization come because of 1) Faulty Software, 2) Greedy Companies, or 3) Stupid Consumers.

When you think about it, it's all the same either way you slice it.

Good Night!

QA GURU