I recently had a friend ask for my thoughts on some software testing methodologies / tools and within that the topic of Equivalence Classes and Boundaries came up. The book he was using, while highly recommended, did a rather poor job of providing any practical example of when these tools would be used and why they were useful. Because of this, he wasn’t even sure if they had much practical value and was wondering if I had any input.
I’ll start this article with the same thing I told him: this is one of the most useful software testing tools you could possibly have! It is definitely not esoteric and it is certainly something you want to work through to understand.
Books don’t always do topics justice, and his book in particular didn’t do an “awful” job of presenting things, but it was very “text book”. Using a simple yet realistic example helped him and I’m hoping the same will be true for the readers of this post.
Guess The Number Game Example
Let’s say you have a program that randomly selects a number between 1 and 10 (inclusive) and then asks you to guess the number. Expected input by the user is any integer value between 1 and 10 (inclusive).
Thinking about this logically we have “good values” for the number between 1 and 10. Anything outside of this range is “bad”. This means that any value less than 1 for is “bad”. Any value greater than 10 is “bad”. This yields three “equivalency classes” as shown below.
|Equivalency Class / Partition||Guessed Number|
|Too Low||x <= 0|
|Good||1 <= x <= 10|
|Too High||11 <= x|
The basic idea is this: testing any value from within one of these Equivalency Classes is the same as testing any other value within it. In other words: the program should behave the same way for one value within the Class as it does for all values within the Class. By testing a part of the Class, you are in fact testing the whole Class.
Extrapolating further: to test a single value from each Class is to test every value in every Class. This is the same as testing all possible values but clearly takes a lot less time!
This is obviously an oversimplification because it’s possible the program could react differently to values within the same Class. (There could be logic for x == -5.) However, trying to test all the values would take too much time and testing is about balancing risk vs. reward. Yes, there is risk to miss special logic only understood by the programmer, but the chance is low. The reward for testing so many values would probably in no way compensate for the time spent in testing it. You’ll never find all the bugs, but testing using Equivalency Classes will help find many of them.
Examples of defects that could only be caught by testing every value would include the Pentium Division Bug (per http://www.willamette.edu/~mjaneba/pentprob.html: “Also, only certain numbers (whose binary representation show specific bit patterns) divide incorrectly. Consequently many users may never encounter the division error.”) and the Microsoft Excel Display Bug (note that only 12 out of 18,446,744,073,709,551,616 possible floating point binary numbers demonstrate the bug). Equivalency Classes / Value Partitioning will never catch these kinds of errors and while they make a lot of media attention, they really didn’t impact that many people in real life.
Boundaries and Catching Some of that “Special Logic”
While I just said that we don’t want to spend loads of time trying to find “special logic” within an Equivalency Class, there are certain areas within and between Equivalency Classes where special logic often creeps up. Based on historic evidence and some common sense, it would make sense to test these areas because the risk vs. reward balance makes sense. These areas are the “Boundaries” and are composed of “Boundary Values”.
Looking at the table above, we can begin to think that somewhere in the program, the programmer is going to have to handle the special Boundaries between the Equivalency Classes very carefully. For instance, the program needs to reject 0 but accept 1. The values of 10 and 11 are handled specially in the same way. This allows us to expand our test values as shown below.
|Value Type||Value Name||Number Range||Test Value|
|Equivalency Class||Too Low Class||x < 0||-284|
|Boundary||Too Low to Good Lower Boundary||x = 0||0|
|Boundary||Too Low to Good Upper Boundary||x = 1||1|
|Equivalency Class||Good Class||1 < x < 10||7|
|Boundary||Good to Too High Lower Boundary||x = 10||10|
|Boundary||Good to Too High Upper Boundary||x = 11||11|
|Equivalency Class||Too High Class||11 < x||5,396|
Notice that we’ve narrowed the Equivalency Classes just a little to exclude the boundary values. Since we’re testing those values as special Boundary Cases, we don’t want to pick the same value to represent the Equivalency Class. We’re also tesing 7 data points now rather than only 3. This is the end of our value selection because this is the least number of test points that will yield the highest number of defects. Programmers often make mistakes by not considering an Equivalency Class (“The user will never enter a negative number, so why handle it.”) or by mishandling Boundaries (IF x > 1 AND x < 10). The points above will exercise the weakest areas of the program, exposing any failure on the part of the developer.
The Infamous Integer and Real Life Web Application Madness
So, what if you don’t have any clear Equivalency Classes or Boundaries within a program? What if you’re dealing with a program that accepts an integer input for Total Cost (cents are always “.00”) and as far as you can see “the sky’s the limit”. Without any stated Classes / Boundaries, where do you begin with testing?
This is where a little bit of programming knowledge will take you a long way as a tester. You won’t run into this every day, but when you do you will shine above the rest! (In my ever-so-humble opinion.)
First, a real-life example that occurred while I was in SQA. Someone was testing a web application and ran into an “unbounded” integer field that was for Total Cost or Total Hours or some such. As a way of testing, they entered the value 9,999,999,999. They then clicked Next and Back. When they came back, their outrageously large value had been “transformed” into the value 1,410,065,407!
The questions: What was happening? Why such an odd transformation? Furthermore, if 9,999,999,999 wasn’t accepted, what should be accepted?
The answers: the value was being truncated in binary, binary truncation yields results that are difficult for mortals to comprehend, any value between 0 and 4,294,967,295.
The why: If you look at the number 9,999,999,999 in binary, you get the following value.
What’s very interesting to note about this binary number is that it is 34 bits long. As we’ll discuss below, most programs use standard sized storage units that are either 8, 16, 32 or (more recently) 64 bits large. 34 bits crosses a “Boundary” from a 32-bit integer to a 64-bit integer (from a DWORD to a QWORD). Depending on exactly what size storage unit the program is using, this could pose a problem.
Let’s look at the outrageously long binary value in another way:
|Bits 33 & 34||Bits 25 to 32||Bits 17 to 24||Bits 9 to 16||Bits 1 to 8|
If binary truncation were to occur, it would happen because the back-end storage unit is either 8, 16 or 32 bits long and can’t hold the entire 34-bit number 9,999,999,999. Let’s look at what each of these values would be in decimal given the digits above. Of these values, I’ve highlighted one very interesting one; namely, the very strange value into which the web application was transforming 9,999,999,999!
|Data Type Size||Data Type Name||Binary Digits||Decimal Digits|
|32-bit||DWORD (Double WORD)||
|64-bit||QWORD (Quardruple WORD)||
In other words: even though the user was entering 9,999,999,999 the input was only being stored in a 32-bit storage unit. The 33rd and 34th binary digits were being lost and we were only getting the 1st through 32nd digits. This yielded a truncated value that fit into a 32-bit storage unit but in no way represented the original value!
What Does All This Mean?
It means that many computer programs have Boundaries and Equivalency Classes even if they are not stated. Depending on the storage unit backing a particular input field, only certain values can be accepted. How a program handles input outside of these ranges is critical. In the case of the web application above, it failed to realize the value was outside of its maximum range. It should have checked the value and ensured it was within range before accepting the input.
Floating point ranges (those involving the decimal point) are a bit harder to test, but they are no less important. However, for purposes here, we’ll just focus on integers. Handling integers gets programmers into more trouble on a routine basis than floating point numbers. I speak from experience.
So, here are ranges for the different fundamental integer data types. What makes things a bit more convoluted is that these data types can hold either signed or unsigned data. The funny thing is, the underlying data type is actually identical! Signed vs. unsigned is a matter of “interpretation” of the value (specifically, the Most Significant Bit or MSB). Don’t worry yourself too much with such esoteric programmer rubbish except to the extent to know that you should probably always test both signed and unsigned ranges, just to see if the programmer misinterprets the value somewhere along the way.
|Integer Data Type||Signed Range||Unsigned Range|
|BYTE (8-bit)||-128 to +127||0 to 255|
|WORD (16-bit)||-32,768 to +32,767||0 to 65,535|
|DWORD (32-bit)||-2,147,483,648 to +2,147,483,647||0 to 4,294,967,295|
|QWORD (64-bit)||-9,223,372,036,854,775,808 to +9,223,372,036,854,775,807||0 to 18,446,744,073,709,551,615|
I’m running way too long on this (as usual), but one last word of advice may help get you pointed in the right direction. We can talk about this more later if you’d like. You can use the above table to have “standard” testing points. For example, knowing that something “funny” might happen if a program is using an 8-bit signed integer with the values -129 to -128 means you should probably always test those values. Finding the above error in the web application would come from testing the upper boundary on a 32-bit unsigned integer; namely, 4,294,967,295 and 4,294,967,296. In that case, you would have found that 4,294,967,296 would have turned into 0!
I hope this helps someone out there, it certainly was useful to my friend and a few others who ended up receiving a copy. I’m sure there are other ways to view this are there are definitely plenty more (and probably better) examples out there, but this was something I put together and wanted to share it with others who are interested.
Until next time,