Human Testing is still necessary

fdillon · 7/1/2009 2:19 pm

It always irritates me when I sit down to test out recently developed software (be it a new asset, shop plugin, etc) and the very first success case I try fails. I then notice that there are a slew of automated tests for the thing I'm testing and I think to myself, how is it that the developer could have had enough foresight to have written automated tests and yet didn't even do the bare minimum of making sure the thing actually worked before proclaiming it "complete".

One word: Laziness. Too often we assume that if the smoke tests pass, nothing could possibly be wrong. However automated tests are only as good as the developer writing them and often do not test anything on the UI (unless you are cool enough to know how to use WWW::Mechanize - which I am not). For this reason (and until a full suite of UI tests exists for your software), it is important to write some old school tests to make sure everything functions as you expect.

This doesn't have to be a hard or painful process. One case for each possible success along with some possible failure cases is all you need. It's as easy as sitting down and typing out what you are doing:

Test 1: Make sure form posts correctly
1: Log into site
2: Navigate to the form at url: /my/application/url
3: Enter the following information into the form
     3.1: First Name: Frank
     3.2: Last Name: Dillon
     3.3: Middle Initial: L
4: Click the submit button
Expected Result: Record is added to the database, page is refreshed, new entry appears on the page.

Now that you've got a good success case, you can start to create permutations of it trying to break it

Test 1.1: Ensure form does not allow empty values to be entered
1: Test 1, Steps 1 & 2
2: Enter the following information into the form
     2.1: First Name: [Empty]
     2.2: Last Name: Dillon
     2.3: Middle Initial: L
3. Click the submit button
Expected Result: Record is not added to the database, page is refreshed, Last Name and middle initial are preserved, error message appears indicating that First Name cannot be empty

You can see how easy it can be to write tests! These tests not only benefit you in that they assure you have fully tested your application before the proclaimation of "done" is uttered, but they also give you a base for others to do regression testing when changes are made.

Re: Human Testing is still necessary·

preaction · 7/1/2009 2:38 pm

This is where Test::A8N (name may be changing) comes into play. You define a test almost exactly as you wrote it, and an automated process runs it.

So you write your test:

 ---
 NAME: Test Updating a User Profile
 ID: admin_profile
 SUMMARY: Update a user's profile through the admin user
 TAGS: admin, profile
 PRECONDITIONS:
    - ensure user is: Admin
    - ensure user exists: TestUser
 INSTRUCTIONS:
    - view profile for: TestUser
    - verify form value:
        username:   TestUser
    - change username to: TestedUser
    - save form
    - verify user exists: TestedUser
 POSTCONDITIONS:
    - delete user: TestedUser

and then a "fixture" turns it into runnable tests. (and imagine the actual code that would've been written to implement that test, and then copied for every potential permutation).

So while your human is testing, they're writing one of these things. When they're done, there's an easily-repeatable automated test that can be run after any changes.

Re: Human Testing is still necessary·

patspam · 7/1/2009 7:04 pm

Very cool, that cpan module looks worth checking out!

But you'd still want to run through some of the tests manually, to check that the UI is ok for humans as well as robots.

Re: Human Testing is still necessary·

Albert2 · 7/6/2009 8:15 am

Testing by hand should always be done to some extent. Tools will ignore what the test case creator in a test tool did not take into account. Automated testing is very usefull, but also limited. The human eye will pick up other things.

A must read if you want to know about testing is TestGoal. The website supporting this book is by the way build in WebGUI :-) There is a whole chapter on automated testing including when it makes sense and when it doesn't.

Re: Human Testing is still necessary·

susanb · 7/7/2009 5:53 pm

I also like Selenium. It's a good way to reproduce your clicks in browser. You can download the Selenium IDE for Firefox. We use Selenium's test runner to check our most important site daily, including a 50 question math quiz. You can even test that stuff is not showing up, like the lovely message when a shortcut's original has been deleted out from under it. We have found it invaluable as an upgrade aid. If Selenium tests don't run clean after an upgrade to webgui, we find out about a potential problem right away and can decide whether to fix or revert.

One other good thing, is it runs in your browser, so you can slow it down a bit and check the site visually whenever you need to.

Susan B

Previous Next

Stick

Lock