Rimu Research


Immediate Storytest Feedback


Paper and Talks



Testing Automation and Release Issues

Changes and Bugs:

  • Your system has grown difficult to change.
    • New features take lots of effort to add and are delivered late.
    • Bugs are hard to find and fix, and sometimes return in a new guise.
    • Changes often lead to unexpected problems.
    • Developers seem to be growing restless, uneasy and fearful of making changes
  • The defects reported from the field are increasing and taking lots of effort to fix.
    • The automated tests that are being used are too slow, are expensive to maintain, and don't provide sufficient cover.
    • You need a bug-tracking system to deal with all the problems.
    • Bug finding and fixing is taking time away from development
    • You've tried several tools and approaches to automated testing, but they've not been satisfactory
    • The tests are inadequate, but it's hard to see how to improve them, automate part of the process and get real value for money
    • You'd like to seriously test for concurrency issues (such as when many users are using your web application at the same time)

Delays before Release:

  • It is taking too long to test your system before release
    • You need to improve your test automation, but it's unclear how to do that effectively at a reasonable cost
    • The tests take a long time to execute through the UI
    • There are lots of combinations to take account of
    • Performance/load tests are unsatisfactory and expensive to manage and they slow the release cycle. They take a lot of effort to change once the system is frozen for release.
  • It's taking too long to complete user help and user and installation documentation before release
    • Screen dumps can't be done until the system is frozen, and the user interface is the least stable
    • The document writers are reluctant to document the parts of the system that may change
    • The documentation is based too much around the features of the system and not around the tasks that users will have to carry out
    • Internationalization makes the problem worse as some customers want screen dumps to show the system configured for their use, with appropriate currency, numbers and etc.

Coaching possibilities...

Copyright, Rick Mugridge, Rimu Research, 2006 .. 2009.