I’ve made a lot of mistakes over the years when testing. I try not to make each mistake more than once - which usually means any future mistakes require more creativity or stupidity on my part. Here are some UAT mistakes.
I will describe some of the mistakes I (and other people I have known) have made when doing UAT so you can use more creativity (or more stupidity - you get to choose) when approaching UAT in the future.
- Repeating System Testing
- Select a subset of system test scripts. Stick them in a document and title it ‘UAT test plan’
- Then get the users to follow the scripts and when they pass ’tick’ you passed UAT
- Danger: you deliver a system that they didn’t want and can’t use, because your scripts and data don’t reflect their usage or data
- Doing UAT too early
- The schedule has slipped and you may run out of time
- UAT has scheduled dates but you haven’t finished System Testing - heck you haven’t finished developing yet!
- But the users are ready to go, they will turn up on the day that you asked them to.
- Oh no, what will you do?
- Danger: You have more concern for your schedule than you do for the user’s time. So you ‘assume’ it will “be ok”, you think you can ‘fudge’ it. You end up wasting their time and making the system look bad.
- Making UAT the ’training’ period and make it the first time the user sees the delivered system
- Everyone has concentrated on checking requirements and fixing bugs. No one has actually involved the user
- The last time they involved the user, the user signed off the requirements specification
- The user only gets involved again when your UAT comes up
- But don’t worry - they can learn the system when running the test scripts
- Danger: your UAT schedule go out the window because the user doesn’t know what to do, does not understand or agree with the scripts, and doesn’t have a holistic view of the system. You don’t train people so don’t react well to their questions - after all you didn’t build or design this system, you just pointed out the flaws. The users don’t accept the system.
- Try and do UAT and usability testing at the same time
- When you do usability testing then look for usability issues which need to get fixed
- When you do acceptance testing you try to get the user comfortable enough to accept the system
- Danger: The two goals conflict and you raise a lot of defects that you don’t have time to fix. And the user either doesn’t accept the system or feels pressured into accepting a system that they don’t like.
- Not involving the users in the UAT planning
- It’s a testing phase.
- You test things.
- Busy users have no time to see you.
- Danger: You have no idea what they want to see to accept the system so you build a high level set of tests which cover the whole system. But each user doesn’t do enough to satisfy themselves that the system works. They don’t sign off, or bugs slip through into live usage.
- Not having user “buy in” to execute UAT
- So it’s UAT day.
- You have a prepped and ready test environment.
- You have a system freshly built, clean and shiny.
- Danger: The users don’t turn up. They don’t care. They have a job to do. It’s a busy time at work. Or worse…they send Bob, and he doesn’t have the authority or sense to accept a compliment.
I learned different things from each of those situations and would approach them in different ways now. In fact I now a generalised model of UAT that avoids all of the above issues. I’ll cover that in a later post but for now… What would you do differently?
Hint: Instead of treating each of your answers as ’the’ solution, or even ‘a’ solution. Treat it as a countermeasure - because whatever you come up with, the system will respond to match it, and may require another countermeasure. So as well as identifying what you could do - work out what would happen as a result.
