User testing: You don’t know everything

Looking at road design and engineering as an example of how you cannot possibly anticipate the unintended consequences and outcomes of design in every unique situation. User testing is absolutely necessary.

by Nathanael Boehm on 17 October, 2009

I was driving home from Sydney last night, going along the Hills M2 at around 6:30pm and it got me thinking … why is the traffic moving at about 20 km/h westbound from the tollgate to the tunnel and as soon as it gets past the tunnel the traffic suddenly accelerates to the speed limit of 100 km/h? Not all the time – not in the middle of peak hour – but around the time I go along it that seems the typical behaviour of the traffic.

Do we know why? Is there a rational explanation? Obviously in a logical universe, laws of physics etc there is a reason that can be found if you look deep enough in retrospect … but the fact remains that they built a road, dug a hole underground and set up the tollgate and because of the configuration, the distance, the location etc the traffic moves really slow through that point.

Was the road, the tunnel and tollgate built with the engineers knowing that this would be the outcome? It’s possible – but for the sake of this example let us assume this was a consequence that could not be foreseen.

It’s like accident black zones – are road designers aware that they’re creating a high-risk intersection or stretch of road? I mean, a roundabout is a roundabout right? But no, one roundabout can remain accident-free and yet an identical roundabout somewhere else can see dozens of collisions a year … thus we see signs popping up “Slow down, accident zone” or ripple strips, more reflectors etc. Essentially hacks and workarounds for a bad design.

And yes, they are bad designs. If people were consistently crashing at all roundabouts then perhaps it might be ok to blame drivers or vehicles but that’s not the case. Even if road designers and engineers didn’t know they were creating an accident high-risk intersection it doesn’t negate the fact that people struggle to negotiate particular intersections, and it is the fault of the intersection and the entry ways into that intersection. Or even just a plain stretch of road.

Now, user testing might be a little hard when we’re talking about millions of dollars of blacktop compacted into the ground because there’s little room to adjust, but what I want to get across with these examples is that you can’t know everything and you cannot design and implement a system assuming that because you followed some heuristic design principles that it’s all going to be ok. There are so many variables and factors at play in each situation that every different implementation is going to be quite unique.

Accept the uniqueness of every project and accept that you need users to guide you to make sure you tailor your design to suit the particular and peculiar environment into which it is being deployed.

Heading westBarry Drive is a 3-lane road from Civic to Belconnen in Canberra. The first part of the road coming out of Civic is shadowed by Black Mountain … so you’re driving in shade for a good 5 minutes; and then you come around a sweeping corner and everyone’s blinded by the setting sun and many people hit their brakes because they’re momentarily blinded. Did the road designers anticipate that? There’s a part of the M7 where there are noise-dampening walls alongside the road. They’re slightly angled back, and they’re made of reflective material. Drive along there are 7:00pm and you’re blinded by a full reflection of the sun. Did the designers anticipate that?

What about merge and exit lanes? It’s fine to have an exit lane right after a merge lane but what if due to the location of that stretch of road most of the people merging want to exit immediately after and have to cross 4 lanes of peak-hour traffic over 200 metres?

Doing you research, being informed, learning from history, knowing some principles, developing personas … it’s all going to help. But at the end of the day do not be so arrogant as to believe you can predict exactly how something you design is going to slot into someone else’s life without even asking them, running it past them, testing it with them, incorporating their feedback. They know things you don’t, things you can’t possibly find out. Things that will influence the outcome, acceptance, uptake and usage of your product in ways you couldn’t imagine.

I’m thinking particular internal business software applications but it applies to any sort of design work. How can you be a user experience designer if you never involve users? Sure – you’ll design an experience, but it’ll possibly be a negative, frustrating experience.

Post to Twitter Post to Delicious Post to Facebook

// purecaffeine.com, UX, design, social media and Gov 2.0 blog by designer Nathanael Boehm, Canberra, Australia. Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 2.5 Australia License.

Related posts:

{ 1 trackback }

Max Design - standards based web design, development and training » Some links for light reading (20/10/09)
20 October, 2009 at 9:05 pm

{ 3 comments… read them below or add one }

xtfer 21 October, 2009 at 8:58 am

Interesting observation you make about the tunnels. I have noted when travelling around Sydney that often the speed limit going into the tunnel (e.g. 80 kph) is slower than the speed limit going out of the tunnel (e.g 90 kph). I assume this is to stop traffic jams in the tunnel, however it usually means the road after it is clearer.

Reply

Nathanael Boehm 21 October, 2009 at 2:28 pm

Quite possibly – but do we know why tunnels cause jams? Was the “Do not change lanes in the tunnel” and “Turn your lights on” rules implemented as an afterthought to solve some problem with the tunnel? I’ve never seen those rules implemented elsewhere. Is the particularly harsh type of sodium lighting in that particular tunnel conflicting with vehicles’ yellow/orange lane-change indicators?

There are just so many variables – it must be a nightmare (assuming the engineer actually thought it through) trying to think of everything, anticipate all the consequences of all the design decisions knowing they have little option to change their design once implemented except to put up “Reduce speed!” signs.

Reply

Dave 24 October, 2009 at 2:36 am

There’s actually some really amazing amateur research out there by people who drive the same commute every day and have adjusted their own driving to see what affect it has on traffic around and behind them.

Waves of denser, slower traffic generally are initially caused because some car somewhere has to slow down, maybe because they come up to a car that is moving too slowly or because many cars are trying to merge onto the road. The problem then continues because most drivers do not leave enough space between them and the car in front of them. If you leave too small of a gap in front of you, then you CAUSE traffic problems because it is more difficult for others to move into and out of the lane you are in, and because you must stop more suddenly when the car in front of you slows down.

The answer to traffic jams big or small is to allow a very large gap to accumulate in front of your car BEFORE you enter the traffic jam. This will probably have little effect on your rate of travel through the jam, but will help smooth all of the traffic behind you, which will help the jam clear faster. The worst things you can do are follow too closely and switch lines in such a way that drivers behind you have to slow down.

Reply

Leave a Comment

You can use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Previous post:

Next post: