Always get a Baseline first

As developers, we often spend time optimising, tweaking or redesigning to increase performance. It's fairly easy to measure performance increases when optimizing code or systems, but it's a lot harder to gauge the effectiveness of User Interface changes.

In many cases the UI choices we make are subjective at best. In some cases, our design decisions can actually make things worse, not better. There are some interesting anecdotes about this in this recent post.

In order to make sure we are making the right decisions, rather than just a bunch of assumptions, we need to measure the effectiveness of the current implementation in order to compare with the improved version.

Depending on what it is that you're changing, this could impact on conversions, page hits, or data collection rates. You'll need to decide the best way to measure the effectiveness of the changes, but without data to measure it, you'll never really know if you're doing it wrong.

It seems like a simple idea, but it applies for all optimisation and is often overlooked. A simple rule to follow is to remember to ask "How will we know if this works?" before you implement a change.