Possibility and Probability

Coding Python and making businesses…

5 January 2016

You should do experiments

by admin

Over the last few years I’ve learned that it is one thing to have a good idea, and it is something completely different to have an idea that someone will pay you for. Sadly the overlap between these two can be pretty small. But, there is a way to enlarge the overlap: find out what your customers really want and then give it to them. The best way to do this is with controlled experiments.

Please note that actually solving a problem a customer has is a really great approach also, and probably should be your first step. This post is going to work with assumption that someone (your boss) has already done that leg work and you are just trying to get more people using your solution, or to make them happier.

An experiment is a slight change to your app or website that is measured. If the thing you are measuring (time on the site) goes up, then the experiment should be considered successful and put into use. If the measurements go down, then the experiment should be discarded or modified and retested. So what is a “slight change”? Typically it is something like the color or position of an element on the screen (like the “buy” button), but it can also be text that is used to drive certain behaviors from the users. A perfect example of this is the text that is on a “buy” button on an e-commerce site. In the not-to-distant past, most websites had a button that said “Submit” that was clicked when the user wanted to buy something. That isn’t the most enticing turn of phrase to get someone to hand over their money. If someone was on the fence about purchasing a product, seeing the word “Submit” didn’t really give them any motivation to click the button. Ok, so if that isn’t a friendly word, then what is? In pretty much any office you’ll probably get 10 opinions on what that button should say. But those 10 opinions are just that… opinions. At the end of the day all that really matters is what phrase causes people to click on that button. So this is the perfect experiment: Gather up your phrases, and then try them all out. The one that gets the best response is the one you should go with! Well, that’s the high level view. The more concrete steps would be:

The idea here is that you are letting the customers vote with their actions: They don’t know they are being tested, so their actions should tell you something. For example if you offer two choices, “GIVE YOUR MONEY NOW” and “Let’s do this!”, you should see a pretty significant difference between the number of clicks. The first one… well its what you want to happen, but it doesn’t mean people will do it. The second one is more fun and light hearted. Depending on your app, you might find that you customers respond very well to certain phrases, and in that case you should then try to use that language at every important decision point in order to get the customers to… well, give you money. :) Some other things to keep in mind with experiments:

This type of testing is the key to success in the field of advertising. Over the last few decades the professional marketers have been using this technique to determine the absolute best copy to use in their sales ads. (If you look closely at ads from different companies you will notice that there are a lot of similarities.) If this is working great for them, then you should definitely be doing it for you app too! Experiments are a great way to peek inside of your user’s thought processes. In future posts I’ll detail some ways I’ve used experiments to determine how to best change my apps and sites in order to get better results.