10 rules to guide you through testing new business ideas

I won’t go into how testing concepts prior to investment or full-blown execution is important. You probably know this already or else you would not land here.
What strikes me as odd though is that i have not come across much material about how to go about testing. So here is a short-list that i have to realize summarizes the most common pitfalls:
1. Test the product’s core statement
Focus on the essence of the service. Strip it down from all possible bells and whistles and try to confirm or reject the basic hypothesis about the project.
Image
Essentially try to answer two questions:
a. which problem is the product solving?
b. is this problem important enough for the target group? (to look for it, to return to it, to pay for it)
If either (a) or (b) are not a 100% YES, then you are sure to fail. No doubt about it.
2. Get more educated on the subject
Assuming that (1) is covered then you need to accept that you don’t know as much as you should about the problem you are trying to solve.
For example, let’s say you want to develop a video aggregation service for sports lovers as there is so much sport in the world. And you go through step 1 and the need for a solution is indeed validated. So let’s say that you develop a vision to create an elaborate algorithm to gather data from various sources and incorporate also machine-learning to personalize the service. Stop. First find out what matters most to your target group. What is it that they are lacking now? You should not answer this based on yourself as this would be the equivalent of running a survey with a focus group of one person.
The biggest risk here is to focus on the wrong features, waste time and energy on developing, let’s say, machine learning technology to optimize personalized ranking of videos to find out that users care about something so specific that could be easily pinpointed without the need of developing a self-improving mechanism. Once you research a bit, then you will be in a position to design and build an MVP.
3. Build a prototype (MVP)
In the example above, just do the service manually. Build a tool to facilitate a person to manually place videos in a playlist. You don’t have the time to do it, hire someone. It will cost you far less than building a machine to do it. Also chances are that the person will do the work better than the algorithm will anyway. Do a small ad campaign and direct traffic to a site showing the editor’s playlist of sports videos. Then you study visitors’ behavior: average time on site, number of pageviews per visit, how many visitors actually return, etc. If the concept is interesting it will show some positive signs. In case you believe that personalization is key to the service and the one-editor-for-all approach does not do justice to the concept, then segment users really finely. In the spots video aggregation example, target fans of a particular team in a specific sport. Then the editor should focus on that team, and safely assume that the playlist would be as good as a personalized one for the particular target group.
4. Design a test for maximum knowledge
The test’s goal is to gather knowledge. Aim for that. Let’s assume you want to launch a new service to a userbase you have access to. Don’t send the invite to users randomly. Segment users based on every attribute you have. And do the segmentation separately. One for each attribute. Separate the userbase in three groups based on each of the attributes: low 50%, medium 30% and top 20%. Whatever the results come out to be, you will be able to draw some conclusions about the userbase and the service’s appeal.
5. Change one item at a time 
Otherwise you won’t know where to attribute the difference. So, for example, if you are testing how different usergroups behave, you need to make sure that you keep everything the same. Send the same email, on the same day and time, keep the same landing page, etc. If you want to test two different email subject lines, the usergroups need to be the same, etc.
Image
6. Segment before time = 0
Segment users before the time of the test, not after. What not to do: run the test to a random sample and then see who participates and analyze that group. This could lead to a number of problems. For example, the test could alter the segmentation, e.g. trigger visits to the site, and hence shift users from one usergroup to the other. Or high users will most probably end up being under-represented if the sample is randomly selected, and according to the Pareto rule those aree the most important people that one should focus on.
Image
7. Define what you are looking at
Carefully define the metrics you will be looking at after the done is done. Make sure you will get the information you will need. Write it down in a table. Confirm that all placeholders will be filled in with numbers after the test runs.
8. Put down actual values before hand
Take the time to predict what you are expecting to get. This is necessary also to budget and time the test but it will also help you to evaluate the end result. It will show show you where to focus your attention to: where your assumption proved to be far off. Do not put down too much data to look at as you risk losing focus. Keep your eyes on what confirms/ reject the primary hypothesis at the heart of the product’s value.
Image
Harold Camping (pictured in December 2002), predicted doomsday to arrive on the following day, May 21, 2011 (http://goo.gl/QVVqW)
 9. Be fast.
The faster you design, execute & evaluate a test, the faster you will move on with the next one. The more tests you do, the more educated you become and the better the decisions you make. By working fast you are not being sloppy, you are maximizing the knowledge per time you collect. And it is all about knowledge at this point when you are researching.
10. Look out for interferences
Watch out for outside factors affecting the test. For example, if the testing is done via email then make sure that on that day you don’t also send the weekly newsletter. This is something that could affect behavior in a random manner.
Please share any thoughts or personal experiences of new idea typical pitfalls in the comments section. I would be really interested to find out about them now, before hand.

Hands on Email Marketing for mobile: review of actual email campaigns

See below a screenshot from my mobile inbox. Stats say that mobile inbox is already the nr1 medium for reviewing emails. Most emails get deleted on mobile devices than anywhere else. Hence it makes sense to take a few minutes and review how your emails appear on mobile devices. For the sake of the exercise I am using the most popular single handset: the iPhone. The email senders are chosen at random and include:

  • Saks – the world famous retailer
  • Netrobe – a fashion iOS app  that I am currently getting involved with as an advisor
  • Kotsovolos – Greek subsidiary of Dixons
  • MelinaMay – leading online stock outlet for the Greek market.

Image

 

Saks: apparel retailer

(-) There are actually two emails from Saks in my inbox. That is too much given that I have never ever purchased anything from Saks. Saks should have segmented me as an inactive user, tested multiple messages on that group and then send the best performing one to all inactive users in hope that the maximum of inactive users will convert to paying customers. Two emails on one day is a sure sign that Saks has not thoroughly tested those messages and also, as a recipient, I realize that these messages are not anything special that deserve my attention more than any of the previous messages I have been ignoring.

(+) Sender: varied sender helps as it differentiates the two messages and makes recipients think that the message actually is sent from two different sources – reduces the feeling of getting spammed.

(+) Positive marks also on correct targeting, serving me messages for males while indeed I am a male.

(+) Top message has a fairly effective title kicking off with a most impressive 70% discount. Also makes clear that it refers to male products. Not so keen on the Must-Haves descriptor which is too vague and hence conveys no real info.

(-) Negative marks for the descriptor below the title especially for the top email, which repeats the Sender and then reads like random text without any coherency. The bottom email reads a bit more logically but still it feels like an AdWord campaign unsuitable for a personalized message to an already subscriber.

NETROBE: a cool iOS app to help fashionistas organize their wardrobe

(-) Not cool though that it repeats its brand name in the sender and title – waste of character real estate really

(-) Also not cool that the title conveys no information – the real information, i.e. the topic of the email, is replaced by “…” as it lies beyond iPhone’s character limit

(+) The descriptor is ok as it reads as a logical text and provides brief product descriptors and brands that could engage recipients.

Kotsovolos Dixons: electronics retailer

(+) Good use of title and title effectively communicating who sends the email and what it refers to

(-) Negative marks for the topic of the email being only a CSR message (= look at us how great we are) offering no real incentive to read it. The correct implementation should include an invitation/ CTA on how the recipient can help or maybe what the Kotsovolos Dixons customers have already done to contribute to this good deed.

(-) Big time negative marks for having left a generic text as a descriptor – Kotsovolos Dixons must have never paid any attention to the mobile inbox.

MelinaMay Fashion: apparel discount retailer

(+) Positive marks on using favicons in the title and especially in the start of the title. A sure way to boost engagement. Also it matches well with the title’s copy which is always a plus.

(+) Good use of character real estate informing recipients that new items have arrived. Important incentive for the bargain hunter to read the email and find out what type of products came in.

(-) Missed opportunity to use the descriptor below the title to outline some of the product types, brands etc. that just came in.

Got any other comments on the emails showing in the photo or disagree with any of the points, feel free to say so in the comments section. 


#2 Quick & Dirty Email Marketing Tip: behave in a 1-to-1 manner

If you search for 1 to 1 marketing you will get results such as personalization, CRM technologies, userbase segmentation and so on. Before you go there and invest in a software or recruit an agency, first identify the principle behind all of those buzzwords and try applying it in your email marketing.

The juice: make the recipient feel like this message is intended to him personally rather than give him the i-am-bulking-everyone-i-have-access-to impression.  The more relevant the message, the higher the probability that the recipient will become engaged upon receiving it. That’s the juice.

How do you get recipients to get this feeling? Here a few examples:

-make the content of the message as relevant to the recipient as possible. One way to achieve this is through userbase segmentation. For example, segment in a list users that purchased a specific product and prepare a custom message for them only. Or make a list of customers that spent more than 100 Euros the past 15 days.

-be explicit in what you are doing. If you did segment your users and you are now addressing a specific segment of those, tell them so. And do so as early in the message a possible (see benefits of cutting to the chase). For example, “You have spent more than 100 Euros the past 15 days and that places you in our Golden Club and eligible for a unique offer…”.

-make copy personalized. The trick here is to make all copy seem as if it is written for this recipient only. This does not mean you need to change your brand’s tone of voice, no just imitate how a company representative would address a customer. To go a step further, you can make the message more personalized by addressing the recipient by his or her name (most email marketing software allow this functionality), explicitly explain why the message is relevant to the recipient, or even better make evident how this message is exclusive for the particular recipient.

-be polite. In 1-to-1 relationships manners matter more. Some examples: don’t try to trick them, don’t make unsubscribing difficult, don’t enroll them in the mailing list without their consent, don’t lie, etc.

-do not treat all customers equal. When I walk in a store and I am a regular, I expect to be treated as one. Similarly if I am not a regular, too much intimacy might be annoying. How would that translate in email marketing world? Segment your users in terms of worth, say gold, silver and bronze, and then treat each group differently. What could differ? The contact frequency first of all, the tone of voice, the quality of the offers, anything that signifies preferential treatment in your business

I believe the main pillars on how to make recipients feel special are here. Do all of the above and you will get 90% of the benefits a 5,000 Euro/ month CRM system could offer you. Have you found another trick in order to make recipients feel special? Please write a comment about it.


#1 Quick & dirty email marketing tip: cut to the chase

 

Being creative is cool. So is breaking the codes. Those are good ways to attract attention. But keep in mind that those cannot be but exceptions as one cannot be breaking the codes all the time. Why? Well simply because it is too difficult to get it right every time – unless you are Oliviero Toscani himself.

Image

 

So rule nr. 1 is ‘cut to the chase’ as early as possible. Ideally, this should be in the title, or even better in the Sender’s name. You can try the occasional teaser or unconventional title but the vast majority of your promotional email communication should cut to the chase for the first word. To better explain what I mean, imagine the recipient to be climbing on a moving bus and you have a few seconds to shout something to him that will make him jump off it. This should the test. This why Mega sale! -50% off all designers clothes for next 2 hours is good title and Weekly Digest 20-May featuring Paul Stevenson is a terrible one.

So here are a few tips on how to make your newsletters more direct and effective:

-explain what it is all about as early as possible. Ideally in the subject line. This is the central theme of the newsletter and everything should revolve around this. You can add additional topics in the newsletter but this should be 2ary.

-be consistent: the first phrase of the body should repeat & elaborate on the key message expressed in the subject line

-if you add a photo or graphics, make sure those come after the 1st phrase no matter how interesting or sexy they are

-make sure the photo or graphics used support the key message from the subject line

-identify the common ground among all subject lines and come up with a sender’s name that is relevant. For example, if you are a retailer called Zachs and most newsletters announce flash sales that last for only a few hours, a good sender’s name could be Zach’s sale of the hour. That would leave you space to actually present what is special’s about today’s sale (Designers’ summer skirts at -50% until 15.00) and hence the recipient has all the important info.

To understand the power of being direct, you may do a test of a vague/ teasing title vs. a to-the-point one and compare the results in open and click-through rates. If you do, please share the results in the comments area.

 


eBusiness is not for Gus Hansen

Gus Hansen is maybe the nr1 poker player in the world. But it should not take Gus Hansen to know the first rule of poker: don’t give away the cards you are holding. Keep people guessing instead. Hence the expression poker face. Even better, mislead your opponents to think that you are holding something other to force a wrong decision from their part. This could read like a business book tip. Trick your competition and so on. But when it comes to eBusiness then this is exactly the opposite of what you should be doing. Better lay all your cards on the table and pray for someone to have a look and share an opinion.

To prove my point, let’s take the case of start-ups. Say that it is the year 2005, Facebook does not yet exist and you find yourself in a coffee place and your friend mentions that he is thinking of building a site where visitors could subscribe to create their personal page displaying their picture and a very brief bio. Would you realize they would be talking about the biggest internet phenomenon to date? Would you even grant it as a good idea to start with? Would you feel envy that you did not think of it first? Would you wish he would get you involved? Would you actually expect that person to go ahead and build it in the end? Say there is also another person following the conversation. Would you expect, following this discussion, that few months later there would be two competing Facebook-like sites in the market, your friend’s and the third person’s listening in? What if your friend would go around town and mention that idea to everyone he met? Would you then expect to get a number of Facebook-like sites emerging triggered by your friend pitching idea left and right?

Hell no. One thing is certain to happen, your friend would get great feedback and build on his idea listening to comments and criticism. For example, some person might mention that it would be cool to allow for users to invite friends and get alerts for whenever their friend does anything to his profile. This idea would be gold. Even if 200 Facebook-like sites launched that year, this insight would set it apart. Another person might propose to allow users to upload pictures. Would people really be willing to upload pictures of their personal moments to the wide public? Can you know the answer to this? How could you unless you asked around and got feedback. If all humans had my taste in cuisine, there would only be Italian restaurants and chocolate ice-cream. Tough to predict what people will like and do. Some feedack will be gold, some will be crap, some will just need to be processed. For example, the comment about privacy is valid and could lead you to coming up with limiting access to profiles based on whether they are friends or not. If you don’t do this, you will most certainly get the product wrong and leave room for the next person with a similar idea to capture it. And be certain out of the international online population, that person won’t be too long.

You might be thinking that all this argumentation stands for start-ups and new ideas. How does it apply to running businesses that are already out there? You will be amazed at how much eBusinessmen prefer to keep in the shadows. How many eCommerce businesses share mistakes they have done with social media? Or best practices? How many of them still don’t know how much is right when it comes to investing in Facebook? Quora? Are they even aware of this opportunity? What about mCommerce? Are they correct to not have invested in this or did they waste their money developing a mobile friendly website? Are they really as SE optimized as they should? Are they leveraging email marketing as they should? Are they even segmenting their userbase or is everyone receiving the same email? Still that same message are they doing A-B testing? Is it worth it? Whoever does it won’t admit to it. The rest are to find out about this whole thing on their own.

In the past, a store would open in the high street and would compete with the rest. Now the high street has expanded to include the whole world. The best practice would be for businessmen to group in teams, share all their cards and try to beat the enemies at the gate to make sure they are ahead in the global game as much as possible.

The reason I am making this point is because I am tired of reading articles or going to presentations where businessmen get on stage with the sole intent to sell their company and its products. They don’t aim in sharing the knowledge, letting you in in any secret. This is bad practice that will drag the whole ecosystem behind. Some other group of businesses will outgrow you and you will be left behind. The other alternative is that your ecosystem will outgrow the rest and gradually expand its borders. This is something to aim for.

So you have an idea, you have a good practice or a bad practice to share. Then just share it. Discuss it. Be open about it. It will only benefit everyone around you in the long term. Yourself included.