Apps  Contact  Seminars 

Archive for November, 2010


November 22nd, 2010

The Tipping Point – The “Stickiness” Factor

by Amrinder

Tipping PointIn the “Tipping Point”, Malcolm Gladwell allocates a generous portion of the book to the quality of “stickiness” – does your marketing message stick?  The premise is that small, subtle changes to your message (or website, or product) can make a big difference in its perception.  The book talks about social psychologist Howard Leventhal’s classic “Fear Experiment“, in which 2 versions of tetanus booklet were shown to the university students.  One version was a “high fear” version (containing gory pictures) and the other one was a more objective version containing same factual information but without any gory pictures.  The “high fear” version provoked much higher initial response rate (expression of how many people wanted to get vaccination) than the low fear version, but when it came to actually getting the shots, both versions evoked a pathetic 3% response rate.  However, a simple change in booklet (including a map of the campus with health center circled and times shot were available) tipped the vaccination report to 28%.  This changed the information in the brochure to practical and personal action initiating information, rather than just an academic exercise.

This is not the same as saying that small changes can accumulate to make a big difference (although that is true as well).  Gladwell’s message is that the small changes (even by themselves) can make a big difference.

Applying the principle of stickiness

Having bought into the stickiness factor, question becomes how to apply this to your actual problem – product, blog, website, presentation, etc.

Product

Courtesy Jeff KubinaOne of the known best practices in product design is called the DYNO (Disturb Your Neighbor / Officemate) routine.  The basic idea is that when you finish doing some activity, go ask some neighbor to come take a look at your product (and possibly trash talk it).  This applies whether your “product” is a software, a hardware, a literary article or a kitchen.  This also applies whether your neighbor is a software developer, the receptionist, the CFO, the senior staff scientist or the sales guy.  The kinds of feedback that we can get simply from our neighbors goes significantly beyond what we might be able to get from a very thorough (and possibly expensive) user test.   Back in the days when I was a software consultant at Verizon, I helped rewrite a portion of the billing logic using WebLogic integrating with IBM mainframe application.  After getting some output snapshots, just out of pride, I sent them to 4 unfortunate “neighbors”.  Besides learning that this is simply not the “Verizon” way of doing things (that too without a full QA supported UAT, PAT, SAT, DAT, RAT and 700 other Ts – imagine that!), I also learnt that: (i) my app blew up if there were exactly 99 records, (ii) my app was retrieving 10 records, but showing only 9, (iii) the “Bill” icon was actually the invoice icon, and most importantly that (iv) my neighbors didn’t really mind if I didn’t bother them too often.

In a more mature setting, it is still very possible that you find nuggets of gold from your neighbors testing your product, and it is very likely that all your neighbors will point out different nuggets.  We as human beings are sufficiently different to have different reactions to the same input.  And yes, your lunch buddy or your neighbor’s lunch buddy who just happens to be in the vicinity when you finished your product qualifies 100% as a neighbor for the DYNO routine.  Diversity trumps profile or interest by a significant margin.

In “Glimmer”, author Warren Berger makes a very similar point when talking about Sam Farber’s company OXO Good Grips.

..so many of the company’s products can be traced to someone asking why a common device couldn’t be made a little bit better or why an everyday task couldn’t be made (a little bit) easier.

Websites (and Web Applications)

As a direct application of Gladwell’s stickiness principle, in Steve Krug’s legendary book “Don’t Make Me Think“, every change appears to be a small change.  The material is the same, the flow is roughly the same.  Still, the impact of each change is quite significant.

The first version of the BizMerlin resource project assignment report showed data essentially in the following table format.

User Project Assignment - version 1

The very next version of BizMerlin (I believe v3.0) had this slight improvement to the user project assignment report.  Essentially, when you would hover over a project or a resource, it would highlight all users and projects that that user (or project) are allocated to.  Looking at the picture below, hovering over “Cloud Computing”, we can see that Andy Jones is assigned to it 100% and Don Lee and Melissa Peters are assigned at 50% each.   Simple, subtle change that saves valuable time during each resource allocation review meeting.

User Project Assignment - version 2

Presentations (and Brochures)

However, if there is one place where the stickiness factor applies more than others, it is clearly in case of presentations and printed materials like brochures.  I believe this is due to the fact that with presentations and brochures, the user (the reader, the ideal passer by) has exceedingly short duration during which the material must seize the initiative and stick.  If the moment is gone, it is gone.  With physical products and software applications, the user may be fortunately/unfortunately locked in for a little bit of time, but with a presentation, the speaker usually has less than 2 minutes to captivate the audience, or it will zone you out and decide to work on presentations of their own – even if they have to sit through yours.

Testing the stickiness factor using A/B testing and other techniques

A/B testing (also called split testing) is a marketing method used to test the effectiveness of two options – kind of like when an optometrist asks – “Is A better, or is B better?”  The positive aspect of this method is that it is fairly simple to use and fairly simple to deploy.  Google Adwords even allows a/b testing as an “easy” version within website optimizer.  The negative aspect is that the insights to draw from the results of the A/B test are far from straightforward. While it may be true that version “A” of your product is better than the version “B”, it may be harder to draw insights that apply to other circumstances (other products, etc).  The hidden challenge is that the different versions of website (or product, or presentation) do not evolve in a way that are amenable to local optimization – in other words, a much better of product cannot be created by making many “positive” changes to a product, rather, it must go through some negative changes as well, on the path to a significantly positive change.  One of the early leaders in A/B testing was Amazon.  If you haven’t yet, you should read Brian Eisenberg’s Amazon.com analysis here.

<a href=”http://www.amazon.com/gp/product/B003H4RAO0?ie=UTF8&tag=standwisdo-20&linkCode=as2&camp=1789&creative=9325&creativeASIN=B003H4RAO0″>Glimmer</a>
Tags:


November 9th, 2010

Analytics on large datasets using Hadoop

by Amrinder

Going to the next Hadoop meetup on Thursday, November 17th. Brendan McAdams will be presenting tips and tricks on doing analytics and ETL on large datasets with the ability to load and save data against MongoDB.  One of the items that Brendan will be talking about will be of interest to Java and Scala programmers, who can use Hadoop MapReduce to find a native solution to process their data with MongoDB.

Looking forward to it!

Tags:


November 5th, 2010

In Tech Industry, late is the norm

by Amrinder

Rum Raisin

Tags:


November 5th, 2010

Applying Wilson Score to Ad Variations in Google AdWords

by Amrinder

In Google AdWords (and other ads programs), you can create multiple ad variations, and then Google, in its infinite wisdom, will show the “better performing” one.  You can also use this to conduct simple A/B experiments and compare different versions of landing pages, etc.  However, every time you “tweak” an ad or a landing page, it is considered a new version, so its statistics get reset.  If you create a new ad or landing page, of course, you also start with blank slate of statistics.  The question then becomes: How can you use the low volume of impressions and clicks for various ad variations/landing pages to estimate their performance?

Let us take an example:

Ad 1: 1900 impressions, 8 clicks (0.42% CTR)

Ad 2: 285 impressions, 1 click (0.35% CTR)

Obviously, at this point, Ad 1 seems to be dominating Ad 2, and chances are Google will continue to show that ad more.  However, comparing CTRs at low volumes is problematic due to statistical significance of the data involved (for example, one extra click on Ad 2 would propel its CTR to higher than that of Ad 1).

I don’t know how Google handles it, but one reasonable way to handle this would be by using Wilson score, which in this context can be defined as:

Wilson Score for Ad Variations

where:

  • CTR’ is the Wilson score based CTR,
  • B is the base CTR rate that we would assume about an ad without showing that ad even once (kind of the default rate),
  • z^2 is a measure of how confident we want to be, [For 90% confidence, z^2 is approximately 2.6.] and
  • N is the number of impressions of the ad.

Finding the base CTR rate

This aspect is a bit more interesting.  If we consider all online ads, then perhaps B = 0.1 may be a reasonable number (in either case Google knows this number).  However, the average click through rate is highly dependent on the category of ads.  So it is much more practical to simply take the sum of clicks and divide by sum of impressions. (Although, do note a clear implication of doing this – the CTR’ of your ad with highest CTR will by definition be pushed down, and the CTR’ of your ad with lowest CTR will by definition be pushed up.)

Some Results

Using B=0.1 (10% CTR being the norm), and z^2 = 2.6, and using the Wilson score based CTR formula above, we obtain:

Ad 1: 1900 impressions, 8 clicks (0.42% CTR),  CTR’ = 0.43%

Ad 2: 285 impressions, 1 click (0.35% CTR), CTR’ = 0.44%

So, in this particular case, it does appear that Ad 2 is actually the better performing ad than Ad 1, or at least to say that there is no statistically significant difference between performance of Ad 1 and Ad 2.

However, if we use B as the average value of CTR for these two ads, then the difference between CTR and CTR’ vanishes almost entirely.

Using B=0.004119 (total number of clicks / total impressions for these two ads), and z^2 = 2.6, and using the Wilson score based CTR formula above, we obtain:

Ad 1: 1900 impressions, 8 clicks (0.42% CTR),  CTR’ = 0.42%

Ad 2: 285 impressions, 1 click (0.35% CTR), CTR’ = 0.35%

Further Observations

  1. If B value is used that is the average click through rate, then difference between CTR and CTR’  for most ads is very minor.
  2. Also, it is easy to see that after a few thousand impressions, CTR’ and CTR are virtually the same (as they should be).  So, this calculation is of interest only in the beginning of an ad launch, or after an ad has been modified.

So, after all this, we reach the perhaps anti-climactic conclusion that applying Wilson score to ad variations in Google AdWords may be unnecessary, and CTR may be as good a measure as any of an ad performance.  Null hypothesis holds.

Tags:


November 4th, 2010

Google’s AdWords Seminars (for $299) are rather silly

by Amrinder

Now this email from Google is just silly.

Join us in Washington D.C. for the following:

  • AdWords 101 on Dec 6: Discover how to organize your AdWords campaigns, and craft keywords and ads that give you the most bang for your buck.
  • AdWords 201 on Dec 7: Take your account to the next level. Learn how to turn clicks into customers by identifying keywords that really work, and by improving those that don’t.

Register now

1-day package: $299 and earn $50 in free AdWords advertising

2-day package: $549 and earn $100 in free AdWords advertising

So, you essentially have to pay $299 to Google so you can learn how to pay more money to Google later?

Especially that $50 in free AdWords advertising is just plainly stupid.  (Google is forgetting the KISS principle.)

Tags:


Switch to our mobile site