The Short and Sweet of Cohort Analysis

We all know the story but few have identified it’s patterns in Greek mythology, novels, and film like Joseph Campbell in his book, The Hero With A Thousand Faces.

Tell me if this sounds familiar: a regular joe wakes up in his normal, “old” world and experiences an inciting incident, a person or moment, that will change his life forever.

From there he embarks on a journey into a “new” world, where he’ll encounter obstacles and ultimately an adversary that will test him.

Often, he overcomes this person and is reborn a new, changed being, more luminous, alive, returning to his old world and sharing his journey and lesson for decades to come.

This is the story, we’ve all been taught. True,  “…the hero’s journey is a noble but difficult path.”

Believe it or not, analytics is no different!

If there is one analytics tool that is the perfect combination of powerful and quick, the cohort analysis is your ticket.

The concept, simple: holding everything constant, what changed given a particular “inciting incident?” In short, measuring the “delta,” or change, before and after.

Like our hero returning to his universe a changed human, the benefits of this tool run deeper than you think.

It:

  • Proves return on investment
  • Digs past vanity metrics (read: tells true impact)
  • Keeps things simple (holding time constant, what was the effect of the change?)
  • Finds you the highest performing group (i.e. cohort) and why it happened
  • Is relatively quick and painless

And similar to our hero, your cohort journey isn’t a one-time opportunity, but can be a journey you embark on in every facet of your startup.

Specific to marketing, you could test the before & after of:

  • Targeting different target audiences
  • New ad content
  • New channels
  • Website redesigns
  • New product updates
  • New service offerings
  • Promotions

However, beware of the creeping demons that might linger after your epic journey: do not confuse correlation with causation.

There can be several variables that could cause varying results before and after your inciting incident.

Thus, to ward off these haunting doubts, do 1 of 2 things:

  • Hold other factors constant if possible (ex. Run Facebook ads in June, Twitter ads in July, and Google SEM ads in August)
  • Run randomized control trials (RCTs) where you randomly assign users to the control or variable group

I challenge you to regular embark on this cohero…I mean cohort…journey, and make it a top priority on your startup quest.

Do you have further cohort questions?  Break the ice and share your qualms below.

~JK

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.