Cohort analysis - User retention in a Rails application.

I want my actions to be more data-driven. I want to make Dave McClure, Steve Blank, and Eric Ries proud. Easier said than done.

Analytics is still a pain in the ass.

How can I tell if people are using my product, Draft? (Draft is the tool I’m working on to help people become better writers.)

I could look at user retention. Once people start using Draft, do they come back to use it again?

There’s some great software to help study how users return to your product. They use a method called cohort analysis, which breaks up users into groups of people who “activate” or sign-up at the same time and then you track their progress as a group. Do users that signup in January after one month use your app more than those users who signed up in December after their first month? They do? Awesome, those features and things you did in February might be onto something.

To use these analytics tools, I have to integrate my application with them. I have to figure out how to send even more data to them.

But I don’t want to integrate something else right now. I already have data. It’s coming out of my ears. A database full of it. Log files upon log files. Can’t some of the data I already have power this user retention analysis? Do I really have to start all over again collecting new data?

I don’t want to work harder for more data right now. I want the data to work harder for me.

I got irritated, so I built a simple way to study your Rails app’s user retention using a cohort analysis with the data you already have in your database. It’s called CohortMe.

To get a cohort analysis, just use this in your Rails app:

CohortMe.analyze(activation_class: Document)

That’s it.

Above is an example of studying user retention for Draft. Users are “activated” once they create their first Document. Then they “return” if they create another Document.

CohortMe.analyze will spit out a ruby Hash:

{Mon, 21 Jan 2013=>{:count=>[15, 1, 1, 0], :data=>[#<Document user_id: 5, created_at: "2013-01-22 18:09:15">,

During the week starting on 21 Jan 2013, I had 15 people signup. One week later, I only had 1 user still creating Documents. Yuck. That’s a terrible retention rate. I better explore why they aren’t coming back. Note: it’s just bogus test data.

Here’s some partial view code you can use to show your cohort analysis in a nice table:

Pretty easy. If you have any trouble let me know on Twitter or email.

You should get my newsletter here.

CohortMe Details #

Options you can pass to CohortMe.analyze:

Examples #

First, figure out who your activated users are. Are they simply Users in your database? Could be. But I prefer treating an active user as someone who has signed up AND done a key feature.

For example, if you created a group messaging app, it’s probably a User when they created their first Group.

Next, figure out what a user does to be classified as “returning”. This needs to be another record in your database. Is it a Message they made this week? A Share? A new Document?

For my group messaging tool, my cohort analysis might look like this:

@cohorts = CohortMe.analyze(period: "months", 
                            activation_class: Group, 
                            activity_class: Message)

CohortMe will look at Groups to find activated users: people who created their first Group. Next, CohortMe will look to the Message model to find out when those users have returned to my app to create that Message activity.

This assumes a Group belongs to a user through an attribute called “user_id”. But if the attribute is “owner_id” on a Group, that’s fine, you can do:

@cohorts = CohortMe.analyze(period: "months", 
                            activation_class: Group, 
                            activation_user_id: 'owner_id',
                            activity_class: Message)

Here’s an example from Draft. It’s slightly more complicated because I have guest users, and documents that can be blank from people kicking the tires. I don’t want to count those. My cohort analysis looks like this:

non_guests = User.where("encrypted_password IS NOT NULL AND encrypted_password != ''").all
@period = "weeks"
activation_conditions = ["(content IS NOT NULL && content != '') and user_id IN (?)", non_guests]

@cohorts = CohortMe.analyze(period: @period, 
                                activation_class: Document, 
                                activation_conditions: activation_conditions)

Data Returned #

The data returned looks like this:

{cohort date1 => {
    :count=>[integer, smaller integer, smaller integer], 
    :data=>[user event record, user event record]

Installation #

class Users
   def performance
       @cohorts = CohortMe.analyze(period: "weeks", 
                                   activation_class: Document)
       render action: 'performance'
<%= link_to "#{((row[1][:count][i].to_f/start.to_f) * 100.00).round(0)}%", show_users_retention_path(cohort: row[0], period: i) %>

And I have a Controller action that looks like this:

class RetentionController < ApplicationController

  def show_users
    @cohorts = CohortMe.analyze(activation_class: Document)

    @documents = @cohorts[Date.parse(params[:cohort])][:data].select{|d| d.periods_out.to_i == params[:period].to_i}

    user_ids = @documents.collect{|d| d.user_id }
    @users = User.where("id IN (?)", user_ids)


Feedback #

Source code available on Github. Feedback and pull requests are greatly appreciated.


Now read this

Write Better - Draft

I’ve been working on a new way to help people write better. It’s ready. Too many inconvenient things get in the way of good writing. Version control is overly complicated. Writers are forced to learn git to get collaborative edits on... Continue →