The Virtual Whirl: Is one hour enough to be considered an active user?

It has long been a matter of considerable debate among virtual-environment pundits about what constitutes an 'active user'. In some ways, subscription MMOGs have it a lot easier than many other kinds of virtual environment. You can always count paying subscribers, and that's all that matters.

In a general-purpose virtual environment, free-to-play or 'freemium' model, though, counting active users is important. Trends in active users measure the health of your user communities, as well as allowing you to credibly measure your virtual-world's e-peen compared to that of the competition.

Linden Lab's definition for an active Second Life user is a one that has spent a total of 60 minutes or more logged in during the course of a given month. Some argue that 60 minutes in a month (15 minutes per week, or about 2 minutes per day) is a bit of a lightweight sort of metric.

Is that really reasonable? First, I went back to the source ... specifically to John Zdanowski, currently CFO of Avatar Reality (operators of Blue Mars) and formerly CFO of Linden Lab. Zdanowski is arguably among the most authoritative thinkers insofar as virtual environment metrics are concerned, and oversaw the growth of the Second Life economy from US$10 million to half a billion dollars.

Here's what Zdanowski had to say, when I put the question to him:

"Based on my experience with multiple virtual worlds and casual and social games, one hour of usage in a month is a good way to define "active" because it differentiates between the "drive-bys" and those who actually had an Experience in the world.

Active Users, as defined by more than an hour of usage in a month, is a very high threshold. Measures of usage on the 2D Web such as visitors, page views or banner impressions are more akin to seeing a billboard on the way to a concert versus actually experiencing the concert. One takes a split second, the other is an investment of time in an experience.

As it turns out, users who spend more than an hour in virtual worlds, are the most committed users spending dozens of hours a month on average and making up much more than 90% of the total user hours. There are many other activity metrics for segmenting the user base of a game include, but I think the active user metric serves as a foundation for the rest of them. For example, an activity based segmentation that we use for Blue Mars looks like this:

1. Experimenter - Less than an hour

2. Active User - More than an hour

3. Socializer - Makes friend, has chat or voice session

4. Consumer - Spends Virtual Currency

5. Resident - Owns land

6. Developer/Business Owner - Creates virtual items usually to sell

Each builds on the other to form a picture of a user that's more and more committed to the experience and theoretically more and more important to the overall economy and the publisher.

The interesting insights and trends come when looking at combinations of the metrics such as dollars per hour or dollars per consumer. Within each of these customer segments its important to consider how many people are coming into the segment new each month and how many are churning out. Then the fun part begins when you try to figure out why all that's happening and what to do about it!"

Zdanowski's take is that it is a fairly solid metric, though far from the only critical one. To get a bit of a competitive contrast, I also spoke to Simon Newstead, CEO and co-founder of Frenzoo, a browser-based 3D world with user-generated economy.

Is the one-hour per month measurement really a good one for tracking active versus inactive users?

"It's fine. Assuming filtering for inactivity (bots, client hidden in system tray all day) to avoid skewing, I think it's a good metric and captures those engaging in a world.

There would be a multitude of other internal metrics to count paying customers and other forms of activity, but in my opinion they shouldn't be mixed up with active users, which are the next level down the funnel from those who have registered for the world.

For example on Frenzoo we measure new site visitors, new members (who have decided to try after looking at the landing page), active members (broken down by rate for same month, 1 month after joining, 2 months etc), members selling items, purchasing different revenue products etc.

In our case the definition of active user is one who has logged on 2 or more times in a month, because it's the minimum criteria for someone who is using the site versus someone who has an account but is not."

Lastly, I rounded up a few long-term and prominent virtual environment users (having run some names through the big randomizing machine that we have in the basement at Massively HQ) and asked what they thought of the validity of the metric, and – if they didn't think it was a good one – what they would propose instead.

Answers were a bit thin on the ground, as many found the prospect of proposing an alternative metric to be quite daunting, three declined to participate for one reason or another; and one thought I should "die in a fire".

That's why we get paid the big bucks here at Massively: Those flame-proximity suits don't come cheaply, I can tell you!

Despite a few false starts, Harper Beresford was willing to share her thoughts with me. Beresford is the Marketing and Business Manager for House of RFyre, a content producer in Second Life. She also serves as a consultant to Beta Technologies and is currently volunteering for the Second Life Seventh Birthday celebration (SL7B). She also blogs about Second Life fashion, and quite a few other things.

"I think the metric by which Linden Lab measures retention is a little slippery. It's an easy one and an arbitrary one. It encompasses enough time to say that someone has gotten a grasp of Second Life, especially if it requires monthly log ins to count as retention. It's simple to understand and measure. However, it is irrelevant to Linden Lab's business model. If Linden Lab were receiving money from advertisers and those advertisers were interested in exposure to potential customers, then it would make sense. But there is no such model in place.

Linden Lab's customer base is not advertisers -- its users -- and their time of engagement with Second Life is tangentially relevant to Linden Lab's goals. Instead of their current measurement, Linden Lab should use a statistic that asks how many people are bringing money into the system. Frankly, that's what Second Life needs to be about for Linden Lab, not about how many people come in-world for an hour a month.

Their business model and continued existence is based on continued revenues from users. To keep the lights on, the computers running, the employees paid (and the content producers producing and the land barons uh... "selling" server space), a flow of money from users has to go through Second Life, and this is what Linden Lab needs to focus on right now.

Let's just put the figure at $5 US. For 5 bucks a month, you are considered a retained user, regardless of how much time you spend. That's a pretty minimal commitment. With such a metric, I think we would find the statistics show Second Life in a new light, possibly less sanguine than we have been led to believe. Perhaps such a figure will give more impetus to Linden Lab to not only retain users but retain spending users and bring in more spending users. It's what Second Life needs."

Beresford's not alone in her views – I've heard similar from quite a few other users over the last few years, which leads me to believe that the sorts of numbers that committed users find truly compelling are rather different from the ones that operators routinely share.

Perhaps an hour seems a bit low. Two support sessions for technical issues would easily eat up that hour.

I suppose, ultimately, the important thing is to have common, and well-defined measurements that can be compared and historically tracked, rather than necessarily quibbling about arbitrary cutoffs. That way you can at least track trends, make comparisons, and possibly determine the health of the environment.