Dan Bricklin's Web Site: www.bricklin.com
When The Long Tail Wags The Dog
There are "must have" application areas and data files that will drive adoption of products. Many of them are far down the long tail of popularity. Serving a narrow, most popular set is a losing proposition.
One of the hot concepts mentioned frequently when discussing Internet businesses and applications for the last year or two has been that of the "Long Tail". It was most recently popularized by Chris Anderson's October 2004 article in Wired called "The Long Tail". I've written about some of value of the long tail, as have many others: See "Small Players Matter" from June 2002 and Jakob Nielsen's email there and his essays with data back to 1997.

From what I've seen, most of the writings derived from Anderson's article have to do with the additional value you get by serving the long tail of less-than-very-popular items made more financially viable by the Internet. In this essay I show why it is even more important to serve the long tail in some areas, and how systems that limit the ability to use inexpensively created custom content will have a hard time in the marketplace. I also relate the idea of the long tail to general purpose authoring tools. The essay goes through the background that brought me to these conclusions.

What do we mean today by the long tail?
The standard discussion of the long tail includes a graph like this:


Along the horizontal axis it has the full inventory of items or services available for purchase. The vertical axis shows the number of units of each sold. There is a classic curve with just a few of the products selling in any significant numbers compared to the others. In a traditional store or company only a subset of the products that could be sold is actually made available for sale -- those to the left of the vertical dotted line. There are many reasons for this, such as cost of inventory, lack of shelf space, old fashioned telecommunications architecture, need to focus sales and support in a narrow area, etc.

The long tail comes into play when the cost of making a much wider selection available drops. This may be because of new technology such as the Internet, business models that involve user generated reviews and support, etc. In any case, in areas such as books and music there is a very long tail of additional products, each of value to its purchasers few that they may be. If you look at the total sales you find that the large volume of niche products more than makes up for the few copies sold of each, with those products making up a significant portion of revenue and often profit. You get graphs like this:


Hyperdifferentiation
Last year I heard a speech by University of Pennsylvania Professor Eric Clemons at a DiamondCluster event. (Eric has since become a DiamondCluster Fellow, as am I.) Prof. Clemons talked about "hyperdifferentiation". (See the paper he co-authored titled "Information, Hyperdifferentiation, and Delight: The Value of Being Different".) Hyperdifferentiation is described as "the art of reducing the importance of price as the principal determinant of customers' selection among alternative goods and services...[It's] not about being better in any absolute sense nor [needing to be] more expensive to produce; rather it is about being better for each customer, and thus more profitable to sell."

He gives examples of products such as beer with a high content of hops. Most people don't like the taste, but for those who do like it that beer is something they will pay much more for than a more common brew. He also talks about certain hotels that just do things just the way he likes, and therefore make him feel the extra cost is "worth it".

The issue can be shown graphically this way: The horizontal axis shows the range of items available, sorted by attributes that matter to the purchaser. The vertical dotted line is the point where the product's attributes are "ideal" for a particular purchaser. The vertical axis is the value to that particular purchaser:


Notice how the value peaks at "exactly what they want" and drops off as it meets the ideal less and less. Clemons and his co-authors write: "Assumption: With many choices available, a customer's willingness to pay for products and services falls off quickly when the fit between these offerings and his or her ideal product location decreases."

Uncertainty
What struck me is the following graph, derived from what I remember of his talk and explained in the paper:


When you have uncertainty involved, not being sure that the product will be of consistent attributes or not being sure of your understanding of the likelihood of meeting your ideal, the value goes down. A hotel which is sometimes great and sometimes lousy is not one you will pay as much for as one that is always great. A hotel that you can't check out in advance through pictures or trusted recommendations will make you feel less comfortable about committing in advance to paying a high room rate. Some products (big fast food and hotel chains) often depend upon the value of certainty: Their attributes may not be the greatest, but they give you a very high certainty that you will get exactly what you expect and rarely get surprised on the downside, so they minimize the uncertainty effect.

Hearing this, I realized something. If I have a particular need that can be fulfilled by a tool, the tool I will choose is the one that I know will more likely fulfill that need. If I have to limit my "toolkit" in advance to just a few tools, I try to include those tools that will meet as wide a range of needs as possible from the range of needs that I think I will have in the future. Tools that can be used for a wide range of purposes therefore become very popular: Hammers, screw drivers, pliers, word processors, personal automobiles, etc.

General purpose tools
A digression about general purpose computer-based tools:

I've learned from my work developing spreadsheet and word processing systems over the last 30 years that one of the reasons they are so popular (and have repeatedly beat out competing genres of products aiming for a subset of their uses) is that these tools are so general purpose and well understood.

People's needs with text and mixed text and data documents are quite diverse. When you get down to the details, most uses are custom. While an "income statement" may sound like a standard thing, so many of them are very specific to the company they report upon and the needs of the particular executives and others who use them. A production planning report is even more varied. A "letter" or "resume" may sound standard, but people like to craft each to be exactly their way and to fit the content.

The beauty of a computer tool like a spreadsheet or a word processor is that it provides an understandable framework with which the user can figure out by themselves how to create exactly what they want. (We sometimes call this framework a "metaphor".) The spreadsheet provides a framework by having a row and column system. That system imposes a slight order and restricts things a bit, but it opens up areas like summation on ranges and other operations. The freeform positioning and arbitrary cell references keep the framework from getting in the way of making things exactly as you want them to be. The spreadsheet as a tool takes care of the tedious parts of producing a wide variety of output without defining the problem being solved. It has calculating (and recalculating) abilities, both text and numbers and straight lines, and numeric formatting -- all tedious things otherwise.

General purpose tools such as these are appropriate when you have very diverse situations. They handle unanticipated needs well. They are "tools", not automatons or agents. They allow for customizing (dealing with the unanticipated) as opposed to just choosing (dealing only with anticipated, predetermined needs).

An important aspect of the framework of a good general purpose tool is that once you learn it, the range of areas it can be successfully applied to should be something you internalize. You know which things are easy to get done exactly as you want with a word processor or with a spreadsheet. You may never have used them for that particular application, but you know the "space" it covers well enough to know if the application is in that space. Tools whose applicable "problem solving space" is broad become popular, from word processors to duct tape. A general purpose tool that most people can't use easily or understand how to apply in new situations will be less popular.

General purpose addresses hyperdifferentiation
Back to hyperdifferentiation: If you know that a product can be used for a range of needs, you will have less uncertainty about getting to your ideal. You may not be sure that public transportation will get you to a new friend's house, but you know an automobile can.

Here is another graph:


David Reed points out that this is related to option theory. You would be willing to pay very little for an option to have something that is unlikely to occur compared to the maximum potential value you would get if you exercised the option. On the other hand, an option to get something that is very likely to happen would be worth much closer to the value of buying it with certainty. The question with valuing an option is how do you know the likelihood of what will happen. When you have a tool that you understand its applicable problem space well and that covers your desired area completely you can have greater certainty of success than a tool which you feel has a smaller applicable problem space and that is unlikely to work for many of the problems that you may need solved.

David points out that the key is having a tool whose design is such that you feel confident in knowing what it can be applied to and a problem space which you can understand the range of needs that must be met by such a tool.

So, general purpose tools that people understand are more likely to serve a longer part of the "tail" of potential future needs and also be valued higher because of such likelihood.

Must Haves
While interviewing ex-Palm Computing CEO Donna Dubinsky last year, I asked her about the applications people came up with for use on the original Palm handheld and their effect on the success of the product (starting at 2:30 in the recording). What she said surprised me. In addition to using one or more of the few "popular" built-in applications (calendar, address/phone book, etc.), "...rather than adding 3-5 applications [to those existing built-in ones] on a device, customers would find one that was compelling for them, and that to them was a make it or break it thing. It might have been the stars and what the stars are, it might have been world traveler applications, or it might have been querying a detailed database at work, but there was always some additional compelling application for the Palm owner."

That statement struck me as very special. There are "must have" applications. Good versions of the popular, everybody-needs-them applications were a minimal requirement (and highly valued). But people bought the Palm instead of some cheaper limited device that could only do those popular applications because they had at least one other niche application that made the increment in cost worth it. More importantly, the Palm (joined later by similarly customizable Pocket PC competitors) became dominant beating out the cheaper, non-customizable devices. The "must have" applications were often quite limited in their appeal and often labors of love, but they made the Palm a successful product.

So, it isn't just the number of potential items that's of value, it's the kind of items. Some are of great value and their availability completely drives the purchase. For example, in the late 1990's, many people in their 40's and 50's didn't use personal computers or email. But when their children went away to college and email became a major form of communication available to stay in touch with those children, they got the computers, signed up for email, and started using it. That one application got them started. Once they were email users they would use it to communicate with people other than their children, but it was that one particular application (communicating with a particular loved one) that drove the purchase and adoption.

Applying this to the long tail
So, now we have another graph:


The horizontal axis is the range of things you'll want to do sorted by popularity. The vertical axis is the value to a particular purchaser. For the sake of argument, I'll assume that in general the value of each item is reflected in its popularity. For example, with PDAs, everybody wants a calendar application and an address/phone book. What I've added is the occasional item that is a "must have". These are of great value to the purchaser.

Think of telephones (wired or wireless). Lots of people need to call airline reservations at some point. Fewer, but still many, need to call LL Bean. Many in a city will need to call for concert ticket reservations. Only those who live in the neighborhood will call a particular pizza restaurant. Finally, only a very few will need to call a particular automobile service station and even fewer a particular individual. However, there are some individuals (my friends, relatives, etc.) that I really, really want to talk to. When my car is making some funny sound and I'm about to start a long trip or when it breaks down I really want to talk to my car mechanic. That's why I pay for the phone service. I don't pay for it to just make calls to LL Bean or receive calls from a Verizon representative. Much of my use is only of interest to me. See my "What will people pay for?" essay from July 2000.

Now, let's add the "must haves" that I don't know about yet but can easily anticipate and want to be prepared for. With the phone it includes friends I haven't met, future customers, and perhaps a doctor for a disease I haven't come down with yet. Here's the new graph:


If I'm buying a system to meet this general area of need, how do different system architectures fit? Here are two types: One is a restricted, special purpose system that meets the popular needs quite well. It is of high value for those needs, but not for the others. The other is a general purpose system that I know will meet many, many more of my current and potential needs. It is of high value for a wider range. As David Reed says, general purpose gives you more options and in the right cases more higher valued options because they are for the right kinds of high value potential uses. Here's the graph:


Another way to look at it:


I believe that for many areas, only products that serve both the left and right side of the graph -- providing popular applications/content as well as specific, niche ones -- will be successful in the long run. The niches can be personal (my family photos, for example), commercial (internal-use-only corporate stuff), or by other means (buyers of Pez dispensers).

I also believe that even when the general purpose system produces somewhat lesser value for those popular applications, the fact that it covers the "must have tail" more than makes up for that drop in value. I'll put up with the lower quality of the sound on a telephone over an FM-radio to listen to my friends rather than just professional radio personalities. Many more people carry a cell phone with them than an FM radio, and they willingly pay much more for the phone.

Here is an example that has been in my thinking: You have a cell phone that can receive video and display video. Should it be tuned for the top 100 video feeds and make it hard or impossible to see the birthday videos of your grandkids or grandparents? Or should it let anybody inexpensively share content composed using any of the popular tools available? I maintain that given a choice, the latter products will win. There will be some content you need (for example, created by a loved one or on the topic of a hobby you are most passionate about) that would only be available on the general purpose system, and you know it (or will find out quickly when you find out it's available). It may be that the niche content is the reason you buy the product in the first place.

The conclusion
So, here is my theory which I call "When The Long Tail Wags the Dog":

In areas where results are often customized to people or situations and that can't be addressed directly in advance, such as when there is a high value to most users of at least some user-generated material, "open" general purpose systems that address all these niches will dominate.

I believe this helps explain the emergence of Internet email, cell phones, automobiles, and many other systems we take for granted today and is useful for evaluating systems and business models for the future.

Companies that develop closed systems that ignore generally created content and applications and that don't take advantage of widespread "open format" content/application creating tools (that is, those that produce output that is easily consumed by a wide range of different content-accessing or application running tools) will be eventually wiped out by those that do.

-Dan Bricklin, 19 February 2006
© Copyright 1999-2018 by Daniel Bricklin
All Rights Reserved.