The Deployment Post Mortem

Your product just released. Customers are buying your product in droves and they love it. It saves them time and money, makes them better at their work, and it even slices bread!

That's wonderful and these moments are what we Product Managers live for. However, things don't always go according to plan. What happens when things don't go so well? Inevitably, a customer deployment will go poorly. They won't be happy with your product, and may even uninstall it. What can you do in a situation like this? One approach is what I will call the Leo Tolstoy approach. The great writer said: "All happy families resemble one another, each unhappy family is unhappy in its own way." In other words, treat the failure as an outlier, an anomaly, rely on your customer support to patch things up as best they can, and move on.

A better approach is to look at the failure as an early-warning mechanism. This is your opportunity to understand whether the failure is indeed an anomaly (as some will inevitably turn out to be) or whether it points to a systemic problem that you need to take steps to address. As Product Managers, we are attuned to issues in the field and actively search for patterns or trends that point to a broader problem. Depending on the type of problem, PM's should have a checklist handy to make sure they get the information they need. The two most common types of issues I have come across are:

Failure in the field. The software caused an outage in the customer's IT environment. A PM checklist should include:

• What was the problem exactly? Get a detailed explanation.
• Was it due to an unexpected software/hardware configuration?
• Was the configuration explicitly not supported? If so, why was it installed? Where did our internal process fail?
• Was the configuration explicitly supported? If so, go back to problem determination. What must be done to include this use-case in future QA plans?
  
• Was the configuration neither explicitly supported nor unsupported? If so, is this a configuration we did not expect to see and did not plan for? Is it an unusual combination that we're unlikely to see again? If it is likely we will see it again we should add to QA plans. In the case that this is a fairly common configuration that was not part of the QA plan, we need to go back and revisit the entire QA process. How closely does the QA test matrix map to what is encountered in the field? Do we need to do more research to ensure adequate QA coverage?
  
• In the problem determination phase, as we uncover the root cause, can we generalize the effects? I.e. Could the same problem manifest itself in other ways? How to we prevent those other problems as well?
• Once we understand the root cause, can we generalize a "graceful degradation" principle from it? Can we build in checks so that when faced with an analogous unknown situation, the software is able to adhust or "degrade" its functionality rather than cause an outage?
  

Mismatched expectations. The product works as specified, but it doesn't do what the customer thought it would do. They don't see value in the product. A PM checklist should include:

• Did we oversell or overpromise product functionality? If we did, this could point to at least two different problems:
  
• If we oversold, was it because the sales force was not trained adequately to know exactly what the product does and does not do? We will then need to work out a more comprehensive training plan for the sales team.
• If we oversold, was it because the sales team felt cornered into "improvising" in the field? This could be an early signal that the product is not meeting the needs of the target market very well, and the sales team feels pressured into setting unrealistic expectations.
  
• Figuring out whether it is a training issue or a product mismatch issue is extremely important - one can be fixed relatively easily, the other may require major realignment.
• Did we demonstrate value? Even if the product is implemented and works flawlessly, did the customer realize the benefits and/or return on investment they were looking for?
  
• If the customer did realize benefits, but is not aware of them or unable to articulate them, then we need to showcase these benefits better and make it easy for the customer to see it. Perhaps this needs additional reports, a dashboard, or some set of running statistics that enables customers to quantify the value they have received.
• If they did not realize their expected benefits, why not? If the software was meant to replace some manual activity, is the manual activity still being carried out? Is there duplication of effort because some component was not correctly or adequately integrated?

Product Bytes Newsletter

I've been a fan of Rich Mironov's newsletter, Product Bytes, for a couple of years now and wanted to provide a link to it for anyone who may be interested. It's a refreshingly BS-free take on the art and science of Product Management, and I always learn something new in every issue. It's a bit skewed towards enterprise software, reflecting Rich's long experience in that area, but I think the basic ideas apply quite broadly.

Long Bets

The Ladies Home Journal from December 1900 contained an article listing predictions for the year 2000. The author writes:

These prophecies will seem strange, almost impossible. Yet, they have come from the most learned and conservative minds in America. To the wisest and most careful men in our greatest institutions of science and learning I have gone, asking each in his turn to forecast for me what, in his opinion, will have been wrought in his own field of investigation before the dawn of 2001 - a century from now. These opinions I have carefully transcribed.

Some of the more remarkable themes:

• Telecommunications: "Man will See Around the World. Persons and things of all kinds will be brought within focus of cameras connected electrically with screens at opposite ends of circuits, thousands of miles at a span."
  

• Spy satellites: "Balloons and flying machines will carry telescopes of one-hundred-mile vision with camera attachments, photographing an enemy within that radius. These photographs as distinct and large as if taken from across the street, will be lowered to the commanding officer in charge of troops below."

• Genetically modified food: "peas as large as beets" and "strawberries as large as apples."

There were some exercises in wishful thinking, such as the view that we would all be athletic and fit in a hundred years:

Gymnastics will begin in the nursery, where toys and games will be designed to strengthen the muscles. Exercise will be compulsory in the schools. Every school, college and community will have a complete gymnasium. All cities will have public gymnasiums. A man or woman unable to walk ten miles at a stretch will be regarded as a weakling."

We'll try to get there in 2100. And my favorite, the somewhat mystifying prediction:

There will be No C, X or Q in our every-day alphabet. They will be abandoned because unnecessary.

The full article is available here.

Fight Poverty with Connectivity

The notion that large-scale handouts of aid hasn't worked to alleviate poverty is well documented. In the worst case, it enriches corrupt, autocratic kleptocracies (e.g. as it did with Mobutu in Zaire). More commonly it's simply wasted because the institutions necessary to use it are not effective, and a sort of low-grade ineffeciency and corruption takes hold. Even the biggest provider of such development aid, the World Bank, has now recognized that aid must be linked to governance to be successful (championed by Paul Wolfowitz, whose current woes do not invalidate this notion).

The basic underlying lesson, according to Iqbal Qadir, founder of GrameenPhone, is that poverty can be reduced only by empowering individuals, not governments. His own involvement in setting up a cell phone company in rural Bangladesh is testament to the individual-centered, connectivity-based model of economic development. Qadir is currently a director at the MIT Center for Developmental Entrepreneurship, which has already brought to market several innovative products for developing economies.

See below for a talk that Mr. Qadir gave at the TED conference in 2005, explaining his ideas about ending poverty through connectivity. (If you don't see the embedded video, click here).

Living La Vida Local

Things are different here. The first thing I noticed were the number of "IIT " bumper stickers. Around here there are more IIT bumper stickers than for all other colleges put together! The pool in our building is full of Indian kids splashing about while their parents call out to them protectively in Hindi, Gujarati or Tamil. And the once-familiar sight of clothes hanging out to dry on window ledges and balconies, is once again common.

We can walk to not one, but two, different Indian grocery stores. Steaming hot platefuls of idli and dosa are available in abundance - and cheaply! My wife and I love Keralan food, so we were delighted when we found a great little place with fantastic pepper-fried chicken. Even though we don't have kids, our friends in the neighborhood tell us about the cultural center they take their kids to on the weekends - to learn Indian classical music and dance. For us less cultural pursuits, like watching Guru, suffice. Or, if we feel like staying in, we can go pick up an Indian movie at the local video store.

No, I'm not back in Bangalore - we live in Sunnyvale, CA! Although it wasn't planned this way, it's turned out to be a great transitional place for us when we moved from San Francisco last year, and get ready to move to Bangalore next month. Sunnyvale and Fremont in the East Bay are the two centers of Indian life in the Bay Area. Everyone planning to relocate from here to India, whether Indian or not, should come and spend a couple of months here to make the transition smoother!

Coda

Now that I'm moving to India in a few weeks, I find myself reflecting on the fifteen or so years I have spent in the US. When I left India in 1991, it was a country that bears little resemblance to the country I will return to next month.

The most apparent changes are the superficial ones: glitzy shopping malls (there were none in 1991) and the profusion of cable TV networks (there was no cable TV in 1991), among others. There are deeper changes too. The one that strikes me more than anything else is the sense of possibility and confidence I see in today's high school and college students. When I was in high school, the limitless possibilities that follow from rapid economic expansion was not something we really conceived of in any meaningful manner.

A nice little illustration of all this can be found in the foreign exchange situation then, and now. In 1991, right was I was getting ready to leave for the US, India's balance of payments weaknesses suddenly caused a crisis. The government was close to defaulting on its debt, and foreign exchange reserves had dropped to about three weeks worth of imports - about $6 billion. As part of a package of reforms, India moved from a fixed to a floating exchange rate, which immediately caused a severe devaluation. I remember my father being quite upset, as my education in the US suddenly became 50% more expensive than it had been a month before!
(If you're interested, you can read more about the 1991 currency crisis in this IMF paper).

Contrast that to today. For those of you who deal in India-US cross border issues, you're probably already aware of the rupee's appreciation against the dollar over the last six months. In fact, the rupee is at an at an 8-year high against the dollar.

Take a look at this exchange rate chart from Oct 06 to today:

The rupee has appreciated from 45.7 per USD in October last year, to 42.6 per USD currently. Foreign exchange reserves have ballooned to almost $200 billion today. It's a world away from 1991.

Long Lived by Design

Picking up where my last post left off, the issue of data longevity is specially important to developing economies where the process of digitizing government records and making government services available online is just beginning. The lessons to be learned from data loss from the early adopters in more developed economies should be taken to heart.

There's no question that this is a well-understood problem and many projects are tackling different aspects of it. For example, open standards such as ODF reduce the risk of unreadable data. Digitization and hosting of content by service providers offloads the problem from individual consumers to service providers who are, presumably, better suited to deal with it. At the cutting edge, there's even talk of using bacteria for long-term data storage!

My point is not that adequate steps are not being taken. My point is that product design should incorporate principles with longevity in mind. What might these principles look like? A great place to start is by looking at the Clock of the Long Now, conceived by Danny Hillis (of Thinking Machines fame). As he explains the genesis of the project:

I want to build a clock that ticks once a year. The century hand advances once every one hundred years, and the cuckoo comes out on the millennium. I want the cuckoo to come out every millennium for the next 10,000 years. If I hurry I should finish the clock in time to see the cuckoo come out for the first time.

This seemingly quirky endeavor is actually a deeply insightful way to examine our notion of time and its impact on technological progress. Hillis lays out a list of design principles for his 10,000 year clock.

• Longevity: The clock should be accurate even after 10,000 years, and must not contain valuable parts (such as jewels, expensive metals, or special alloys) that might be looted.

• Maintainability: Future generations should be able to keep the clock working, if necessary, with nothing more advanced than Bronze Age tools and materials.

• Transparency: The clock should be understandable without stopping or disassembling it; no functionality should be opaque.

• Evolvability: It should be possible to improve the clock over time.

• Scalability: To ensure that the final, large, clock will work properly, smaller prototypes must be built and tested.

These principles seem to me to be broadly applicable. If your product, software or hardware, may be used for any duration longer than five years, you would do well to consider each of these issues (adapted for your specific situation) and how you plan to address them in your product.

Microsoft Office v. 2100?

Those of us who have an interest in technology and want to build widely adopted products will do well to consider the issue of longevity. Building products with a long-term view is a good practice in itself; but also thinking long-term yields design requirements that tend to be beneficial for the overall product.

As an example, consider the Domesday Book, a comprehensive survey of England, commissioned by William I of England, and completed in the year 1086. The book still survives and can be viewed in Britain's National Archives. Today, more than 900 years later, the book is readable, understandable, and provides valuable historical, social and economic insights to historians and scholars. Inspired by the Domesday Book, the BBC undertook to create a similar survey of England in 1986. The contents were stored as 12-inch video discs (remember those?) which are now obsolete. The digital version of the Domesday book lasted all of 20 years!

Think about how much of the information generated in the last 40 years or so is already lost to us. Data created in some proprietary format for which the program is no longer available, or stored in obsolete media which can no longer be read. Clearly we generate a lot more content that we did before, and perhaps not all of it needs to be preserved. However, since there is no way to distinguish what should be preserved from what should not (and who gets to decide this anyway?), they are treated identically. There are groups, such as the Digital Preservation Coalition, based in the UK (started as an attempt to save the 1986 version of the Domesday book as their top priority!) that are tackling the issue. But a lot more remains to be done.

Even more important than preservation efforts are design principles that allow for the creation of long-lived content. Products designed with longevity principles in mind are the only proactive and scalable way to solve the problem. Preservation will always be a reactive and expensive fall-back option and will never offer a complete solution to the problem.

How Not to Get a Job

Some time ago, I blogged about connecting with your passion to find the best job opportunity for you, and also about innovation in recruiting techniques that enable companies to find these passionate people.

In an interesting, and unintentionally humorous, example of what *NOT* to do, a colleague actually received this cover letter from a job applicant today:

To Whom It May Concern:

This is an opportunity I've been searching for. I carefully have been selecting what Company’s to send my Resume. I’m honored to send my resume to your Company and possibly be considered as a team member. My skills along with personality make a perfect match for what you’re seeking. I look forward to meeting with you to discuss my future with your company. I'm confident, coach able, persistent, and consistent in achieving success independently or with a company that has a positive direction. Please call me so we can discuss a time to meet.

While I have no idea what this person's situation is, or how qualified he is for the position, this is a letter that is guaranteed to lead to failure. What was astounding was that it came through a recruiter! How could this have made it past even the most cursory due diligence? While this is obviously an extreme example, I've seen more polished cover letters that say essentially the same thing. Where's the passion?

The Decoy Effect

I just finished reading "A Beautiful Mind" about the game theorist John Nash. If you haven't read the book, or seen the movie, I highly recommend that you do. Nash's life is as dramatic as it gets: genius, fame, tragedy and finally redemption.

Game theory is an interesting, if somewhat dispiriting, lens with which to view decision-making processes. It's astonishing how predictable we are. Even when we're unpredictable, we tend to be unpredictable in a systematic (i.e. predictable) manner.

How does any of this relate to software, technology and innovation? Only this: I think it provides a useful set of tools for product managers and marketers to have at their disposal as they think about their target markets and adoption of their products. There's one interesting example that I remembered reading about in grad school. It's a phenomenon called "The Decoy Effect" and provides an insight into how consumers value different attributes in a given product set. The wikipedia article explains it very well so I won't discuss this in any more detail here.

Thinking about current products within this framework can be a clarifying experience. As an example, consider the market for web-based meetings. There are two major players: Webex and GoToMeeting. Customers value two attributes: ease of use (to set up as well as to participate in meetings) and collaboration features. Based on my experience with both products, Webex is harder to use but has more powerful collaboration features than GoToMeeting. What sort of decoy product would increase adoption of Webex over GoToMeeting? What about vice-versa?

By the way, the decoy effect can be used in a wide variety of circumstances. Take a look at this fascinating article in the Washington Post about the decoy effect applied to the 2008 U.S. Presidential Elections.