05/10/2013
Exrept | 'Innovate or Die' by Jack V. Matson
CHAPTER 15 - IDEA EVALUATION
Evaluation is something you do every day. You evaluate what clothes to wear every morning and the route to take to work. These evaluations may seem automatic, but the risks and consequences of each decision are weighed. Try the following short exercise to gain insight on how you evaluate ideas.
1. Write down at least two recent risks you have taken which had successful outcomes. Now decide why they were successful. Be specific. How did you arrive at your conclusions?
2. Next, write down at least two risks you have taken that were unsuccessful. Write down why the outcome of each risk was unsatisfactory. Again, be specific. How did you arrive at these decisions? What did you learn from these experiences? How can you improve your judgment?
There are almost as many ways to evaluate ideas as there are to generate them. The easiest way to evaluate an idea is to trust your instincts, to go with your intuition. Here you're deciding which ideas are the best based on what feels right.
Your intuition often tells you what is right about an idea. But you don't want to dismiss an idea out of hand, simply because it didn't grab you at first glance. Ask yourself, "What's wrong with this idea?"
Snap judgments are often guesses. It's hard to be objective about your own ideas simply because they are yours. One way to achieve some measure of objectivity about your ideas is to seek outside opinions. While honest feedback is invaluable, bypass "critics" who may not tell you what they really think of an idea because there's a relationship at stake. Instead, when you want an objective opinion, try consulting an expert, someone who's knowledgeable about the idea you're interested in. Experts don't usually have the biases of family and friends, and sometimes their suggestions can be valuable.
However, since experts are human and humans aren't perfect, build in some safeguards. Get more than one expert opinion and remember that experts can make mistakes, especially if they have a stake in your ideas. In fact, sometimes an expert's very expertise can limit his judgment and cause him to declare prematurely "It can't be done."
And one last point on idea evaluation. Often it helps to let some time pass between generating ideas and evaluating or acting on them. The old adage "sleep on it" has value. Quick turnaround is important, but everything must come in its own time. Go at your own pace (not too slow), but don't feel pressured to commit to action before you are ready. Ideas look different after a few days or weeks have passed. Ideas that you once thought had no value may take on new meaning with the passage of time. And hot ideas may not seem so hot. A fresh perspective can be invaluable.
Ask yourself what would happen, realistically, if an idea were implemented right now. Evaluate the realistic outcomes. What's the best thing that could happen if you went with this idea? What's the worst thing? Could you survive the worst outcome? Is the best outcome what you really want? Does it meet your goal or solve your problem? Do long-term advantages overcome short-term adjustments?
You can use checklists to evaluate ideas as well as to generate them. The following questions should be useful:
Is it effective? Will your idea do what you want it to do? Will it work? Will basketball shoes with cantilevers really enable people to jump higher? Will becoming the teacher's pet really result in a better economics grade?
Is it efficient? Is your idea better in some way(s) than anything else already on the market? Is it significantly different from the status quo? Will it result in gains that would make someone want to adopt it?
Is it compatible with human nature? An idea won't help anyone if people won't adopt it. A product may be unnecessary if people won't use it For example, the best-selling computers on the market are those that require the least amount of effort to successfully operate. They are "user friendly," adapted as much as possible to our tendency as human beings, to avoid that which is difficult or distasteful.
Is the timing right? Would your idea be practical right now? Will it keep pace with future trends? How would it have looked in the past-better than today? Can it be easily adapted to change? Good timing is essential for success in business, personal relationships, sports - almost every area of life. The right idea at the wrong time usually equals failure.
Is it feasible? (Can it be done? If it can be done, is it worth it? Compromise is sometimes an important part of creativity. If you have a marvelous idea, make sure that the means to implement it exist and that you have access to them. Leonardo da Vinci filled his notebooks with drawings of flying machines, but technology took another four centuries to catch up. Is implementing your idea worth what it would cost? Do a cost-benefit analysis.
Is it easily understood or capable of being explained? Make sure you haven't reinvented the wheel or the polyester suit.
Checklists can also be specific, tailored to the idea you're evaluating. For example, aesthetics would be a concern in evaluating some ideas, but not others. The word "cost" has many meanings, not all of them monetary. Be innovative - make your own checklists. No one knows your ideas better than you.
RATING SYSTEM
Set up a numerical rating system with ten as the best and zero as the worst score. The rating system is based on your best judgment Write down your best ideas in no particular order in tabular form.
Add up the scores from Idea Value and Feasibility for the total in the fourth column. Toss out the ideas with the lowest total scores. Compare the results of the numerical rating system with the checklist or other system you used. Each evaluation technique yields a different angle. Do not get caught in an endless evaluation cycle. Choose the ideas to go with and move on to experimentation. In the example above, five names for a new toothpaste were being considered. Ideas 2, 3, and 4 were tossed out because of low scores.
EXPERIMENTATION
Consider yourself Chief Experimentalist in the laboratory of your innovations. You are a scientist, poking around and testing, receiving feedback, learning enough to plot your next set of experiments. As in the process of scientific inquiry, most of your experiments may fail. It took Thomas Edison thousands of experiments to find a suitable filament for the light bulb. You may not work on projects with such a high rate of failure, but science operates on the same principles advocated here - trial and error.
As Chief Experimentalist, you will be influencing others to participate in your experiments. Your attitude determines how well you can convey the spirit of inquiry and adventure to others.
EXPERIMENTING WITH OATMEAL RISKS
Oatmeal risks are ordinary risks you undertake to get through daily life. You know from experience that you are likely to fail at certain things. Maybe you hesitate to work on any problem involving math, such as balancing your checkbook, because you've always been told and believed that you were bad at math, or that you have "math phobia." Your car may not start because the engine under the hood is a complete mystery to you.
As they accumulate, these oatmeal failures can have a profound impact on your life. They limit your range of opportunities, close off possibilities, and undermine your confidence in yourself and your" abilities. These "small" risks of your life are good opportunities to begin experimenting.
Take out your Idea Journal and start with a blank sheet. At the top of the page write:
EXPERIMENTS DATE
The four subheadings are:
I. NATURE OF THE EXPERIMENT
What are you trying to do?
II. UPSIDE POTENTIAL
What is the best possible outcome? The most probable outcome?
III. DOWNSDE RISK
What is the worst possible outcome? What would happen if all your experiments failed? Evaluate the worst-case scenario.
IV. SETUP
A. IDEAS
Go through the idea generation process, using the techniques that seem most
applicable to this experiment. List the best options.
B. EXPERIMENTS
Order your experiments and estimate the time and resources required.
Here's how a sample experiment might look on paper. Let's say your experiment is to look at business opportunities associated with environmental protection by joining an environmental group.
EXPERIMENT: Environment-Get Involved 01/01/95
NATURE OF EXPERIMENT
Get involved in an environmental group.
UPSIDE POTENTIAL
I'll learn more about the environment, hopefully improve the environment, meet like-minded people, and uncover business opportunities.
DOWNSIDE RISK
Wasted time and effort.
SETUP
1. IDEAS
Get involved with a citizens' group on a hazardous waste problem.
Join the Sierra Club and participate in a committee on a current environmental issue.
Research a local environmental problem and present it to City Council.
Start a recycling drive in my neighborhood.
Volunteer to go to a grade school to speak about the environment.
2. EXPERIMENTAION
Each idea needs to be defined and refined on paper.
Smaller experiments can be done in your head. For instance, think about all the possible routes you can take to work. Your mind might generate ideas for six ways you haven't ever tried. Then experiment. Over the next six weeks, take each new route several times and note the time and hassle factors involved. Your mental mapping will give you not only alternative routes but a chance to visit new terrain and see stores and neighborhoods that may be beyond your normal vision. Finally, briefly document what you did and the results in your Idea Journal.
Use your Idea Journal to document your successes, your failures, and your own style of experimentation. On the average, how many failures does it take you to hit a success? How far from the stake do you stand in your personal ring-toss game? Examine the records of your experiments and outcomes in your Idea Journal.
Experimenting on a small scale can have large rewards. One of my friends, Joe Smith, makes it a practice on a daily basis to call or write someone he hasn't spoken to for a long time, or listen to a new type of music, or read a different magazine, or try some new activity. He remarks: "Experimenting has made me realize how precious life is. The important thing, and my thoughts on this are constantly evolving, is to experience life and all the dazzling variety it has to offer: love, happiness, friendship, success, and failure. Many small experiments may not work out, but you can just try them anyway, realizing that they will give you information well worth having. When you succeed, a new opening to the world is made."
Once you begin to experiment, the process will have an effect on you. You may become more of a maverick. You will not want to accept pat answers or canned responses to your questions. You will insist on finding out for yourself, on running experiments instead of accepting conventional wisdom. You will become more childlike and playful, and your sense of humor will deepen. Your vision will become peripatetic, always scanning the horizon for the next set of experiments to run. You will show creative dissatisfaction with everything. You will begin questioning everything you do, and the questions will prompt you to begin experimenting.
Learning will become active, instead of passive. You will become self-expressive and passionately involved in your life and in what you do. An experimental life demands all you can give, including emotional commitment. You will experience pain and misery, as well as pleasure and joy, as experiments fail and succeed. The process of intelligent fast failure isn't easy. Nothing worthwhile ever is. An experimentalist recognizes negative and positive feelings for what they are and moves on as quickly as possible, learning from each experiment.
ACCELERATION EXPERIMENTATION
Experiments are usually run in sequential order. You prioritize your ideas and start experimenting with what you think is the best one. If the idea does not work, you abandon it and try the second idea. The sequential process looks like this:
Experiment 1 → Fail; Experiment 2 → Fail; Experiment 3 → Fail
The huge problem with this approach is the time it takes to run the experiments.
What if you ran the three experiments simultaneously? The process looks like this:
Experiment 1 → Fail
Experiment 2 → Fail
Experiment 3 → Fail
The time taken to run three experiments simultaneously instead of sequentially is much less. The feedback you receive is accelerated so that your decision time is quicker. You maintain a steeper learning curve that is absolutely essential to maintain pace with competitors.
The slower sequential experiments lengthen decision times and create emotional frustration, and they can be more costly. However, sequential experiments are essential when a test result indicates that a modification of an idea shows promise.
Combinations of simultaneous and sequential experiments may be mixed as the process reveals itself. Your goal as Chief Experimentalist is to run the experiments quickly, which means do as many simultaneously as possible.
Joe Sanchez was in the collision-damage repair business. He needed an experienced estimator. Through friends and an ad in the newspaper, he developed a list of six good prospects. At this point, he had a choice. He decided to prioritize the six candidates and interview the top one on down until he found the one he wanted. He thought the sequential method would save him time because if one of the top candidates was the right one, he could stop the process.
But as it turned out, none of the candidates satisfied Joe, so he turned to an employment agency. They sent over four prospects within a day for him to interview. None of them fit the bill. Joe then decided to train several of his best repairmen to estimate part-time. That worked.
Joe lost three weeks in the first round of sequential interviews. He lost only a day with the simultaneous interviews. Quick feedback from the failure to find a suitable estimator allowed Joe to rapidly make a business decision and move on.
Denise Torres had a problem: She could not figure out how to price her home- delivered cakes. She knew how much the cakes cost to produce and deliver, and what kind of profit margin she wanted. But the price seemed too high. She wanted the right price to move her product. Her first thought was to lower the price in increments over a period of time until the cakes sold in adequate volume. Then she had a better idea. She simultaneously advertised the cakes under different names and prices in three different newspapers after verifying that readership demographics for each paper were similar. Sure enough, she quickly found the optimum price for her cakes.
Joe and Denise realized the best way to experiment was to accelerate the trial and error process. The knowledge they gained allowed them to make intelligent decisions quickly and move on.
DOWNSIDE RISK ANALYSIS
Using the strategy of intelligent fast failure, you should assume at the outset that most of the experiments will fail. Therefore, the resources (time and money) that you put into each experiment should be proportionate to how much you can afford to lose. Further, it should be an amount that allows for realistic tests. The critical question is: how controlled can you make the experiments to minimize the downside risk? There is no formula answer; your experience and intuition must respond. You must look constantly for ways to reduce downside risk so that you can continue to experiment at a high rate.
Elizabeth Matthews is a real estate agent in vacation home sales. Constantly she looks for innovative ways to market her properties. Standard forms of advertising in newspapers and magazines cost a lot of money, so she looks for offbeat and inexpensive forums such as university newspapers, trade association and professional society newsletters, and company news organs in targeted cities. Advertisements in these forums are inexpensive and free of competitors. After three months of ads, if no response is generated, she stops. This is the essence of intelligent fast failure: low risk, multiple experiments to explore innovative possibilities.
Sam Chamberlain had an idea for an innovative snow shovel. He built one in his basement, and after many modifications, he was ready to market it. He found a local manufacturer who could produce the shovels in increments of one thousand, but he was stymied; he did not have the ten thousand dollars necessary to get into production. His alternative was to take his shovel around to independent hardware store owners and see if they would take it on consignment. The response was terrible; none would. Sam has not given up. He is producing fliers to pass out in neighborhoods. The point is he found out the response of middlemen quickly without investing a large sum of money. One could argue that Sam would be much more persistent if he invested the money. Possibly. However, the prudent tactic is to hold on to your resources until the experimentation process has yielded sufficient knowledge for confident investing.
THE GOLDEN EGG AWARD
One of the greatest obstacles to experimentation on any level, organizational or personal, is the cover-up that often follows failure. This behavior is particularly destructive because it guarantees that the same mistakes will be made over and over. No one learns from anyone else in such situations because no one reports problems. We are all afraid that mistakes will be used against us.
Phil Alexander, a management consultant in Ann Arbor, Michigan, found a device to encourage experimentation and the sharing of ideas - and mistakes. He calls it the Golden Egg Award. Here is the concept in Phil's own words:
"I founded a group twelve years ago called the Ann Arbor President's Seminar. We would get together once a month for one day.
One of the participating company presidents wanted to hear from the members with egg on their faces so he could learn from their failures. So we created an award. Atop a trophy was a used pantyhose container that had been reclaimed for a higher purpose and painted gold. A newspaper reporter looked at it and remarked that we must be turning goose eggs into golden eggs. Inspired, we added an inscribed motto to the base:
"Sharing can turn a goose egg into a Golden Egg."
The afternoon portion of our seminar was for sharing mistakes, screw ups, and problems. The Award was then presented to the participant who was judged by the rest to have the classic of the day. It became the traveling award of the corporation president who shared the biggest goof.
One president took the trophy back to his office and set it on his desk. The chief financial officer came in and asked him about it. The president described the goof that had gotten him the award. To the president's utter surprise, the officer owned up to a problem the president knew about.
At the next meeting, this president shared with the group that he had known about the problem in his company but did not know how to get at it. He stated: when I opened up and showed the mistake I had made, the financial officer came back and shared his. What was happening was the beginning of a new form of openness in the business culture.
The participants in the seminar then created a new rule. The winner of the Golden Egg Award had to set it on his desk for one month. Anyone who came in and inquired about the trophy had to be told about the screw up. Slowly, the culture changed from one in which screw ups were problems to be buried into one in which they became learning experiences for everyone."
We can learn from our own mistakes. And in the process, we can gain important knowledge to progress and improve the well-being of everyone involved.
Your experiments will fail because of your ignorance, lack of preparation, stupidity, and many reasons not related to your efforts. This is nothing to be ashamed of. Give yourself credit for taking risks and for giving yourself the opportunity to make mistakes. The discomfort, pain and anger will dissipate rapidly as you move on to the next set of risks.
