What Is Research?

November 15, 2008

Intuition in research

Filed under: Thinking and research — vipulnaik @ 10:06 pm

I recently read a couple of books by Gary Klein: Sources of power and The Power of Intuition. Klein is a decision researcher who started work in the 1980s studying high-stakes high-pressure decision-making. His research team began by studying how firefighters make high-stakes decision in the face of severe time constraints, incomplete information, high pressure, and unclear goals. The research team found that the firefighters rarely compared multiple options. Rather, when faced with a particular situation, the experienced firefighters typically found an immediate first response, simulated it mentally, and executed it if the simulation seemed to work fine.

While there were certain situations where the firefighters rejected one course of action and selected another, Klein found that the courses of action were usually considered sequentially: the first course of action was simulated mentally, and if the firefighter sensed it to be good, he or she executed it. If the course of action didn’t seem good, another course of action was simulated. The hallmark of experienced firefighters was their ability to pick out a good first option and execute it.

According to Klein’s book, his findings contradicted the formal decision-making strategies considered good by decision researchers at the time. Many decision researchers warned people against using their intuitions, which were prone to being misleading, and to instead consider multiple options and compare them across multiple dimensions. In recent years, there has been greater acknowledgement of the infeasibility of comparing multiple options as well as the advantages of strengthening intuition in order to pick good first options.

Through studies of firefighters, marines, NICU nurses, and many other decision makers, Klein came up with a model he called the Recognition-Primed Decision (RPD) model. The core idea is that the repository of experience that a worker builds creates certain templates, and when the worker is thrust into a new situation, the new situation is matched against these templates. If a good match is obtained, the course of action suitable for that template is tried. The rich repository of experience helps with the initial recognition of the appropriate template, with the mental simulation that follows, and with collecting feedback once a course of action has been sought.

For instance, a firefighter, over the years, becomes sensitive to different cues such as the smell, floor temperature, room temperature, way in which the fire is spreading, and numerous other small indicators. By gauging these cues, the firefighter subconsciously develops a “story” around how the fire developed and what the priority should be (rescue people, douse the flames, call for more help). Similarly, nurses in intensive care units for children (The NICU nurses) develop a repository of experiences on subtle cues as well as combinations of cues that ill children provide. An experienced nurse can thus size up a situation based on the many cues he or she (usually, she) sees, and develop a story that immediately suggests a next course of action.

The emphasis is thus not on analysis but on building a story, and judging the story by how well it fits the facts, where the extent of the fit is determined by feedback from past experience.

Do similar principles apply to research?

In terms of speed, research is the opposite of firefighting. For a firefighter, the situation is live and demands immediate action, with high stakes and usually very immediate feedback about success (either the flames get doused or they don’t). Research, on the other hand, is a slow process, with very little riding on decisions made on the spur of the moment and very rare opportunities for instantaneous feedback. If the research problem I pick is too hard for me, I don’t get to know the consequences and feel the pain for quite a bit of time. I might suffer the delusion that I am making steady progress on the problem and figure out that it is too hard after several years of trying.

Given the obvious differences, it is natural to be suspicious of an assertion models that help with high-stakes decision-making are prima facie suitable for researchers. However, I make the case here that intuition is important in research, albeit in a different way.

In general, intuition is important in situations where either the information explicitly and clearly available is inadequate, or the effort needed to process all this information is infeasibly high. In a firefighting situation, the information available is inadequate at the time the decision needs to be made, even though the story usually does become clearer in a short while. When working on a research problem, we again have a gap: the information available (as to whether or not I should work on the problem) is inadequate at the time I start work on the problem, though it is likely to become more clear once I have worked on the problem. The difference is in the time scale. But in both cases, there is inadequate information at the time of decision-making.

Strengthening one’s intuitions

Klein’s book on the power of intuition offers many concrete suggestions on strengthening intuitions. Klein begins with the (obvious) observation that practice improves intuition. More importantly, he identifies two aspects: frequency of exposure to situations, and feedback that helps in correct model-building. Frequency alone is not enough.

Of course, there are obvious problems with providing practice for emergency situations — the kind that shouldn’t occur anyway. How do novice doctors get practice in performing critical surgical procedures and making critical medical diagnoses, without risking the lives of patients more than necessary? Atul Gawande, a Massachussetts surgeon, discusses these issues in his bestselling book Complications, where he points out that doctors have a learning curve, and this necessitates that some patients receive substandard care because they get treated by residents and new doctors rather than more experienced ones.

Similarly, how do firefighters get experience fighting fires? Again, this experience is provided through apprenticeship and observing other firefighters — the less experienced firefighter accompanies the more experienced firefighter and sees the more experienced firefighter make the critical decisions while offering support.

Apprenticeship is one approach, but it may be expensive, and it is best complemented with other approaches that are less expensive. Other approaches (some of which are mentioned in Klein’s book) include simulation and training exercises, where some of the features of the real experience are captured through simulation. An example is flight simulation. In addition, there is the crucial aspect of experienced people documenting their experiences, and sharing these experiences with others, so that the many little nuggets of wisdom get passed on.

Now let’s move from high-stakes, instantaneous decision-making to the world of research. The same ideas seem to apply: newer researchers generally have less experience, and they learn more through apprenticeship and through interaction with more experienced people. The timescales are, of course, very different. A Ph.D., which is like an apprenticeship under a guide (the thesis advisor) may take anywhere between three and eight years. Even after completing a Ph.D., researchers generally tend to work under the guidance and tutelage of more experienced people before fleshing it out totally on their own.

So the same questions seem to apply: what are the skills that experienced researchers have, that their less experienced counterparts may lack? And how can new researchers pick up these skills faster?

The importance of metacognition

Klein claims that people with experience and expertise in a topic not only have a richer repository of experience to draw upon — they are also capable of analysis of their own thoughts on the subject. In other words, they not only know more of the territory they are exploring, they also know more about their own tendencies in exploring the territory. For instance, an experienced athlete not only knows how to run long distances, but is also aware of how it will impact his or her mood and energy levels, and can thus plan ahead accordingly. Awareness of their own proclivities thus helps experts plan around as well as exploit their human strengths and weaknesses.

Thus, a person who has just learned a subject, say group theory, and doesn’t have much experience with it, may not be able to distinguish between two problems in group theory in terms of their level of difficulty without attempting the two problems. A more experienced person may be able to look at both problems and get a feel as to which one is likely to be harder, even without attempting either problem. This comes from a stronger intuitive grasp of the territory, including its highs and lows.

Of course, there are times when expertise of this sort can be misleading, because it may lead the more experienced person to be less adventurous in exploring some things based on false preconceptions. This needs to be watched out for.

The upshot is that there are certain skills: a clearer understanding of the specific territory of the subject being researched, as well as a broader intuitive feel of the subject that helps one decide what direction to go in. The question is: can these skills be developed faster? Are there any obvious ways of doing this?

Some concrete suggestions

The goal is that new researchers should get a sense of how to think about specific problems, be better at metacognition (understanding their own thought processes) and get a broader intuitive idea about the direction that certain approaches will lead to. This should not be done at the cost of diversity in thinking — new researchers should not be handed down the right way of thinking about something. I discuss here some interesting suggestions to help improve the intuitions of new researchers.

  • The two-problem faceoff: This is a game of sorts, where two problems are presented to the novice researcher. Neither of these problems is trivial; one of them, however, is easy (i.e., it can be solved by the researcher) and the other is hard (i.e., it either requires a lot of ingenuity or some new machinery). The researcher has to decide which of the problems to try, and try that, and succeed.

    This faceoff game has some interesting aspects. First, researchers are forced to develop their intuitions not just on how to solve a problem, but also on how to pick a problem to solve among a collection of two. Thus, researchers are forced to consider metacognitive questions: which path should I tread?, and to look ahead and predict what will happen. Second, it may actually turn out that the purportedly harder problem is the one the researcher picks and actually does solve (perhaps coming up with an easier solution). Perhaps this indicates that the new researcher is particularly good with problems of that kind.

  • What came first?: Here, a researcher is presented with two proofs of the same theorem, both of which arose at different historical points, and is asked to do a comparison: which proof came first? Which one is more useful? Which one would be the kind of proof you’d come up with?
  • Spotting relations, thinking creatively: New researchers should constantly confront reflective questions regarding different aspects of the work they are doing or learning about. For instance: what does the statement of this result tell me? What does the structure of proof tell me? Are there corollaries of the statement? Are there other related statements? Are there other statements whose proof follows the same structure? Can the proof idea be transferred to a totally new subject? Can I come up with similar-sounding statements that are false?

I have been exploring some of these possibilities for spotting relations and encouraging reflective thinking that builds intuitions for some time. I’ve implemented some of these ideas in the structure I’m using for the Group properties wiki. For instance, see this page about a property of normal subgroups, or this page about a cute fact regarding unions of two subgroups of a group.

Advertisements

4 Comments »

  1. […] I mentioned in a previous post, the main advantage of experience is the presence of a large repository of knowledge that allows […]

    Pingback by Small fry or big fish? « What Is Research? — November 15, 2008 @ 11:17 pm

  2. […] (See also my post on intuition in research). […]

    Pingback by Knowledge matters « What Is Research? — February 13, 2009 @ 3:59 pm

  3. […] Thinking and research — vipulnaik @ 7:25 pm In previous posts titled knowledge matters and intuition in research, I argued that building good intuition and skill for research requires a strong knowledge and […]

    Pingback by Doing it oneself versus spoonfeeding « What Is Research? — February 23, 2009 @ 7:26 pm


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: