What Is Research?

February 13, 2009

Knowledge matters

It is fashionable in certain circles to argue that, particularly, for subjects such as mathematics that have a strong logical and deductive component, it is not how much you know that counts but how you think. According to this view, cramming huge amounts of knowledge is counter-productive. Instead, mastery is achieved by learning generic methods of reasoning to deal with a variety of situations.

There are a number of ways in which this view (though considered enlightened by some) is just plain wrong. At a very basic level, it is useful to counter the (even more common) tendency to believe that in reasoning problems, it is sufficient to “memorize” basic cases. However, at a more advanced level, it can come in the way of developing the knowledge and skills needed to achieve mastery.

My first encounters with this belief

During high school, starting mainly in class 11, I started working intensively on preparing for the mathematics Olympiads. Through websites and indirect contacts (some friends, some friends of my parents) I collected a reasonable starting list of books to use. However, there was no systematic preparation route for me to take, and I largely had to find my own way through.

The approach I followed here was practice — lots and lots of problems. But the purpose here wasn’t just practice — it was also to learn the common facts and ideas that could be applied to new problems. Thus, a large part of my time also went to reviewing and reflecting upon problems I had already solved, trying to find common patterns, and seeing whether the same ideas could be expressed in greater generality. Rather than being too worried about performing in an actual examination situation, I tried to build a strong base of knowledge, in terms of facts as well as heuristics.

In addition, I spent a lot of time reading the theoretical parts of number theory, combinatorics, and geometry. The idea here was to develop the fact base as well as vocabulary so that I could identify and “label” phenomena that I saw in specific Olympiad problems.

(For those curious about the end result, I got selected to the International Mathematical Olympiad team from India in 2003 and 2004, and won Silver Medals both years.)

At no stage during my preparation did I feel that I had become “smarter” in the sense of having better methods of general reasoning or approaching problems in the abstract. Rather, my improvements were very narrow and domain-specific. After thinking, reading, and practicing a lot of geometry, I became proportionately faster at solving geometry problems, but improved very little with combinatorics.

Knowledge versus general skill

Recently, I had a chance to re-read Geoff Colvin’s interesting book Talent is overrated. This book explains how the myth of “native talent” is largely just a myth, and the secret to success is something that Colvin calls “deliberate practice”. Among the things that experts do differently, Colvin identifies looking ahead (for instance, fast typists usually look ahead in the document to know what they’ll have to type a little later), identifying subtle and indirect cues (here Colvin gives examples of expert tennis players using the body movements of the person serving to estimate the speed and direction of the ball), and, among other things, having a large base of knowledge and long-term memory that can be used to identify a situation.

Colvin describes how mathematicians and computer scientists had initially hoped for general-purpose problem solvers, who knew little about the rules of a particular problem, but would find solutions using the general rules of logic and inference. These attempts failed greatly. For instance, Deep Blue, IBM’s chess-playing computer, was defeated by then world champion Garry Kasparov in a tournament, despite Deep Blue’s ability to evaluate a hundred million of moves every second. What Deep Blue lacked, according to Colvin, was the kind of domain-specific knowledge of what works and where to start looking, that Kasparov had acquired through years of stored knowledge and memory about games that he had played and analyzed.

A large base of knowledge is also useful because it provides long-term memory that can be tapped on to complement working memory in high-stress situations. For instance, a mathematician trying to prove a complicated mathematical theorem that involves huge expressions may be able to rely on other similar expressions that he/she has worked with before to “store” the complexity of this expression in a more simple form. Similarly, a chess player may be able to use past games as a way of storing a shorter mental description of the current game situation.

A similar idea is discussed in Gary Klein’s book Sources of Power, where he describes a Recognition-Primed Decision Model (RPD model) used by people in high-stress, high-stakes situation. Klein says that expert firefighters look at a situation, identify key characteristics, and immediately fit it into a template that tells them what is happening and how to act next. This template need not be precisely like a single specific past situation. Rather, it involves features from several past situations, mixed and matched according to the present situation. Klein also gives examples of NICU nurses, in charge of taking care of babies with serious illnesses. The more experienced and expert of these nurses draw on their vast store of knowledge to identify and put together several subtle cues to get a comprehensive picture.

Knowledge versus gestalt

In Group Genius: The Creative Power of Collaboration, Keith Sawyer talks about how people solve insight problems. Sawyer talks about gestalt psychologists, who believed that for “insight” problems — the kind that require a sudden leap of insight — people needed to get beyond the confines of pre-existing knowledge and think fresh, out of the box. The problem with this, Sawyer says, is that study after study showed that simply telling people to think out of the box, or to think differently, rarely yielded results. Rather, it was important to give people specific hints about how to think out of the box. Even those hints needed to be given in such a way that people would themselves make the leap of recognition, thus modifying their internal mental models.

I recently had the opportunity to read an article, Understanding and teaching the nature of mathematical thinking, by Alan Schofield, published in Proceedings of the UCSMP International Conference on Mathematics Education, 1985 (pages 362-379). Schofield talks about how a large knowledge base is very crucial to being effective at solving problems. He refers to research by Simon (Problem Solving and Education, 1980) that shows that domain experts have a vocabulary of approximately 50,000 “chunks” — small word combinations that denote domain-specific concepts. Schofield then goes on to talk about research by Brown and Burton (Diagnostic models for procedural bugs in basic mathematical science, Cognitive Science 2, 1978 ) that shows that people who make mistakes with arithmetic (addition and subtraction) don’t just make mistakes because they don’t understand the correct rules well enough — they make mistakes because they “know” something wrong. Their algorithms are buggy in a consistent way. This is similar to the fact that people are unable to solve insight problems, not because they’re refusing to think “outside the box”, but because they do not know the correct algorithms for doing so.

Schofield then goes on to describe the experiences of people such as himself in implementing George Polya’s problem-solving strategies. Polya enumerated several generic problem-solving strategies in his books How to solve it, Mathematical discovery, and Mathematics and plausible reasoning. Polya’s heuristics included: exploiting analogies, introducing and exploring auxiliary elements in a problem solution, arguing by contradiction, working forwards, decomposing and recombining, examining special cases, exploiting related problems, drawing figures, and working backward. But teaching these “strategies” in classrooms rarely resulted in an across-the-board improvement in students’ problem-solving abilities.

Schofield argues that the reason why these strategies failed was that they were “underspecified” — just knowing that one should “introduce and explore auxiliary elements”, for instance, is of little help unless one knows how to come up with auxiliary elements in a particular situation. In Euclidean geometry, this may be by extending lines far enough that they meet, dropping perpendiculars, or other methods. In problems involving topology, this may involve constructing open covers that have certain properties. Understanding the general strategy helps a bit in the sense of putting one on the lookout for auxiliary element, but it does not provide the skill necessary to locate the correct auxiliary element. Such skill can be acquired only through experience, through deliberate practice, through the creation of a large knowledge base.

In daily life

It is unfortunately true that much of coursework in school and college is based on a learn-test-forget model — students learn something, it is tested, and then they forget it. A lack of sufficient introspection and a lack of frequent reuse of ideas learned in the past leads students to forget what they learned quickly. Thus, the knowledge base gets eroded almost as fast as it gets built.

It is important not just to build a knowledge base but to have time to reflect upon what has been built, and to strengthen what was built earlier by referencing it and building upon it. Also, students and researchers who want to become sharper thinkers in the long term need to understand the importance of remembering what they learn, putting it in a more effective framework, and making it easier to recall at times when it is useful. I see a lot of people who like to solve problems but then make no effort to consolidate their gains by remembering the solution or storing the key ideas in long-term memory in a way that can be tapped on later. I believe that this is a waste of the effort that went into solving the problem.

(See also my post on intuition in research).



  1. The Deep Blue example is a bad one, because Deep Blue actually won the series against Kasparov, as everyone remembers it.

    I agree with the main content of the post. Some remarks in Peter Norvig’s Teach Yourself Programming in Ten Years are related. All the anecdotal evidence I know leaves the suspicion that Polya’s book (How To Solve It) has never actually helped anyone get better at solving problems. 🙂 It think it is highly acclaimed because many people recognize that it’s a good description of much of the problem-solving process: it might have been better titled “How It Is Solved”.

    Comment by Shreevatsa — February 13, 2009 @ 9:16 pm

  2. Hi,

    Kasparov won the first series. In the second series, Kasparov won one match, and drew three others, against a computer that could perform 200 million operations per second. The very fact that a human could win once is strong enough for my point!

    I guess I should have elaborated this (as Geoff Colvin had done in his book). Thanks for the link to Peter Norvig’s page; it references some of the studies on which the books that I’ve talked about are based, and makes up for the complete absence of hyperlinks from my original post :).


    Comment by vipulnaik — February 14, 2009 @ 6:22 pm

  3. “consolidate their gains by remembering the solution or storing the key ideas in long-term memory in a way that can be tapped on later”

    This is probably the key statement you have made.

    I would like to point out that there is a very large gap between “knowledge” (as in facts) and “long term memory” (as in general and specific structures/insights which can be called upon at any time to attack new questions). Ability to bridge this gap is what differentiates the upper crust from the mediocre. Lots of mediocre students have lots of knowledge but can’t sieve out the key structures from them to store in their minds forever. Or they fail at a more elementary level to simply be able to remember anything because of lack of memory. The ability to bridge this gap is probably one of the immediate manifestations of lack of intelligence. {know this because of harsh personal experiences}

    Further this ability to remember *both* specific results/expressions AND generic logic structures ultimately manifests itself in efficiency to perform under time constrained situations (like most Indian examinations). Again intelligence is directly the factor that helps inculcate these 2 things.

    Probably there is some scientific reasoning behind why almost always intelligence and memory are correlated in most people.

    Things that you have said like “reflecting upon the solutions” etc are again processes which most mediocre students are unable to do because of the lack of intelligence. This deficiency cause tire once the question is solved and there is very little energy left for the person to reflect on what has been done and absorb the key ideas.

    Comment by Anirbit — February 15, 2009 @ 5:53 pm

  4. It’s true about Kasparov winning the first series and all that; I’m not saying that the example doesn’t fit your point: just that it’s a bad choice rhetorically, because the only thing most people remember is “In Kasparov v/s the computer, the computer won”. 🙂

    Comment by Shreevatsa — February 20, 2009 @ 1:17 am

  5. […] and assimilation, Thinking and research — vipulnaik @ 7:25 pm In previous posts titled knowledge matters and intuition in research, I argued that building good intuition and skill for research requires a […]

    Pingback by Doing it oneself versus spoonfeeding « What Is Research? — February 23, 2009 @ 7:26 pm

  6. hi,

    i am looking to go to math grad school next year; my background is very minimal(btech in engg in india). what would you consider the barest minimum that an entering math grad student must know ( of course i wouldn’t be able to go to a place like chicago or princeton but even for the smaller schools) could you also suggest books for the same and how to go about aquiring the kind of knowledge that one needs esp considering that i wouldnt have access to other people to discuss to(for the most part)

    p.s: i have taken one intro course each in algebra (linear and absract) and (real + complex ) analysis but that is pretty much it.

    thanks in advance,

    Comment by anon — March 3, 2009 @ 12:31 am

  7. Hi,

    You should send me an email with your questions and I’ll try to answer to the best of my knowledge. Email me at vipul@math.uchicago.edu



    Comment by vipulnaik — March 6, 2009 @ 12:17 am

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: