UPDATE: See the polymath project backgrounder for the latest information.
It’s been some time since I last wrote about the polymath project (see this, this for past coverage), and an even longer time since I wrote an extremely lengthy blog post about Michael Nielsen’s ideas about collaborative science.
The first polymath project, polymath1, was about the density Hales-Jewett theorem. This was declared a success, since the original problem was solved within about a month, though the writing up of the paper is still proceeding. The problem for the project was proposed by Timothy Gowers.
Terry Tao (WordPress blog) has now started a polymath blog discussing possible open problems for the polymath project, strategies for how to organize the problem-selection and problem-solving process, and other issues related to writing up final solutions and sharing of credit.
In this blog post, Jon Udell reflects on how the introduction of LaTeX typesetting into wordpress was a positive factor in getting talented mathematicians like Terence Tao and Timothy Gowers into the blogosphere, and leading to innovative user projects such as the polymath project. Udell notes that introducing existing typesetting solutions into new contexts such as Internet blogging software can have profound positive effects.
Christina Pikas offers her take. It seems to me that her post is a summary plus some cautious optimism and historical context.
This post on the backreaction blog considers the negative side of open science. Influenced by James Surowiecki’s book The Wisdom of Crowds, this talks about the pitfalls of too much openness leading to science developing only in certain directions that attract more immediate attention, compromising on independence during the decision-making process of what direction to pursue research in. The author says that it may in fact be the case that there is already too much collaboration in science leading to the underdevelopment of lines of research that have long-run promise but will not attract collaborators in the short term because others may not think that highly of them.
The backreaction blog post is a response, not specifically to the polymath project (in fact, it does not even mention polymath and is not specific to mathematics). Rather, it is a reaction to this post on the Open Science blog.
I think there are two somewhat differing and confused aspects here. The first is whether scientific findings should be open and widely available. The second is whether science should be done more collaboratively, with early sharing of findings and solicitation of inputs from others, rapid publishing, more “polymath”-style research. These are somewhat different issues.
Massive collaboration (of the kind that Nielsen talks about) is admittedly possible only when scientific findings are openly available. But the converse is not necessarily true. Greater open availability, accessibility, and ease of interaction with scientific content do not form a sufficient prerequisite for massive collaboration. Thus, it is possible to believe that openness and accessibility of findings is a good thing with few pitfalls while massive collaboration is a mixed blessing.
The key point is that open availability of research that is in progress can be achieved without making such research prominent to other researchers, except those who specifically express their interest in it. At the same time, well-established information should be made available in a manner that makes it very easy to find and use for all, as I’ve argued before. My paradoxical view is that the maximum openness can be achieved with the minimum downside if we concentrate on making sure that the material that is already well-known and clearly areed upon is made very widely available and “in-your-face”.
What about new and counterintuitive ideas?
For newer research and speculative findings, we need modes of availability that minimize the dangers of groupthink. One way of doing this would be to have incomplete information, betting, challenges, and competition.
For instance, in sharp contrast to polymath-style projects, one thing that could be done is that one person announces that he/she is going to conduct a certain experiment with a certain methodology and also explains how different outcomes for the experiment will result in different conclusions. This is put up openly and inconspicuously. The person then withdraws into his or her inner shell (or research lab) to do the experiment, and then puts up the results. In the meantime, the formulation of the experiment attracts some people (though not a lot) and they post their varying ideas on whether there could be problems with the methodology.
In this case, the people evaluating the experiment (who would be the select few with interest in that area) are not biased by the actual outcomes of the experiment when questioning the potential merits or demerits of the methodology. This could in principle reduce the possibility of herding.
This “betting” kind of approach could provide greater incentives to work on less apparently useful things. Suppose person A has an alternative explanation of some hypothesis, and designs an experiment to test it. But person A is uncertain about getting collaborators because nobody believes her hypothesis.
So she designs an experiment and puts up the methodology online (without actually carrying out the experiment). She challenges detractors to come up with possible flaws in the methodology. Detractors come up with such potential flaws. The methodology is accordingly fixed. Once the methodology is fixed to the comfort of the detractors, both she and the detractor are convinced that the experiment will yield opposing outcomes. This is an ideal time to place a bet. (This brings us to ideas of futures markets for ideas, which is an intriguing topic).
Here, openness contributes not so much by encouraging cooperative collaboration but by encouraging competitive collaboration.
Such a system could be adapted in many cases to mathematics, even though mathematics has fewer of these “experiments”.
Again, however, this runs the risk of getting people too much into wanting to find “counterintuitive” things rather than doing the good old solid work of progressing on the existing foundations by proving that epsilon more (too much cooperative collaboration probably leads to the opposite problem). Further, even with this, there may be some ideas where the problem isn’t that they’re counterintuitive but rather that they seem totally unimportant because people aren’t used to thinking in terms of them at all.
Here, a “futures market” type of idea may be useful. A person who is convinced that an apparently unimportant idea could be very important when developed further, can make such a claim. The exchange (or community, or market) then asks for something (the stake of the person’s reputation) in exchange for a large reward if that person turns out to be successful in developing the idea to a great level.