The recent publishing debate -- The economic power of publishers

I have been trying to find the time to continue my posts on the publishing debate, discussing the other posts from the original timeline. Suffice it to say, I've not yet given up getting around to it... But there were two posts in the last couple of days that made me want to write something.

If you follow me on twitter, you might have seen my remarks on the IMU's strange attempt at entering the blogosphere. A couple of weeks ago, they started a Blog on Mathematical Journals. It's not a blog about writing, but about commenting. The first post (which, horror horribilis, was named BLOG on Mathematical Journals) was aimed at getting comments on the IMU's development of a journal ranking to battle the dominance of impact factors. I was under the impression it's not really working out -- 28 comments in three weeks from the mathematical community of the entire world(!) just seems a little unimpressive. Then again, the only way I heard from it was via the EMS news blog (of course through mathblogging.org), go figure.

But this week, the IMU surprised me with a follow up post, which, as promised, was possible for the discussion of specific related topics. It is titled "What might be done about high prices of journals?".

I starting writing a comment over there, but for several reasons decided not to post it. But this is a topic that is related to what I see to be at the center of the publishing debate so I parked it in the ongoing draft here. So here it is:

It seems to me that the pricing issue cannot be solved as long hiring decisions are almost exclusively based on the publication track record in traditional journals (and even worse on the metrics of those journals).

In other words, when it comes down to it, we only value researchers by the metrics of the traditional journals they've published in. This gives the publishers incredible power over the development of our community and there's no conceivable reason why for-profit publishers wouldn't use this power to maximize their profits.

I think the mathematical community needs to spend time and money on finding ways to evaluate other activities of researchers.

For example, wikipedia has de facto a peer review system, especially the "Good Articles". It's not as easy to evaluate a wikipedia author, but an active author offers a lot of activity to evaluate them by and there is precedent for considering this a research activity.

Similarly for MathOverflow and other research-level Q&A sites. The so-called reputation points are a poor metric per se, but they nevertheless indicate an activity that is worth evaluating, i.e., a high "reputation" indicates at the very least a high activity, an amount of data that could be used to evaluate a researcher.

The best thing about these new platforms of academic activity is: we can get started right away! On the one hand, mathematicians can (and should) invest time in making their efforts outside of traditional journal publishing more apparent, in particular on their professional homepage. On the other hand, the professional societies should start taking the web serious, support and invest in new ways of doing research online and, above all, investigate the development of standards for evaluating such activities.

So why am I writing this now? Well, Noam Nisan mentioned a powerful rant by Danah Boyd with many interesting comments (if only because people there aren't shocked that (social) media activity might be worthwhile for researchers).

So I just left a comment at Noam Nisan's g+ post (repeating some of the above).

I very much agree with your comment there "... but the current system will stay unless we develop an alternative to this grading, an alternative to which there is an incremental transition path"

However, many alternatives already exist, I think -- from MathOverflow to Wikipedia to blogging to video lectures.

What seems to be missing is

a) ourselves considering our own non-standard research activities as research activities (e.g. http://blog.wikimedia.org/2011/04/06/tenure-awarded-based-in-part-on-wikipedia-contributions/ )

b) tools to analyze the data generated by non-standard activities (e.g. MO reputation points is a poor metric, but the actual questions and answers of a user are an incredibly rich source)

I also think that we should try not to replace the monoculture of journal publications with a different monoculture (say MO or Gowers's arxivMO-idea) -- it's too easy to game one system.

Instead, valuing many activities will allow people to find more activities they excel in without the pressure to excel in all of them.

Any thoughts on this?

Addendum Dec22: John Baez encouraged me to post my comments on the IMU's blog after all (and I saw this morning that he also encouraged the n-Category Cafe crowd). I extended it a bit due to the comments already visible. But it's not yet through the moderation process... So, here it is:

It seems to me that the pricing issue cannot be solved as long hiring decisions are almost exclusively based on the publication track record in traditional journals (and, even worse, on the metrics of those journals).

In other words, when it comes down to it, we only value researchers by the metrics of the traditional journals they’ve published in. This gives the publishers incredible power over the development of our community and there’s no conceivable reason why for-profit publishers wouldn’t use this power to maximize their profits.

The pressure to publish is immense and getting published has turned into a game rather than an effort of communication -- with many adverse effects to a functioning community.

None of the comments so far have addressed the issue that the monoculture of "publishing papers" is limiting the way a scientific community can develop. Journals used to be a necessary evil to enable a minimal degree of communication within a community that is spread around the globe. But now the community is fully connected, in real time, and there is no difficulty to stay in touch with any researcher as long as they use the internet to some degree. In turn, we can now communicate every detail of our academic work effortlessly; not just individual papers, but refereeing, student interaction, Q&A's, video lectures, expository writing, research exchanges, live-broadcasting talks and seminars etc

I think the key problem is that we need to find ways reduce the pressure to publish the traditional way. The only way this is possible is if we find a way to publish less.

Therefore we need to spend time (and money) on finding ways to evaluate other activities of researchers.

For example, wikipedia has de facto a peer review system, especially the “Good Articles”. It’s not as easy to evaluate a wikipedia author, but an active author offers a lot of activity to evaluate them by and there is precedent for considering this a research activity.

Similarly for MathOverflow and other research-level Q&A sites: The so-called reputation points are a poor metric per se, but they nevertheless indicate an activity that is worth evaluating, i.e., a high “reputation” indicates at the very least a high activity, an amount of data that could be used to evaluate a researcher.

Fortunately, these new platforms of academic activity allow us to get started right away.

the one hand, mathematicians can (and should) invest time in making their efforts outside of traditional journal publishing more visible, in particular on their professional homepage and their CVs (in the research section, that is!)

On the other hand, the professional societies should start taking the activity of mathematicians on the web seriously. They could support and invest in new ways of doing research online and, above all, investigate the development of standards for evaluating such activities.

===
I thank John Baez for encouraging me to make this comment.

Comments.

• Igor Carron, 2011/12/12

  “…. it’s too easy to game one system…”
  the question is really what kind of point system can be put in place in order to avoid most gaming.

  • sam, 2011/12/12
    I agree with Peter: it is probably a theorem that any point system can be gamed. Even in sports, where betting is a big motivation to be fair, there are still a lot of problems. Think about the BCS.
    • François, 2011/12/12
      That theorem would be Arrow’s Theorem. The theorem boils down to the fact that ultrafilters on finite sets are always principal, so the only system that can’t be gamed would be the one where the Monarch of Mathematics decides everything.
• Peter, 2011/12/12
  Igor, for me the “can’t be gamed” is a more of a positive side-effect. The main thing is: we really need to value more than journal papers.
  I understand that historically, journals allowed for a minimal amount of communication between researchers in a time when actual communication was not feasible. But this has changed now — with very little effort can we now contact any researcher anywhere anytime, in real time, live, with video, with online whiteboards, in public via broadcasts or one-on-one.
  This finally allows us to communicate our entire research activites — results as well as reviews, refereeing, Q&A’s, student interactions, mailing list activities, wikipedia activity, video lectures, code examples, expository notes, research exchanges — everything.
  Your amazing efforts at Nuit Blanche are the perfect example that it is possible to present an entire research area, always up to date, always dynamic, always changing. It’s an amazing work — yet it is not valued less than a single paper in a glamour mag.
  I think we need to move beyond from the minimalistic monoculture that is the journal system. Papers can be great and their will always be an important place for them.
  But we need to develop tools to actually evaluate the content people create outside of journals. I remember the amazing TED talk you linked to on Nuit Blanche — what they do for TV and twitter could mean a lot for research communication on social networks.
  • Igor Carron, 2011/12/12
    Peter,
    I absolutely agree that as soon as we have more discussions taking place electronically, we will be able to develop more tools that can provide a more accurate image of the contribution to some areas of research.
    I am really convinced that focusing on the peer review process is the way to unleash these conversations, hence the need to figure out how a Q&A could be construed as least gamable as possible, even though I agree, all processes can be gamed eventually.
    • Peter, 2011/12/12
      Hm… I think we do disagree on something. I’m not sure.
      Something to clarify: it’s clear that research succeeds only if peers look carefully at each others work. So all ideas that I spew out are always under the assumption that feedback from peers is involved (peer review has the unfortunate connotation of “peer-reviewed journals” which limits the scope of peer review).
      That being said: I think we need to have many, many more ways that the community will officially consider research activities.
      A Q&A-site of any kind has its dangers — The Accidental Mathematician wrote about it. (As you know I don’t like the idea of a centralized site, I favor independence + aggregation but I’ll actively support any project.)
      As an example, here’s my personal problem with MathOverflow: there are too many(!) top researchers in my field active on MO — I never get to a question before they do! It makes it near impossible to get any reputation points myself and, much worse, to experience the incredibly positive exchanges happening on MO.
      Another example: we should start valuing peer review! The referee’s side that is. And teaching. And exposition. And history. And work with students. And Q&A sites.
      It used to be that we couldn’t communicate and aggregate information on these activities very well. So we only had journals.
      But now we’re still only valuing papers even though we could value much more — and I think it is bad for the development of our community that only people that can play the journal-game, i.e., can produce enough good-looking papers in a short amount of time, are able to get jobs.
      Is it impossible that we are loosing excellent researchers because they cannot (or refuse to) play the journal game at the pace that is expected of them but who excel at other activities that help maintain a functioning research community?
      Then again, the way the funding development is going in the UK and Canada, there might just not be any pure mathematics left in a generation or two — that will allow for a fresh start eventually…
      —
      Ok, I got a little carried away here. I agree with you: peer-to-peer interaction should be the focus. Peer review of papers and preprints would be a substantial first step.
      So we should write more about our own papers on blog-like websites — and we should seek out the opportunity to comment on other people’s sites. I don’t think we need a new platform for that.
• Santo D'Agostino, 2011/12/12
  I agree with you wholeheartedly, Peter. For the health of the community, we need to value other activities beyond the narrow few that are now recognized. And teaching, and activities that support good teaching, ought to be included in a way that is beyond just lip service. (At the university at which I last taught, there are a number of superb teachers who will never be promoted to full professor because their research record is not up to the level judged appropriate, forget about what else they have done for students and the rest of the university community.)
  Part of the problem is that everyone is busy nowadays, and we judge by what can be measured easily. We count publications, weighting them by which journals they appear in, rather than assess the overall quality of research, because it’s easier to do, and we can thereby avoid arguments. We foist the same injustices on students, for the same reasons, grading them because it’s easy to do, even though a single number is a ridiculous assessment.
  Another argument that a number of peole have made is that citizens have already paid for research via their taxes, and so it’s not right that they are made to pay again for aceess to the published results.
  • Peter, 2011/12/12
    Thank you for comment, Santo. I absolutely agree that we’re not valuing teaching enough. This is very damaging in the long term for our research community — how can we expect talented students to become interested in mathematics when we wait until graduate level to show them actual mathematics? It might be that mathematics used to have a singular intellectual attraction among all scientific fields — I think this is not the case anymore, other fields offer a similar level of complexity and beauty. Therefore, we need to actively seek out the most talented students. Initiatives regarding women and minorities in mathematics have shown that we’re missing out on very talented people by ignoring those that do not show the kind of interest in mathematics we’re used to seeing.
    Of course, an even bigger problem with not appreciating good teachers and their methods is the effect it has on students that are supposed to have mastered some mathematical skill to pursue other careers — here, the lack of appreciation of good teachers is effecting our societies as a whole and, most dramatically, the students who suffer through our curricula.
    You are right to point out that everybody is busy and that any change will take a huge effort, both in work and money. I do think that this is connected to the publish-or-perish pressure — if nothing else counts, I understand that people ignore everything else. But I think we can start nonetheless.
    On the one hand, I think individual researchers can start by making their work visible — and help each other do so! Booles’ Rings is precisely based on this idea (so is mathblogging.org). On the other hand, the professional societies seem a natural starting point for investigating broader standards and best practice; they might, however, have serious conflicts of interests because of their current revenue models.
    Finally, I also agree that publicly funded research should be open access — I have yet to meet a mathematician who does not feel that way.
• Santo D'Agostino, 2011/12/12
  Bravo for your efforts on several fronts, Peter. And best wishes for continued success.
  • Peter, 2011/12/12
    Thank you for your kind words, Santo. It’s all a team effort, with Sam here at Booles’ Rings and with Felix and Fred at mathblogging.org. I’ll be sure to pass your compliments along :)
• François G. Dorais, 2012/01/16
  This New York Times article is of interest.