Tuesday, August 11, 2015

The impact-factor introduction

Last week, I went to the Penn MSTP retreat (for MD/PhD and VMD/PhD students), which was really cool. It truly is The Best MSTP Program in the Galaxy™, with tons of very talented students, including, I'm proud to say, four in our lab! There was lots of interesting and inspiring science in talks and posters throughout the day, and I also got to meet with a couple of cool incoming students, which is always a pleasure.

One thing I noticed several times, however, was the pernicious habit of mentioning of what journals folks in the program were publishing in or somehow associated with, emphasizing, of course, the fancy ones like Nature, etc. I noticed this in particular in the introduction of the keynote speaker, Chris Vakoc (Penn alum from Gerd Blobel's lab), because the introduction only mentioned where his work was published and didn't say anything about what science he actually did! I feel it bears mentioning that Chris gave a magnificent talk about his work on chromatin and cancer, including finding an inhibitor that actually seems to have cured a patient of leukemia. That's real impact.

I've seen these "impact-factor introductions" outside of the MSTP retreat a few times as well, and it really rubs me the wrong way. Frankly, being praised for the journals you've published in is just about the worst praise one could hope for. In a way, it's like saying "I don't even care enough to learn about what you do, but it seems like some other people think it's good". Remember, "where" we publish is just something we invent to separate out the mostly uninteresting science from the perhaps-marginally-less-likely-to-be-uninteresting-but-still-mostly-uninteresting science. If you actually are lucky enough to do something really important, it won't really matter where it's published.

What was even more worrisome was that the introduction for the speaker came from a (very well-intentioned) trainee. I absolutely do not want to single out this trainee, and I am certain the trainee knows about Chris's work and holds it in high regard. Rather, I think the whole thing highlights a culture we have fostered in which trainees have come to value perceived "impact" more than science itself. As another example, I remember bumping into a (non-MSTP) student recently and mentioning that we had recently published a paper, and rather than first asking what it was about, they only asked about where it was published! I think that's frightening, and shows that our trainees are picking up the worst form of scientific careerism from us. Not that I'm some sort of saint, either. I found it surprising to read BioRxiv recently and feel a bit disoriented without a journal name on the paper to help me know whether a paper was worth reading. Hmm. I'm clearly still in recovery.

Now, I'm not an idealist, nor particularly brave. I still want to publish papers in glossy journals for all the same reasons everyone else does, mostly because it will help ensure someone actually reads our work, and because (whether I like it or not) it's important for trainees and also for keeping the lab running. I also personally think that this journal hierarchy system has arisen for reasons that are not easy to fix, some of which are obvious and some less so. More ideas on that hopefully soon. But in the meantime, can we all at least agree not to introduce speakers by where they publish?

Incidentally, the best introduction I've ever gotten was when I gave a talk relatively recently and the introducer said something like "... and so I'm excited to hear Dr. Raj talk about his offbeat brand of science." Now that's an introduction I can live up to!

7 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. I find such introductions very disrespectful to the presenters. I am happy that you are raising the issue and focusing attention to this regrettable practice.

    ReplyDelete
  3. Two pieces of the puzzle included in the conundrum are
    * the increasingly common phenomenon of fake peer reviews justifying the publication of crud, and
    * the 'publish or perish' paradigm on the track to hiring and tenure

    Without proposing a replacement for the old paradigm, it's hard to see where hope for a fundamental change to the core motivation for being enthused about "publication" will come from.

    ReplyDelete
  4. I do think there are specific contexts in which this sort of introduction might be justified. For instance, if you're introducing the speaker to a very broad audience, referring to their many Nature and Science papers serves the same purpose as referring to their Nobel Prize or their MacArthur "genius" fellowship or whatever. It's says that the speaker is an outstanding scientist who's done amazing work, and says it in a way everybody in the audience can appreciate. Not that that's the only possible way to introduce someone to a broad audience--but it's one way, and in that context I think it's reasonable.

    ReplyDelete
    Replies
    1. I think I disagree. I think a very broad audience may or may not appreciate a Nature or Science paper as much as we think–I know in many other fields, it's not considered remotely as important. If someone has really done amazing work and was selected for speaking to the broad audience, presumably that's because they did something really cool for which the import is relatively easy to convey, like "they cured someone of cancer" or something.

      Delete
    2. We might only disagree a bit. I'm an ecologist, and yeah, if whoever's introducing the departmental seminar tells me that Chris Vakoc cured someone of leukemia, I'm impressed and eager to hear more from Chris Vakoc. But if the introduction says is something like "Chris Vakoc has done fundamentally important work on the relation of chromatin to...", sorry, that doesn't give me any handle to grab onto. And it works the other way too. If I were to give a talk at a biology dept., I can imagine that for many people, an introduction that mentions the fact that I've got a Nature paper (or maybe that I've got one of the most widely-read professional science blogs in the world) is going to resonate a lot more than "Jeremy Fox has done important work on spatial synchrony of..."

      Now, will mentions of the speaker's Nature or Science paper resonate equally well with everyone in the audience? No, of course not. But there aren't many things that would. Heck, there are people who think the Nobel Prize is overrated and who don't give a rat's a** if the speaker has one. I mean, sure, if nobody in the audience is going to care about a speaker's Nature or Science papers, don't mention them. But I dunno, my experience is that if the audience is a typical biology dept. at a research university, most of the people in the audience are going to appreciate Nature or Science papers.

      Perhaps what this comes down to is the question of what an introduction is for. One answer is that it serves no purpose, or no important purpose--it's a mere formality. The host might as well just say "Please welcome this week's seminar speaker, Dr. So-and-so." I don't like that answer, and I'm guessing you don't either, but I suppose some people might. Just off the top of my head, I'd say that a good introduction should do a couple of things. It should say nice things about the speaker, as one way of thanking the speaker for their time and effort (and depending on the sort of seminar it is, honoring the speaker for their achievements). And it should get the audience in the mood and frame of mind to listen attentively. (Not the audience wouldn't otherwise pay attention, of course.) But that really is just off the top of my head. So what do you think--what are seminar introductions for, what makes for a good one, and how do the answers to those first two questions depend on the composition of the audience?

      Delete
    3. Very good points. I have been thinking about what would make a good speaker introduction, and what speaker introductions I have given in the past (and how I might have improved them). I think the introduction depends on the forum and the audience (and the speaker, hehe!). To me, the ideal introduction mostly just gets out of the way, but also gives people maybe some sense of a speaker’s history (potentially including major awards), and perhaps a very brief description of any big discoveries they may have made along the way. Also, I would for sure mention any interesting “extracurriculars”–for instance, running such a cool blog as yours would definitely be worth a mention!

      I appreciate the argument that mentioning papers in Nature or Science would catch the attention of most people attending, say, a biology colloquium. The question is whether it’s actually informative beyond just saying that the speaker has done important work in whatever topic. I mean, if someone did “Nature quality” work on a topic, then presumably it’s import is broadly applicable. I think it’s fundamentally more informative to say “Chris Vakoc has done exciting work linking the structural aspects of DNA to the biology of cancer” than just to say “Chris had a Nature paper”. If a succinct summary of why his work is important is not compelling, then the question is really why are you attending this seminar anyway? And assuming that you probably aren’t going to leave at that point, well, what difference would it really make anyway?

      I think what I find most worrisome about impact-factor introductions is that it insinuates to trainees that publishing in fancy journals is what we value as scientists. Publishing well is not per se bad, but is only an imperfect proxy for doing good work. If we want to mention papers, I think it would be fine to say that the speaker recently published an important paper in XYZ. I would perhaps just leave it at that, then let their talk do the talking.

      Delete