Kicking Ass and (Not) Taking Names

by JS

I’ve been reading a number polemics that provide examples of problem science but stop short of naming the people responsible. Take for example this:

Too many research papers in Computer Science are nonsense: they convey no worthy message. Yet, they pass a Turing test of sort: at a glance, they are indistinguishable from interesting research papers. In fact, they are designed as nonsense from the beginning: the authors mimic the output of good research. The goal is to appear to be doing valued research. Some of these papers appear in top conferences, and even go on to be highly cited.

No actual examples of papers or researchers responsible for these papers are given in the rest of the post.

Or in another field (optics, apparently) one scientist’s epic efforts to get a comment published stops short of naming names:

I’ve withheld the names of the various individuals in this story because my purpose is not to make accusations (as much as I would like to; they’re certainly deserved), but instead to effect some social change.

My point is not to call bullshit on these examples of scientific misconduct. Both these scenarios sound plausible, indeed even likely. The problem is that the issues these academics identify are not separable from the people whose decision making led to them. I understand that there are a number of serious disincentives to calling out individuals for bad behavior. Future grant money, future positions for your graduate students, or ability to successfully navigate future reviews might all suffer from making public enemies by naming people whose papers are nonsense or whose approach to journal editing is ethically suspect.

However, I always thought that, in exchange for tenure, an academic had a certain obligation to be a sort of public curmudgeon. As a graduate student, who certainly has not mastered the art of scholarly work, I would benefit quite a bit from publicly available criticism of published papers. The problem is that so much of what scientific criticism does exist is buried under private anonymous reviewing. Criticism has become something impolitic in science, and as a consequence, the only examples that graduate students see of scholarship are the positive ones. Except that, according to some people, this set of positive examples is full of secret negative examples. We don’t get to know which ones.

In other words, teasing apart the difference between good and bad scholarship becomes a multiple instance learning problem instead of an easier supervised learning problem.