Burning down the house II- not-so-bad ideas

This Sticky Wicket article first featured in Journal of Cell Science. Read other articles and cartoons of Mole & Friends here.

“No visible means of support, and you have not seen nothing yet. Everything stuck together. Dum dee dum dum dum dee dum, Baby what do you expect?” Hey, we’re back. And no, I’m not still listening to that song. I’m listening to it again. I do that.

If you’re just joining us, we’ve been talking about bad ideas. Things that keep coming up – observations, conclusions, hypotheses – that we don’t believe but that just don’t go away. Even when they’ve been disproven or replaced by better ideas, here they are again – being cited in reviews, making their way into summary figures, being pointed to as an explanation for something in a paper, being used to make arguments. The field moves past them but they keep coming up for more exposure. And with ‘no visible means of support’ you wondered why I was listening to that song.

So how do we get rid of them? Well, before I tell you my ideas on this, let’s have a look at what is being suggested in the blogosphere (online). No, I don’t pay attention to the blogosphere. But I do listen to the ‘beer-o-sphere’, which I experience at meetings all the time –after the sessions, over a beer, when everyone weighs in on issues like this. The beer-o-sphere is way better than the blogosphere.

So here is the solution from the beer-o-sphere on how to get rid of bad ideas:

Let us publish negative results! Okay, we hear this all the time, not only in the beer-o-sphere. If there were a place to publish negative data, data that say that something doesn’t work, the weight of this will bury the bad idea in a deep grave made of electrons (no, nobody thinks negative data have to be published on paper; it isn’t worth killing trees to kill a bad idea). That should do it. Right?

Well, I agree and I don’t agree. A definitive experiment that proves that an accepted idea cannot be correct, certainly, should be published. But experiments of this type are extremely hard to design, and I suspect that when they are done, they do get published. The sort of negative data that are being discussed in the beer-o-sphere is generally of the sort, “We tried to repeat the experiment and it didn’t work.” People. I’ve said this before and I’ll say it again: Personally, I have trouble reproducing anything – I am not sure I could reproduce an experiment that shows that objects fall down and not up. I do notwant to read about your failures. There are many, many reasons for things not to work and, generally, very few reasons why they do. Yes, having a way for everyone to complain that something doesn’t work, or isn’t reproducible, or gives a different result, is all very cathartic, especially when others have the same problem, but it doesn’t actually prove anything. I don’t mind the idea of publishing negative data but it won’t burn down the house of bad ideas.

Let me give you a counterexample, how we can be wrong about wrong ideas, even when we have compelling negative data. Professor Echidna publishes that after a long search, he has actually found a mammal that is venomous. Quickly the community tries to reproduce this, but no mammals are venomous. So he shows them where he found it. More reports state that all mammals in that area are non-venomous, although many have pouches. Echidna identifies the animal as a duck-billed platypus. Professor Platypus herself publishes that she is definitely not venomous. The community, and the beer-o-sphere, are satisfied that this is a bad idea that needs to go away. But Echidna persists, “I meantmale platypuses.”

Which brings me to a question. Who says a bad idea is wrong? This is one thing to consider. Just because the consensus in the field says something is wrong, doesn’t actually make it wrong. Work can be difficult to reproduce, contain artifacts and look shaky, and still be correct. Male platypuses are, in fact, venomous. I bet you knew that?

But Mole, you say (I’m listening). If something is wrong, and most people in a field know it’s wrong, isn’t it dangerous to give it credibility? At the very least, people will waste their time on it. At the worst, they’ll use the idea to influence policies that could hurt people. Just because something could be correct, despite the experience of other experts in the field, must we give credit to everything that is published? How can we ever get anywhere?

Okay so let’s get something straight. Just because a bad idea seems to persist, it doesn’t mean youhave to use it. In fact, not only shouldn’t you use it but you shouldn’t use anything that isn’t useful for understanding the problem you have set yourself to do (or been set to do, not everyone gets to pick). Be critical, by all means, not only of ideas you question, but all ideas. Examine the evidence. And this goes both ways – if you are told something is wrong, find out why some folks believe it. There are lots of things in your own field you ‘don’t believe.’ But can you make a clear argument for why not? I’m often surprised at how many of us cannot.

So let’s get back to what we can do about this. First of all, be an intellectual. If you doubt an idea, have clear reasons for why. Make an argument. The thing is, while we don’t care (much) to see lots of failed experiments, we are very open to hearing your opinion as to what is wrong and why. And what the correct answer might be. When we write a review, or a commentary, or an opinion piece, be critical and make a case for what is correct and what is not. Be diligent. I promise I’ll be interested in reading that.

My friend Professor Wombat feels it is important to publish contrary data, and he does it regularly. Actually, he does it very well. He also publishes lots of interesting observations that are rigorous and move things forward (it would be a shame if he only set about showing other people are wrong). And he often publishes the contrary positions in places where people will read it (you know, the journals with the nice, soft pages). But here’s the thing – he gets the contrary positions wrong, too. A few years ago, he published a very influential paper that disproved a prominent idea, to great attention. But with time, we came to realize that he had not disproven the idea but, rather, found a counterexample. It did not ‘burn down the house’ of the idea, but refined it. He’s good with that. We’ve talked about it, and I know that he is interested in having the discussion, not proving his own point. This is how we make progress.

So yes, challenge those bad ideas – hold them up in the light and show us why they are bad. And here’s the thing. We have to fight with editors and reviewers, who tell us that challenging the idea is “not new.” As long as it is out there, the arguments against it are just as fresh as ever. And maybe, instead of citing the bad idea, someone will think twice and cite something else. Maybe even you. It’s about having the discussion.

It’s raining and starting to move into evening, and I haven’t had any ‘tea.’ Time to, um, put the kettle on and get back to reading. I’m not too worried about bad ideas when there are so many good ones out there. Many papers are, in the end, wrong on some level, but it is the bits that are right that move us forward. We can burn the house and live in it too – it’s what we do.

(4 votes)

Loading...