Header picture by Ken Teegardin
Trying to make sense of nonsense
Tag Archives: research
June 19, 2013Posted by on
There are a lot of scary things to face when doing a PhD: supervisor’s ideas of ‘normal’ working hours, reviewers whose sole aim in life is to reject as many papers as possible, or the experimental equipment that only works when the right amount of blu-tack is in the right place and you karate chop the on-button. But possibly the scariest of all is the journalist.
This is why sense about science has set up their Standing up for Science media workshop: a one day workshop, specifically for early career scientist that gives a bit of insight into how science gets translated into news. It’s a great workshop that combines a session of scientists talking about successful (and less successful) experiences with journalists, with a session of journalists talking about what they actually do during their busy days. But most of all, it gets us early career scientists away from our lab benches for a day to talk about why we think it is so scary in the first place.
Most of us grad students (and scientists in general) are funded by public money, so it is a reasonable expectation that we try to feed our results back to the public. That’s easier said than done though. As scientists, we spend a lot of time getting the right results, and even more so, getting them just right on paper. Even though a scientific article might be only 3,000 words, it has to represents years of blood, sweat and tears.
So it might be understandable that we can be a bit hesitant when we have to hand this over to a journalist not familiar with our particular brand of science. We’ll just have to stand by while they condense it into a catchy headline and accompanying article that is often shorter than any summary we could write ourselves. Everyone knows someone for whom this has gone horribly wrong. Stories are abound about how a basic science paper on cells in a petri dish ended up promising to have found the cure-all pill for cancer, or how bacon is apparently responsible for doubling our (already 100%) risk of death.
It’s great to hear from experienced people like Dr Deirde Hollingsworth and Prof Stephen Keevil that talking to media gets easier after a while, and that the mess-ups are rarely remembered by anyone but yourself. Even talking to a news outlet with a reputation like Fox News can be a good experience, according to Dr Emily So who talked to them live on air after the Fukushima earthquake and tsunami.
In the Q&A session afterwards, Dr Hollingsworth advises us not be afraid of silence (unless you’re The Doctor, in which case you’re right to be afraid). It’s up to the journalist to ask questions, and if you try to fill the void you might end up saying things you didn’t intend to.
The journalist session is equally enlightening. Jason Palmer (BBC), Richard Van Noorden (Nature) and Jane Symons (former health editor at the Sun) assure us they’re not out to get us: they want to get the science right as much as we do. However, they do have a product to sell and a deadline to make (not to mention a mythical sleepy granny to keep awake), so it would be helpful to them if we do pick up the phone when they call. If we don’t, they might go for someone even less qualified to answer their question.
Helpfully, sense about science has provided a booklet with some easy tips (and even a checklist) on talking to the media. Sense about science are organising the Standing up for Science workshop again in September (London) and in November (Glasgow).
August 21, 2012Posted by on
Well, thank you all for reading the first part – I got more visitors than I normally get in a month! Hope you like the second half as much as the first! Any comments/other tips are of course very welcome.
Step 3: Submission and waiting
Before you submit, you should make some final checks, for which most journals supply a handy checklist (sometimes you don’t run in to these until you actually register to their submission system, so have a look around there early on). Are you complying with the relevant reporting guidelines? Do you have all necessary forms, such as conflict of interest statements, and the perfect abstract and cover letter (those are two things an editor is sure to look at) to convince the journal your article is worth reviewing? Right then: hit those buttons and submit.
And now the wait starts. If you don’t hear from the journal within the next couple of days, your paper has probably been sent out for review, which is good, but could take weeks, if not months. Luckily, there are things you can do to slightly speed this process up, such as suggesting potential reviewers in your cover letter (if the journal doesn’t provide that option in their submission system). Even though the journal might have published lots of similar studies, it is always helpful to make some recommendations.
Step 4: Results!
Unless your are submitting to the journal of universal rejection, you can never be sure what the outcome of a submission is going to be. There are three possible outcomes: you’re paper gets rejected, the editor wants you to revise your paper, or your paper is accepted without changes (it’s theoretically possible, I’ve been told). In case of rejection, you can either appeal the decision or move on to the next journal.
Personally, I have never appealed, but it is possible to do so when you feel you’ve been unfairly rejected. Maybe the reviewers didn’t display any knowledge of the topic area (you’d be surprised how many reviewers accept to review paper on topics they have no or little expertise on), or the decision of the editor doesn’t seem to add up with the opinions of the reviewers. It happens. One thing to keep in mind is that an appeal can take a long time: if the editor appears to have made the wrong call, associate editors will have to make a decision, which can be fairly quick. However, if the reviewers were in the wrong, the editor will have to assert their incompetence, and find new reviewers. It might be faster to submit to a different journal (which might be preferable in the case of looming grant application deadlines).
The third option, revision, comes in two flavours: minor and major. Although the first one gives you a better chance of eventual acceptance, it’s still not sure you’re going to get in. The vestiges of published, peer-review science are guarded well, or that is the intended function of peer-review at least. Major revisions will require re-analysis, new tests or experiments, rewrites or explanations of unexplained concepts: the list is endless really.
Step 5: Responding to reviewers’ comments
One thing to take into account when responding to reviewers’ comments is that is not personal, and that reviewers rarely agree. A large meta-analysis  actually found that peer reviewers only agree about 1 in 3 times (or even less if you focus on the larger studies with smaller confidence intervals). However, the editor would like to see your study published (that’s what is paying
his their salary) and the reviewers’ comments are meant to be constructive, so it’s important to stay in character and be polite when you answer.
This might seem like pointing out the obvious, but under the guise of anonymity, some reviewers tend to lose composure. Although you might be tempted to give in and give such reviewers a piece of your mind, it will be the editor who will read your response first, so it’s better to hold your guns. Some reviewers might have a vested interest in whether or not your paper gets published: they work in your field, so they will have an opinion on whether what you’re doing is correct and line with their work. Other peculiar behaviour might happen when someone remarks you only cited one single paper by the distinguished Dr Scientist. Maybe you could also cite these other eight (barely relevant) papers by the honourable Dr Scientist, who you’re not supposed to guess is the actual reviewer?
Working through comments can get very frustrating. Here’s a beautiful pair of comments I got back from some reviewers (both on the same paper):
- Reviewer #1: “The analysis and purpose of the study is confusing. The quality of the data is likely suspect.”
- Reviewer #2: “I found the paper to be well written, the analysis rigorous and well conceived and the conclusions supported by the data and analysis.”
And this was just the start of both reviews, the disagreements between both reviewers got worse with every paragraph they dealt with. As mentioned before, these inconsistencies between reviewers are common, which is why it is an editor making the final decision, rather than the reviewers battling it out amongst themselves. Working through them can be become a bit tedious to say the least.
The final part of responding to reviewers’ comments (and you have to respond to all of them), is writing the rebuttal or revision letter. I like to start by thanking the editor for giving me the opportunity to revise and respond to the comments. That will take up one page, which I structure a bit like my cover letter (department-headed paper and all). Then I start the actual rebuttal:
“We thank the reviewers for their comments on our paper. We have changed our paper accordingly and addressed all the comments as listed below:”
[Short summary of major changes]
[Copy and paste comments from reviewers and write a short response to each of them for instance:]
1. In figure 1, the authors have not included in which units the y-axis is labelled.
We thank the reviewer for noticing this omission. We have now correctly labelled our y-axis in rate per 100 person-years.
(the colour helps to distinguish between reviewer’s and my words)
It might take a few pages to get through all of them, but it makes it easier for the editor to see what I did and why I did it – hopefully shortening my waiting time a bit. Then it’s time to resubmit the whole thing again. If the comments were only minor, it’s usually the editor who will make the final decision. If there were any major comments, the paper is likely to go back to the initial reviewers and you’ll have to wait a bit longer.
Alternative ways to get published
Writing papers isn’t the only way to get your name out there: give blogging a go! Or offer to write a book review (free books!), write science news articles (a good way to keep on top of what is happening in your field, and to practice those abstract writing skills) or enter a science writing competition. (I’m obviously not entirely subjective here). Significance is always on the look out for new bloggers, so why not try them if you’re tempted?
So every journal on your list rejected your paper? Why not try the Journal of Negative Results in Biomedicine, the All Results Journal, the Journal of Pharmaceutical Negative Results or even the Journal of Articles in Support of the Null Hypothesis?
And now go read some author guidelines! They’re likely to be shorter than this post.
Resources & Reference:
1. Bornmann L, Mutz R, Daniel H-D (2010) A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability and Its Determinants. PLoS ONE 5(12): e14331. doi:10.1371/journal.pone.0014331
Twitter hashtags: #PhDchat / #ECRchat / #acwri <- useful to ask questions and find other good resources. If you don’t use Twitter, no worries: PhD-chat has an off-twitter wiki, and ECR (early career researcher) chat has a blog.
August 20, 2012Posted by on
With only 3 papers with my name on it, I’m definitely not an expert when it comes to getting papers published. However, those 3 papers (and one that’s waiting in some reviewers’ inboxes right now) have been rejected a total number of 12 times, giving me at least some experience in preparing and submitting them. Maybe that’s why I got invited to give a talk on publishing papers at the student skills workshop at the ICPE – the International Conference on Pharmacoepidemiology and Risk Management. Or it might have been that when a publisher dropped out last minute, and the organisers (one of whom I happen to share a supervisor with) really needed someone who had already booked their tickets to Barcelona.
Either way, I’ve got a presentation to prepare for, and in doing so, I’ve found that actually I have developed a bit of a five-step system when it comes to preparing papers. Even more so: when talking with other students and staff around my department, I’ve found some interesting tips and thoughts on how to get published. I felt it might be worth it to actually write all of this up in a blog (and get some last minute considerations to add to my talk?), as a reference to build on, so here we go:
Step 1: Selecting a journal
I’ll start at the point where you’ve done all your analyses and have pretty good idea what you want your paper to be about. Maybe you’re working on that first draft, or you’re already on version 17.3, but at some point you’ll have to start considering what journal to submit to. As my first supervisor told me when my very first paper was rejected by JAMA: “If the first journal you submit your paper to accepts it, you didn’t aim high enough”.
And there you immediately have your first problem: what constitutes aiming high? Impact factors are one determinant of ‘high’, though we all know now that using those in any decision making will only prove you are statistically illiterate. Rather, you could aim at submitting to one of the general medicine journals, such as the New England Journal of Medicine, JAMA, the Lancet or BMJ. All of those boast large regular reader counts and even larger rejection rates.
The scopes of these journals are wide, but they will only consider the studies that will keep their impact factors up, so it might be good to consider some more specialist journals as well. You might not reach as many researchers, but you are more likely to reach the right ones. To find out which are best for your research, go over the papers your citing: there are bound to be some relevant journals there. Or ask an expert; you’re collaborators (if you have any) will probably be able to make some suggestions.
These can ideas can then form a list of potential journals to submit to. Being rejected becomes a lot less painful if you’ve your plan B at the ready. Final considerations will depend on your funder (should the journal be open access?) and funding (yes, it costs money to be published).
Step 2: Formatting and editing
However much I’d like it, there is no getting out of formatting or weeding through formatting guidelines (at least not until you’re senior enough to have someone do it for you), but there are some little things that can make it easier. One of these is Wordle, which creates a ‘word cloud’ highlighting the words you’ve used most often. The first time I copied a paper of mine in there (luckily just before I meant to send it to my supervisors), one word stood out like a sore thumb: However. Without really noticing it, I had started using the word in every other sentence in the discussion. Apart from highlighting unnecessary repetitions, it’s also a very nice tool for identifying key words in your paper: if the right ones don’t come up, you’re probably using to many different terms to describe one phenomenon.
Also important: use a reference manager (I like Zotero – it’s free and integrates with Firefox/Chrome, so you can you use it on any computer without needing to bring the most recent database file with you). Different programs will have different (dis)advantages, so shop around a bit before you decide upon one, there are a lot of options out there.
Another tip is to read your paper aloud. After taking six years of Latin, I’ve really come to love subordinate clauses and the dactylic hexameter. Unfortunately, they don’t work so well for academic writing, and reading sections aloud really helps in locating the overly complicated sentences I can come up when left to myself for too long (enter joke about ablative absolutes). It works even better when you leave your paper for a few days or even weeks, and then come back to it. Instead of reading what you think is there, you’ll suddenly be able to see what it is actually there.
When you finally come round to sending it to your co-authors make sure you give them enough time. Or even better: decide on a revision plan. How many times will each co-author see the paper, and in which order will you send it round? It can be hugely ineffective to send it to everyone at the same time, as you will end up with lots of similar or contradictory comments. Of course this will get more difficult with increasing numbers of co-authors, but it is important to keep at least the PI and supervisors involved.
One last formatting tip: LaTeX. It’s amazing. Like reference managers format your references, LaTeX can format your entire paper. It will take a bit coding (unless you opt for a program like LyX – thank you @JStreetley), but it will be worth it. One downside: the resulting text will be in PDF, making it harder for some reviewers to write comments or make changes.
Naturally, publishing involves a lot of waiting. So as my post is already past the 1,000 word mark, I’ll leave you to wait for part 2 (submitting & final checks, results!, and responding to reviewers’ comments) tomorrow.