Why we should turn to journals

September 22, 2019 | by Nathanaël Fijalkow

Why do I avoid submitting to highly selective conferences?

What do I mean by highly selective conferences? I refer to the publication system which consists in regular (often yearly) events where researchers submit “extended abstracts” of their latest results to go through a highly competitive reviewing process resulting in a so-called “conference publication” in the proceedings and a talk slot during the conference. In some communities (mostly in computer science) conference publications are the most important part of your CV, and researchers focus their efforts on being published in prestigious conferences.

My complaints below concern these top conferences, but more specifically their competitive selection process, which has the following consequences: unfair selection process, ineffective dissemination of the research, time and energy demanding publication and reviewing processes, and counter-productive normative rules.

I think it is time we break with this toxic system.

I strongly believe that seminar talks, specialised workshops, week-long meetings, weeks-long research visits, and journal publications are the best way to disseminate research. The distinction between prestigious conferences and specialised workshops is the competitive selection process implying that conference papers are prizes to be voraciously collected. I find small scale conferences with little or no selection process often very interesting events to attend (as an example, the Highlights conference of Logic, Games, and Automata).

My ideal timeline for a scientific publication is: submit on arXiv, which hopefully raises some interest; present and discuss the results in various scientific events, inducing a healthy maturation process; once this has stabilised, usually a few months to a year later, submit to a journal.

To read more on the topic but with a different take, I suggest Moshe Vardi’s opinion pieces, see Scalable Conferences and Divination by Program Committee.

Counter-arguments to the usual reasons for submitting to prestigious conferences.

Click on an argument to expand it.

I want my results to be visible and quickly accessible

Put your results on arXiv. This will give them a wide and immediate visibility. Everyone knows and uses arXiv, it is well referenced by scholar websites. To strengthen that effect I often send my latest arXiv link to a few colleagues.

I need the stamp of approval from top conferences for my CV and being a member of my research community

This is an unfortunate truth, but this does not mean that it cannot be fought. The obstacle is just a community mindset. There are two things to consider here: the recognition for a specific paper, or the need to build up your CV and your place in the community. For the research paper although prestigious conferences surely help, this is by far not the only way to advertise your work. Researchers have enough judgment to make their own mind about your results. In my experience giving talks and discussing your results is much more effective than getting published in a top conference. For your CV, indeed a long list of prestigious conference papers looks good. But this is only a first impression, which is quickly replaced by your research interactions, ability to present your results, discuss them, relate them to others, and engage in developping new ideas. In the long run the correlation between your ability to get published in top conferences and your place in the community is not that clear. There are examples of all combinations: in particular, there are (fortunately not too many) researchers who have an impressive list of conference publications but whose research impact is debatable.

I want my students to thrive and get positions

This is arguably the strongest point against not submitting to conferences. In some sense, this is the same point as above, but specialised to students. The question is whether students can be successfully integrated into research communities, which includes getting jobs and grants, without conference publications on their CVs. My first point is that spending the same energy into submitting to journals rather than conferences is actually a good strategy: the outcome is better (a journal paper rather than a conference paper) and more fair, the price being that the process is slower. It should be commonly agreed that a journal publication carries more weight than an "extended abstract" in a conference. This is unfortunately not true in some minds, but this is completely irrational, and this strange bias should be fought against. Submitting to conferences is in pratice not that much faster than journals, once we factor in the risk of getting rejected (which is higher for students!) and the random aspect of the reviewing process for conferences. One may argue that for instance postdocs who are looking for positions need papers now. I sympathise with this as I was there just a few years ago, but again: given the odds, what is better, a noisy shot at a prestigious conference or a more fair try at a journal? My second point is that many initiatives achieve more towards integrating a student to a research community than a long list of conference publications: involving them in research projects, inviting them to scientific events, and organising visits in other research groups.

I want to be informed about the latest important results

Sign up for the daily or weekly arXiv alerting service on your research interests. This will be far more efficient than attending a conference for a broad coverage of the latest results. For more specific results and deeper discussions you want to talk directly to the relevant researchers, this is much easier and natural in a specialised event than in a general conference. For instance, the Logic, Games, and Automata community organises every year a Highlights conference where anyone can submit and everyone is accepted. This great initiative already inspired a similar event in the Algorithms community. Those are the perfect places to get up to date with results in the field.

I do not want to be scientifically isolated

This argument relies on the idea that your research interactions are only or mostly through your conference publications. There are many other ways to share your research, as listed above: seminar talks, specialised workshops, week-long meetings, weeks-long research visits, and journal publications. I am tempted to add another one: I started writing this research blog, and found that it is a useful platform for stimulating research interactions.

I want to publish smaller results which are not yet ready for a journal publication

This is wrong. We should aim at publising less, not more. For a simple reason: the more we publish, the less visible our results are, drowned in the number. On the other hand, nothing prevents you from publishing small results on arXiv. If I have a result which I think is interesting but not yet a full-blown satisfactory answer, I write a short note and put it on arXiv. The actual publication may come much later, be a merge of different notes, and often is enriched by interactions stimulated by the arXiv paper.

Arguments in favour of submitting to journals rather than conferences.

Click on an argument to expand it.

Consider environmental issues

Travelling around the world for a few days to present a paper is not good for the environment. See for instance the No Fly Climate Sci initiative and the Labos 1.5 initiative (in French).

Escape the unfair system of conference reviews

My experiences both as author and as PC member have convinced me that the conference reviewing system is bad. Although most PC members spend a lot of effort trying to evaluate articles, find expert reviewers, and select the best papers, the outcome is heavily biassed. Another problem is that tight deadlines and high competition imply that reviewers lose their benevolence. Something is not to their taste, the presentation, outline or notations not ideal, that's a perfect reason to reject. The scientific part is sometimes less important than pure presentation aspects. In a journal submission, would this be the case, the reviewer would ask for revisions and make useful suggestions for improvements. Conferences and their tight deadlines do not allow this simple process.

Avoid the huge waste of time in the conference system

The first main time waste is in writing conference papers. I have spent way too much time trying to fit my research results into conference papers (page limits, style, story, take away message, and other stupid constraints). This is wrong. This is a normative approach killing creativity. The second aspect is the time we spend as a community reviewing conference papers, which is in my opinion and experience a bad use of our time. The standard of conference reviews is way below that of journals; it is commonly understood that conference reviews are not bound to check proof details or even correctness. The time saved in not writing and reviewing conference papers can be reinvested in writing and reviewing for journals, which may also have the effect of making the journal publication process faster.

Beat the untenable pace of conference deadlines

This is wrong that deadlines dictate when a paper must be ready. It encourages unpolished submissions, rushed proofs, and the practice of subdividing papers.

Break free from the random selection process of prestigious conferences

At the heart of general prestigious conferences is an unspoken contradiction in terms: the selected papers should be at the same time the best results and the most accessible to a wide audience. I have two issues with this. The first is that it assumes that there is such a thing as a "result of interest to a wide audience". Reading reviews and participating in PC discussions convinced me that every researcher has a different understanding of what this means: either simple, or technical, or impressive, or solving an open problem, or relating two fields, or introducing a new notion... This point is in my opinion partly responsible for the random aspect of the selection process. The second is that this implies that the best results should be accessible and of interest to a wide community. This implicit idea means that PC discussions are filled with rejection arguments of the kind "the ideas are hard to grasp for someone outside of the field", "the notions seem to be very specific to the problem solved in the paper", or even "it will be very hard to present the results to a wide audience". As a consequence, a simple result easy to present in a few pages becomes more valuable than a technical and difficult one.

Mitigate the stress of rejection

Submitting to conferences means regular notifications, and often rejections. Notification dates are stressful experiences, which personally affect me too much. There is no need for that.

Write a scientific article, not a piece of advertisement

Because of the format and the competition, conference publications tend to contain less science and more advertisement: as little technical details as possible, no proofs, no deeper explanations. The journal paper is the place where the science is carefully explained, all aspects are considered, and all details are explored. On a scientific level, this has a much greater value.

Attending prestigious general conferences is not that productive

In some of the top general conferences I may be able to understand about 30% of the talks (rough estimates). What is the point?

Ignore the trendy topics and dive into your niche interest!

Conferences encourage trendy topics: it is hard to get a paper about a niche question published at a major conference. The best evidence of this being wrong is the decades during which neural networks was considered a model of minor importance with little applications, which meant that it was hard to publish results in major machine learning venues. Perseverance paid off, as Yann LeCun, Geoffrey Hinton and Yoshua Bengio will explain when receiving their 2019 Turing award!