^{1}

^{2}

^{3}

^{1}

^{4}

^{5}

The authors have declared that no competing interests exist.

When researchers complete a manuscript, they need to choose a journal to which they will submit the study. This decision requires to navigate trade-offs between multiple objectives. One objective is to share the new knowledge as widely as possible. Citation counts can serve as a proxy to quantify this objective. A second objective is to minimize the time commitment put into sharing the research, which may be estimated by the total time from initial submission to final decision. A third objective is to minimize the number of rejections and resubmissions. Thus, researchers often consider the trade-offs between the objectives of (i) maximizing citations, (ii) minimizing time-to-decision, and (iii) minimizing the number of resubmissions. To complicate matters further, this is a decision with multiple, potentially conflicting, decision-maker rationalities. Co-authors might have different preferences, for example about publishing fast versus maximizing citations. These diverging preferences can lead to conflicting trade-offs between objectives. Here, we apply a multi-objective decision analytical framework to identify the Pareto-front between these objectives and determine the set of journal submission pathways that balance these objectives for three stages of a researcher’s career. We find multiple strategies that researchers might pursue, depending on how they value minimizing risk and effort relative to maximizing citations. The sequences that maximize expected citations within each strategy are generally similar, regardless of time horizon. We find that the “conditional impact factor”—impact factor times acceptance rate—is a suitable heuristic method for ranking journals, to strike a balance between minimizing effort objectives and maximizing citation count. Finally, we examine potential co-author tension resulting from differing rationalities by mapping out each researcher’s preferred Pareto front and identifying compromise submission strategies. The explicit representation of trade-offs, especially when multiple decision-makers (co-authors) have different preferences, facilitates negotiations and can support the decision process.

Researchers often face the decision problem of where to submit a manuscript. In addition to potential professional benefits, the quick publication of papers where they are likely to be widely read improves the public good obtained by the generation and dissemination of scientific knowledge.

Some authors and institutions focus on the impact factor of the journals in which a researcher publishes as a measure of the reach of the researcher’s work [

However, researchers may not only select journals to maximize the potential impact of a publication. Researchers may also seek to minimize the time length of the review process (as well as the frustrations and extra time commitments stemming from multiple revisions and resubmissions), helping to ensure that new knowledge reaches its audience as quickly as possible. The “journal submission decision” problem was introduced by ref. [

Complex decision problems often feature

Here, we expand upon previous studies and offers a formal method to navigate the concerns that result from deep uncertainty and multiple decision-maker rationalities. We employ a Many-Objective Robust Decision-Making (MORDM) approach, using a set of tools designed to inform decision-making through transparency [

We adopt as a starting point the insightful model of the journal submission decision problem described in ref. [

Let

The impact factor (IF) of a journal is calculated as
_{i} represent the number of citations in the current year to items published in the previous two years for publication

Let an enumeration of journals be indexed by _{j}, λ_{j}, and τ_{j} denote the acceptance rate, expected number of citations for an article in the journal over the course of a year, and the expected time from submission to publication for journal _{R} denote the time (in days) required to make revisions and resubmit a manuscript. Following previous work, we adopt the assumption that a manuscript that has been “scooped” is essentially worthless and receives no citations, and that the probability of being scooped is constant in time [^{t} (where

Thus, the total probability of acceptance at a journal in the submission sequence is 1-_{r}. We include this in the results as a quantity to help navigate trade-offs, but do not include it explicitly as an objective because the acceptance rate for each journal is included in the other objectives. The expected number of citations for the manuscript is then
_{r}*0 term in Eq (

There are several simplifying assumptions in the formulation of Eq (

The expected total number of submissions of a manuscript is
_{max} is the expected maximum number of submissions within the time horizon _{max} is the largest index

The formulation of Eq (

The expected total time spent under review along a given journal submission sequence is

This can be interpreted as the expected time until the final decision, which occurs at one of three points: (i) acceptance by one of the _{R} between each resubmission. If the time horizon _{R}, and _{j}, λ_{j}, and τ_{j} are properties of the journals to which the manuscript might be submitted.

The model of Eqs (_{j}), impact factor (λ_{j}), and mean time from submission to final decision (τ_{j}) for each potential journal the researchers might submit to. We use the data set from ref. [

In the original formulation [_{r} = 0). When all 61 journals in the data set are included in each submission sequence, this is a reasonable assumption; the probability of being rejected from all 61 journals is about 2x10^{-13} (neglecting scooping). Here, we consider only journal submission sequences of length

The “XLRM” framework is a simple approach to synthesize the formulation of a decision analysis [

Illustration of the external factors (X), levers the decision-maker can manipulate to affect the outcome (L), the modeling relationships (R) and the performance metrics (M) in the formulation of the journal submission decision problem [

In our formulation of the journal submission decision problem, the external factors consist of the journal-specific data (acceptance rates (α_{j}), impact factors (λ_{j}), and time-to-decision (τ_{j})), the scooping probability (_{R}). The scooping probability, time needed to make revisions, and time horizon are uncertain factors that may vary from researcher to researcher. The decision lever in this problem is the chosen ordered sequence of

Choosing a strategy in the face of multiple objectives often requires the navigation of trade-offs among the objectives. In other words, different solutions may perform better than others with respect to particular objectives. For these types of problems, decision-makers are often interested in characterizing the entire set of solutions that are

For even relatively simple problems, it can be computationally intractable to enumerate all possible solutions to find the “true” Pareto front. In this relatively stylized example, there are over 700 million possible permutations of five journals out of the total set of 61 journals; direct identification of the exact Pareto front would involve simulating all of these permutations and comparing their objective values. While technically still feasible (but arguably cumbersome) in this case, we apply a method to identify high-quality approximations of the Pareto front. One class of these computational methods are Multi-Objective Evolutionary Algorithms (MOEAs) [

In the face of deep uncertainties, MOEAs can be combined with methods from robust decision-making to form the Multi-Objective Robust Decision-Making (MORDM) framework [

First, we compare the approximate Pareto front from the original Metropolis algorithm-based method [_{R} = 30 days. The NSGA-II algorithm finds the basic shape of the approximate Pareto front within 10^{4} function evaluations (^{7} function evaluations to converge to a similar approximate front. Even with 10^{7} function evaluations, several solutions on the Metropolis front are dominated by solutions on the NSGA-II front, suggesting that the NSGA-II algorithm outperforms the Metropolis algorithm in this case.

The approximate Pareto front between expected citations (horizontal axis), expected time under review (vertical axis) and expected number of submissions (size of points), as discovered by the Metropolis algorithm (red points) or NSGA-II MOEA (blue points) for (a) 10^{4}, (b) 10^{5}, (c) 10^{6} and (d) 10^{7} function evaluations. The model was run with _{R} = 30 days. The gold star in the lower-right corner represents the target direction for the represented objectives.

The overall geometry of the approximate Pareto front is mostly invariant with respect to the time horizon (^{7} function evaluations, scooping probability _{R} = 30 days, unless otherwise stated.

Approximate Pareto fronts for the (a) junior researcher (

We find three distinct regions that represent different strategies a researcher could follow in submitting a manuscript (

The Efficient Strategy aims to balance the effort the researcher must dedicate to revisions and resubmissions, while pushing the expected citation count as high as possible before the upwards inflection in expected time under review (seen in

Time horizon | Submission sequence | Expected citations | Expected submissions | Expected time under review | Acceptance probability | |
---|---|---|---|---|---|---|

Citation-Maximizing Strategy | 3 years | 1. |
6.4 y^{-1} |
3.8 | 251.7 days | 0.728 |

7 years | 1. |
7.0 y^{-1} |
3.8 | 251.7 days | 0.728 | |

20 years | 1. |
7.4 y^{-1} |
3.8 | 268.0 days | 0.716 | |

Compromise Strategy | 3 years | 1. |
5.5 y^{-1} |
2.9 | 149.0 days | 0.840 |

7 years | 1. |
5.9 y^{-1} |
2.9 | 149.5 days | 0.833 | |

20 years | 1. |
6.0 y^{-1} |
2.9 | 148.1 days | 0.841 | |

Efficient Strategy | 3 years | 1. |
4.3 y^{-1} |
1.9 | 94.2 days | 0.873 |

7 years | 1. |
4.5 y^{-1} |
2.0 | 95.7 days | 0.871 | |

20 years | 1. |
4.6 y^{-1} |
1.9 | 94.2 days | 0.873 |

We identify submission sequences for each researcher that maximize the number of citations under each of the three submission strategies (

There are strong trade-offs between the citations objective and each of the two effort objectives, depending on the submission strategy pursued (

Parallel axis plots for the (a) junior researcher (

For all three researchers, Efficient and Citation-Maximizing submission sequences that perform well on expected citations (i.e., near the top of the axis) tend to perform poorly in expected number of submissions and time in review, and vice versa. This demonstrates a strong trade-off between the expected number of submissions (and time in review) and the expected number of citations (

It may be counterintuitive that

Consider a multiple author manuscript with a junior author (

Now consider the case of a graduate student who is approaching her Ph.D. defense, with a research focus that is at the forefront of her field that is also quite crowded. This graduate student may assume a relatively high scooping probability _{R} = 45 days, while her advisor assumes _{R} = 30 days.

The higher perceived probability of scooping (and thus accumulating zero citations) compresses her range of possible citation counts (

Pareto-approximate solutions for a scooping-averse junior researcher (_{R} = 45 days) in red and a less scooping-averse senior researcher (_{R} = 30 days) in blue. The

These different rationalities require discussion and compromise between the two decision-makers, so that the senior author’s desire for maximum exposure can be reconciled with the junior author’s aversion to being scooped. One approach is to look for common Pareto-approximate submission sequences, though these may not always exist, requiring deviation from the approximate Pareto front for one or more decision-makers. In this case, three sequences are non-dominated for both researchers. These sequences and the values for each objective for each researcher are given in

Submission sequence | Expected citations | Expected submissions | Expected time |
|||
---|---|---|---|---|---|---|

Junior author | Senior author | Junior author | Senior author | Junior author | Senior author | |

1. |
4.3 y^{-1} |
6.4 y^{-1} |
3.7 | 3.1 | 233 days | 164 days |

1. |
4.2 y^{-1} |
6.4 y^{-1} |
3.3 | 3.0 | 216 days | 166 days |

1. |
4.1 y^{-1} |
6.0 y^{-1} |
3.2 | 2.9 | 200 days | 148 days |

These solutions come close to maximizing the expected citations for the junior researcher, while the senior author can expect more than 75% of the maximum citations he could obtain (indeed, the third sequence in

From the perspective of the conditional impact factor heuristic described earlier, the solutions presented in

It may not always be the case that these compromise solutions occur across all of the collaborators’ Pareto-approximate sets, particularly as the number of coauthors increases. It may also be the case that the compromise solutions still perform relatively poorly for certain co-authors (for example, the junior author in our example may not be satisfied with overall acceptance probabilities below 60%). In this case, methods such as the Analytic Hierarchy Process [

The analyses of submission decision problem, as the objectives are formulated here and in ref. [

We do not intend to explicitly recommend that a researcher should follow some of the Pareto-approximate sequences (e.g., from Tables

The results presented here are intended to illustrate a framework for understanding the trade-offs and the balancing of competing decision-makers’ interests that are inherent in the journal submission decision problem. These methods show how a general understanding of the trade-offs can provide insight into non-obvious ways to generate journal submission sequences that balance researchers’ values (such as in _{R}) and whose efforts go into the resubmissions. These can be important considerations when honing submission sequences to decision-makers’ particular goals.

The analysis presented here is a limited version of the robust decision-making analysis present in the full MORDM framework (16,19,21). Under this analytic framework, after finding the approximate Pareto front for expected values of uncertain parameter values (such as revision time and scooping probability), selected solutions are tested against a variety of states of the world to determine under what combinations of parameters they perform acceptably well or unacceptably poorly. This full analysis, which is beyond the scope of this didactic example, can more fully inform decision-makers about the risks they may face if they have made incorrect assumptions about the “true” parameter values [

We use a multi-objective optimization analysis to explicitly illustrate the trade-offs inherent in deciding on which journal(s) to send a manuscript. Our results expand upon previous work by representing the tension brought on by multiple rationalities surrounding co-author values such as time horizon and perceived probability of being scooped. This analysis serves as an introduction to the more rigorous robust decision-making framework which often accompanies this method of decision analysis. We implement an open source and freely available multi-objective optimization framework and demonstrate the relative efficiency of our approach over that of previous work (

(EPS)

(EPS)

(EPS)

(PDF)

We gratefully acknowledge Caitlin Spence, Julianne Quinn, Rob Lempert, and Bella Forest for invaluable inputs. This study was partially supported by the Penn State Center for Climate Risk Management. Any conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the funding agencies. Any errors and opinions are, of course, those of the authors.