From ac07d62ceee6af958adaaca1681d42ef0cd0e3d4 Mon Sep 17 00:00:00 2001 From: mpadge Date: Tue, 2 Dec 2025 13:04:45 +0100 Subject: [PATCH 01/32] review EiC section for #487 --- softwarereview_editor.Rmd | 48 +++++++++++++++++++++++++++------------ 1 file changed, 33 insertions(+), 15 deletions(-) diff --git a/softwarereview_editor.Rmd b/softwarereview_editor.Rmd index 441dc989b..8137f680a 100644 --- a/softwarereview_editor.Rmd +++ b/softwarereview_editor.Rmd @@ -8,7 +8,7 @@ aliases: ```{block, type="summaryblock"} Software Peer Review at rOpenSci is managed by a team of editors. The Editor-in-Chief (EiC) role is rotated (generally quarterly) amongst experienced members of our editorial board. -Information on current and recent editors and their activities can be viewed on our editorial dashboard at [dashboard.ropensci.org](https://dashboard.ropensci.org). +Information on current status of all editorial team members is presented on our [*Editorial Dashboard*](https://dashboard.ropensci.org). This chapter presents the responsibilities [of the Editor-in-Chief](#eicchecklist), and of [handling editors in charge of submissions](#editorchecklist). It also describes [how to respond to an out-of-scope submission](#outofscoperesponse), and provides guidance on [answering reviewers' questions](#reviewersupport). @@ -21,34 +21,52 @@ If you're a guest editor, thanks for helping! Please contact the editor who invi ## EiC Responsibilities {#eicchecklist} Rotating Editors-in-Chief (EiCs) generally serve for 3 months or a time agreed to by all members of the editorial board. -The EiC is entitled to taking scope and overlap decisions as independently as possible (but can still request help and advice). -Information on current status of all editorial team members is presented on our [*Editorial Dashboard*](https://dashboard.ropensci.org). -The EiC is responsible for the following tasks: +The EiC is responsible for [general management](#eic-general) of the review process, and for the initial stages of all submissions. + +### General EiC duties {#eic-general} + +The EiC provides general management of all [software-review issues](https://github.com/ropensci/software-review/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen) with the help of our [editorial dashboard](https://dashboard.ropensci.org), as described in the following [dashboard sub-section](#eic-dashboard). +The EiC is responsible for the following general tasks: -- On assuming EiC rotation, reviewing the status of current open reviews as detailed on the [*Dashboard* page](https://dashboard.ropensci.org/reviews.html), and issuing reminders to other editors or package authors as needed. See [the following sub-section for more details](#eic-dashboard) +- On assuming EiC rotation, reviewing the status of current open reviews as detailed on the [*Dashboard* page](https://dashboard.ropensci.org/reviews.html), and issuing reminders to other editors or package authors as needed. -- Watching all new issues posted to the software-review repo, for which the EiC must either subscribe to repo notifications on GitHub, or watch the `#software-peer-review-feed` channel on Slack. +- Watching all new issues posted to the software-review repo, for which the EiC must either subscribe to repo notifications on GitHub, or watch the `#editors-only` channel on Slack (ideally both). -- Tagging each new full submission with ` 0/editorial-team-prep` +- Regularly (for instance weekly) monitoring the pace of all open reviews by keeping an eye on the [*Dashboard* page](https://dashboard.ropensci.org/reviews.html), and reminding other editors to move packages along as needed. + +- Responding where appropriate to issues posted to the [software-review-meta repo](https://github.com/ropensci/software-review-meta/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen). -- Calling `@ropensci-review-bot check srr` on pre-submission enquiries for statistical software. See corresponding [*Stats Dev Guide* chapter](https://stats-devguide.ropensci.org/pkgsubmission.html#editor-in-chief) for details. +### EiC duties for each initial submission {#eic-submission} -- Finding an editor (potentially including yourself) to handle each submission. Currently available editors are indicated on the [*Editorial Dashboard*](https://dashboard.ropensci.org), and editorial workloads should be distributed as evenly as possible, through referring to the [*Dashboard* charts of recent editorial load](https://dashboard.ropensci.org/editors.html#past-ed-load). +The EiC is reponsible for initial processing of all new submissions. +The primary duties are to decide whether or not a submission should be considered in scope and proceed to review, and if so, to assign a handling editor. +The EiC is entitled to take scope and overlap decisions as independently as possible, but is recommended to request opinions on the #editors-only channel on Slack. +For each new pre-submission or submission, the EiC should: -- Assigning editors by issuing the command: +- Make decisions on scope and overlap for pre-submission inquiries, referrals from JOSS or other publication partners, and submissions. Discussions should be initiated in the rOpenSci Slack `#editors-only` channel through summarising the (pre-)submitted/referred software, along with any concerns the EiC might have. If the EiC feels they haven't received enough answers after a day or two, they can ping all editors. + + - Any editor should feel free to step in on these. See [this section](#outofscoperesponse) about how to respond to out-of-scope (pre-) submissions. + - After explaining an out-of-scope decision, write an issue comment `@ropensci-review-bot out-of-scope`. + +- Tag each new full submission with ` 0/editorial-team-prep` + +- Find a handling editor (potentially including yourself). Currently available editors are indicated on the [*Editorial Dashboard*](https://dashboard.ropensci.org), and editorial workloads should be distributed as evenly as possible, through referring to the [*Dashboard* charts of recent editorial load](https://dashboard.ropensci.org/editors.html#past-ed-load). + +- Assign the handling editor to the review issue by issuing the command: ``` @ropensci-review-bot assign @username as editor ``` This will also add the tag `1/editor-checks` to the issue. -- Regularly (for instance weekly) monitoring the pace of all open reviews by keeping an eye on the [*Dashboard* page](https://dashboard.ropensci.org/reviews.html), and reminding other editors to move packages along as needed. +#### Statistical software submissions -- Responding to issues posted to [the software-review-meta repo +Statistical software should be considered in scope as long as it complies with > 50% of all [applicable standards](https://stats-devguide.ropensci.org/standards.html). +If a pre-submission inquiry indicates that standards have already been complied with, the EiC should: -- Making decisions on scope and overlap for pre-submission inquiries, referrals from JOSS or other publication partners, and submissions. Discussions should be initiated in the rOpenSci Slack editors-only channel through summarising the (pre-)submitted/referred software, along with any concerns the EiC might have. If after the EiC feels they haven't received enough answers after a day or two, they can ping all editors. +- Call `@ropensci-review-bot check srr` to confirm that standards have been complied with. +- Consider whether the package is best placed in the nominated statistical category, or whether alternative or additional categories might be appropriate. - - Any editor should feel free to step in on these. See [this section](#outofscoperesponse) about how to respond to out-of-scope (pre-) submissions. - - After explaining an out-of-scope decision, write an issue comment `@ropensci-review-bot out-of-scope`. +Details for EiC handling of statistical software submissions are provided in the corresponding [*Stats Dev Guide* chapter](https://stats-devguide.ropensci.org/pkgsubmission.html#editor-in-chief). ### The rOpenSci Editorial Dashboard {#eic-dashboard} From 8f908f354d34f8240fad5ffa2b0fd97f4e0035c3 Mon Sep 17 00:00:00 2001 From: mpadge Date: Tue, 2 Dec 2025 16:55:46 +0100 Subject: [PATCH 02/32] further edits of EiC section for #487 --- softwarereview_editor.Rmd | 44 ++++++++++++++++++++++++++------------- 1 file changed, 30 insertions(+), 14 deletions(-) diff --git a/softwarereview_editor.Rmd b/softwarereview_editor.Rmd index 8137f680a..6d75d817a 100644 --- a/softwarereview_editor.Rmd +++ b/softwarereview_editor.Rmd @@ -21,36 +21,50 @@ If you're a guest editor, thanks for helping! Please contact the editor who invi ## EiC Responsibilities {#eicchecklist} Rotating Editors-in-Chief (EiCs) generally serve for 3 months or a time agreed to by all members of the editorial board. -The EiC is responsible for [general management](#eic-general) of the review process, and for the initial stages of all submissions. +The EiC is responsible for [general management](#eic-general) of the review process, and for the [initial stages](#eic-submission) of all submissions. ### General EiC duties {#eic-general} The EiC provides general management of all [software-review issues](https://github.com/ropensci/software-review/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen) with the help of our [editorial dashboard](https://dashboard.ropensci.org), as described in the following [dashboard sub-section](#eic-dashboard). -The EiC is responsible for the following general tasks: +EiC responsibilities include the following general tasks: -- On assuming EiC rotation, reviewing the status of current open reviews as detailed on the [*Dashboard* page](https://dashboard.ropensci.org/reviews.html), and issuing reminders to other editors or package authors as needed. +- At the start of a rotation, the EiC should review the status of current open reviews on the [editorial dashboard](https://dashboard.ropensci.org/reviews.html), and issue reminders to other editors or package authors as needed. -- Watching all new issues posted to the software-review repo, for which the EiC must either subscribe to repo notifications on GitHub, or watch the `#editors-only` channel on Slack (ideally both). +- Watch all new issues posted to the software-review repo, for which the EiC must either subscribe to repo notifications on GitHub, or watch the `#editors-only` channel on Slack (ideally both). -- Regularly (for instance weekly) monitoring the pace of all open reviews by keeping an eye on the [*Dashboard* page](https://dashboard.ropensci.org/reviews.html), and reminding other editors to move packages along as needed. +- Regularly (for instance weekly) monitor the pace of all open reviews by keeping an eye on the [*Dashboard* page](https://dashboard.ropensci.org/reviews.html), and reminding other editors to move packages along as needed. -- Responding where appropriate to issues posted to the [software-review-meta repo](https://github.com/ropensci/software-review-meta/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen). +- Respond where appropriate to issues posted to the [software-review-meta repo](https://github.com/ropensci/software-review-meta/issues?q=sort%3Aupdated-desc%20is%3Aissue%20is%3Aopen). ### EiC duties for each initial submission {#eic-submission} The EiC is reponsible for initial processing of all new submissions. -The primary duties are to decide whether or not a submission should be considered in scope and proceed to review, and if so, to assign a handling editor. -The EiC is entitled to take scope and overlap decisions as independently as possible, but is recommended to request opinions on the #editors-only channel on Slack. +The primary duties are: +1. to decide whether or not a submission should be considered in scope and proceed to review, and if so, +2. to assign a handling editor. + +The EiC is entitled to take scope and overlap decisions as independently as possible, but is recommended to request opinions on the `#editors-only` channel on Slack. +Scope decisions for statistical software are generally easier than for general (non-statistical) submissions, as [desribed below](#eic-stats-submissions). For each new pre-submission or submission, the EiC should: -- Make decisions on scope and overlap for pre-submission inquiries, referrals from JOSS or other publication partners, and submissions. Discussions should be initiated in the rOpenSci Slack `#editors-only` channel through summarising the (pre-)submitted/referred software, along with any concerns the EiC might have. If the EiC feels they haven't received enough answers after a day or two, they can ping all editors. +- Refer to the categories described in the [_Aims and Scope_](#aims-and-scope) section to make decisions on scope and overlap for pre-submission inquiries, referrals from JOSS or other publication partners, and submissions. + Initiate discussions in the rOpenSci Slack `#editors-only` channel through summarising the (pre-)submitted/referred software, along with any concerns the EiC might have. + If the EiC feels they haven't received enough answers after a day or two, they can ping all editors. - - Any editor should feel free to step in on these. See [this section](#outofscoperesponse) about how to respond to out-of-scope (pre-) submissions. +- If a pre-submission, referral, or submission is deemed out of scope, the EiC should thank authors for their submission, explain the reasons for the decision, and direct them to other publication venues if relevant. + Use wording from [Aims and scope](#aims-and-scope) in particular regarding the evolution of scope over time. - After explaining an out-of-scope decision, write an issue comment `@ropensci-review-bot out-of-scope`. -- Tag each new full submission with ` 0/editorial-team-prep` +- If a pre-submission inquiry is considered within scope, the EiC _may_ perform preliminary checks. + The [_Editors Template_](#editortemplate) may be used for this. + To aid authors' responses to EiC comments, it helps to use unambiguous notation like [this example from one of our editors, Mauro Lepore](https://github.com/ropensci/software-review/issues/673#issuecomment-2559753922). + +- Once the EiC is satisified that a package may proceed to a full submission, invite the authors to open a full submission issue, and then close the pre-submission enquiry. + +- Initially tag each new full submission with ` 0/editorial-team-prep` -- Find a handling editor (potentially including yourself). Currently available editors are indicated on the [*Editorial Dashboard*](https://dashboard.ropensci.org), and editorial workloads should be distributed as evenly as possible, through referring to the [*Dashboard* charts of recent editorial load](https://dashboard.ropensci.org/editors.html#past-ed-load). +- Find a handling editor (potentially including yourself). + Currently available editors are indicated on the [*Editorial Dashboard*](https://dashboard.ropensci.org), and editorial workloads should be distributed as evenly as possible, through referring to the [*Dashboard* charts of recent editorial load](https://dashboard.ropensci.org/editors.html#past-ed-load). - Assign the handling editor to the review issue by issuing the command: ``` @@ -58,7 +72,7 @@ For each new pre-submission or submission, the EiC should: ``` This will also add the tag `1/editor-checks` to the issue. -#### Statistical software submissions +#### Statistical software submissions {#eic-stats-submissions} Statistical software should be considered in scope as long as it complies with > 50% of all [applicable standards](https://stats-devguide.ropensci.org/standards.html). If a pre-submission inquiry indicates that standards have already been complied with, the EiC should: @@ -66,7 +80,9 @@ If a pre-submission inquiry indicates that standards have already been complied - Call `@ropensci-review-bot check srr` to confirm that standards have been complied with. - Consider whether the package is best placed in the nominated statistical category, or whether alternative or additional categories might be appropriate. -Details for EiC handling of statistical software submissions are provided in the corresponding [*Stats Dev Guide* chapter](https://stats-devguide.ropensci.org/pkgsubmission.html#editor-in-chief). +If standards have not yet been complied with, the EiC should ask the authors to judge whether they think their package will be able to comply with at least half of all general and category-specific standards. +This may require discussion of the appropriate category or categories. +Full details for EiC handling of statistical software submissions are provided in the corresponding [*Stats Dev Guide* chapter](https://stats-devguide.ropensci.org/pkgsubmission.html#editor-in-chief). ### The rOpenSci Editorial Dashboard {#eic-dashboard} From 755de8ba3fecdce833ca079ee8f0aa6a8e2323da Mon Sep 17 00:00:00 2001 From: mpadge Date: Tue, 2 Dec 2025 17:14:05 +0100 Subject: [PATCH 03/32] two new sub-sections in EiC section for #487 And rm previously separate 'Asking for more details' sub-section --- softwarereview_editor.Rmd | 68 +++++++++++++++++++++------------------ 1 file changed, 36 insertions(+), 32 deletions(-) diff --git a/softwarereview_editor.Rmd b/softwarereview_editor.Rmd index 6d75d817a..86e40d561 100644 --- a/softwarereview_editor.Rmd +++ b/softwarereview_editor.Rmd @@ -38,10 +38,13 @@ EiC responsibilities include the following general tasks: ### EiC duties for each initial submission {#eic-submission} -The EiC is reponsible for initial processing of all new submissions. +The EiC is responsible for initial processing of all new submissions. The primary duties are: + 1. to decide whether or not a submission should be considered in scope and proceed to review, and if so, -2. to assign a handling editor. +2. to proceed to a full submission and assign a handling editor. + +#### Deciding on scope and overlap The EiC is entitled to take scope and overlap decisions as independently as possible, but is recommended to request opinions on the `#editors-only` channel on Slack. Scope decisions for statistical software are generally easier than for general (non-statistical) submissions, as [desribed below](#eic-stats-submissions). @@ -52,13 +55,34 @@ For each new pre-submission or submission, the EiC should: If the EiC feels they haven't received enough answers after a day or two, they can ping all editors. - If a pre-submission, referral, or submission is deemed out of scope, the EiC should thank authors for their submission, explain the reasons for the decision, and direct them to other publication venues if relevant. - Use wording from [Aims and scope](#aims-and-scope) in particular regarding the evolution of scope over time. + Where relevant, use wording from [_Aims and scope_](#aims-and-scope) regarding the evolution of scope over time. - After explaining an out-of-scope decision, write an issue comment `@ropensci-review-bot out-of-scope`. - If a pre-submission inquiry is considered within scope, the EiC _may_ perform preliminary checks. The [_Editors Template_](#editortemplate) may be used for this. To aid authors' responses to EiC comments, it helps to use unambiguous notation like [this example from one of our editors, Mauro Lepore](https://github.com/ropensci/software-review/issues/673#issuecomment-2559753922). +Decisions on scope may be require further information from submitting authors. +The EiC should minimally ensure that documentation is sufficient to judge scope, including an accompanying website. +If not, please ask for more details; even if a package is deemed out-of-scope, requesting improvements to package documentation can only help others to understand the package. +This is an example request: + +```markdown +Hello and many thanks for your submission. + +We are discussing whether the package is in scope and need a bit more information. + +Would you mind adding more details and context to the README? +After reading it someone with little domain knowledge should have been informed about the aim, goals and functionality of the package. + + +If a package has overlapping functionality with other packages, we require it to demonstrate in the documentation [how it is unique in comparison to similar packages](https://devguide.ropensci.org/policies.html#overlap). +Could you add a more detailed comparison to the packages you mention in the README so we can evaluate? + +``` + +#### Initial EiC duties for full submissions + - Once the EiC is satisified that a package may proceed to a full submission, invite the authors to open a full submission issue, and then close the pre-submission enquiry. - Initially tag each new full submission with ` 0/editorial-team-prep` @@ -90,40 +114,20 @@ The [*rOpenSci Editorial Dashboard*](https://dashboard.ropensci.org) is updated The dashboard provides an up-to-date overview of our editorial team, their recent responsibilities, and the current state of all software review issues. The EiC (or any editors who are interested) can gain an overview of the editorial team status, availability, and recent workloads on [the *editors* page](https://dashboard.ropensci.org/editors.html). This should be used to find and assign editors for new software review issues. -An overview of all current software reviews is on [the **Software Review* page](https://dashboard.ropensci.org/reviews.html). -Entries on this page are colored by a measure of "urgency", summarised in the [table at the bottom of that page](https://dashboard.ropensci.org/reviews.html#urgency-of-reviews). +An overview of all current software reviews is on [the *Software Review* page](https://dashboard.ropensci.org/reviews.html), with entries are colored by a measure of "urgency", summarised in the [table at the bottom of that page](https://dashboard.ropensci.org/reviews.html#urgency-of-reviews). -Specific tasks for reviews in the specific review stages include: +Tasks for reviews in the specific review stages include: -- Looking over submissions in "0\/presubmission" and "1\/editorial-team-prep", to check whether any action needs to be taken (such as polling editors, making decisions, putting issues on hold, pinging for updates, or finding and assigning editors). +- Look over submissions in "0\/presubmission" and "1\/editorial-team-prep", to check whether any action needs to be taken (such as polling editors, making decisions, putting issues on hold, pinging for updates, or finding and assigning editors). -- Looking over submissions in "2\/seeking-reviewer(s)" to ensure things are progressing quickly. -If the reviewer search has been going for unusually long (red color), check whether the submission is on hold, read the thread to gather context, and contact the editor in private to ask for more information. +- Look over submissions in "2\/seeking-reviewer(s)" to ensure things are progressing quickly. + If the reviewer search has been going for unusually long (red color), check whether the submission is on hold, read the thread to gather context, and contact the editor in private to ask for more information. -- Looking over submissions in "3\/reviewer(s)-assigned". If there are still missing reviews after an unusually long time (red color), check whether the submission is on hold, read the thread to gather context, and contact the editor in private to ask for more information. +- Look over submissions in "3\/reviewer(s)-assigned". + If there are still missing reviews after an unusually long time (red color), check whether the submission is on hold, read the thread to gather context, and contact the editor in private to ask for more information. -- Looking over submissions in "4\/review(s)-in-awaiting-changes". If some are still lacking an author response after an unusually long time (red color), check whether the submission is on hold, read the thread, and contact the editor in private to ask for more information. - -### Asking for more details {#asking-for-more-details} - -In some cases online documentation is sparse. Minimal README, no pkgdown website make assessment harder. -In that case please ask for more details: even if the package is deemed out-of-scope, the package docs will have gotten better so we are fine asking for these efforts. - -Example text - -```markdown -Hello and many thanks for your submission. - -We are discussing whether the package is in scope and need a bit more information. - -Would you mind adding more details and context to the README? -After reading it someone with little domain knowledge should have been informed about the aim, goals and functionality of the package. - - -If a package has overlapping functionality with other packages, we require it to demonstrate in the documentation [how it is best in class](https://devguide.ropensci.org/policies.html#overlap). Could you add a more detailed comparison to the packages you mention in the README so we can evaluate? - - -``` +- Look over submissions in "4\/review(s)-in-awaiting-changes". + If some are still lacking an author response after an unusually long time (red color), check whether the submission is on hold, read the thread, and contact the editor in private to ask for more information. ### Inviting a guest editor {#guesteditor} From b067302b9e4232743b0e04c35c471d08a79e05a3 Mon Sep 17 00:00:00 2001 From: mpadge Date: Tue, 2 Dec 2025 17:28:33 +0100 Subject: [PATCH 04/32] revise guest editor section for #487 --- softwarereview_editor.Rmd | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/softwarereview_editor.Rmd b/softwarereview_editor.Rmd index 86e40d561..b38f248db 100644 --- a/softwarereview_editor.Rmd +++ b/softwarereview_editor.Rmd @@ -89,6 +89,7 @@ Could you add a more detailed comparison to the packages you mention in the READ - Find a handling editor (potentially including yourself). Currently available editors are indicated on the [*Editorial Dashboard*](https://dashboard.ropensci.org), and editorial workloads should be distributed as evenly as possible, through referring to the [*Dashboard* charts of recent editorial load](https://dashboard.ropensci.org/editors.html#past-ed-load). + The EiC may also recruit a guest editor to handle any submission, as described in the [sub-section below](#guesteditor). - Assign the handling editor to the review issue by issuing the command: ``` @@ -131,16 +132,17 @@ Tasks for reviews in the specific review stages include: ### Inviting a guest editor {#guesteditor} -After discussion with other editors the EiC might invite a guest editor to handle a submission (e.g. if the submission volume is large, if all editors have a conflict of interest, if specific expertise is needed, or as a trial prior to inviting a person to join the editorial board). +After discussion with other editors the EiC may invite a guest editor to handle a submission. +Guest editors may be desired or needed if specific domain expertise is needed, if the current submission volume is large, if potential editors have conflicts of interest, or as a trial prior to inviting a person to join the editorial board. When inviting a guest editor, - Ask about conflicts of interest using the [same phrasing as for reviewers](#coi), - Give a link to the [guide for editors](#editorchecklist). -If the person said yes (yay!), +If a guest editor accepts an invitation (yay!), -- Make sure they [enabled 2FA for their GitHub account](https://help.github.com/articles/securing-your-account-with-two-factor-authentication-2fa/), +- Make sure they have [enabled 2FA for their GitHub account](https://help.github.com/articles/securing-your-account-with-two-factor-authentication-2fa/), - Invite them to the `ropensci/editors` team and to the ropensci organization, - Once they've accepted this repo invitation, assign the issue to them, - Ensure they're (already) invited to rOpenSci Slack workspace, From dfeca5b0efd840e2a5eb4cc24d8ba656760111aa Mon Sep 17 00:00:00 2001 From: mpadge Date: Wed, 3 Dec 2025 11:04:18 +0100 Subject: [PATCH 05/32] rm 'out-of'scope' section, merge into EiC section #487 --- softwarereview_editor.Rmd | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/softwarereview_editor.Rmd b/softwarereview_editor.Rmd index b38f248db..c788bd915 100644 --- a/softwarereview_editor.Rmd +++ b/softwarereview_editor.Rmd @@ -56,6 +56,9 @@ For each new pre-submission or submission, the EiC should: - If a pre-submission, referral, or submission is deemed out of scope, the EiC should thank authors for their submission, explain the reasons for the decision, and direct them to other publication venues if relevant. Where relevant, use wording from [_Aims and scope_](#aims-and-scope) regarding the evolution of scope over time. + + - [Examples of out-of-scope submissions and responses](https://github.com/ropensci/software-review/issues?q=is%3Aissue+is%3Aclosed+label%3Aout-of-scope). + - After explaining an out-of-scope decision, write an issue comment `@ropensci-review-bot out-of-scope`. - If a pre-submission inquiry is considered within scope, the EiC _may_ perform preliminary checks. @@ -153,17 +156,20 @@ After the review process is finished (package approved, issue closed), - Thank the guest editor again, - Remove them from the `ropensci/editors` team (but not from the ropensci organization). -## Editors' responsibilities {#editors-responsibilities} +## Editors' general responsibilities {#editors-responsibilities} -- In addition to handling packages (about 4 a year), editors weigh in on group editorial decisions, such as whether a package is in-scope, and determining updates to our policies. We generally do this through Slack, which we expect editors to be able to check regularly. +- In addition to handling submissions as described in the following section, editors weigh in on group editorial decisions, such as whether submissions are in-scope, and guiding updates to our policies. + We generally do this through Slack, which we expect editors to be able to check regularly. -- You only need to keep track of your own submissions, but if you do notice an issue with a package that is being handled by another editor, feel free to raise that issue directly with the other editor, or post the concern to editors-only channel on Slack. Examples: - - - You know of an overlapping package, that hasn't been mentioned in the process yet. +- You only need to keep track of your own submissions, but if you do notice an issue with a package that is being handled by another editor, feel free to raise that issue directly with the other editor, or post the concern to editors-only channel on Slack. + Examples might include: + + - You know of an overlapping package that hasn't been mentioned in the process yet. - You see a question to which you have an expert answer that hasn't been given after a few days (such as linking to a blog post which may answer a question). - - Concerns related to general review progress, including aspects such as the speed of the process, should be directed to the current Editor-in-Chief. -## Handling Editor's Checklist {#editorchecklist} +## Handling editors' responsibilities {#editorchecklist} + +Handling editors are responsible for guiding each assigned submission through the entire process from initial submission to final acceptance. ### Upon submission: {#upon-submission} @@ -276,12 +282,6 @@ For package authors who wish to retain their repositories in their original GitH - Direct the author to the chapters of the guide about [package releases](#releases), [marketing](#marketing) and [GitHub grooming](#grooming). -## Responding to out-of-scope submissions {#outofscoperesponse} - -Thank authors for their submission, explain the reasons for the decision, and direct them to other publication venues if relevant, and to the rOpenSci discussion forum. Use wording from [Aims and scope](#aims-and-scope) in particular regarding the evolution of scope over time, and the overlap and differences between unconf/staff/software-review development. - -[Examples of out-of-scope submissions and responses](https://github.com/ropensci/software-review/issues?q=is%3Aissue+is%3Aclosed+label%3Aout-of-scope). - ## Answering reviewers' questions {#reviewersupport} Reviewers might ask for feedback on e.g. the tone of their review. From ebfdfa5a04fefff4e1852b73707c84e38a150e96 Mon Sep 17 00:00:00 2001 From: mpadge Date: Wed, 3 Dec 2025 11:31:38 +0100 Subject: [PATCH 06/32] start revision of handling editor sections for #487 --- softwarereview_editor.Rmd | 50 ++++++++++++++++++++++++++------------- 1 file changed, 33 insertions(+), 17 deletions(-) diff --git a/softwarereview_editor.Rmd b/softwarereview_editor.Rmd index c788bd915..3687beaf9 100644 --- a/softwarereview_editor.Rmd +++ b/softwarereview_editor.Rmd @@ -170,42 +170,50 @@ After the review process is finished (package approved, issue closed), ## Handling editors' responsibilities {#editorchecklist} Handling editors are responsible for guiding each assigned submission through the entire process from initial submission to final acceptance. +Editors should be assigned no more than one issue per quarter-year, or maximum of four per year. ### Upon submission: {#upon-submission} -- Submission will automatically generate package check output from the ropensci-review-bot. The check results should be examined for any outstanding issues (most exceptions will need to be justified by the author in the particular context of their package). Checks can be re-run after any package change with the comment `@ropensci-review-bot check package`. +- Submission will automatically generate package check output from the ropensci-review-bot. + All check failures (❌) should be rectified prior to proceeding, although exceptions may be justified at the handling editor's discretion. + Encourage authors themselves to trigger checks by calling `@ropensci-review-bot check package` to confirm that all checks pass (✅). +- Examine all items flagged with 👀, and request further information or action from author where appropriate. - For statistical submissions (identifiable as "Submission Type: Stats" in issue template), add the "stats" label to the issue (if not already added). -- Check that the issue template has been properly filled out. Most common oversights and omissions should be caught and noted by the bot, but a manual check always helps. Editors can edit templates directly for minor fixes, or may direct authors to fill any mandatory template fields that may be missing. -- The checking system is rebuilt every Tuesday at 00:01 UTC, and can take a couple of hours. If automatic checks fail around that time, wait a few hours and try again. -- After automatic checks are posted, use the [editor template](#editortemplate) to guide initial checks and record your response to the submission. You can also streamline your editor checks by using the [`pkgreviewr` package created by associate editor Anna Krystalli](https://docs.ropensci.org/pkgreviewr/articles/editors.html). Please strive to finish the checks and start looking for reviewers within 5 working days. + - If the author has identified that they have "incorporated documentation of standards ... via the [srr](https://docs.ropensci.org/srr) package", then call `@ropensci-review-bot check srr` to confirm that at least 50% of all standards have been complied with. +- Check that the issue template has been properly filled out. Most common oversights and omissions should be caught and noted by the bot, but a manual check always helps. + Editors can edit templates directly for minor fixes, or may direct authors to fill any mandatory template fields that may be missing. +- The checking system is rebuilt every Tuesday at 00:01 UTC, and can take a couple of hours. + If automatic checks fail around that time, wait a few hours and try again. +- After automatic checks are posted, use the [editor template](#editortemplate) to guide initial checks (if not already covered by the EiC) and record your response to the submission. + You can also streamline your editor checks by using the [`pkgreviewr` package created by associate editor Anna Krystalli](https://docs.ropensci.org/pkgreviewr/articles/editors.html). + Please strive to finish the checks and start looking for reviewers within 5 working days. - Check against policies for [fit](#aims-and-scope) and [overlap](#overlap). Initiate discussion via Slack #software-review channel if needed for edge cases that haven't been caught by previous checks by the EiC. If rejected, see [this section](#outofscoperesponse) about how to respond. - Ensure that the package gets tested on multiple platforms (having the package built on several operating systems via GitHub Actions for instance; see [criteria in this section of the CI chapter](#whichci) for further details and options). -- Wherever possible when asking for changes, direct authors to automatic tools such as [usethis](https://usethis.r-lib.org/), [Air](https://posit-dev.github.io/air/) and [styler](https://styler.r-lib.org/), and to online resources (sections of this guide, sections of the [R packages book](https://r-pkgs.org/)) to make your feedback easier to use. See this [example of editor's checks](https://github.com/ropensci/software-review/issues/207#issuecomment-379909739). +- Wherever possible when asking for changes, direct authors to automatic tools such as [usethis](https://usethis.r-lib.org/), [Air](https://posit-dev.github.io/air/) and [styler](https://styler.r-lib.org/), and to online resources (sections of this guide, sections of the [R packages book](https://r-pkgs.org/)) to make your feedback easier to use. + See this [example of editor's checks](https://github.com/ropensci/software-review/issues/207#issuecomment-379909739). - Ideally, any remarks you make as editor should be addressed before assigning reviewers. -- If initial checks show major gaps, request changes before assigning reviewers. If the author mentions changes might take time, [apply the holding label by calling `@ropensci-review-bot put on hold`](#policiesreviewprocess). You'll get a reminder in the issue every 90 days to check in with the package author(s). +- If initial checks show major gaps, request changes before assigning reviewers. + If the author mentions changes might take time, [apply the holding label by calling `@ropensci-review-bot put on hold`](#policiesreviewprocess). + You'll get a reminder in the issue every 90 days to check in with the package author(s). - If the package raises a new issue for rOpenSci policy, start a conversation in Slack or open a discussion on the [rOpenSci forum](https://discuss.ropensci.org/) to discuss it with other editors ([example of policy discussion](https://discuss.ropensci.org/t/overlap-policy-for-package-onboarding/368)). ### Look for and assign two reviewers: {#look-for-and-assign-two-reviewers} -#### Tasks {#tasks} +#### Finding reviewers {#finding-reviewers} - Comment with `@ropensci-review-bot seeking reviewers`. - Use the [email template](#reviewrequesttemplate) if needed for inviting reviewers - When inviting reviewers, include something like "if I don't hear from you in a week, I'll assume you are unable to review," so as to give a clear deadline when you'll move on to looking for someone else. -- Assign reviewers with `@ropensci-review-bot assign @username as reviewer`. `add` can also be used instead of `assign`, and `to reviewers` (plural) instead of `as reviewer` (single). The following is thus also valid: `@ropensci-review-bot add @username to reviewers`. One command should be issued for each reviewer. If needed later, remove reviewers with `@ropensci-review-bot remove @username from reviewers`. -- If you want to change the due date for a review use `@ropensci-review-bot set due date for @username to YYYY-MM-DD`. - -#### How to look for reviewers {#how-to-look-for-reviewers} - -##### Where to look for reviewers? {#where-to-look-for-reviewers} + - You may send multiple invitations at the same time, especially as this often helps find reviewers more quickly. + You can always reply to say a reviewer has already been found, and thank people for offering regardless. -As a (guest) editor, use +Sources of information for finding reviewers include: -- the potential suggestions made by the submitter(s), (although submitters may have a narrow view of the types of expertise needed. We suggest not using more than one of the suggested reviewers); -- the Airtable database of reviewers and volunteers (see next subsection); -- and the authors of [rOpenSci packages](https://ropensci.org/packages/). +- Potential suggestions made by the submitter(s), (although submitters may have a narrow view of the types of expertise needed. We suggest not using more than one of the suggested reviewers); +- Our Airtable database of reviewers and volunteers (see next subsection); +- Authors of similar [rOpenSci packages](https://ropensci.org/packages/). When these sources of information are not enough, @@ -239,6 +247,14 @@ Here are criteria to keep in mind when choosing a reviewer. You might need to pi Each submission should be reviewed by *two* package reviewers. Although it is fine for one of them to have less package development experience and more domain knowledge, the review should not be split in two. Both reviewers need to review the package comprehensively, though from their particular perspective. In general, at least one reviewer should have prior reviewing experience, and of course inviting one new reviewer expands our pool of reviewers. +#### Assign reviewers {#assign-reviewers} + +- Assign reviewers with `@ropensci-review-bot assign @username as reviewer`. `add` can also be used instead of `assign`, and `to reviewers` (plural) instead of `as reviewer` (single). + The following is thus also valid: `@ropensci-review-bot add @username to reviewers`. + One command should be issued for each reviewer. + If needed later, remove reviewers with `@ropensci-review-bot remove @username from reviewers`. +- If you want to change the due date for a review use `@ropensci-review-bot set due date for @username to YYYY-MM-DD`. + ### During review: {#during-review} - Check in with reviewers and authors occasionally. Offer clarification and help as needed. From a9b3d7ae3631cd511e70a32e04a59faac891737e Mon Sep 17 00:00:00 2001 From: mpadge Date: Wed, 3 Dec 2025 11:47:05 +0100 Subject: [PATCH 07/32] update 'during review' section of editors chapter #487 --- softwarereview_editor.Rmd | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/softwarereview_editor.Rmd b/softwarereview_editor.Rmd index 3687beaf9..18d4bf8d7 100644 --- a/softwarereview_editor.Rmd +++ b/softwarereview_editor.Rmd @@ -249,31 +249,40 @@ Each submission should be reviewed by *two* package reviewers. Although it is fi #### Assign reviewers {#assign-reviewers} -- Assign reviewers with `@ropensci-review-bot assign @username as reviewer`. `add` can also be used instead of `assign`, and `to reviewers` (plural) instead of `as reviewer` (single). +- Assign each reviewer with `@ropensci-review-bot assign @username as reviewer`. `add` can also be used instead of `assign`, and `to reviewers` (plural) instead of `as reviewer` (single). The following is thus also valid: `@ropensci-review-bot add @username to reviewers`. One command should be issued for each reviewer. If needed later, remove reviewers with `@ropensci-review-bot remove @username from reviewers`. +- Due dates for reviews are set by default to [21 days](https://github.com/ropensci-org/buffy/blob/20c5ac630ce436a9cd7b698d217a0f680f368aa0/app/responders/ropensci/reviewers_due_date_responder.rb#L162-L164) (3 weeks) after the date of assignment. - If you want to change the due date for a review use `@ropensci-review-bot set due date for @username to YYYY-MM-DD`. ### During review: {#during-review} - Check in with reviewers and authors occasionally. Offer clarification and help as needed. -- In general aim for 3 weeks for review, 2 weeks for - subsequent changes, and 1 week for reviewer approval of changes. +- In general aim for 3 weeks for review, 2 weeks for subsequent changes, and 1 week for reviewer approval of changes. - Upon each review being submitted, - Write a comment thanking the reviewer in your own words. - - Record the review via typing a new comment `@ropensci-review-bot submit review time