Skip to content

Commit 39fa69c

Browse files
authored
Merge pull request #344 from lwasser/ai-policy
enh(policy): add Gen AI policy to review guide
2 parents 83e0ed1 + 2b2eb10 commit 39fa69c

File tree

4 files changed

+86
-31
lines changed

4 files changed

+86
-31
lines changed

appendices/editor-in-chief-checks.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,12 @@ to work on them before the review process begins.
99
Please check our [Python packaging guide](https://www.pyopensci.org/python-package-guide) for more information on the elements
1010
below.
1111

12-
- [ ] **Installation** The package can be installed from a community repository such as PyPI (preferred), and/or a community channel on conda (e.g., conda-forge, bioconda).
12+
- [ ] **Installation** The package can be installed from a community repository such as PyPI (preferred), and/or a community channel on conda (for example, conda-forge, bioconda).
1313
- [ ] The package imports properly into a standard Python environment `import package`.
1414
- [ ] **Fit** The package meets criteria for [fit](https://www.pyopensci.org/software-peer-review/about/package-scope.html#what-types-of-packages-does-pyopensci-review) and [overlap](https://www.pyopensci.org/software-peer-review/about/package-scope.html#package-overlap).
1515
- [ ] **Documentation** The package has sufficient online documentation to allow us to evaluate the package's function and scope *without installing the package*. This includes:
1616
- [ ] User-facing documentation that overviews how to install and start using the package.
17-
- [ ] Short quickstart tutorials that help a user understand how to use the package and what it can do for them.
17+
- [ ] Short, quickstart tutorials that help a user understand how to use the package and what it can do for them.
1818
- [ ] API documentation (documentation for your code's functions, classes, methods, and attributes): this includes clearly written docstrings with variables defined using a standard docstring format.
1919
- [ ] Core GitHub repository Files
2020
- [ ] **README** The package has a `README.md` file with a clear explanation of what the package does, instructions on how to install it, and a link to development instructions.
@@ -28,13 +28,14 @@ NOTE: We prefer that you have development instructions in your documentation too
2828
- [ ] **Package overlap** The package doesn't entirely overlap with the functionality of other packages that have already been submitted to pyOpenSci.
2929
- [ ] **Archive** (JOSS only, may be post-review): The repository DOI resolves correctly.
3030
- [ ] **Version** (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?
31+
- [ ] The package authors have disclosed the use of Generative AI tools in the development and/or maintenance of their package.
3132

3233
**Optional:** Let projects know that it's a great idea for projects to have a .github repository for the project organization where they can host a commonly used LICENSE, Code of Conduct, and even a YAML file with label definitions. These items will then be automatically applied to every repository in the organization to ensure consistency (but can be customized within repos too). The [SunPy project](https://github.com/sunpy/.github/) has a great example of this.
3334

3435
---
3536
- [ ] [Initial onboarding survey was filled out ](https://forms.gle/F9mou7S3jhe8DMJ16)
3637
We appreciate each maintainer of the package filling out this survey individually. :raised_hands:
37-
Thank you, authors, in advance for setting aside five to ten minutes to do this. It truly helps our organization. :raised_hands:
38+
A big thank you in advance to authors for setting aside five to ten minutes to do this. It truly helps our organization! :raised_hands:
3839
---
3940

4041
*******

appendices/gen-ai-checklist.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
```markdown
2+
- [ ] Generative AI was used to produce some of the material in this submission.
3+
- [ ] If generative AI was used in this project, the authors affirm that all generated material has been reviewed and edited for clarity, correctness, and completeness. The authors are responsible for the content of their work and affirm that it is in a state where reviewers will not be responsible for primary editing and review of machine-generated material.
4+
5+
If you checked the first box above, please describe how generative AI was used, including:
6+
7+
- **Which parts** of the submission were generated (e.g., documentation, tests, code). In addition to a general description, please specifically indicate any substantial portions of code (classes, modules, subpackages) that were wholly or primarily generated by AI.
8+
- **The approximate scale** of the generated portions (e.g., "all of the tests were generated and then checked by a human," "small routines were generated and copied into the code").
9+
- **How the generative AI was used** (e.g., line completion, help with translation, queried separately and integrated, agentic workflow).
10+
11+
If you have a policy around generative AI use in your project, please provide a link to it below:
12+
13+
- _Your link here (remove this line if you don't have a link)_
14+
```

how-to/author-guide.md

Lines changed: 16 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ Curious about the general timeline for pyOpenSci reviews?
6666
:link-type: doc
6767
:class-card: left-aligned
6868

69-
Read about our peer review policies.
69+
Read through our peer review policies before submitting a package to us.
7070
:::
7171
::::
7272

@@ -87,9 +87,10 @@ Before you begin this process, [please be sure to read the review process guidel
8787
8888
1. Please be sure that you have time to devote to making changes to your
8989
package. During review, you will receive feedback from an editor and two reviewers. Changes could
90-
take time. Please consider this before submitting to us. You can read more about the timeline to make changes in our [peer review policies page](../our-process/policies).
90+
take time. Please consider this before submitting. You can read more about the timeline for making changes on our [peer review policies page](../our-process/policies).
9191
2. A diverse group of volunteer editors and reviewers leads peer review.
9292
Please be considerate when engaging with everyone online.
93+
3. All participants in our peer review process need to follow the [pyOpenSci Code of Conduct](https://www.pyopensci.org/handbook/CODE_OF_CONDUCT.html). Please review it before submitting to us.
9394
```
9495

9596
## 1. Do you plan to continue to maintain your package?
@@ -129,7 +130,7 @@ emphasis on long-term software maintenance and focuses more on
129130
publication quality and citation/credit.
130131
```
131132

132-
## 2. Does Your Package Meet Packaging Requirements?
133+
## 2. Does your package meet packaging requirements?
133134

134135
Before submitting your project for review with pyOpenSci, make sure that
135136
your package meets all of the requirements listed in the editor checks (see below).
@@ -148,6 +149,11 @@ for Python packaging, including discussions of:
148149
149150
```
150151

152+
:::{important}
153+
Given the increased use of Generative AI tools (LLMs), we have developed a disclosure policy for all packages submitted to pyOpenSci (generative-ai-policy). Please review this before submitting to us.
154+
:::
155+
156+
151157
```{hint}
152158
**Do you have questions about Python packaging or our peer review process?**
153159
@@ -170,7 +176,7 @@ that you can use to create a high-quality
170176
Python package.](https://www.pyopensci.org/python-package-guide/package-structure-code/python-package-build-tools.html)
171177
```
172178

173-
## 3. Is Your Package in Scope for pyOpenSci?
179+
## 3. Is your package in scope for pyOpenSci?
174180

175181
Next, check to see if your package falls within the topical and technical scope of pyOpenSci. If you aren't
176182
sure about whether your package fits within pyOpenSci's scope (below), submit
@@ -188,13 +194,13 @@ Our current categories for determining package scope are below:
188194
Click here to view our technical and domain scope requirements.
189195
```
190196

191-
## 4. Submit Your Package for Peer Review
197+
## 4. Submit your package for peer review
192198

193199
To submit your package for peer review, you can
194200
open an issue in our [pyopensci/software-review repo](https://github.com/pyOpenSci/software-review/issues/new/choose/)
195201
repository and fill out the [Submit Software for Review](https://github.com/pyOpenSci/software-submission/issues/new?template=submit-software-for-review.md) issue template.
196202

197-
## 5. Editor-in-Chief Reviews Package for Scope and Minimal Infrastructure Criteria
203+
## 5. Editor-in-Chief reviews package for scope and minimal infrastructure criteria
198204

199205
Once the issue is opened, our editor-in-chief and an editor from our editorial board will review your submission within
200206
**2 weeks** and respond with next steps. The editor may request that you make updates
@@ -207,7 +213,7 @@ package if it does not fall within our scope.
207213
Click here to view the editor checks that will be used to evaluate your package.
208214
```
209215

210-
## 6. The Review Begins
216+
## 6. The review begins
211217

212218
If your package meets the minimal criteria for being
213219
reviewed, it may then be given to an editor with appropriate domain experience
@@ -217,7 +223,7 @@ issue within **3 weeks**. Reviewers can also open issues in your package reposit
217223
We prefer issues that link back to the review as they document changes made to your
218224
package that were triggered by our review process.
219225

220-
## 7. Response to Reviews
226+
## 7. Response to reviews
221227

222228
You should respond to reviewers’ comments within **2 weeks** of the
223229
last-submitted review. You can make updates to your package at any time. We
@@ -231,7 +237,7 @@ Once the reviewers are happy with the changes that you've made to the package, t
231237
editor will review everything and accept your package into the pyOpenSci ecosystem.
232238
Congratulations! You are almost done!
233239

234-
## My Package is Approved, Now What?
240+
## My package is approved, now what?
235241

236242
Congratulations on being accepted into the pyOpenSci community of maintainers!
237243
Once your package is approved, a few things will happen:
@@ -246,7 +252,7 @@ Once your package is approved, a few things will happen:
246252

247253
If you'd like to submit your package to JOSS, you can do so now. Remember that JOSS will accept our review as theirs, so **you DO NOT need to go through another review**. Read more below.
248254

249-
### Journal of Open Source Software (JOSS) Submission
255+
### Journal of Open Source Software (JOSS) submission
250256

251257
pyOpenSci has a [partnership with JOSS](JOSS), where our review is accepted by JOSS by
252258
default if the package fits into the JOSS scope.

our-process/policies.md

Lines changed: 52 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -3,15 +3,15 @@
33
## Review process guidelines
44

55
pyOpenSci packages are reviewed for quality, fit, scope, documentation, and
6-
usability. The review process is similar to a manuscript review, however, it
6+
usability. The review process is similar to a manuscript review; however, it
77
has a stronger focus on Python packaging best practices.
88

99
Unlike a manuscript review, our peer review process is an ongoing conversation.
10-
Once all major issues and questions are addressed, the review editor will make
11-
a decision to accept, hold, or reject the package.
10+
Once all major issues and questions are addressed, the review editor will decide
11+
to accept, hold, or reject the package.
1212

1313
Rejections are usually done early in the process, before the review process
14-
begins. In rare cases, a package may also not be on-boarded into the pyOpenSci
14+
begins. In rare cases, a package may also not be onboarded into the pyOpenSci
1515
ecosystem after review & revision.
1616

1717
It is ultimately the editor’s decision on whether or not to reject the package
@@ -20,26 +20,27 @@ based on how the reviews are addressed.
2020
## Review communication approach
2121

2222
Communication between authors, reviewers, and editors takes
23-
place on GitHub. You can, however choose to contact the editor by email if
23+
place on GitHub. However, you may contact the editor by email if
2424
needed.
2525

2626
When submitting a package, please make sure that your GitHub notification
27-
settings are setup to notify you when you receive feedback on the review issue.
27+
settings are turned on for the software-submission repository to notify you
28+
when you receive feedback on a review issue.
2829

2930
## Submitting your package for review in other venues
3031

3132
We recommend submitting your package for review with pyOpenSci before
3233
submitting a software paper describing the package to a journal.
3334

34-
Review feedback may result in major improvements and updates to your package,
35-
including changes that could be break package functionality.
35+
Review feedback may result in significant improvements and updates to your package,
36+
including changes that could break package functionality.
3637

3738
Applying reviewer or editor recommendations to your package can improve your
38-
users' experience with future versions of your package even if your package is
39+
users' experience with future versions of your package, even if your package is
3940
already published on `PyPI` or `conda-forge`.
4041

4142
> Please do not submit your package for review while it or an associated
42-
> manuscript is also under review at another venue, as this may result on
43+
> manuscript is also under review at another venue, as this may result in
4344
> conflicting requests for changes from two sets of reviewers.
4445
4546
### Publication with Journal of Open Source Software (JOSS)
@@ -105,6 +106,37 @@ a conflict of interest if:
105106
In the case where none of the associate editors can serve as editor, an
106107
external guest editor will be recruited to lead the package review.
107108

109+
(generative-ai-policy)=
110+
111+
## Policy for use of generative AI / LLMs
112+
113+
:::{admonition} How this policy was developed
114+
:class: important
115+
116+
The policy below was co-developed by the pyOpenSci community. Its goals are:
117+
118+
- Acknowledgment of and transparency around the widespread use of Generative AI tools, with a focus on Large Language Models (LLMs) in Open Source development.
119+
- Ensure an equitable balance of effort in the peer review process: Authors acknowledge that a human has carefully reviewed parts of the package that are AI-generated. Generated material should be in a state that minimizes review time. Our reviewers are not responsible for correcting errors in machine-generated content.
120+
- Disclosure allows reviewers and editors to make informed decisions around the types of packages that they wish to review code for.
121+
- Raise awareness of challenges that Generative AI tools present to the scientific (and broader) open source community.
122+
123+
[Please see this GitHub issue for a discussion of the topic.](https://github.com/pyOpenSci/software-peer-review/issues/331)
124+
:::
125+
126+
### Disclosure of generative AI use in pyOpenSci reviewed packages
127+
128+
- When you submit a package to pyOpenSci, please disclose any use of LLMs (Large Language Models) in your package's development by checking the appropriate box and describing your use of generative AI in its development and/or maintenance on our software submission form. Disclosure should include what parts of your package were developed using Generative AI tools.
129+
- Please also disclose this use of Generative AI tools in your package's `README.md` file and in any modules where generative AI contributions have been implemented.
130+
- We require that all aspects of your package have been reviewed carefully by a human on your maintainer team. Please ensure all text and code have been carefully checked for bugs and issues before submitting to pyOpenSci.
131+
- Your acknowledgment of using Generative AI will not prejudice the success of your submission. However, a reviewer can and will ask you to revisit your package's content if it appears that sections have been copied and pasted from other sources without human review.
132+
- If the review team (comprised of the editor and reviewers) determines that the code and text in the package are too challenging to review, they can decide to pause and/or discontinue the review following this policy’s guidelines.
133+
134+
Below is the checklist that you will need to respond to in our submission form:
135+
136+
```{include} ../appendices/gen-ai-checklist.md
137+
138+
```
139+
108140
## Review timelines and on-hold reviews
109141

110142
At any time, an author can choose to have their submission put on hold
@@ -143,9 +175,9 @@ including issues, pull requests, and dates of the last release and last commit
143175
to the package repository. Activity is defined as a repository commit, pull
144176
request, or release.
145177

146-
We will flag packages that haven't been updated within a 1 year/ 12 month time
178+
We will flag packages that haven't been updated within a 1-year/12-month time
147179
period based on activity. Packages with no activity after 12 months will be
148-
flagged. At that time, pyOpenSci editorial team member will contact the package
180+
flagged. At that time, a pyOpenSci editorial team member will contact the package
149181
maintainers to evaluate the maintenance status of their package.
150182

151183
(archive-process)=
@@ -160,15 +192,15 @@ ecosystem.
160192
In cases where the community heavily uses a package, we may
161193
collaborate with the community to identify reasonable next steps, such as
162194
assisting in finding a new maintainer. If a solution for the ongoing package
163-
maintenance is not found, the package will be archived within the pyOpenSci
164-
ecosystem.
195+
maintenance of your package is not found, the package will be archived within
196+
the pyOpenSci ecosystem. [See section on archiving below.](package-archive)
165197

166198
If a sub-community decides to fork and maintain the package, we are open to
167199
working with the new maintainers to register the newly forked package within
168200
our ecosystem. The original package will be archived with a link to the new
169201
fork.
170202

171-
We will also add a note to any blogs written that highlight your tool that
203+
We will also add a note to any blogs written that highlight your tool, that
172204
the package is no longer maintained.
173205

174206
### Quality commitment
@@ -186,19 +218,21 @@ in touch with us if they do need to step down from maintaining a tool.
186218
### Requesting Package Removal from the pyOpenSci Ecosystem
187219

188220
In the unlikely scenario that a contributor of a package requests removal of
189-
their package from our ecosystem, we retain the right offer the last / most
221+
their package from our ecosystem, we retain the right to offer the last/most
190222
recently released version of that package in our ecosystem for archival
191223
purposes only.
192224

225+
(package-archive)=
226+
193227
### Archiving a package
194228

195229
If a package appears to be no longer maintained, we will mark it as
196-
archived which moves the package from our
230+
archived, which moves the package from our
197231
[main package listing](https://www.pyopensci.org/python-packages.html#all-packages)
198232
to our [archived packaging](https://www.pyopensci.org/python-packages.html#archived-packages)
199233
listing section.
200234

201-
To archive a pyOpenSci approved package, add the
235+
To archive a pyOpenSci-approved package, add the
202236
[archive label](https://github.com/pyOpenSci/software-submission/issues?q=label%3Aarchived)
203237
to the original review issue. Once this label is applied to the issue, the
204238
website will automatically update to reflect this status. If at any point

0 commit comments

Comments
 (0)