Decouple linting, formatting, and pre-commit from external dotfiles

[mark-pitblado]

Fixes #7402

This PR removes the link between Max Platiiuk's dotfiles and the Specify repository for formatting, linting and the pre-commit config. I have tried to stay as close to a copy and paste operation as possible, however the following two changes were made:

1. Previously, flake was pulling from a repository on GitLab, which required a login when I was testing with pre-commit. Since developers and testers for Specify are developing on GitHub, it seemed to make sense to switch to a GitHub based repo so that no additional setup is required.
2. The markdownlint used in pre-commit required ruby during my testing. Since ruby isn't one of the languages used by the Specify project, it seemed prudent to swap this out for a markdownlint based on nodejs. Markdownlinters should be pretty similar, there are very few rules for markdown.

Eslint is definitely where things get the most complicated, particularly because I don't think eslint has been a part of the ci pipeline for a while due to the high number of errors it produces?

specify7/.github/workflows/test.yml

Lines 270 to 274 in cbff898

	run: |
	# TODO: Once most errors are fixed remove "set +e" to not ignore errors
	set +e
	npx eslint --fix --color `echo "${{steps.changed.outputs.changed}}" | tr " " "\n"`
	set -e

I think it may be prudent to think about where in the pipeline each type of error is expected to be caught, and which tools are mandatory to setup. For example, if a developer has pre-commit setup, are they good to go? Or should they also run npm run test on their machine? Which jobs are handled by the ci runners, compared to local testing? A good candidate for developer documentation.

Note

Gitguardian has a pre-commit config that can catch secrets before they are commited as a pre-commit check (or pushed as a pre-push check), whereas I believe the PR integration will only catch the secret after it has been committed (it can be then rotated/revoked, but that's a pain). An example of a case where it matters where a check is run.

Dependency changes

The following dependencies were added to the Specify frontend. These were always dependencies for the project, they were just further upstream because they were dependencies for the dotfiles.

• prettier-plugin-sh
• prettier-plugin-package
• prettier-xml

Checklist

• Self-review the PR after opening it to make sure the changes look good and
  self-explanatory (or properly documented)

Testing instructions

• These changes are meta-changes, and shouldn't change any functionality of application code. The only test to be done is to run the ci and pre-commit pipelines and observe that there are no big changes suggested for files that have already been through the pipeline. Note that running the existing pre-commit configuration over the entire codebase will result in a lot of changes, it appears like many directories have never been touched by pre-commit (perhaps because they haven't seen changes in a while, or pre-commit is not consistently used).

[mark-pitblado]

04cc77c installs the latest versions of prettier and the plugins, don't see a reason not to use the latest version, as these are just formatters.

[grantfitzsimmons]

Thank you so much for doing this! @CarolineDenis, we should discuss what changes need to be made to address

Note that running the existing pre-commit configuration over the entire codebase will result in a lot of changes, it appears like many directories have never been touched by pre-commit (perhaps because they haven't seen changes in a while, or pre-commit is not consistently used).

[mark-pitblado]

This pull request isn't ready to merge just yet, mostly because I'm trying to work out a proposal for what to do with eslint. The current config is using a version of eslint (version 8) that is depracated. However, just switching back to the recommended configuration¹ will litter the codebase with comment strings that say to ignore a previous rule. In a perfect world I could get all the plugins and rules to line up.

As far as I can tell, the following are done at each stage of the pipeline:

Stage	Testing
Pre-commit	Runs frontend jest tests, regex testing, python typechecking, python linting, python formatting (black), prettier formatting, doctstring linting, markdown linting, docker compose linting.
On PR	GitGuardian (secret checking)
CI Workers	Eslint, prettier formatting, frontend tests, backend tests

Currently, the only thing that eslint will do is autofix what it can (which isn't supported for all rules, just some). Errors are ignored for both prettier and eslint in the pipeline, presumably because they were too restrictive at some time. Warnings are... well just warnings.

# TODO: Once most errors are fixed remove "set +e" to not ignore errors

Example: https://github.com/specify/specify7/actions/runs/18353184533/job/52278324256

I don't think stylelint is used anywhere, but there is a config for it, I guess if an individual developer wanted to set it up in their IDE they could use it.

Footnotes

1. Since the current ruleset is being ignored, I think it would be better to pull back to just the recommended config and see if that is something that should be adhered too, but that's just my two cents. Using the recommended would reduce dependencies by eliminating the need for all the plugins. ↩