[ADR] context + utcnow + fs

[utnapischtim]

problem description

• utcnow: is the deprecation warning and we move to utc aware timestamps.
• context: marshmallow deprecated the usage of self.context
• fs: pkg-resources is deprecated and the package fs still uses it and it seems that the package is unmaintained

utcnow

there was a discussion if we should keep the change from utcnow to something new backwards compatible. during the discussion we came to the conclusion that since the API offers a UTC aware timestamp the whole implementation should use a UTC aware timestamp.

so the main change is to move from datetime.utcnow to datetime.now(timezone.utc)
invenio-db provides now a Timestamp model base class which holds createdand updated with a new introduced UTCDateTime column type. this column type verifies that what goes into the database and what goes out has utc in it.

the new column type enforced some changes for flask-admin

context

for the self.context problem there two main solutions:

• context_schema which is a ContextVar variable which is set in ServiceSchemaWrapper replaces self.context for load and dump
• other configurations like is_parent, object_schema_cls etc are moved to the constructor of the schema where it is used

the problem is that it was easy to find a place where the context could be created for load and dump but it was difficult to find a place to set the context, if the variable was given to the constructor of a Schema. Most of the constructor context's are used directly in the schema itself

fs

• pkg-resources is deprecated with a endoflife date
• the package fs seems unmaintained so @mesemus implementd a solution for invenio-files-rest

PR's

utcnow

branch: fix-utcnow-with-custom-type
"invenio-db", # branch: add-utcdatetime-type PR
"flask-oauthlib-invenio",# tests are green PR
"flask-kvsession-invenio",# tests are green PR
"flask-security-invenio",# tests are green PR
"invenio-banners",# tests are green PR
"invenio-files-rest", # see combined with fs
"invenio-records",# tests are green PR
"invenio-stats", # tests failing <-- the only where tests are failing PR
"invenio-formatter",# tests are green PR
"invenio-jobs",# tests are green PR
"invenio-drafts-resources",# tests are green PR
"invenio-checks",# tests are green PR
"flask-iiif",# tests are green PR
"invenio-oaiserver",# tests are green PR
"invenio-oauth2server",# tests are green PR
"invenio-oauthclient",# tests are green PR
"invenio-audit-logs",# tests are green PR
"invenio-collections", PR
"invenio-accounts", PR
"invenio-cache", PR

combined

"invenio-files-rest", # utcnow-plus-fs-plus-context # tests green PR
"invenio-records-resources", # context-plus-fs-plus-utc # tests are green PR
"invenio-communities", # context-plus-utc # tests are green PR
"invenio-requests", # context-plus-utc # tests are green PR
"invenio-rdm-records", # context-plus-utc # tests are green PR
"invenio-rest", # context-plus-utc # tests are green PR
"invenio-users-resources", # context-plus-utc # tests working PR
"marshmallow-utils", # context-plus-utc # tests are green PR

fix context problem

fix: remove-context-usage
"invenio-rdm-records", # object_key # see combined
"invenio-communities", # object_key + identity + context_field_permission_check # see combined
"invenio-users-resources", # identity # tests working # see combined
"invenio-requests", # identity # tests working # see combined
"invenio-records-resources", # removing usage of context # see combined
"flask-resources", # object_schema_cls, object_key and providing for object_key and object_schema_cls # tests working PR
"marshmallow-utils", # context_field_permission_check # see combined

fix fs (pkg-resources deprecation)

"invenio-files-rest" # see combined

a pyproject.toml test file

to test the whole thing

[project]
name = "InvenioRDM"
requires-python = ">= 3.13"
dynamic = ["version"]


dependencies = [
  "invenio-app-rdm[opensearch2]~=14.0.b1.dev2",
  "uwsgi>=2.0",
  "uwsgitop>=0.11",
  "uwsgi-tools>=1.1.1",

  # utcnow
  # branch: fix-utcnow-with-custom-type
  "invenio-db", # branch: add-utcdatetime-type
  "flask-oauthlib-invenio",# tests are green
  "flask-kvsession-invenio",# tests are green
  "flask-security-invenio",# tests are green
  "invenio-banners",# tests are green
  #  "invenio-files-rest", # see combined with fs
  "invenio-records",# tests are green
  "invenio-stats", # tests failing
  "invenio-formatter",# tests are green
  "invenio-jobs",# tests are green
  "invenio-rest",# tests are green
  "invenio-drafts-resources",# tests are green
  "invenio-checks",# tests are green
  "flask-iiif",# tests are green
  "invenio-oaiserver",# tests are green
  "invenio-oauth2server",# tests are green
  "invenio-oauthclient",# tests are green
  "invenio-audit-logs",# tests are green
  "invenio-collections",
  "invenio-accounts",

  # combined
  "invenio-files-rest", # utcnow-plus-fs-plus-context # tests green
  "invenio-records-resources", # context-plus-fs-plus-utc # tests are green
  "invenio-communities", # context-plus-utc # tests are green
  "invenio-requests", # context-plus-utc # tests are green
  "invenio-rdm-records", # context-plus-utc # tests are green
  "marshmallow-utils", # context-plus-utc

  # fix context problem
  # fix: remove-context-usage
  # "invenio-records-lom", # included into branch: fix-demo, object_key
  # "invenio-app-rdm", # identity
  # "invenio-rdm-records", # object_key
  #  "invenio-communities", # object_key + identity + context_field_permission_check # tests working
  "invenio-users-resources", # identity # tests working
  # "invenio-requests", # identity # tests working
  # "invenio-access", # identity # tests working
  # "invenio-records-resources", # removing usage of context # see combined
  # "flask-celeryext", # identity for tasks # not necessary anymore, because of context_schema
  "flask-resources", # object_schema_cls, object_key and providing for object_key and object_schema_cls # tests working
  # "marshmallow-utils", # context_field_permission_check # tests working
]

[tool.setuptools]
py-modules = []

[tool.uv.sources]
flask-iiif = { git = "git+https://github.com/utnapischtim/flask-iiif", branch = "fix-utcnow-with-custom-type" }
flask-kvsession-invenio = { git = "git+https://github.com/utnapischtim/flask-kvsession", branch = "fix-utcnow-with-custom-type" }
flask-oauthlib-invenio = { git = "git+https://github.com/utnapischtim/flask-oauthlib-invenio", branch = "fix-utcnow-with-custom-type" }
flask-resources = { git = "git+https://github.com/utnapischtim/flask-resources", branch = "remove-context-usage" }
flask-security-invenio = { git = "git+https://github.com/utnapischtim/flask-security-invenio", branch = "fix-utcnow-with-custom-type" }

invenio-accounts = { git = "git+https://github.com/utnapischtim/invenio-accounts", branch = "fix-utcnow-with-custom-type" }
invenio-audit-logs = { git = "git+https://github.com/utnapischtim/invenio-audit-logs", branch = "fix-utcnow-with-custom-type" }
invenio-banners = { git = "git+https://github.com/utnapischtim/invenio-banners", branch = "fix-utcnow-with-custom-type" }
invenio-check = { git = "git+https://github.com/utnapischtim/invenio-check", branch = "fix-utcnow-with-custom-type" }
invenio-collections = { git = "git+https://github.com/utnapischtim/invenio-collections", branch = "fix-utcnow-with-custom-type" }
invenio-communities = { git = "git+https://github.com/utnapischtim/invenio-communities", branch = "context-plus-utc" }
invenio-db = { git = "git+https://github.com/utnapischtim/invenio-db", branch = "add-utcdatetime-type" }
invenio-drafts-resources = { git = "git+https://github.com/utnapischtim/invenio-drafts-resources", branch = "fix-utcnow-with-custom-type" }
invenio-files-rest = { git = "git+https://github.com/utnapischtim/invenio-files-rest", branch = "utcnow-plus-fs-plus-context" }
invenio-formatter = { git = "git+https://github.com/utnapischtim/invenio-formatter", branch = "fix-utcnow-with-custom-type" }
invenio-jobs = { git = "git+https://github.com/utnapischtim/invenio-jobs", branch = "fix-utcnow-with-custom-type" }
invenio-oaiserver = { git = "git+https://github.com/utnapischtim/invenio-oaiserver", branch = "fix-utcnow-with-custom-type" }
invenio-oauth2server = { git = "git+https://github.com/utnapischtim/invenio-oauth2server", branch = "fix-utcnow-with-custom-type" }
invenio-oauthclient = { git = "git+https://github.com/utnapischtim/invenio-oauthclient", branch = "fix-utcnow-with-custom-type" }
invenio-rdm-records = { git = "git+https://github.com/utnapischtim/invenio-rdm-records", branch = "context-plus-utc" }
invenio-records = { git = "git+https://github.com/utnapischtim/invenio-records", branch = "fix-utcnow-with-custom-type" }
invenio-records-resources = { git = "git+https://github.com/utnapischtim/invenio-records-resources", branch = "context-plus-fs-plus-utc" }
invenio-requests = { git = "git+https://github.com/utnapischtim/invenio-requests", branch = "context-plus-utc" }
invenio-rest = { git = "git+https://github.com/utnapischtim/invenio-rest", branch = "context-plus-utc" }
invenio-stats = { git = "git+https://github.com/utnapischtim/invenio-stats", branch = "fix-utcnow-with-custom-type" }
invenio-users-resources = { git = "git+https://github.com/utnapischtim/invenio-users-resources", branch = "context-plus-utc" }

marshmallow-utils = { git = "git+https://github.com/utnapischtim/marshmallow-utils", branch = "context-plus-utc" }

[utnapischtim]

the open PR's are merged and released