Fix ignore_existing=True for mirroring Digital Metadata files

.github/workflows/cd.yml

@@ -13,6 +13,9 @@ concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
 
+permissions:
+  contents: write
+
 env:
   # Many color libraries just need this to be set to any value, but at least
   # one distinguishes color depth, where "3" -> "256-bit color".

news/fix_ignore_existing.rst

@@ -0,0 +1,23 @@
+**Added:**
+
+* <news item>
+
+**Changed:**
+
+* <news item>
+
+**Deprecated:**
+
+* <news item>
+
+**Removed:**
+
+* <news item>
+
+**Fixed:**
+
+* Fix ignore_existing=True when mirroring Digital Metadata files. Previously, ignore_existing=True would ignore all existing Digital Metadata files, which would skip mirroring the most recent file(s) containing metadata that still applies through forward-fill logic. This modifies the ignore_existing=True behavior to *not* ignore the latest existing Digital Metadata files.
+
+**Security:**
+
+* <news item>

python/digital_rf/mirror.py

@@ -365,6 +365,26 @@ def start(self):
                     include_dmd_properties=False,
                 )
                 paths = chain(paths, more_paths)
+            elif self.include_dmd:
+                # mirror dmd files that still apply currently (most recent)
+                # do this by setting starttime to max possible (or endtime) which
+                # through forward-fill logic of DMD matching will get the last files
+                if self.endtime is not None:
+                    lasttime = self.endtime
+                else:
+                    lasttime = datetime.datetime.max.replace(
+                        tzinfo=datetime.timezone.utc
+                    )
+                more_paths = list_drf.ilsdrf(
+                    self.src,
+                    starttime=lasttime,
+                    endtime=self.endtime,
+                    include_drf=False,
+                    include_dmd=self.include_dmd,
+                    include_drf_properties=False,
+                    include_dmd_properties=False,
+                )
+                paths = chain(paths, more_paths)
 
             # add events for existing files, skipping observer and event queue
             # and dispatching directly so we can dispatch from this thread