Period Types as time interval sequences

[jbee]

Motivation

ATM periods and period types are pretending to work within any calendar system. The calendar used is a system setting (that can never change once used). This IMO is a confusion around the concept of time "buckets" (slicing up a continuous timeline into a sequence of intervals). A bucketing system such as "week of year" is very tightly connected to one calendar system. Other calendars might have a similar concept that is also similarly named but these would be 2 distinct sequences that have nothing in common other than the terms we use to refer to them. Specifically any ISO conform period is always a Gregorian calendar period. In contrast a weekly period in the traditional Chinese calendar would have 10 day weeks in a variable length Year. This is an entirely different sequence to the ISO weeks. It does not make sense for these 2 to share any code or for data of the two to be compared to each other. Attempting to do so has lead to the very complicated and fragile date related code in DHIS2 where the attempt to make it "multi-calendar ready" has moved it to a state that is inconsistent, incorrect and incomplete and very hard to reason about.

The complication is further stressed by the need for custom period types. These are essentially sequences that do not follow a calendar but a lookup table of decided dates and names for the intervals they hold. A classic example would be a school year with Spring term, Autumn term and alike.

Last but not least, the idea that all periods in a DHIS2 system must follow a single calendar is wrong on many levels. It is an unnecessary limitation, source of great confusion and risk to break the data. Imagine a country now using non-ISO weeks but entering a transition phase to join the ISO world. Such a system would clearly have multiple weekly (start Monday) types. The non-ISO one for old existing data and ISO weeks for new data. This cannot be solved in a system where one calendar rules them all.
In contrast, it is easy to see that data points entered for time intervals belonging to a specific calendar should never change their semantics. The semantics implied by the type and its implied calendar is what anchors the data in time.

Solution Outline

• remove the calendar setting and replace it with a selection setting where users select the period types they want in their system out of the offered range of sequences we have implemented. This changes period types from being a matrix of type and calendar to be a list where a calendar is implied in each type.
• make any period type a sequence within a specific calendar system, e.g. Gregorian week of the year (a.k.a. ISO week)
• make the date string (ATM wrongly called "ISO period") the external ID of a period and as such the canonical identifier (as opposed to the date interval and type)
• create a minimal interface for a sequence that has the essential functions
  • now (creates the period that contains "now" for the type)
  • date interval => ID string
  • ID string => date interval
  • next (period after a given one)
  • previous (period after a given one)
  • typical length (replaces getFrequencyOrder, e.g. week = 7)
  • infix (the letters following the year to identify the type)

Custom Period Types

A custom type is essentially a lookup table with the columns

• sequence number (1, 2, ...)
• string ID
• first day (year, month, day)
• last day (year, month, day)

The main challenge with custom period types is to make sure the ID string is unique and does not collide with the identification of other period types. At the same time it should be as readable and self explanatory as possible. It seems reasonable to limit custom periods to such sequences that split a year into intervals. Meaning they also can follow the general pattern YYYY$? where $ marks the infix that is unique for a period type. For example, for a school term type, the infix may be ST, a term can be written as 2026ST1, 2026ST2 and so forth. Like in this example, digits can follow the infix.

Prototype

enum PeriodType {
  WEEKS(new GregorianWeeklyPeriodSequence())
  //...
 ;
 final PeriodSequence sequence;
}

record FixPointDay(int daysSince2000) {}
record SpanOfDays( FixPointDay first, FixPointDay last) {}

interface PeriodSequence {
  String infix();
  
  SpanOfDays decode(String period);
  
  String encode(SpanOfDays period);

  SpanOfDays now();

  SpanOfDays next(SpanOfDays period);
  
  SpanOfDays previous(SpanOfDays period);
  
  int lengthInDays();
}

Timestamps

As the prototype indicates periods and period types or sequences can and should work without a link to an absolute timeline (like a timestamp would provide). They track days independent of when exactly such a day would have started on an absolute timeline somewhere. The reason is simply that the start of day on a absolute timeline is always linked to a place or timezone. This confuses and complicates the world or time intervals with the world of time-zones. Such complication is entirely unnecessary. Instead everything can be described in days since a fix-point. The idea is similar to a timestamp except that the fix-point is a local date without time zone.
This also has the advantage of being able to calculate the length of a period in days without understanding the calendar. It is simply the difference between the last and the first day fix-point offset. In addition it is trivial for most types to advance the next interval in the sequence.

Ideally DHIS2 would have used such a model for periods from the beginning but unfortunately we have to deal with the past of start and end dates being stored as timestamps in the database. While a big task, it is an option to also refactor the database. More realistically we maintain timestamps by making them absolute assuming the servers time-zone to compute the corresponding timestamps.

Time-zones

By its nature aggregate numbers for a span of days does not have to deal with time-zones. While it did matter for the counting to consider when days start and end once the number for a day or a span of days is counted and then reported to DHIS2 the system does not have to care about absolute time. Users will usually select days with a date picker specifying the day or day span independent of time-zones. There might be special cases where the API does not require the user to give a day but will implicitly assume that the current one is to pick. This is where time-zones matter when it is determined what the current day is in the calendar that is implied in a period sequence. Maybe we can do without such a feature. It is unclear if such a period creation is used beyond tests.