Future Tariff Updates Fail Due to NULL Emergency Credit Values

[damonrand]

Problem Summary

Future tariff updates are failing when customer-specific tariffs are created without corresponding microgrid_tariffs entries. The job crashes attempting to convert NULL emergency credit values to Decimal types.

Current Behavior

• Customer creates a tariff for Sep 1, 2025
• No corresponding microgrid_tariffs entry exists for Sep 1
• The meters_missing_future_tariffs RPC returns NULL for emergency credit fields
• Python code crashes with: decimal.InvalidOperation: [<class 'decimal.ConversionSyntax'>]
• Tariff update fails completely, leaving old tariff active on meter

Root Cause

The system assumes emergency credit values always exist in microgrid_tariffs table, but:

1. Individual customer tariffs can be created without microgrid tariffs
2. The SQL function returns NULL when no future microgrid_tariffs exists
3. The Python code doesn't handle NULL values: Decimal(str(None)) fails

Evidence from Logs

Aug 30 02:20:10 - Job found meter EML2137580797 with NULL emergency credit values:
{
  'serial': 'EML2137580797',
  'customer_unit_rate': Decimal('0.0'),
  'customer_standing_charge': Decimal('0.0'),
  'emergency_credit': None,        // NULL
  'debt_recovery_rate': None,      // NULL
  'ecredit_button_threshold': None // NULL
}

Aug 30 02:20:10 - Job crashed:
File "/mediator/simt_emlite/jobs/future_tariffs_update.py", line 55, in update
    Decimal(str(self.tariff["emergency_credit"])),
decimal.InvalidOperation: [<class 'decimal.ConversionSyntax'>]

Proposed Solution: Make Emergency Credit Writes Optional

Since tariffs_future_write sends 11 separate OBIS commands to the meter, we can conditionally skip the emergency credit writes when values are NULL, preserving the meter's existing settings.

Code Changes

1. Update the mediator client method

File: simt-emlite/simt_emlite/mediator/client.py (around line 662)

def tariffs_future_write(
    self,
    from_ts: datetime.datetime,
    standing_charge: Decimal,
    unit_rate: Decimal,
    emergency_credit: Decimal | None = None,  # Make optional
    ecredit_availability: Decimal | None = None,  # Make optional
    debt_recovery_rate: Decimal | None = None,  # Make optional
) -> None:
    # ... existing threshold and rate writes (keep unchanged) ...
    
    # Only write prepayment amounts if provided (not None)
    if emergency_credit is not None:
        self.log.debug(f"set emergency_credit to {emergency_credit}")
        self._write_element(
            ObjectIdEnum.tariff_future_prepayment_emergency_credit,
            emop_encode_amount_as_u4le_rec(emergency_credit),
        )
    else:
        self.log.debug("skipping emergency_credit write (None value)")
    
    if ecredit_availability is not None:
        self.log.debug(f"set ecredit_availability to {ecredit_availability}")
        self._write_element(
            ObjectIdEnum.tariff_future_prepayment_ecredit_availability,
            emop_encode_amount_as_u4le_rec(ecredit_availability),
        )
    else:
        self.log.debug("skipping ecredit_availability write (None value)")
    
    if debt_recovery_rate is not None:
        self.log.debug(f"set debt_recovery_rate to {debt_recovery_rate}")
        self._write_element(
            ObjectIdEnum.tariff_future_prepayment_debt_recovery_rate,
            emop_encode_amount_as_u4le_rec(debt_recovery_rate),
        )
    else:
        self.log.debug("skipping debt_recovery_rate write (None value)")
    
    # ... rest of writes (keep unchanged) ...

2. Update the job to pass None values through

File: simt-emlite/simt_emlite/jobs/future_tariffs_update.py (around line 47-58)

def update(self) -> bool:
    """
    Update the future tariff on the meter.
    Returns True if successful, False if failed.
    """
    try:
        # Convert tariff date
        tariff_start_date = datetime.strptime(
            self.tariff["tariff_period_start"], "%Y-%m-%d"
        ).replace(tzinfo=timezone.utc)
        
        # Convert values, preserving None for emergency credit fields
        standing_charge = Decimal(str(self.tariff["customer_standing_charge"]))
        unit_rate = Decimal(str(self.tariff["customer_unit_rate"]))
        
        # Handle optional emergency credit values - pass None if not provided
        emergency_credit = (
            Decimal(str(self.tariff["emergency_credit"])) 
            if self.tariff["emergency_credit"] is not None 
            else None
        )
        ecredit_threshold = (
            Decimal(str(self.tariff["ecredit_button_threshold"])) 
            if self.tariff["ecredit_button_threshold"] is not None 
            else None
        )
        debt_recovery = (
            Decimal(str(self.tariff["debt_recovery_rate"])) 
            if self.tariff["debt_recovery_rate"] is not None 
            else None
        )
        
        # Write to meter (emergency credit params now optional)
        self.emlite_client.tariffs_future_write(
            tariff_start_date,
            standing_charge,
            unit_rate,
            emergency_credit,
            ecredit_threshold,
            debt_recovery,
        )
        
        self.log.info("future tariff set")
        return True

Benefits of This Approach

1. Preserves Meter Settings: Emergency credit values remain unchanged when not specified
2. Enables Flexible Tariffs: Allows customer-specific tariff changes without requiring microgrid_tariffs
3. Backward Compatible: Existing code that provides all values continues to work
4. Clean Separation: Tariff rates and emergency credit are logically separate concerns

Testing

1. NULL emergency credit values: Create customer tariff without microgrid_tariffs, verify update succeeds
2. Provided emergency credit values: Create both customer and microgrid tariffs, verify all values update
3. Mixed NULL/provided: Some customers with microgrid tariffs, some without, verify both work
4. Meter state preservation: Verify meters retain existing emergency credit when NULLs passed

Impact

• Immediate: Fixes the crash preventing Sep 1 tariffs from being applied
• Long-term: Enables more flexible tariff management without requiring synchronized microgrid_tariffs