Create batch of adjuncts

[donaldgray]

What does this change?

Adds support for creating batches of adjuncts. The concept of Batch already exists for assets, so this introduces a new AdjunctBatch concept - it is very similar to asset/batches but different enough to warrant new entities etc.

Extend Hydra Model

Resolves #1150

Adjunct HydraModel gains Asset property

API + Engine handle AdjunctBatch

Add new entities:

• AdjunctBatch - this is the overall record of items/success/failed etc counts for group of adjuncts (equivalent of Batch for assets)
  • Batch and AdjunctBatch are IDeliverableBatch as there is some commonality between them.
  • Rename BatchAssetStatus -> BatchStatus as it's shared between the 2 different batch types.
• AdjunctBatchAdjunct - collection of all Adjuncts in an AdjunctBatch, maintains a record of processing for adjunct-batches (equivalent of BatchAsset)
  • BatchAsset and AdjunctBatchAdjunct are both IDeliverableBatchItem

Refactored adjunct processing code out of CreateOrUpdateAdjunct and into AdjunctUpsertService. This handles mapping of fields and raising notifications.

As part of extracting to AdjunctUpsertService I updated engine notifications to be sent in batches, which is more efficient. This has resulted in a slight loss of identifying which actual message sending failed as we're no longer sending one at a time but this should be a very rare occurrence so the trade-off seems worthwhile.

New controller for adjunctQueue - currently only supporting POST operations to create a batch.

CreateAdjunctBatch mediatr request, this

• Validates incoming payload and finds existing adjuncts from DB.
• Creates a db transaction
• Iterates through all adjuncts in payload and calls adjunctUpsertService to process
• Creates batch + assigns batchId to all adjuncts
• Creates AdjunctBatchAdjunct record with correct status.
• Commits transaction
• Send notifications (Engine + Create/Update)

Extend AdjunctConverter to include an overload of ToDlcsModel() - this allows either the short or long-form assetId to be passed.

Use "adjunct" queue name, rather than "default". Saving "default" for assets, introducing a new one will make it clearer the differences. This change won't be immediately obvious but is surface in customer-queue endpoints, it allows customers to see how many items are outstanding in each type of queue.

Refactor IEngineClient to make use of IDeliverable for single async ingest requests. Stick to Asset and Adjunct for batch requests as there are more differences (namely priority or not). Internally refactor EngineClient to share more internally based in IDeliverable type.

Resolves #1149

Other Changes

Add HydraExceptionFilter - this catches any unhandled exceptions. It's essentially replacing HydraController.HandleHydraRequest() so calls to that were removed. The resulting controller code is now clearer to read.

Database Migration

Note

• Migration name: Add AdjunctBatch
• What it does: Adds AdjunctBatches table (with sequence adjunct_batch_id_sequence and index on (Customer, Submitted)), AdjunctBatchAdjuncts junction table, and a nullable Batch FK column on Adjuncts.
• Breaking Change? No — all new tables/columns; existing data unaffected.