DKB: persist() produces truncated serialized output (HICAZSv1 mid-string), unserialize fails

[hajoeichler]

Summary

For DKB (BLZ 12030000, FinTS URL https://fints.dkb.de/fints, TAN mode 940), FinTs::persist() produces a deterministically truncated PHP serialize output. The byte stream ends mid-string within an s:136:"..." declaration of an HICAZSv1 segment, with no closing quote, no semicolon, no closing braces. Calling unserialize() on the result fails:

Error at offset 877 of 923 bytes in vendor/nemiah/php-fints/src/FinTs.php line 213

Reproducibility: 100% deterministic — same DKB account, multiple successful interactive runs, every captured persistence blob is bit-identical (sha256 the same, length 1231 base64 chars, decoded 923 bytes ending at the same offset).

Repro

1. Connect to DKB with a fresh FinTs (empty $persistedInstance).
2. Login with TAN mode 940 + pushTAN approve.
3. Call $fin_ts->persist() after successful login.

The returned string, when base64-decoded, ends:

...{i:0;s:136:"HICAZS:16:1:3+1+1+0+450:N:N:urn?:iso?:std
                                                       ^-- stream ends here, mid-string

s:136 declares a 136-byte string but only the first 40 bytes are emitted; no "; terminator, no closing braces for the parent Fhp\Segment\CAZ\HICAZSv1, Fhp\Protocol\BPD, or the outer a:9:{...}.

What I ruled out (in isolation)

• serialize() of a fresh HICAZSv1 populated by Parser::parseSegment(<wire>, $seg) and then re-emitted via Serializer::serializeSegment($seg) round-trips correctly (137-byte input → 137-byte output → 186-byte PHP-serialize → unserialize OK).
• Wrapping such a segment inside a synthetic BPD->parameters = ["HICAZS" => [1 => $seg]] and PHP-serializing the BPD also round-trips cleanly (359 bytes, ends with }, unserialize OK).
• So the bug needs the production-time FinTs state assembled by a real DKB dialog. Reproducing only via the synthetic path didn't surface it.

Environment

• nemiah/phpFinTS commit 4800d6a (HEAD of master, 2026-05-23)
• PHP 8.4.19 inside the bnw/firefly-iii-fints-importer Docker image (php -S server)
• Linux/arm64 (Raspberry Pi 5, 16 GB), rootless Docker

Why this matters

It defeats the persistence model: any caller that wants to reuse a 90-day SCA-exempt session has to retry the login (and consume a fresh TAN challenge) every time, because the saved blob is unloadable. For DKB specifically, this combines badly with their per-day TAN-challenge quota — failed cron jobs quickly trigger error 9040 "Authorization method is locked because too many challenges were requested," sometimes severe enough to invalidate the mobile-app session too.

Happy to help bisect

If you have a hypothesis for which segment property / element is producing the silent truncation, I can instrument my local copy and re-run against my DKB account (carefully — one attempt per day at most). Or if you can point me at the test harness pattern you'd use to replay a captured BPD response, I can capture mine and produce a synthetic repro.

[ampaze]

I would suggest to add a test for GetStatementOfAccountXML to the DKB integration test and compare how the dialog initialization in the test differs from your fresh data.

[hajoeichler]

Quick update.

I took @ampaze's suggestion and added an integration test for GetStatementOfAccountXML to the DKB suite: PR #565 (draft). It includes a testWithTanPersist variant whose explicit assertion is assertNotFalse(@unserialize($finTs->persist())) - i.e. exactly the property this issue says is violated in production.

All three test variants pass against fixtures. That tells us the truncation isn't reproducible from the BPD/UPD shape encoded in DKBIntegrationTestBase.

Side-by-side diff (HICAZS slice of persist() output):

	Fixture	Production-broken (this issue)
Wire-format length	82 bytes	136 bytes
s:N declaration	s:82	s:136
Segment header	HICAZS:87:1:4	HICAZS:16:1:3
sicherheitsklasse	1	0
camt descriptors	1 (camt.052.001.02)	likely 2-3

I also separately constructed an HICAZSv1 whose wire-format is exactly 136 bytes (HICAZS:16:1:3+1+1+0+450:N:N:urn?:iso?:std?:iso?:20022?:tech?:xsd?:camt.052.001.02:urn?:iso?:std?:iso?:20022?:tech?:xsd?:camt.052.001.08') and pushed it through Parser::parseSegment -> Serializer::serializeSegment -> serialize -> unserialize. Round-trips cleanly. So the bug is not in the segment's wire format, the multi-descriptor shape, or any character that DKB plausibly puts on the wire.

That leaves the truncation happening outside phpFinTS proper, somewhere between persist() returning and what I observed at the consumer side. My current top suspects, in order:

1. Symfony Session storage interaction. The downstream importer (bnw/firefly-iii-fints-importer) stores $finTs->persist() in $_SESSION['persistedFints']. PHP's default php session serializer concatenates key|serialize(value); and reads back by scanning for |. The persist output contains |-equivalent magic in PHP's nested format, and the session serializer is documented to corrupt values containing |. I'll switch the importer to session.serialize_handler=php_serialize and re-run; if that fixes it, the bug is in the consumer not phpFinTS, and I'll close this issue.
2. mbstring.substitute_character runtime config. Serializer::serializeDataElement calls mb_convert_encoding($value, 'ISO-8859-1', 'UTF-8'). If PHP runtime has mbstring.substitute_character = none and $value is already binary ISO-8859-1 (not valid UTF-8), mb_convert_encoding would silently return false / empty / a short string in some configurations. Less likely to be a hard truncation but worth checking - the wire string is short of its declared length by exactly 96 bytes (136 declared, 40 actually emitted) which is suspicious.
3. OPcache file storage corruption. Lower probability but cheap to rule out: opcache_reset() + retry.

I'll do (1) and (2) on my side and report back. The new test in #565 is independent of root cause - it just guards the property and gives anyone bisecting later a starting point in the DKB suite.

If you'd prefer I keep the draft open until we identify the cause or convert it to "ready for review" now, let me know.

[hajoeichler]

Quick update — ruled out the session-serializer hypothesis from the draft PR's diagnostic.

I wrote a small standalone probe that round-trips three payloads through PHP's session storage with each of the three session.serialize_handler modes (php, php_serialize, php_binary):

1. A synthetic binary blob with NULs, pipes, and control bytes — the kinds of bytes the default php handler is famously bad at.
2. A realistic persist() output built from a synthetic-but-realistic BPD containing a HICAZSv1 with the full 136-byte DKB wire-format string.
3. The actual 923-byte broken persistence pasted by Hajo (the one ending at s:136:"HICAZS:16:1:3+1+1+0+450:N:N:urn?:iso?:std).

All nine combinations round-trip byte-for-byte. The session layer is not the source of the truncation.

Hex of the last 30 bytes from the broken production persistence:

3a 33 2b 31 2b 31 2b 30 2b 34 35 30 3a 4e 3a 4e 3a 75 72 6e 3f 3a 69 73 6f 3f 3a 73 74 64
:  3  +  1  +  1  +  0  +  4  5  0  :  N  :  N  :  u  r  n  ?  :  i  s  o  ?  :  s  t  d

No NULs, no control bytes, no multibyte UTF-8 issues — the stream just stops after std with no "; terminator and no closing braces. So whatever produced these 923 bytes returned them already-truncated.

What we've now ruled in/out:

• ❌ Serializer::serializeSegment(HICAZSv1) — round-trip on a synthetic segment with the same wire is clean.
• ❌ serialize(BPD) with a synthetic parameters = ['HICAZS' => [1 => $seg]] BPD — clean.
• ❌ PHP session.serialize_handler (php / php_serialize / php_binary) — all preserve the bytes.
• ❌ Base64-encode/decode in transit — round-trip is clean.
• ❌ NUL-byte / control-char display cutoff at byte 40 in the user's clipboard — no such byte present.
• ✅ FinTs::persist() itself, with a specific real-DKB BPD shape — this is what the synthetic fixtures don't reproduce. The 923 bytes ARE what persist() returns in production.

So the next step really is to capture a real DKB BPD and feed it through persist() in isolation to reproduce. I don't want to do that against Hajo's live account (TAN-cost concerns), but I can try to extract a recorded DKB dialog from a previous successful run if you can point me at a known-good capture format. Alternatively, if you have a contributor with DKB access willing to record one BPD response and share it as a fixture in the repo, that would unblock this.

In the meantime I'll leave PR #565 in draft. Let me know if you'd prefer a different angle.

[ampaze]

It seems to me you are already logging the wire format?

That is the capture format you need to write a test that fails. Which is what would be needed right now and if the existing BPD for DKB is not triggering it, you/we need to use the data from your sessions.

I also noticed that you use PHP 8.5, have you tried with PHP 8.4 as well?

---

In case you are not logging right now, here is how I did the Atruvia tests

$fints->setLogger(new SanitizingLogger($this->fintsLogger, [
    $options,
    $finTsCredentials->getUsername(), $finTsCredentials->getPin(),
    $account->getKto(),
    $account->getIban(),
    ....
]));

Where fintsLogger is defined like (using Symfony)

monolog:
    channels: ["fints"]

    # Wird nur für den $fintsLogger angewendet, siehe "getMonolog_Logger_FintsService" bzw. "monolog.logger.fints"
    handlers:
        fints:
            type:  stream
            path:  "%kernel.project_dir%/fints.log"
            level: debug
            channels: [fints]
            bubble: false

And then I copied the relevant requests and responses to the test file(s) and further anonymized parts as needed.

[hajoeichler]

Thanks for the nudge on PHP 8.4. Turns out a misread on my side — re-checked the image: the importer (bnw/firefly-iii-fints-importer overlay) is actually built on php:8.4-cli (8.4.19), not 8.5 as the issue body claimed. Corrected the body. So the broken persistence WAS produced on PHP 8.4.

I also ran a controlled comparison anyway. Built a synthetic BPD by parsing the existing DKB fixture (ANONYMOUS_INIT_RESPONSE in DKBIntegrationTestBase, 167 segments → 133 parameter keys including HIAUBS/HIBMES/HIBSES/HICAZS/HITANS/HIPINS/etc.), substituted a production-matched 136-byte HICAZS wire (HICAZS:16:1:3+1+1+0+450:N:N:urn?:iso?:std?:iso?:20022?:tech?:xsd?:camt.052.001.02:urn?:iso?:std?:iso?:20022?:tech?:xsd?:camt.052.001.08'), and ran serialize([2, $bpd, ...]) on both PHP 8.4.19 and 8.5.6.

Result: bit-identical 25,948-byte output on both versions, both unserialize cleanly. No truncation either way. So PHP version isn't the variable, and the fixture-derived BPD shape (even with a production-matched HICAZS wire) isn't enough to reproduce.

The clue I think we should chase next is the size disparity:

Source	persist() output
Synthetic (fixture-derived 133-segment BPD)	25,948 bytes
Production (broken)	923 bytes

That's ~28× smaller. The production blob can't contain a fully-populated DKB BPD — at most a handful of segment entries before it truncates mid-HICAZS. So either $bpd->parameters was mostly empty when persist() ran (and the truncation is just "less data than expected"), OR the serializer was interrupted mid-stream.

Per your earlier suggestion, I'm setting up SanitizingLogger in the bnw importer to capture the wire format of a real DKB session, and will paste an anonymized replay into the test in #565 so we have something that actually fails. Will follow up with that capture once it's in hand.

[hajoeichler]

Big update — captured the wire format you asked for, and the data points to this not being a phpFinTS bug. The persistence is fully intact end-to-end on the server side. The truncation lives downstream in bnw's done.twig.

What I measured

Built a Fhp\Logging\SanitizingLogger overlay into the bnw container, gated by a sentinel file so it only logged the one capture run. username / PIN / productName / bank-code redacted at log time; manually scrubbed IBAN / customer name / counterparty / Verwendungszweck before sharing.

Layer	Size	State
DKB wire (HIBPA + 50+ HIxxxS through HIZDLS)	~5 KB segment	well-formed, parses cleanly
$session->get('persistedFints') after import	13,038 bytes	full PHP-serialize, validates
base64_encode(...) rendered in <pre> on done.twig	17,384 chars	full DOM value
Manual browser copy from that <pre>	1,231 chars	mid-base64 cut, invalid padding
What ended up in three dkb-*.json files	1,231 chars	byte-for-byte identical to the manual copy

Decoded blob ends cleanly with i:8;i:6;} — top-level array properly closed. So persist() is doing exactly what it should. The cut happens during the user's browser copy from a single 17K-char no-whitespace base64 string inside a Bootstrap <pre> (overflow:auto, no wrap). Cmd-A on the <pre>, triple-click, click-and-drag all return different fractions of the string. In our case the result was deterministic at 1,231 chars across three separate copy attempts into three different config files.

Where the truncation lands exactly

Cut lands at decoded byte 923, mid-HICAZS-segment, inside urn?:iso?:std?...camt.052.001.08. The base64 char count (1231) isn't divisible by 4, so the stored blob fails base64_decode(..., strict=true) — explaining the deserialize blowup on the next run.

Verification

Wrote the intact 17,384-char value back into the JSON file by hand (via Python, not browser copy) and re-ran the bnw cron path headless. Container log:

[INFO] Bank supports CAMT XML format (HICAZS)
[INFO] Using statement format: camt
[INFO] Parsing CAMT XML format
[INFO] No transactions found for date range: 2026-05-01 to 2026-05-29

Full FinTS dialog completed using the saved persistence — no TAN prompt. SCA-exempt session reload works correctly when the right bytes reach the JSON.

Fix

Opened bnw/firefly-iii-fints-importer#236 — auto-saves the persistence back to the loaded ?config=<name>.json so users never copy/paste at all, plus a <textarea readonly> + Copy-to-clipboard fallback for the no-config interactive path. 9 PHPUnit tests covering the round-trip, the 17K size class explicitly as a regression test, all-other-key preservation, atomic write, and file-permission preservation.

A side-effect the patch surfaced: RunImportWithoutJS was previously dropping the persistence value entirely (headless users had no way to save it at all). Both render paths now go through the same auto-save helper.

Probably safe to close #564 here as "not phpFinTS — downstream UX in bnw" once you've had a moment to look. Thanks for the wire-log nudge; that was the unlock.