High template render times

[ninja-]

The templates are very slow to load...
Rendering map.jinja takes 0.5s and that is repeated for each php plugin requested.
So in my case fpm, mongo, mysql, curl, soap...

[PROFILE ] Time (in seconds) to render '/var/cache/salt/minion/files/base/mchc/web/php.sls' using 'jinja' renderer: 0.0007643699645996094
[PROFILE ] Time (in seconds) to render '/var/cache/salt/minion/files/base/mchc/web/php.sls' using 'yaml' renderer: 0.0010766983032226562
[PROFILE ] Time (in seconds) to render '/var/cache/salt/minion/files/base/php/ng/mongo.sls' using 'jinja' renderer: 0.5516741275787354
[PROFILE ] Time (in seconds) to render '/var/cache/salt/minion/files/base/php/ng/mongo.sls' using 'yaml' renderer: 0.00038695335388183594
[PROFILE ] Time (in seconds) to render '/var/cache/salt/minion/files/base/php/ng/curl.sls' using 'jinja' renderer: 0.5393562316894531
[PROFILE ] Time (in seconds) to render '/var/cache/salt/minion/files/base/php/ng/curl.sls' using 'yaml' renderer: 0.00039315223693847656
[PROFILE ] Time (in seconds) to render '/var/cache/salt/minion/files/base/php/ng/soap.sls' using 'jinja' renderer: 0.5391628742218018

I tried really hard to optimize this, by caching the map.jinja content in a variable and then re-using that variable (via pillar.get('php', {}) + .update({'cache:': php}) ), but that didn't help.
No matter what I do it takes 0.5s to render map.jinja unless I remove "from map.jinja" completely for a test.