Twice escaping in `default` and `default_if_none` filters when printed variable and default value both taken from render and printed variable is undef

[pavelsr]

use lib './blib/lib';
use lib './blib/arch';
use DTL::Fast;

my $x = 0;
my $z = \&DTL::Fast::html_protect;
*DTL::Fast::html_protect = sub {
    $x++;
    goto &$z;
};

my $tpl = DTL::Fast::Template->new('Hello, {{ username }}!');
warn $tpl->render({ username => '<xmp>Hello world</xmp>' });	# ok
warn $x; # 1

###

my $tpl = DTL::Fast::Template->new('Hello, {{ foo|default:username }}!');
warn $tpl->render({ foo => undef, username => '<xmp>Hello world</xmp>' }); # not ok
warn $x; # must be 2, but it's 3

###

my $tpl = DTL::Fast::Template->new('Hello, {{ foo|default_if_none:username }}!');
warn $tpl->render({ foo => undef, username => '<xmp>Hello world</xmp>' }); # not ok
warn $x; # 5

Seems like we need to make some fixes at lib/DTL/Fast/Variable.pm

[pavelsr]

Investigated code a little bit, found that if we wrap construction with default filter with autoescape off tag like

{% autoescape off %}{{ foo|default:username }}{% endautoescape %}

everything will be ok.

UPD: not everything, if foo is not empty it will not be escaped.

But I don't think that it's expected behavior.

[hurricup]

This was too long time ago and the only way here to read the code and debug.
I'd suggest to:

1. check with Django, how this should work in it
2. If it differs from current behavior, debug your samples, find the second escape and fix + test

[pavelsr]

Checked in Django

version 1.8

$ docker run -it python:3 bash
root@17b5ee27c1a1:/# pip install Django==1.8 
Collecting Django==1.8
  Downloading https://files.pythonhosted.org/packages/4e/1c/17a429cfb79c1814d1ec31939fc5cf4a8ac68fe934279e095fb6160123a9/Django-1.8-py2.py3-none-any.whl (6.2MB)
     |████████████████████████████████| 6.2MB 2.3MB/s 
Installing collected packages: Django
Successfully installed Django-1.8
root@17b5ee27c1a1:/# django-admin startproject mysite
root@17b5ee27c1a1:/# python mysite/manage.py shell
Python 3.8.0 (default, Nov 23 2019, 05:36:56) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from django.template import Context, Template
>>> t = Template("Hello {{ foo|default:username }}!")
>>> c = Context({"foo": None, "username" : "<i>Vasya</i>"})
>>> t.render(c)
'Hello &lt;i&gt;Vasya&lt;/i&gt;!'

latest version (3.0)

$ docker run -it python:3 bash
root@4740ca372dd6:/# pip install Django==3.0
Collecting Django==3.0
  Downloading https://files.pythonhosted.org/packages/43/d6/0aed0b12c66527748ce5a007da4618a65dfbe1f8fca82eccedf57d60295f/Django-3.0-py3-none-any.whl (7.4MB)
     |████████████████████████████████| 7.4MB 3.5MB/s 
Collecting pytz
  Downloading https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl (509kB)
     |████████████████████████████████| 512kB 4.9MB/s 
Collecting asgiref~=3.2
  Downloading https://files.pythonhosted.org/packages/a5/cb/5a235b605a9753ebcb2730c75e610fb51c8cab3f01230080a8229fa36adb/asgiref-3.2.3-py2.py3-none-any.whl
Collecting sqlparse>=0.2.2
  Downloading https://files.pythonhosted.org/packages/ef/53/900f7d2a54557c6a37886585a91336520e5539e3ae2423ff1102daf4f3a7/sqlparse-0.3.0-py2.py3-none-any.whl
Installing collected packages: pytz, asgiref, sqlparse, Django
Successfully installed Django-3.0 asgiref-3.2.3 pytz-2019.3 sqlparse-0.3.0
root@4740ca372dd6:/# jango-admin startproject mysite
bash: jango-admin: command not found
root@4740ca372dd6:/# django-admin startproject mysite
root@4740ca372dd6:/# python mysite/manage.py shell
Python 3.8.0 (default, Nov 23 2019, 05:36:56) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from django.template import Context, Template
>>> t = Template("Hello {{ foo|default:username }}!")
>>> c = Context({"foo": None, "username" : "<i>Vasya</i>"})
>>> t.render(c)
'Hello &lt;i&gt;Vasya&lt;/i&gt;!'

So it's a definetely bug in DTL::Fast

[darviarush]

Hi. Looks like a bug in render / https://github.com/hurricup/DTL-Fast/blob/master/lib/DTL/Fast/Filter/Default.pm#L24

$context is expression on DTL::Fast language. It is necessary that render does not escape.
To do this it needs to pass the second parameter 1: return $value || $self->{default}->render($context, 1);.