Double newlines in HTML formatter after Elixir comments

When formatting newline whitespace in HTML inside a `<pre>`, newlines are sometimes rendered twice. 

- First the original newline token is rendered, and then 
- the token has another newline to close the span.

This only seems to happen after comments in Elixir code, so this issue might be a tokenizer issue and not an HTML formatter issue. 

---

For example, using makeup 1.0.5 and makeup_elixir 0.15.1, given this string:

```
defmodule BestStructEver do\r\n defstruct [:a]\r\nend\r\n\r\n# Struct to simple map:\r\niex> Map.from_struct(%BestStructEver{a: \"foobar\"})\r\n# => %{a: \"foobar\"}\r\n\r\n# Map to a struct:\r\niex> struct(BestStructEver, %{a: \"foobar\"})\r\n# => %BestStructEver{a: \"foobar\"}\r\n\r\n# Note: The struct function is from Kernel, so `Kernel.` can be omitted.
```

It is rendered this way using a Phoenix template `<%= raw(Makeup.highlight(my_code_here)) %>`:

```html
<pre class="highlight"><code>defmodule BestStructEver do
 defstruct [:a]
end

# Struct to simple map:

iex&gt; Map.from_struct(%BestStructEver{a: "foobar"})
# =&gt; %{a: "foobar"}


# Map to a struct:

iex&gt; struct(BestStructEver, %{a: "foobar"})
# =&gt; %BestStructEver{a: "foobar"}


# Note: The struct function is from Kernel, so `Kernel.` can be omitted.</code></pre>
```

appearing like this image:

![image](https://user-images.githubusercontent.com/643967/112741358-1d34f700-8f53-11eb-94e0-4d1650c5077e.png)

When it should appear more like this (ignoring the theme -- this is not Makeup):
![image](https://user-images.githubusercontent.com/643967/112741485-515ce780-8f54-11eb-8f55-c657caf81ef6.png)

---

When I fork makeup, I _think_ I fix it if I treat the newline as if it needed escaping, rendering an empty string instead, but I feel like this fixes it the wrong way. Instead I feel the span closing tag should not have a newline before it. 

```diff
diff --git a/lib/makeup/formatters/html/html_formatter.ex b/lib/makeup/formatters/html/html_formatter.ex
index 1c28d51..912f30e 100644
--- a/lib/makeup/formatters/html/html_formatter.ex
+++ b/lib/makeup/formatters/html/html_formatter.ex
@@ -38,6 +38,8 @@ defmodule Makeup.Formatters.HTML.HTMLFormatter do
 render_token(escaped_value, css_class, meta, highlight_tag)
 end
 
+ defp escape_for(?\n), do: ""
+
 defp escape_for(?&), do: "&amp;"
 
 defp escape_for(?<), do: "&lt;"
```

It produces this HTML:

```html
<pre class="highlight"><code>defmodule BestStructEver do
 defstruct [:a]
end

# Struct to simple map:
iex&gt; Map.from_struct(%BestStructEver{a: "foobar"})
# =&gt; %{a: "foobar"}

# Map to a struct:
iex&gt; struct(BestStructEver, %{a: \"foobar\"})
# =&gt; %BestStructEver{a: "foobar"}

# Note: The struct function is from Kernel, so `Kernel.` can be omitted.</code></pre>
```

```diff
diff --git a/before b/after
index 55d1d44..37f2cc3 100644
--- a/before
+++ b/after
@@ -1,17 +1,13 @@
-<pre class="highlight"><code>defmodule BestStructEver do
- defstruct [:a]
-end
+<pre class="highlight"><code>defmodule BestStructEver do
+ defstruct [:a]
+end
 
 # Struct to simple map:
-
-iex&gt; Map.from_struct(%BestStructEver{a: "foobar"})
+iex&gt; Map.from_struct(%BestStructEver{a: "foobar"})
 # =&gt; %{a: "foobar"}
 
-
 # Map to a struct:
+iex&gt; struct(BestStructEver, %{a: \"foobar\"})
+# =&gt; %BestStructEver{a: "foobar"}
 
-iex&gt; struct(BestStructEver, %{a: "foobar"})
-# =&gt; %BestStructEver{a: "foobar"}
-
-
 # Note: The struct function is from Kernel, so `Kernel.` can be omitted.</code></pre>
```

![image](https://user-images.githubusercontent.com/643967/112741754-4b680600-8f56-11eb-99fe-048bda8e1683.png)



The same result happens when I `String.replace(code, "\r\n", "\n")` before passing it to makeup, so I don't believe that's causing it yet.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Double newlines in HTML formatter after Elixir comments #36

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Double newlines in HTML formatter after Elixir comments #36

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions