Replace spaces with nonbreaking spaces?

[Rhababo]

Hi!

We currently replace newlines with <br/>, which solves the issue of truncating blank lines.

Do we want it to also replace blank spaces (U+0020) with non-breaking spaces (U+00A0) or  ?

edit: forgot to put my <br/> in a code block, but you caught it!

[Alteras1]

Our plan is to replace newlines with <br/>. This matches with the behavior of how Xenforo processes newlines. Not sure if you're also migrating from something similar, but forcing unicode characters should work as well.

We also want to see if we can disable the markdown-it's paragraph rule.

[Rhababo]

Discourse's TextCleaner will change all whitespace to regular spaces, which markdown/html then trims/truncates. Noted here:
https://meta.discourse.org/t/no-break-space-and-narrow-no-break-space-are-replaced-by-space/169410/8

What I was working on before coming here was to disable this function in the text cleaner, and then (somehow) change replace normal spaces with non-breaking spaces, which would then be passed to markdown. - I was pretty excited to see that this setup with BBob was already doing this with newlines.

I've just posted my code as a plugin here: https://github.com/Rhababo/discourse_normalize_whitespace

But the core functionality is really just:

class ::TextCleaner
    module NormalizeNormalize_whitespace
      def normalize_whitespaces(text)
        text
      end
    end
    singleton_class.prepend NormalizeNormalize_whitespace
  end

in the "after_initialize do" block

BTW: I had found this documentation on discourse really useful in figuring out how to override this:
https://meta.discourse.org/t/override-existing-discourse-methods-in-plugins/83389

So with this function, if we replaced the spaces with nonbreaking spaces in BBob, those spaces should persist through markdown.

[Alteras1]

Oh! I didn't realize that's where it's from. I've been staring at the markdown-it engine this whole time thinking it was caused by the paragraph tag generation.

Non breaking whitespace should work, but hoping to see if we can do without text replacement. Any luck with disabling it?

[Rhababo]

I think that markdown actually retains the whitespace, but just puts in in a <p> tag and html is what truncates it.

<p>Markdown is kind of lazy that way</p> -->

Markdown is kind of lazy that way

Bbob already does whitespace checks when building its tree, I don't know if it would be that inefficient to inject a character replacement during those checks. But it would require modifying a custom fork of Bbob...or maybe some kind of special plugin format (I'm still getting familiar with the Bbob code).

We could try injecting a <pre> tag.
<pre><p> It looks like Markdown makes this an html block? </p></pre> -->

 It looks like Markdown makes this an html block?
 

So I went to test this in discourse:

This also works on the markdown-it demo page

Not sure how this is processed exactly yet.

[Mondrethos]

A plugin be preferred over a custom fork of bbob, but I could be convinced.

[Rhababo]

Having looked at how the Bbob tree is parsed, I think a plugin for this issue would be fine. We'd only need to check for nodes that are just multiple spaces and replace those characters since nodes that are single spaces will be retained anyway.

Interestingly, the parser treats a nonbreaking whitespace (U+00A0) as a normal character, so you can get array nodes that "look     like        this          . " if the nonbreaking space is entered before parsing.

[Alteras1]

Huh, interesting. A plugin should work fine in this case. If not, you can brute force it with a bit of client side js that just does a regex check for multiple spaces.