js syntax highlighting for markdown (not for codeblocks in markdown)

[Fletcher T. Penney]

I agree with John -- what you need for syntax highlighting and what you 
need for conversion (to HTML or whatever) are different.  There are 
performance optimizations you can make when converting that you cannot 
make for syntax highlighting (depending on your code).

I have a modified version of MMD-4 that handles syntax highlighting as 
well (not open source, used in MultiMarkdown Composer) and it definitely 
required a different approach.

When you start getting into character offsets, multibyte characters, 
etc. it definitely was better to separate the workflows for syntax 
highlighting and conversion, especially from a performance standpoint. 
(At least in my situation.)


Fletcher



On 4/24/15 12:06 PM, John MacFarlane wrote:
> +++ Tobias Bengfort [Apr 24 15 11:14 ]:
>> I am not sure if source mapping is the best approach. It sounds strange
>> to me that the highlighter would need the AST *and* the original input.
>>
>> Wouldn't it be easier to include enough information in the AST to be
>> able to reconstruct the original input (e.g. whitespace)? It should then
>> be easy to add a second renderer for highlighting.
>
> In principle one might be able to include enough information in the
> AST to reconstruct the exact source input, but this would make the
> AST immensely more complex.  For example, for a block quote we'd have
> to store the exact pattern of leading `>`s on every line, including any
> initial indentation.  For any character that could be escaped we'd
> need to store whether it was escaped in the source.  I could go on and
> on.  I think that source mapping is a much simpler approach.  It also
> has applications beyond highlighting (e.g., for "click the formatted
> text and an edit box appears" style online editing).
>
> One thing to note is that the source text for a particular element
> can be discontinuous.  Take, for example:
>
>     > This is *emphasized
>     > text*.
>
> Here the emphasized node corresponds to columns 10-20 in line 1
> and columns 2-6 in line 2.  That's why my original approach of storing
> start and end positions for each node is not enough.  So, the revisions
> needed to commonmark.js are bigger than I thought, but still feasible.
>
> _______________________________________________
> Markdown-Discuss mailing list
> Markdown-Discuss at six.pairlist.net
> https://pairlist6.pair.net/mailman/listinfo/markdown-discuss

-- 
Fletcher T. Penney
fletcher at fletcherpenney.net