evolving the spec (was: forking Markdown.pl?)

[John Fraser]

On Feb 29, 2008, at 10:13 AM, Michel Fortin wrote:


> I think the syntax needs to be defined unambiguously, not  

> necessarily as a formal grammar, but certainly not with code either.  

> My idea, currently, is to write a parsing procedure which is easy to  

> read and implement in various ways, using a formal grammar to define  

> various constructs of the syntax and plain english to link things  

> together. I also intend to keep the spec implementable as an  

> incremental parser, but that will require backtracking.



I agree that Markdown needs to be defined unambiguously, but I don't  
think that's feasible with plain English in the loop.  For something  
as complex and flighty as Markdown, we need working code.

One possibility I've been thinking about is to define Markdown as a  
dressed-up parsing expression grammar, which would be fed to a simple,  
portable packrat parser.  The result would be a "spec" that makes  
sense to machines as well as humans, so implementors would have  
something to test against.

We could take it a step further and define rewrite rules to transform  
the parse tree into the final output.  Then we'd have a readable  
definition of both syntax and semantics that doubles as a reference  
implementation.  It might be too slow for many uses; I'd probably have  
to do a hand-written JavaScript parser for [Showdown] 2.0, for  
example.  But it's much easier to optimize a working system than a  
paper spec.

[Showdown]: http://www.attacklab.net/showdown-gui.html

The biggest win here would be extensibility: adding rules to a PEG is  
simple enough that Markdown would finally be flexible.  Users could  
add custom rules for wiki links, footnotes, or whatever else they need  
-- without the risk of fragmenting the language.

If there's interest in this approach, I'll try to do a quick hand-wavy  
prototype next week to show what I'm talking about.

John Fraser
http://wmd-editor.com/ (going open source soon)