evolving the spec (was: forking Markdown.pl?)

John Fraser john at attacklab.net
Fri Feb 29 11:40:34 EST 2008

On Feb 29, 2008, at 10:13 AM, Michel Fortin wrote:

> I think the syntax needs to be defined unambiguously, not

> necessarily as a formal grammar, but certainly not with code either.

> My idea, currently, is to write a parsing procedure which is easy to

> read and implement in various ways, using a formal grammar to define

> various constructs of the syntax and plain english to link things

> together. I also intend to keep the spec implementable as an

> incremental parser, but that will require backtracking.

I agree that Markdown needs to be defined unambiguously, but I don't
think that's feasible with plain English in the loop. For something
as complex and flighty as Markdown, we need working code.

One possibility I've been thinking about is to define Markdown as a
dressed-up parsing expression grammar, which would be fed to a simple,
portable packrat parser. The result would be a "spec" that makes
sense to machines as well as humans, so implementors would have
something to test against.

We could take it a step further and define rewrite rules to transform
the parse tree into the final output. Then we'd have a readable
definition of both syntax and semantics that doubles as a reference
implementation. It might be too slow for many uses; I'd probably have
to do a hand-written JavaScript parser for [Showdown] 2.0, for
example. But it's much easier to optimize a working system than a
paper spec.

[Showdown]: http://www.attacklab.net/showdown-gui.html

The biggest win here would be extensibility: adding rules to a PEG is
simple enough that Markdown would finally be flexible. Users could
add custom rules for wiki links, footnotes, or whatever else they need
-- without the risk of fragmenting the language.

If there's interest in this approach, I'll try to do a quick hand-wavy
prototype next week to show what I'm talking about.

John Fraser
http://wmd-editor.com/ (going open source soon)

More information about the Markdown-Discuss mailing list