evolving the spec (was: forking Markdown.pl?)

[Michel Fortin]

I'm going to comment both on Yuri and Waylan's message here.

Le 2008-03-01 à 0:31, Waylan Limberg a écrit :


> On Fri, Feb 29, 2008 at 6:04 PM, Yuri Takhteyev  

> <qaramazov at gmail.com> wrote:

>> Since Joe called for procedural suggestion, here is what I think we  

>> should do:

>>

>> 1. First, I think there were valid concerns about whether it would be

>> ok for us to come up with a spec and call it "Markdown 2.0".  I

>> suggest we put the question of naming aside.  Once we agree on a  

>> spec,

>> we'll ask for John's permission to call it "Markdown 2.0" or

>> Markdown-Something-Else, and in the worst case we'll call it  

>> something

>> different, like "M-Spec" or "FooBar7.0"  For now, let me refer to it

>> as the M-Spec.


My solution to this problem was to call it the Markdown Extra spec.  
What do you think?


>> 2. As Thomas suggested, we should first reach some agreement as to

>> what if anything needs to change from the original Markdown or how  

>> the

>> "holes" are to be filled. _Then_ try writing a grammar.


I don't think that's a good process. My idea for writing a Markdown  
Extra spec was to write something as a draft, call for comments,  
improve things, call for new comments, and so on until we have  
something stable enough (a few months later).


>> I suggest that we do this first part in plain English (and perhaps  

>> some

>> pseudo-code), using the wiki to record the decisions and the

>> highlights of the dissenting views, and using the mailing list for  

>> the

>> actual discussion.  Let's _try_ to do it by consesus.  It might just

>> work.  If we can't agree and have to resort to voting, we can then

>> figure out how to handle that.  (We've got a voting expert among us:

>> Joe.)

>

> Something tells me this is going to be the ugly part. As long as it

> doesn't turn into something like writing html specs has become. Uhg.


I think the HTML spec is going on well, thanks to an editor who can  
make decisions. Voting on an issue-to-issue basis isn't something I'd  
like to try. The problem being that once something has been elected in  
or out of the spec, it's problematic and conter-productive to ask  
everyone to vote again on that decision some time later because  
something changes (unexpected side effects are found, new research  
data shows its a bad idea, some change elsewhere made the thing a  
little silly; those things happen all the time). Beside, on a list  
like this one, simply voting again on an issue at a later time can  
cause the result to change since people will have joined and other  
left. I wouldn't want to be the editor of a spec with such a strict  
voting process.

Beside, it's a better idea in my opinion to make a decision  
considering the technical merit of an argument rather than a  
popularity contest.


>> 3. I suggest that be start by breaking "M-Spec" into two levels.

>> Level 1 will aim to clarify the original Markdown syntax and "fix" it

>> in cases where we agree it is broken.  This may involve incorporating

>> some ideas from Markdown Extra (like emphasis_in_the_middle fix).  It

>> will try to stay true to the "spirit" of Markdown and to not add any

>> new "features."  Level 2 will add certain features, such as  

>> footnotes,

>> tables, definition lists, unindented code blocks, etc.


In my Markdown Extra spec, level 1 is "Markdown" and level 2 is  
"Markdown Extra".


> Excellent. Without this separation between Level1 and 2, I'd say the

> spec is a really bad idea.


I disagree with this statement: having a spec with no separation  
between the two is better than no spec at all since one could take the  
given spec, remove some features, and get a good specification for how  
to parse plain Markdown. Most of the work would already have been done.

That said, I agree that it's a good idea to have a way to read the  
spec ignoring the extra features if you want to base yourself on it to  
implement plain Markdown.


>> 4. We'll agree that Level 1 is required for all "M-Spec"

>> implementation, while Level 2 is optional.


I wonder where you would put PHP Markdown's no-markup mode (ignoring  
HTML) into this. Is this a level 2 feature?


>> M-Spec implementations can choose whether to implement

>> some (or all) Level 2 features, but they should avoid implementing

>> similar-but-not quite features.  E.g., you don't have to implement

>> footnotes, but if you do, please do it as specified in Level 2 spec.

>> All implementations should also be clear as to which Level 2 features

>> are supported and which are not.


That's a good idea in general, but I'm not sure it should be  
explicitly forbidden in the spec. If the spec defines something and  
you decide to do it differently, then you're obviously not following  
the spec, and probably for a reason.

But anyway, I'd rather begin by defining something that works, and  
leave conformance requirements for later.


>> It could be as simple as giving the

>> user a check list.  All implementations that support Level 2 features

>> should have an option of turning them of.


Again, I'm not keen to enter conformance requirement before having a  
working spec. But while I agree with you sentiment that implementors  
should make an effort in that direction (as I'm doing by maintaining  
PHP Markdown and PHP Markdown Extra side by side), I don't think it's  
fair for implementors to require them to implement that kind of  
customizability. Keep in mind that having various modes makes a  
program harder to test, more bug-prone, and may prevent some  
optimizations.


> Although I would like to add one thing, not only should the various

> implementations be able to turn Level2 on or off, is should be

> preferred (or maybe required?? thought anyone?) that each Level2

> feature can be turned on or off individually. For example, assuming

> wikilinks become a Level2 feature, I dislike them (yes I know, I wrote

> the wikilink extension for python-markdown), so I don't want them at

> all even if I need footnotes.


While I partly agree with that sentiment, I don't think the spec  
should require this for the reason above. The more modes a program  
has, the more difficult to implement and test the program will be. I  
don't feel the spec should burden implementors more than necessary.


>> 5. I suggest that for everyone's sanity we divide both levels of the

>> specs into a "macro" and "micro" part.  The first ("macro") part will

>> tell us how the text is to be chunked into headers, paragraphs,  

>> lists,

>> quotations, etc.  I think this part would be best described as an

>> algorithm for turning markdown text into a tree of "block-level"

>> nodes, where each node has certain "type" ("paragraph", "list item",

>> "quotation", "code")  The second ("micro") part will tell us what to

>> do with the text inside those nodes.


The tree is a good idea (and is how I intend to spec Markdown Extra).


>> I think that this would be best

>> described as a list of substitution rules that would be run in a

>> particular order (to clarify precedence).  This also would allow us  

>> to

>> divide the work: we could have a "macro" working group and a "micro"

>> working group.


Two separate groups working on block-level and inline-level syntaxes  
in parallel would be like having two people each engraving a different  
side of a coin at the same time: there are too much interactions  
between the two for them to be defined separately, and I don't think  
the spec is going to be big enough to justify this anyway.


> Works for me. But why not just call them "block-level" and "inline"

> rather than "macro" and "micro"?


Indeed.


Michel Fortin
michel.fortin at michelf.com
http://michelf.com/