Optional features (was: Markdown Extra Specification (First Draft))

[Aristotle Pagaltzis]

* Sherwood Botsford <sgbotsford at gmail.com> [2008-05-07 22:10]:

> THAT said, however, maintaining perfect backward compatibility

> slows down progress.


I don’t know. It seems to me perfect backward compatibility is
not even possible, considering that Markdown.pl is not set in
stone (John takes bug reports and writes fixes, every so often)
and yet is not formally defined anywhere. As such, there is no
way to say what is backward compatible and what isn’t. I think
at most, backcompat for the purposes of a spec for Markdown can
only be defined as targetting a particular feature set, but not
an exact implementation of it.

That is, after all, the entire reason for the spec effort in the
first place.


> Can markdown extra have a configuration file:

> The default behaviour is to emulate markdown.

> The configuration file allows for new features that don't fit

> well into the old set.


Optional features are dangerous and impede interoperability.

Everyone who ever thinks about chipping in on the design of
a spec should read [section 5 of RFC 3339][1]. That RFC is
a spec for a particular datetime format, but section 5 is
largely agnostic of the nature of the format, and lays down
the principles according to which the design decisions for
this format were made. [Section 5.3][2] is the part with
direct relevance to your stipulation, but the entire section
is readworthy.

[1]: http://tools.ietf.org/html/rfc3339#section-5
[2]: http://tools.ietf.org/html/rfc3339#section-5.3

One problem is that every new option leads to a geometric
increase in the number of feature combinations that have to be
tested.

Another issue is that Markdown is a document format. If it has
many optional features, what are the chances that if I send you
a document ostensibly written in Markdown that will work in your
implementation of Markdown exactly as it did in mine? You really
really don’t want to have to wonder.

This was a major reason why SGML mostly failed, f.ex., and only
gained traction when it was restandardised as XML. SGML had
legions of optional author-friendly features that it made it an
extreme amount of work to implement a parser that correctly
implemented the entire spec. The XML working group sat down and
basically chucked out 95% of the optional features and made the
rest mandatory. The rest is history.

Optional features in a document format are an invitation for
interoperability problems. Since the entire point of the Markdown
spec effort was to reduce existing interoperability problems,
I strongly advise that as little as possible in the spec be made
optional. Ideally, nothing would be.

It is, mind, perfectly fine to have two (or maybe three?) specs
of which one is a superset of the other, as seems to be Michel’s
current thrust with Markdown vs Markdown Extra. Assuming that no
feature in either spec is optional, that means you would be able
to expect Markdown Extra documents to work in all Markdown Extra
processors, and all Markdown documents to work in all Markdown
and Markdown Extra processors. The scope of the problem is much
smaller in such a scenario, enough so to be perfectly tractable.

Regards,
-- 
Aristotle Pagaltzis // <http://plasmasturm.org/>