[ANN] Babelmark

[Aristotle Pagaltzis]

* Tomas Doran <bobtfish at bobtfish.net> [2008-03-23 13:10]:

> I've got [Devel::Cover] to measure coverage, and it (on [my

> trunk], which is close to 1.0.17) says that my coverage is

> almost perfect:

>

> File                                stmt   bran   cond    sub   time  total

> ---------------------------- ------ ------ ------ ------ ------ ------

> blib/lib/Text/Markdown.pm      99.8   94.2   72.0  100.0  100.0   87.0   

> 96.9

>

> Which is *just wrong* - as I know there are piles of corner

> cases that my tests don't adequately cover. :(

>

> Probably a lesson to point out that automated coverage analysis

> isn't too useful in some cases...

>

> [Devel::Cover]: 

> http://search.cpan.org/~pjcj/Devel-Cover-0.63/lib/Devel/Cover.pm

> [my trunk]: http://svn.kulp.ch/cpan/text_multimarkdown/trunk/


Yes, well. If you write one test case that causes the code to
perform a particular pattern match, then the match expression is
considered covered… even if uses a 500-line regex that parses an
entire file format.

This problem basically occurs whenever a separate language is
embedded into the main code: SQL queries, XPath expressions,
templates in some template mini-language, whatever. If the
coverage metric doesn’t cross the language barrier right along
with the code, it breaks down at these points.

Regards,
-- 
Aristotle Pagaltzis // <http://plasmasturm.org/>