[RSS 0.91]
Monday 29 October, 2007
#XSLT: Test case update

A couple of months I published some results running Arabica against part of the OASIS XSLT conformance test suite. I've done a bit of work since then, and so it's time to update the numbers

Run Failures Errors Skips
attribvaltample12001
axes130002
boolean90001
conditional23000
conflictres35001
copy62000
dflt4000
expression6006
extend4004
impincl29302
lre221100
match321401
math107100
mdocs18007
message16202
modes17000
namedtemplate19001
namespace1333900
node21000
output1087801
position1117015
predicate58000
processorinfo1001
reluri11102
select85006
sort377010
string133408
variable70700
ver5004
whitespace220010
Total1421174093

Since the last published results, I have one more skip and 20 less fails. My little spreadsheet (the first I have ever constructed, career fact fans) says I'm running 1328 tests altogether, with a pass rate of 86.9%.

A failure means the test ran, but did the wrong thing. An error means it threw an exception, didn't compile the XSLT, or something similarly unexpected. A skip means the test deliberately wasn't run because of some known deficency in my code. It might be a feature I haven't implemented, the test is just plain wrong (there are a couple of these), the test is Xalan specific, or some other thing. Skips come in three flavours - don't bother at all, shouldn't compile, or shouldn't run. If a test that's not expected to compile does, or one that shouldn't run suddenly starts working, that's actually flagged as a failure. There aren't any tests doing this in these results.

Not every failure represents a unique bug. Similarly not every skip represents a unique deficiency. The biggest set of failed tests, the 78 output failures, I haven't investigated in depth but I suspect many of those are related to either HTML output (which I don't do) or text output (which the test harness can't currently compare).

These results are from current Subversion head, built on Windows XP using Visual Studio 8 and expat.

Enter your comment

Name Email
URL
If you have an account, please log in.
No account? Just create one.


SourceForge Project Page

Jez Higgins