SAXON: Change History

This file describes changes for versions 6.0 through to 6.5.4. Earlier history is logged here:

• Changes from version 5.0 to version 5.5.2
• Changes from version 2.0 to version 5.0

At the time of writing the most recent release of Saxon is version 8.4. For information about current Saxon versions, see http://saxon.sf.net/.

Changes in version 6.5.4 (2005-06-22)

The TransformerFactoryImpl (Saxon's implementation of the JAXP TransformerFactory class) now supports the method setFeature introduced in JAXP 1.3. However, the "secure processing" feature which the specification says all implementations must provide is not supported.

The Controller (Saxon's implementation of the JAXP Transformer class) now implements the reset() method introduced in JAXP 1.3. Note that the reset() method does not clear the document pool (the collection of documents loaded using the document() function. This is because the purpose of resetting a Transformer rather than creating a new one is in order to reuse resources.

The Saxon tree models implement DOM interfaces. This support has been upgraded so that all DOM level 3 methods are present, as required in order to compile the code under JDK 1.5. In many cases, the new methods are trivial implementations, that is, they typically return null or throw an UnsupportedOperationException if called.

In the NodeInfo interface, the method isSameNode() has been renamed isSameNodeInfo() to avoid conflict with the DOM Level 3 interface of the same name.

To allow compilation under JDK 1.5, variables named "enum" have been renamed.

The algorithm for generating IDs for attribute and namespace nodes has been changed in both the tiny tree and the standard tree. The previous algorithm did not guarantee that the IDs consisted of ASCII alphanumeric characters, as required by the XSLT 1.0 specification.

In version 6.5.3 the following bug was present, and has now been fixed: When comparing two nodes for identity (e.g. when evaluating the union operator |), an element, text, comment or PI node may be considered identical to an attribute or namespace node if they happen to be at the same offset in their respective data structures. This problem applied to the TinyTree only.

A performance bug in the implementation of result tree fragments has been fixed. The code in FragmentValue.java used the construct new Vector(20, 20) to allocate space for nodes on the tree; the effect of this is that a fixed allocation unit of 20 items is used, meaning that the cost of constructing the tree increases as the square of the number of nodes.

Several bugs in xsl:number have been fixed:

• The implementation now correctly handles a format pattern that contains no formatting tokens, for example format="*" (the resulting output takes the form *12*). {test numb26}

• The first non-alphanumeric token in the format picture is no longer treated as a separator token, so if multiple numbers are output using a format picture of (1), the output is now (1.2.3) rather than (1(2(3). {test numb35}

• Saxon now returns the correct results for <xsl:number level="any"> when the context node is an attribute that does not match the count pattern, and when its parent is an element that does match the count pattern. {test numb32}

An error is now reported if there are two templates in different stylesheet modules with the same name and the same import precedence, provided (a) that there is no template with that name and higher import precedence, and (b) that the template is actually referenced in an xsl:call-template instruction. {test error052}

An error is now reported if a namespace prefix used in the exclude-result-prefixes attribute of the xsl:stylesheet element of an imported or included stylesheet module has not been declared, unless the module is in forwards-compatible mode (for example, because it specifies version="2.0". {test error235}

Within xsl:for-each, any xsl:sort elements must now precede any other instructions. {test error172}

Some previously unreported errors have been found as a result of running Saxon 6.5.4 against the test suite for Saxon 8.5. The following bugs have been fixed:

• Saxon 6.5.3 did not fail cleanly when there are too many nested xsl:apply-templates calls. {test error051}

• Saxon 6.5.3 threw a NullPointerException if an xsl:stylesheet or xsl:transform element appeared as a child of another element in the stylesheet. {test error236}

• Saxon 6.5.3 failed when processing a literal result element appearing within an xsl:fallback instruction. {test ver23}

• Saxon 6.5.3 produced an ArrayIndexOutOfBoundsException when serializing an element that (a) is included in the cdata-section-elements attribute of xsl:output, and (b) contains a Unicode character above 65535 (a "supplementary character"). {test output176}

• Saxon 6.5.3 threw an ArrayIndexOutOfBounds exception when calling an extension function if the target class contained both a matching static method with N arguments and a matching non-static method with N-1 arguments.

• Saxon 6.5.3, when operating in forwards compatibility mode, does not ignore all 1.0 errors (for example, XSLT 2.0 features) appearing within the children of an unrecognized top-level element such as the XSLT 2.0 xsl:function element.

• Saxon 6.5.3, when operating in forwards compatibility mode, does not always treat an optional attribute with an unrecognized value as if the attribute were not specified. This is a difficult problem to fix across the board, but it has been fixed for some particular cases that arise when Saxon 6.5.4 is presented with an XSLT 2.0 stylesheet, for example if the mode attribute of xsl:template or xsl:apply-templates is not a valid QName then the attribute is ignored {test cnfr18}. A match pattern that XSLT 1.0 does not allow is now treated (when in forwards compatibility mode) as a pattern that no nodes will match {test ver24} (this is not an explicit rule in the XSLT 1.0 specification, but seems to be the best thing to do given the intent behind forwards compatibility mode.)

• In Saxon 6.5.3, the JDOM adapter (when used with the JDOM 1.0 release) did not correctly handle the namespace axis, nor the DocType node in a JDOM tree. It also did not allow access to attributes in the XML namespace (such as xml:space)

Support for FOP has been dropped.

Saxon 6.5.4 is no longer supported under JDK 1.1. In consequence, Instant Saxon, which relied on the Microsoft JVM, is no longer available.

The applet support module XSLTProcessorApplet, and the sample HTML pages illustrating use of Saxon as an applet, have been dropped.

Changes in version 6.5.3 (2003-08-10)

Added the EXSLT functions in package math: abs, acos, asin, atan, atan2, constant, cos, exp, log, power, random, sin, sqrt, tan. Thanks to Simon St. Laurent for these.

Changes in version 6.5.2 (2002-04-11)

The mechanism for keeping stylesheet signatures in the namepool has been removed. It caused a creeping "memory leak" in continuously running services, and is not really needed. It was invented to allow namepools to be copied, but this facility has never been properly documented or tested. Instead, there is now a simple check that the source document and stylesheet are using the same namepool.

Changes in version 6.5.1 (2002-02-08)

Added extension functions saxon:pause-tracing() and saxon:resume-tracing()

Suppress lazy evaluation of assignable variables. (This was designed to prevent a stack overflow, it didn't succeed, but it seems a good idea anyway).

Changes in version 6.5 (2001-11-15)

This is primarily a maintenance release for error clearance, however, it introduces an important incompatible change for XSLT conformance reasons.

XSLT 1.0 conformance

To use XSLT version 1.1 features, the stylesheet should now specify version="1.1". The W3C XSL Working Group has announced that XSLT 1.1 will not be progressed beyond the working draft stage. Therefore, in the interests of standards conformance and stylesheet portability, Saxon 6.5 allows version 1.1 features to be used only if the stylesheet specifically indicates that it intends to use them. Specifically, this affects three features:

• The xsl:document element (but not saxon:output, which is a synonym)
• The xsl:script element (but not saxon:script, which is a synonym)
• The ability to use a result tree fragment (also known as a temporary tree) in a context where a node-set is required, typically a path expression or filter expression.

To use these features, the stylesheet must invoke forwards-compatible mode. This is done by specifying any value other than "1.0" as the value of the version attribute of the xsl:stylesheet element, or as the value of the xsl:version attribute on a containing literal result element. For clarity, I recommend setting version="1.1".

Note that for the third feature, the ability to use a result tree fragment as a node-set, it is the xsl:variable element that creates the result tree fragment that must be executed in forwards-compatible mode, not the expression where it is used.

When the stylesheet specifies version="1.0", the saxon:node-set() or exslt:node-set() function can be used to convert a result tree fragment to a node-set.

Security advice: disabling extension functions

A new switch has been added to disable use of extension functions, other than the system-supplied Saxon and EXSLT extension functions. The switch can be set from the command line (-noext option) or by calling:

setAttribute(FeatureKeys.ALLOW_EXTERNAL_FUNCTIONS, new Boolean(false))

on the TransformerFactory object. This switch is useful when running Saxon as part of a servlet environment, if the stylesheets to be executed are untrusted. It prevents the security risk of an untrusted stylesheet invoking arbitrary Java methods on the server machine that use, modify, or delete privileged system resources.

JAXP 1.1 Support

The reference implementation of JAXP 1.1 available from Sun has not changed for some time, and has one or two annoying bugs. A much more recent version, in which these bugs are cleared, is available from Apache as part of the Xalan product. I have therefore updated the JAXP library in Saxon using the implementation included with Xalan-j_2_2_D11. An additional benefit is that the Apache distribution includes source code, which is not available from Sun. I have made no changes to this source code (not even to change the default parser and XSLT processor).

Miscellaneous

• Added support for utf-16 output encoding
• Added support for CP852 output encoding, used in Czech and Slovak. Thanks to Zdenek Wagner [wagner@cesnet.cz] for supplying these. Other extensions for these languages (for example, collating sequences) can be found at icebearsoft.
• The saxon.jar file is now built with a manifest that identifies its main class, so you can invoke Saxon with the command line:
  java -jar saxon.jar source.xml style.xsl

Defects cleared

• 474923: Failure to reset parameters when using a Transformer for more than one transformation.
• 472844: AElfred fails to handle byte-order-mark at start of UTF-8 files. Also fixed some problems with AElfred diagnostics: reporting of byte position for a bad UTF-8 byte, and reporting of failures to resolve a URL for a DTD or other external entity.
• 441078: The original fix for this in Saxon 6.4.3 had the unfortunate side-effect that result tree fragments were referenced in the document pool for the duration of a transformation, meaning that they could never be cleared from memory by the garbage collector. This is now fixed, by rewriting the DocumentPool class. Note that it remains true that the DocumentPool is not reset automatically after doing a transform() - this caching can be useful when several transformations use the same lookup documents. To clear the pool, either create a new Transformer for each transformation, or call the new clearDocumentPool() method.
• 481258: id() function in complex predicates (an expression reduction bug)
• 463578: sum() function in complex predicates (another expression reduction bug)
• 481381: When a Writer is supplied to accept Saxon output, with encoding "UTF-8", Saxon writes an XML declaration saying encoding="UTF8".
• 472842: AElfred reports that PE reference isn't allowed in the internal DTD subset, in certain situations where it should be allowed.
• 445881: AElfred incorrectly resolves a relative URI for an external entity. It resolves it relative to the resource containing the entity reference, not the resource containing the entity declaration.

Changes in version 6.4.4 (2001-09-18)

This is primarily a maintenance release for error clearance.

Defects fixed (details at SourceForge):

• 443487 - No document number generated for a document supplied using a TransformerHandler
• 444070 - Command line -a option fails if PI contains "title"
• 444173 - NullPointer exception when stylesheet supplied as DOMSource with no SystemID
• 444075 - Multiple namespace nodes when processing a DOMSource
• 444530 - ArrayIndexOutOfBoundsException while evaluating distinct() function
• 445161 - Overriding a global parameter using xsl:import
• 448377 - dayInWeek() gives wrong answer
• 448711 - NullPointerException with misspelled attribute name in xsl:strip-space
• 449219 - week-in-year() misnamed as week-number()
• 457166 - AbstractNode doesn't implement DOMLocator
• 457176 - Identity Transformer loses comments
• 459543 - Intensional node-set dependency problem involving an AxisExpression
• 462241 - Identity transformer loses element names when using Crimson XML parser

Preview mode (the <saxon:preview> extension) was never designed to work when using a TransformerHandler in the JAXP interface, only when using the transform() method. This is now documented as a restriction.

FOP integration has been upgraded to work with FOP 0.20.1. It no longer works with previous FOP releases. However, if you need to use Saxon 6.4.4 with FOP 0.19.0, you should be able to do so by using the saxon-fop.jar issued with 6.4.3, along with the 6.4.4 version of saxon.jar.

JDOM integration has been upgraded to work with JDOM 0.7. It no longer works with previous JDOM releases. However, if you need to use Saxon 6.4.4 with JDOM 0.6, you should be able to do so by using the saxon-jdom.jar issued with 6.4.3, along with the 6.4.4 version of saxon.jar. Saxon's JDOM integration now relies on the JDOM tree being built with entities expanded (which is the default). Saxon does not yet support the new JDOMSource and JDOMResult classes included in JDOM 0.7.

Writing an attribute node when there is no open element start tag is now a recoverable error: the action depends on the recovery policy (by default, a warning is output and processing continues).

Writing a text node with disable-output-escaping when it includes a character outside the character set supported by the output encoding was previously treated as an unrecoverable error; this has changed so the processor recovers silently (by not disabling output escaping).

This means that all errors for which the XSLT specification describes a recovery action are now either (a) handled according to the recovery policy selected by the user, or (b) recovered silently, using the recovery action described in the XSLT Recommendation. None of these errors are signalled unconditionally.

The DTDGenerator, previously issued as a sample Saxon application, has been rewritten as a pure SAX application. Since it no longer makes use of Saxon, but works with any JAXP 1.1 compliant XML parser, it is now issued as a free-standing package under the Saxon project at SourceForge, with its own version number (initially 7.0).

The handling of xsl:message terminate="yes" has changed. Previously the transform() method output a message to System.err, and then returned normally. Now the transform() method outputs no message (other than the message output by xsl:message itself) but throws a com.icl.saxon.style.TerminationException, which is a subclass of jaxp.xml.transform.TransformerException. The command-line driver com.icl.saxon.Stylesheet now handles this exception, so the only change in the behavior of the command line is that a result code (1) is now returned. When using the Java API, the application can now detect the condition by catching the TerminationException.

There are a couple of new optimisations, prompted by an example submitted by Evan Lenz:

• The document() function, if it has constant arguments, is now evaluated when the expression is reduced - this typically means that it is evaluated only once if it is used in a predicate or a sort key. I still avoid evaluating the document function at compile time, in the way that all other constant expressions are pre-evaluated, on the grounds that the contents of the referenced document may change.
• Any path expression whose first component is a constant node-set (typically, a path expression starting with document() or current() or with a variable reference) is now converted to a "node-set intent" when it is reduced. This means that if such an expression is used in a predicate, the node-set will be evaluated the first three times it is used, but if it is used more than thrice, memory will be allocated for it so that it doesn't need to be evaluated again. This behaves the same way as if the node-set is assigned to a variable and the variable is used within the predicate.
• Certain path expressions and filter expressions are now statically recognized as always returning nodes from the same document as the context node. These include most path and filter expressions, with exceptions such as those beginning with document() or with a variable reference. Within such an expression, any absolute path expression (beginning with "/", id(), or key()) within a predicate is now evaluated once only, since it will have the same value for all the nodes being filtered.

A new extension function has been added for parsing the content of processing instructions: getPseudoAttribute().

Changes in version 6.4.3 (2001-07-13)

This is a maintenance release for error clearance.

Defects cleared

The following defects are cleared in this release. Full details of all Saxon bugs are now placed on the SourceForge register as soon as the problem is acknowledged to be a bug.

• 440384 Overriding an imported variable causes an exception
• 440452 The number() function accepts strings containing '+' and 'e'
• 440992 Identity transform fails
• 441000 date:dayOfWeekInMonth() gives wrong answer
• 441078 Union on nodes in temporary trees eliminates nodes that aren't duplicates

EXSLT extension library

The exslt:function element now implicitly declares the namespace http://exslt.org/functions to be an extension namespace, so that exslt:result is recognized. Similarly, saxon:function implicitly declares http://icl.com/saxon to be an extension namespace.

Changes in version 6.4.2 (2001-07-07)

Defects cleared

For details of defects cleared in this release, see the project page at SourceForge (http://www.sourceforge.net/projects/saxon).

The document() function

A failure to retrieve the identified URI, or a failure to parse the XML, is now treated as a recoverable error. By default, the error is reported to System.err, and an empty node-set is returned. As with other recoverable errors, you can change this by setting the recovery policy (e.g. -w2 on the command line makes the error fatal), or by supplying your own ErrorListener.

It is now possible for a user-defined URI Resolver to return a DocumentInfo object directly. This is illustrated by the sample URI Resolver included in the TraxExamples sample application, which is used to resolve URIs referring to simple text files. (This applies only to the document() function; when used by xsl:include or xsl:import, the URIResolver should return a SAXSource, StreamSource, or DOMSource).

Error reporting

XSLT 1.0 says that is an error to output a node other than a text node while evaluating xsl:comment, xsl:attribute, or xsl:processing-instruction; the processor is allowed to recover by ignoring the offending nodes and their descendants. Saxon previously recovered silently from this error. At this release, this error is handled in the same way as other recoverable errors: by default, it results in a warning message.

XML parsing failures detected in the source document or stylesheet are now directed to the JAXP ErrorListener, unless a SAX errorHandler has been nominated.

A PrintStream can now be supplied to the standard error listener to define the destination for error messages (the default is System.err).

There has been some tidying-up of error messages, to prevent the same message being output several times.

EXSLT Support

The core extension functions defined in the EXSLT dates-and-times module are now available: specifically date-time(), date(), time(), year(), leap-year(), month-in-year(), month-name(), month-abbreviation(), week-number(), week-in-month(), day-in-year(), day-in-month(), day-of-week-in-month(), day-in-week(), day-abbreviation(), hour-in-day(), minute-in-hour(), second-in-minute(). For details see http://www.exslt.org/

Changes in version 6.4.1 (2001-07-03)

This was an emergency patch release of Instant Saxon only. It fixed a problem whereby Instant Saxon would not run unless certain other JAXP software (for example, the Crimson parser) was present on the classpath.

Changes in version 6.4 (2001-07-03)

Documentation

I have given the API Guide a much needed overhaul, including improved descriptions of the APIs for invoking XPath expressions from Java code.

Defects cleared

The following errors were found version 6.3, and have been cleared.

JAXP 1.1 support

Saxon no longer sets itself as the default DocumentBuilderFactory for use when building a DOM. This is because the Saxon DOM implementation, being read-only, is suitable only for specialized use.

Saxon still sets itself as both the default XSLT transformer and the default SAX2 ParserFactory.

FOP integration

Saxon's FOP integration has been updated to use FOP 0.19.0

Two new attributes are available on the xsl:output and xsl:document elements, for use when method="saxon:fop":

• fop:renderer specifies the name of a FOP Renderer class, for example fop:renderer="org.apache.fop.render.pdf.PDFRenderer".
• fop:configuration specifies the name of a FOP user configuration file, for example fop:configuration="c:\config\fop.xml"

Here fop: is the prefix of a namespace whose URI must be http://icl.com/saxon/fop

These two attributes have not been fully tested.

Internal API changes and code reorganisation

These changes are made partly to improve maintainability of the code, partly to reduce its size, and partly to enable the future support of a wider variety of data structures that the XPath implementation can access (for example, non-SAXON DOM structures, databases, etc). Some of the changes will affect Java applications, especially those that make intimate use of internal Saxon implementation classes.

The NodeInfo interface

The main change is a major simplification of the NodeInfo interface, greatly reducing the number of methods and subclasses that need to be implemented to support a new kind of tree structure, but hopefully without reducing the usability of the interface or the performance of its implementations.

The interface classes that are subclasses of NodeInfo have been eliminated, (for example the old favorite ElementInfo). The only exception is DocumentInfo (representing the root node or the document as a whole). This reflects the fact that in the XPath data model, all methods are available on any kind of node. Tests that were previously written if (node instanceof TextInfo) should now be written if (node.getNodeType()==NodeInfo.TEXT). In other cases, simply replace the specific interface (for example ElementInfo) by the general class NodeInfo.

The NodeInfo interface, which is the main interface to Saxon's tree model, no longer extends the DOM Node interface. This means that methods such as getNextSibling() are no longer available on this interface. Navigation from a node should be done instead by creating an enumeration using one of the XPath axes, using the getEnumeration() method.

However, the two implementations of the NodeInfo interface, that is the standard tree and the tiny tree, continue to implement the DOM Node interface. To achieve this, the two implementation types (NodeImpl and TinyNodeImpl) both inherit from a new abstract class called AbstractNode. This class implements both the Saxon NodeInfo interface and the DOM Node interface; it also includes methods needed only for element, text, comment, or root nodes. (This is done to make these methods shared between the two tree implementations: it is not possible in Java for a class such as TextImpl to inherit both from NodeImpl containing the Saxon methods and from a generic AbstractTextImpl containing the DOM methods.)

The NodeInfo class now implements the JAXP Source interface, which means that any NodeInfo can be used directly to define the source of a transformation, with no need to wrap it in a DOMSource object. Note that if you supply the source tree in this way, it is your own responsibility to strip any unwanted whitespace nodes before XSLT processing begins. The xsl:strip-space and xsl:preserve-space instructions in the stylesheet will be ignored.

Saxon still uses DOM methods such as getNextSibling() to navigate the stylesheet tree, which is always implemented using the standard tree model. However, Saxon no longer relies on the source document providing DOM interfaces.

As well as the DOM methods, a number of other methods on the NodeInfo interface have been removed. Many of these were "shortcut" methods that weren't really needed, and which were the same in all implementations. In all cases there are alternatives available.

The getValue() method in the NodeInfo class has been renamed getStringValue(), to better reflect its meaning, and to avoid clashing with the getValue() method of the org.w3c.dom.Attr class.

The AxisEnumeration classes are now logically part of the tree implementation, so they are implemented differently for each tree structure. This allows the implementation to use the navigation mechanisms that are most efficient in each data structure.

The subclasses of Axis, which existed essentially to provide information about each axis, have been removed. Instead the Axis class itself provides this information in the form of a number of arrays, indexed by axis number. The Axis class has been moved to the com.icl.saxon.om package.

The unused utility methods in class com.icl.saxon.om.Navigator, for example isFirstInGroup() and getAncestor(), have been deleted. If you need these methods in your application, I suggest reconstructing them within your application code, based on the Saxon 6.3 source code.

The interface com.icl.saxon.om.ExtendedAttributes has been removed from the object model, as the preferred way of accessing all the attributes of an element is now to enumerate the attribute axis.

Multiple documents

In previous releases, certain information held within a document was required to be unique across all documents used within a single transformation: in particular, the document number, and the node sequence numbers. This potentially causes problems when the same source document is used in multiple transformations, perhaps running in parallel. The problems were previously avoided by rebuilding the document for each transformation, which is inefficient.

In Saxon 6.4, a document no longer contains a unique document number. The methods generateId() and getSequenceNumber() now generate numbers which are required to be unique only within a single document; making them globally unique is done by the calling code, with the aid of the document pool maintained by the Controller.

A tree implementation is no longer required to provide sequence numbers for the nodes. Instead, it is required to implement a compareOrder() method that determines the relative ordering of two nodes within the same tree. Comparison of nodes in separate trees is now done at a different level of the software.

The only extra data that a source document now contains to support Saxon transformations is:

• A reference to the NamePool used to translate between the string representations of node names and their numeric representations;
• A reference to the indexes that implement key structures. These are indexed by an internal identifier for the key definition within the stylesheet. Key definitions in different stylesheets will always have different internal identifiers, but if the same source document is transformed repeatedly using the same stylesheet (typically with different parameters), the indexes can now be re-used.

The NodeHandler interface

NodeHandler was previously an abstract class in package com.icl.saxon.handlers; it is now an interface in package com.icl.saxon. This may affect user-written node handlers, used either in a Java (non-XSLT) application, or via the saxon:handler extension element.

There is an extra method requiresStackFrame() whose value is a boolean. You can generally return false. Return true only if the node handler maintains variables or parameters that can be accessed from XPath expressions - something that is not especially easy to do.

This change also means that any user-written TraceListener will need to be recompiled.

Sorting

I have made internal changes to the sorting routines to reduce the memory used, especially when the sort involves only a single sort key. The changes are unlikely to affect many users. However, some of the methods in internal classes such as SortedSelection have changed.

JDOM support

To illustrate the way that the new NodeInfo interface can be used to create adapters for other document formats, I have built an adapter for JDOM (see http://www.jdom.org/). Although the code for this is included in the main source tree, it is issued in a separate Jar file, saxon-jdom.jar. The code is still at beta quality. A sample application showing how to use Saxon with JDOM is provided. The JDOM interface requires JDK 1.2.

This facility allows a JDOM tree to be used as the input to an XSLT transformation, or as the target for XPath expressions issued from your Java code. You can direct the output to a JDOM tree by using JDOM's SAX driver as the SAXResult destination object for the transformation.

Saxon currently makes no attempt to merge adjacent text nodes in the JDOM tree: these can arise if the two text nodes are separated by an entity boundary or by a CDATA section boundary.

Using SAXON with JDOM is not likely to be especially efficient; it requires extra memory for the wrapper data structures, and some XPath navigation routes are quite inefficent because they are not supported directly in JDOM (for example, JDOM provides no direct way of getting from a node to its siblings). It is provided partly as an illustration of how to interface other data sources, and partly for users who already have data in JDOM format. It is particularly useful to enable XPath access to JDOM from Java applications.

Ælfred XML parser

I have reviewed the changes made by David Brownell in his version of the Ælfred XML parser (available as project xmlconf in www.sourceforge.net), and have incorporated those that are relevant into the Saxon version. This is basically all changes except those required to report validation errors. Most of the changes are very minor, but there are some enhancements in the handling of character encoding: if an input file is in an encoding that Ælfred itself does not understand, it now attempts to get the Java VM to decode it. The set of character encodings available in the Java VM is platform-dependent.

SQL extension functions

Following a suggestion from René Jansen, I have changed the xsl:insert code so it now prepares the SQL statement only the first time it is executed, and reuses the prepared statement thereafter. Also, it can now handle columns that are not strings.

xsl:attribute and friends

The three instructions xsl:attribute, xsl:comment, and xsl:processing-instruction have been speeded up. Where the content of the instruction is a single text node, or an xsl:value-of instruction, Saxon now avoids the overead of setting up a new output destination; instead of processing the content as a general template body, it evaluates it directly as an expression. Where this is not the case, a streamlined output method is used that avoids many of the overheads previously incurred.

Variables and parameters

Global variables and parameters are no longer evaluated if there is another variable or parameter with the same name and higher import precedence.

An xsl:variable element containing a single text node is now treated specially, bringing the performance close to that of a String variable.

Changes in version 6.3 (2001-05-03)

JAXP 1.1

Saxon now implements the javax.xml.parsers package in JAXP 1.1 as well as the javax.xml.transform package.

If you have the system property javax.xml.parsers.SAXParserFactory set to the value com.icl.saxon.aelfred.SAXParserFactoryImpl, then any call on JAXP 1.1 interfaces to get an XMLReader will select AElfred. Moreover, Saxon itself uses the JAXP 1.1 interfaces to get an XMLReader if none has been explicitly requested, so you can now determine the parser to be used by setting this system property. The default for this property, defined by a services file in saxon.jar, selects the AElfred parser.

Similarly, if you have the system property javax.xml.parsers.DocumentBuilderFactory set to the value com.icl.saxon.om.DocumentBuilderFactoryImpl, then any call on JAXP 1.1 interfaces to get a DOM Document builder will select the Saxon tinytree implementation. However, Saxon does not call JAXP interfaces to get a Document builder: it will always choose its own. Note that Saxon's DOM implementation is an immutable DOM: you can construct the DOM by parsing a source document, but you cannot build it or modify it through the DOM API methods.

Saxon's Builder and Stripper classes have been moved to the package com.icl.saxon.om.

When a Saxon document is supplied as input to the transform() method (using a DOMSource object), in previous releases the tree was rebuilt. At this release the tree is used as is, provided that either (a) the stylesheet does not require whitespace nodes to be stripped, or (b) whitespace stripping has been disabled by calling the new Controller.disableWhitespaceStripping() method. In the cases where the tree does need to be rebuilt, a "fast path" routine has been introduced to do this: previously the same code was used as for a third-party DOM, which incurred unnecessary costs because there are so many different ways namespaces can be represented in a DOM.

When performing multiple transformations on a single source document, it is best to do the whitespace stripping once as a separate operation: this is made possible by a new method PreparedStyleSheet.stripWhitespace(), which uses the xsl:strip-space directives in a stylesheet to remove whitespace from a document (in fact, it returns a new document that is a copy of the original, with relevant whitespace nodes removed; if no whitespace stripping needs to be done, it returns the original document unchanged).

It is now possible to supply a Saxon document as the output of a transformation. (This didn't work at previous releases, though the restriction was undocumented.) The document must be empty, and the node supplied in the DOMResult object must be the document (ie. root) node.

It is now possible to start a transformation at a node other than the root node, if the input is supplied in the form of a DOM (in a DOMSource object). Global variables are still evaluated with the root node as context node, and the entire tree is available to the transformation, but the first template rule applied is not, as is usual, the match="/" rule, but the rule that matches the supplied node. The DOM supplied as input must not contain CDATA or entity reference nodes that are parents or preceding siblings of the start node.

Extension Functions

Saxon's support for Java extension functions has been brought into line with the working draft XSLT 1.1 specification.

Polymorphic methods are now fully supported. If the relevant class has several methods (or constructors) with the same name, the one that is chosen is the one that gives a "best match" to the types of the supplied arguments, following the rules in the XSLT 1.1 draft. If there is no unique method that provides a best match according to these rules, an error is reported.

Methods that return void, null, char, or byte are now handled as described in the XSLT 1.1 working draft.

There is still a restriction that extension functions cannot construct a new DOM tree and return nodes from this tree by using DOM methods. They can only return existing nodes that were constructed by Saxon itself.

Methods that expect a node-set as input can now declare the argument type as com.icl.saxon.expr.NodeEnumeration, as an alternative to com.icl.saxon.expr.NodeSetValue. This is likely to be a bit more efficient. The enumeration will always be positioned at the start when the function is called, and its position on exit can be anywhere. It is also possible to return a NodeEnumeration as the result of a function. Again, the enumeration must be positioned at the start. Returning a NodeEnumeration is especially efficient if the result is then converted to a String or a Boolean.

The rules for spelling of external function names have been brought into line with XSLT 1.1. This may require stylesheet changes. For example, the function has-same-nodes() must now be spelt as "has-same-nodes()" or "hasSameNodes()", it can no longer be spelt as "hassamenodes()" or as "HAS-SAME-NODES()". For backwards compatibility, the node-set() function may be spelt with or without the hyphen (or as "nodeSet()").

EXSLT

EXSLT is an initiative to define a standardized set of extension functions and extension elements that can be used across different XSLT processors.

Saxon now supports the EXSLT modules Common, Math, Sets, an Functions. The full list of extension functions is:

• exslt:node-set()
• exslt:object-type()
• math:min()
• math:max()
• math:highest()
• math:lowest()
• set:difference()
• set:intersection()
• set:distinct()
• set:leading()
• set:trailing()
• set:has-same-node()

plus the following new elements:

• func:function
• func:result

These have considerable overlap with functions that have previously been provided in the Saxon namespace. The Saxon versions of the functions remain available, for the time being, but the EXSLT versions are preferred.

The saxon:function and saxon:return elements have been changed slightly to conform to the EXSLT rules. Specifically: saxon:return can now appear inside xsl:for-each, provided the xsl:for-each iterates at most once. There is now a check that saxon:return is not used inside the definition of a variable or inside another saxon:return. It is an error to instantiate more that one saxon:return within a function.

saxon:closure

Following a suggestion from Christian Nentwich, I have implemented a new extension function saxon:closure(), which forms a node-set by taking the transitive closure of a node-set expression. The function does NOT detect cycles.

Defects cleared

The following errors were found version 6.2.2, and have been cleared. Many of these relate to incorrect handling of error cases, and reflect the fact that I have greatly increased the test coverage of error handling.

Changes in version 6.2.2 (2001-03-15)

Upgraded to the latest JAXP ("version 1.1 final release") dated 6 Feb 2001. Saxon now uses the JAXP binaries exactly as issued by SUN. Unfortunately the TransformerFactory issued by Sun invokes Xalan as the "platform default" XSLT processor. The saxon.jar file includes a META-INF file to override this, so there should be no problems unless you have other things on the classpath that conflict. If you want to be absolutely sure of loading Saxon rather than any other XSLT processor, set the system property javax.xml.transform.TransformerFactory to the value "com.icl.saxon.TransformerFactoryImpl", either from your application (by calling System.setProperty()), or from the command line (java -Djavax.xml.transform.TransformerFactory=com.icl.saxon.TransformerFactoryImpl classname)

Make sure you remove any older versions of jaxp.jar from your classpath to prevent any incompatibilities.

I have changed the packaging of the FOP integration, to reduce the problems this causes for people who want to rebuild Saxon or load it into a development environment such as IBM's Visual Age for Java. The FOP integration module, FOPEmitter, is now part of a separate package, com.icl.saxon.fop, and is not included in saxon.jar, but is in a separate JAR file, saxon-fop.jar. This must be on the class path if you want to use Saxon with FOP, but you can ignore it otherwise. There are no longer any compile-time references to FOPEmitter from the rest of the Saxon code, so you can recompile the product without first installing FOP, provided that you remove FOPEmitter from the source library first.

I have reinstated the ability to call Java extension functions using the namespace xmlns:ext="full.class.Name" as an alternative to xmlns:ext="java:full.class.Name". However, the "java:" form is preferred.

I added extension functions saxon:before() and saxon:after(), based on the BEFORE and AFTER operators defined in XQuery. These take two node-sets as arguments and return all the nodes in the first node-set that are before/after at least one node in the second node-set, in document order. This provides an alternative to saxon:leading(), e.g. saxon:before(*, s[1]) gets all the child elements that precede the first child <s> element.

A further refinement to class loading: if the loader returned by getContextClassLoader() fails to load a class, we now try to load the class using Class.forName(). This is all something of a black art: different things appear to work in different environments.

I have re-instated saxon:output as a synonym of xsl:document. The reason for this is that some XSLT processors object to finding an xsl:document element in the stylesheet, even when running in forwards compatible mode. Using saxon:output is therefore more portable. Note, however, that the new saxon:output is not completely compatible with the old: attribute names have changed, especially "file" to "href".

Defects cleared

The following errors were reported for version 6.2.1, and have been cleared:

Changes in version 6.2.1 (2001-02-20)

Support for Running Saxon in an Applet: I have shamelessly copied the XSLTProcessorApplet module from Xalan, which was written to run any TrAX processor from a Java applet, and have adapted it to work with Saxon. The only changes were to remove a call on a Xalan error-handling routine, and to change the package name. I have also copied and adapted the Xalan sample application which shows how to incorporate this applet into an HTML page. To run a transformation using Saxon requires saxon.jar to be downloaded to the client. At 550Kb this is fairly substantial.

There are some sample applications using Saxon as an applet in the samples/applet folder.

It is now possible to specify the CharacterSet class to be used for a named output encoding by setting the system property, e.g. -D"encoding.EUC-JP"="EUC_JP"; the value of the property should be the name of a class that implements the PluggableCharacterSet interface.

Saxon has been modified to work with FOP 0_17_0; it no longer works with earlier versions of FOP. This has required some extensions to the Emitter interface, to cater for the fact that FOP now requires an OutputStream rather than a Writer as its output destination. Note also that FOP attempts to load Xerces as its default XML parser; if you want to use Saxon's AElfred parser istead, set the system property -Dorg.xml.sax.parser=com.icl.saxon.aelfred.SAXDriver. To run FOP, include the supplied JAR files fop.jar and w3c.jar on your classpath (FOP uses the DOM SVG package which is not included in saxon.jar).

Defects cleared

The following errors were reported for version 6.2, and have been cleared:

Changes in version 6.2 (2001-02-06)

Towards XSLT 1.1

The xsl:script element is now available. It is ignored unless the language is "java". This element can be used to identify the Java class implementing an extension function as defined in the XSLT 1.1 specification. The archive attribute can be used to specify a list of URLs to be searched, but only with a JVM that supports JDK 1.2 interfaces (i.e. not with the Microsoft JVM, and therefore not with Instant Saxon). NOTE: the rules for selecting a method within this class are unchanged. In particular, where there are several methods with the same name and number of arguments, it is not predictable which will be chosen. The native Saxon techniques for identifying a Java class will continue to be used if there is no xsl:script element for the relevant prefix, with one exception: the form xmlns:prefix="fully.qualified.ClassName" is no longer supported; use xmlns:prefix="java:fully.qualified.ClassName" instead.

The element name saxon:script can be used as a synonym of xsl:script. The advantage of using saxon;script is that other processors will ignore it. This allows you to define the way Saxon will implement an extension function which may be different from the way other processors implement it. This is epecially useful if your stylesheet uses functions such as xx:intersection() which are now offered by several different XSLT processors. Note that the built-in Saxon extension functions are all implemented in the same way as user extension functions, in class com.icl.saxon.functions.Extensions; so you can use src="java:com.icl.saxon.functions.Extensions" to locate the Saxon implementation of these functions.

The Saxon class com.icl.saxon.Context now implements the org.w3c.xsl.XSLTContext interface, as defined in the XSLT 1.1 working draft. This can now be used as the first argument of a method that implements an extension function (but you can continue to use com.icl.saxon.Context if you prefer). A consequence of this change is that getContextNode() and getCurrentNode() now return a org.w3c.dom.Node rather than a com.icl.saxon.om.NodeInfo; if you want to use Saxon methods on the returned node, you will have to cast it to a NodeInfo. Note that although the getOwnerDocument() method of XSLTContext is implemented, the resulting document will not be updateable.

The xsl:apply-imports element may now take parameters, that is, it may have child xsl:with-param elements.

The xml:base attribute is implemented. This can be used to change the base URI of an element (in either the source document or the stylesheet) for the purposes of the document() function. A new extension function is provided (largely for diagnostic purposes): saxon:base-uri() returns the base URI of the context node. Note that the terms "base URI" and "system ID" have in the past been used synonymously. This has been tidied up. The System ID refers to the entity (ie. file) in which an element was found, and is useful for diagnostics in conjunction with the line number. The Base URI defaults to the System ID, but may be changed using xml:base, and is used for resolving relative URIs appearing in calls to document() or to xsl:include and xsl:import.

If you supply your own URIResolver, you can use the base URI any way you like. For example, if the relative URI is the key of a record in a database, you could use the base URI to hold information identifying the database, e.g. the JDBC connection details.

Performance

I have changed the algorithm used for generate-id(). The existing algorithm was very inefficient, which was proving a problem with Muenchian grouping algorithms that rely on this function. It performed particularly badly when using the tinytree data structure with a large source document. The new algorithm is much faster, especially with the tinytree structure. It produces different results from the old algorithm, and is different for the two tree implementations.

Defects cleared

The following errors were reported for version 6.1, and have been cleared:

Changes in version 6.1 (2001-01-09)

Towards XSLT 1.1

The saxon:output element is renamed xsl:document, and its file attribute is renamed href. (At this stage, though, it still takes a filename rather than a URI). The next-in-chain attribute is renamed saxon:next-in-chain and is now available on both xsl:output and xsl:document. The href attribute is mandatory: if saxon:next-in-chain is also present, ot determines the destination of the output of the chained stylesheet. The indent attribute must now be either "yes" or "no"; the previous option to specify the level of indentation is now replaced by saxon:indent-spaces="integer", on both xsl:output and xsl:document. The omit-meta-tag and character-representation attributes, similarly, are prefixed "saxon:" and are available on both elements.

The xsl:output element (like xsl:document) now allows all its attributes to be specified as attribute value templates.

A side-effect of this change is that xsl:output properties are now ignored when running in preview mode, because the properties cannot be evaluated until the source document is available.

The saxon:user-data attribute of saxon:output is removed. Instead, any number of user-defined attributes may be defined on both xsl:output and xsl:document. These attributes must have names in a non-null namespace, which must not be either the XSLT or the Saxon namespace. These attributes are interpreted as attribute value templates. The value of the attribute is inserted into the Properties object made available to the Emitter handling the output; they will be ignored by the standard output methods, but can supply arbitrary information to a user-defined output method. The name of the property will be the expanded name of the attribute in JAXP format, for example "{http://my-namespace/uri}local-name", and the value will be the value as given, after evaluation as an attribute value template.

The special provisions in XSLT 1.1 for defining what happens when you use xsl:document while the current output destination is a temporary tree are not yet implemented.

URI handling

The standard URI resolver now accepts URIs containing a fragment identifier. The fragment identifier must be the value of an ID attribute within the referenced XML document. The effect is to return a tree containing the subtree rooted at the element with that id. This facility works for URIs contained in the document() function and in xsl:include and xsl:import. If there is no element with the required ID, an empty tree is returned (i.e. a root node with no children).

As a result, embedded stylesheets are now working again. In fact, there is no special code to handle embedded stylesheets: anywhere a stylesheet module can be referenced by URI (including the command line, the xml-stylesheet processing instruction, and the href attribute of xsl:include and xsl:import), a URI containing a fragment identifier can be used, and this will select the relevant subtree in the same way as for any other XML document

In response to complaints about Saxon incompatibility with Xalan, and in order to get the JAXP 1.1 example programs working, I have changed the behaviour of both the AElfred SAX2 driver, and the SAXON standard URI resolver, so that if no systemId is specified for a document, then relative URIs are interpreted relative to the user's current directory. Equally, if the base systemId specified for the document is a relative URI, this is expanded using the current directory as the base. Arguably this behaviour is non-compliant with the SAX2 specification, which states that the systemId must be an absolute URI, but it seems to be a useful convenience.

This means that every document, and every node, now has a base URI: it can never be null. A minor side-effect is that I have withdrawn the ability for saxon:node-set() to take a string (or number, or boolean) as an argument: it must now be a result tree fragment or an existing node-set. The reason is that there is no obvious way of constructing a base URI.

JAXP 1.1

Saxon 6.1 implements the new TrAX interface, now defined as part of JAXP 1.1: see JSR-63. Saxon implements the javax.xml.transform interfaces. Saxon does not implement (or use) the javax.xml.parsers interfaces.

This has involved fairly extensive changes to the Java API for invoking Saxon. Some of the main implications are:

• Error handling is now via an ErrorListener object which may be user-supplied. The standard error listener will now report any number of compile-time errors, but will stop at the first run-time error (XSLT recoverable errors are reported as warnings).

• The package names for classes such as Templates and Transformer have changed.

• The exceptions that are thrown have changed: in general, internal routines now throw javax.xml.transform.TransformerException or one of its subclasses, where previously they threw a SAXException. However, I have removed the exceptions from many internal methods entirely, where they were not needed. In some cases this is achieved by throwing an Error instead, which simplifies exception handling in calling routines.

• The Transformer (and hence Saxon's Controller class) is no longer a SAX XMLFilter; instead it is possible to get an XMLFilter that performs the transformation using the SAXTransformerFactory class.

• The com.icl.saxon.StyleSheet class no longer acts as the TrAX factory; its only responsibility now is to implement the command line interface on top of the TrAX Java API. It now does this almost entirely using TrAX-defined interfaces rather than Saxon internal interfaces, so the class provides a good demonstration of how the TrAX API can be used.

• The classes OutputDetails and OutputFormat have disappeared; output properties are now represented throughout the system using a java.util.Properties object as defined in the TrAX API. Unlike the old OutputDetails class, the Properties object only describes the format of the required output, not its location. The location is represented by a Result object.

• The class OutputManager has disappeared: its functions have been merged into the Outputter class. Code that switches output to a new destination should now call Controller.changeOutputDestination() to get a new Outputter, and should remember the previous Outputter so that it can be reinstated when calling resetOutputDestination().

• The Emitter class is now an abstract class rather than an interface; this enables null implementations of many of its methods to be provided. It also enables Emitter to implement the TrAX Result interface, which means that wherever the TrAX API allows a Result object to be supplied defining the destination of a transformation, Saxon allows you to supply an Emitter as the destination. (A side-effect of this is that FragmentValue, which implements result tree fragments, is no longer an Emitter, as it cannot be both an Emitter and a SingularNodeSet; instead, it is possible to call FragmentValue.getEmitter() which provides an emitter front-end for building the tree).

• If a user-specified ContentHandler is used as the output destination, it will now be notified of requests to disable or re-enable output escaping using special processing instructions inserted into the event stream (the names of these are defined as constants in the class javax.xml.transform.Result). Saxon also now uses a processing instruction to notify the ContentHandler if output has been suppressed because the result tree is not well-formed; if the ContentHandler is prepared to accept ill-formed output, it can reject this notification by throwing a SAXException with the message text "continue".

• Setting of configuration parameters (such as supplying a TraceListener or MessageEmitter) is now generally done through the setAttribute() method of the TransformerFactory. The names of the relevant attributes are defined in com.icl.saxon.FeatureKeys. Methods such as setXMLReader() on the Controller and PreparedStyleSheet classes have therefore disappeared.

• For the same reasons, the ParserManager.properties file has disappeared.

Effect on Java-only applications

I have tried to minimize the impact of the TrAX changes on Java-only applications, but inevitably some incompatible changes have crept in. The main ones are:

• Many methods now throw TransformerException instead of SAXException.
• The Builder.build() method now expects a SAXSource rather than a SAX InputSource. The SAXSource object contains information about both the input source itself, and the SAX2 parser (XMLReader) used to process it.
• The old OutputDetails object is replaced by a Properties object to define the output properties and a Result object to define the output destination; the methods setOutputDetails() and resetOutputDetails() are replaced by changeOutputDestination() and resetOutputDestination().

In future I want to align the Java-only processing model more closely with TrAX, so that the set of processing rules defining the transformation becomes another kind of Templates object.

Defects cleared

The following errors were reported for version 6.0.2, and have been cleared:

Changes in version 6.0.2 (2000-12-08)

Defects cleared

The following errors were reported for version 6.0.1, and have been cleared except where otherwise noted:

Integration with FOP has been restored. Saxon now works with FOP version 0_15_0.

NamePools: I have changed the approach, so that instead of making a copy of the stylesheet name pool for each transformation, the name pool is now shared (which means its updating methods are now synchronized, to ensure thread-safety). This shouldn't affect most users, unless you are manipulating NamePools explicitly. It is still possible to have multiple name pools, but you now need to organise any copying yourself if this is what you want to do. For 99% of users, it should be possible to ignore NamePools entirely and just leave the system to use the single default name pool all the time.

The following changes are for conformance with the (imminent) XSLT 1.0 errata:

• Copying a namespace node using xsl:copy or xsl:copy-of now works as specified.
• When xsl:number level="any" finds no nodes that match the count pattern, the generated sequence number is now an empty list rather than the number zero.
• When the second argument to document() is an empty node-set, this is now signalled as an error. Previously it was handled as if the second argument was omitted.
• It is now an error to use the key() function within the match or use attributes of xsl:key. Previously this was only an error if the reference was circular, and if the key was actually referenced within the stylesheet.
• When xsl:number is called using the value attribute, the behavior for negative, infinite, and NaN values is now as specified.
• In a numeric sort, NaN is now sorted before all other values. Previously it was sorted last.

Changes in version 6.0.1 (2000-11-28)

Defects cleared

The following errors were reported for version 6.0, and have been cleared except where otherwise noted:

Other changes

Warning messages (issued typically when a node matches more than one template rule) are now limited in number: only the first 25 are displayed.

Changes in version 6.0 (2000-11-17)

In Saxon 5.5, I introduced a change that allows a result-tree-fragment to be implicitly converted to a node-set. I did this in anticipation of changes in XSLT 1.1, and to allow interoperability with MSXML3. However, Microsoft have now withdrawn this facility and conform fully to the XSLT 1.0 rules, so in order to protect Saxon's reputation for 100% conformance, I have decided to withdraw the facility too. It can still be used, however, if the stylesheet specifies version="1.1". For more details, see Conformance

Defects in version 5.5.1

The following errors are cleared in version 6.0:

New XSLT facilities at version 6.0

Added support for two new output encodings on xsl:output: iso-8859-2 and cp1250.

Added two attributes to xsl:output (not yet available in saxon:output):

• saxon:omit-meta-declaration="yes" or "no" controls, for HTML output, whether a META tag is added after the HEAD start tag. The default is to include it, setting this attribute to "yes" causes it to be omitted, which is useful if you prefer to generate it yourself.
• saxon:character-representation gives the preferred representation for special characters. For method="xml" the values are "hex" or "decimal" controlling whether character references should be in decimal or hexadecimal notation: this aplies only to characters outside the selected encoding. For method="html" two values may be given, separated by a semicolon. The first gives the representation for non-ASCII characters that are present in the target character set: the values are "native", "entity", "decimal", or "hex". The second gives the representation for characters outside the selected encoding: the same values can be used, except for "native". For example if encoding="iso-8859-1", then saxon:character-representation="native;hex" causes characters in the range 0-255 to be written as themselves (except less-than, ampersand, etc which are always written as entity references, as is non-breaking-space), and causes characters outside this range to be written as hexadecimal character references. By contrast "entity;decimal" causes characters in the range 160-255 to be written using HTML-defined symbolic entities, and characters above 255 to be written in decimal. The default is "entity;decimal".

Added a new extension function saxon:showNodeSet(). It takes a single argument that is a node-set, produces a diagnostic print of the node-set on System.err, and returns an empty string.

Added an extension function saxon:getContext() to get the context object. Only really intended for diagnostic use.

Command line changes

Added an option to choose the tree implementation (see below): -ds for the standard tree, as used in previous releases, -dt for the "tinytree" which is new to this release. The tinytree is the default: it takes up less memory, is faster to build, and generally appears to perform better in most circumstances.

The -a option on the stylesheet, which causes the source document to be processed using the stylesheet identified from its xml-stylesheet processing instruction, now uses the same logic as the getAssociatiedStylesheets() method in the TrAX interface. This means multiple (cascading) stylesheets are now supported. However, embedded stylesheets (identified by href="#id" in the xml-stylesheet processing instruction) are not supported at this release.

Java API changes

There have been a great many internal changes, but relatively few that impact directly on the high-level transformation API. In particular, if you only use TrAX interfaces, there are no changes. Otherwise, the main points to note are:

• If you use the Builder to build a document tree, there are several changes. Firstly, Builder itself is now an abstract class, it has concrete classes for the two kinds of tree. Secondly, the Builder must now be supplied with a NamePool for holding all XML names. All documents used in a transformation must use the same NamePool. Thirdly, the Builder is no longer a SAX2 ContentHandler, instead it implements Saxon's own Emitter interface. You can front-end it with a ContentEmitter if you want to drive it via SAX2 events. Finally, it is no longer possible to strip whitespace nodes from the completed tree, this has to be done using the Stripper while the tree is being constructed. This has implications if you want to use the same tree in memory as input to different transformations controlled by different stylesheets.
• If you have a user-written Emitter to handle output, you will have to change it, as the interface has changed. In particular, names are now passed as integer references to the NamePool, see the XMLEmitter for an example of how these names are handled.
• If you set up parameters to the transformation using a ParameterSet, this has also changed to use integer namecode rather than Name objects. The TrAX interface, which adds one parameter at a time, is unaffected.
• If you use extension functions that access the Saxon tree structure as a DOM, you need to be aware that the representation of namespace nodes has changed. Namespace nodes are no longer presented as attribute nodes in the DOM: in fact they are not presented at all. This simplifies the model, and is consistent with the DOM specifications. You can still access namespace nodes, but only by casting the DOM objects to Saxon's own classes.

  This release adds support for pluggable character sets: if you specify xsl:output encoding="class-name", class-name should be a class that implements com.icl.saxon.output.PluggableCharacterSet. The class must provide two methods, one that determines whether a given character is present in the character set, and one that gives the name of the encoding to be used by the Java VM for translating Unicode characters into a file with this encoding.

  To use free-standing XPath expressions and patterns from a Java application, you now need to supply a StaticContext object when parsing the expression. This object handles the resolution of variable names, namespace prefixes, and function names occurring within the expression. For convenience the StandaloneContext object is provided for this purpose. This class allows namespace prefixes to be declared so they can be used in an expression. It also allows external functions to be called (but not functions defined in your XSLT stylesheet). It does not allow the expression or pattern to contain references to variables.

Internal Changes

These details should only affect you if you access intimate internal interfaces or use the Saxon source code.

There are two big changes to the internals of Saxon at this release: a new implementation of the tree structure, and a new system for handling names.

The tinytree implementation

I have introduced an alternative tree implementation (called "tinytree"). This is designed to reduce the number of Java objects created: the tree is sliced vertically rather than horizontally, so instead of having one Java object per node, there is one Java array for each property of the nodes, with an entry in the array for each node. The effect is to greatly reduce the Java memory management overheads. The existing tree structure remains available, and is always used for the stylesheet tree. It is also currently always used for the intermediate result tree created when saxon:output next-in-chain is used.

To select the standard tree structure, use -ds on the command line. To select the "tinytree" structure, use -dt. The default is -dt. You can also select the tree structure using a method on the Controller class.

The tinytree is smaller than the standard tree, as the name suggests, and it is also faster to build. However, it may be slower to navigate. So if you have a small document that is built once in memory and used repeatedly, the standard tree implementation is probably better. In other cases, however, the tinytree usually wins.

Name pools

I have made radical changes to the way names are managed. Previously, the NamePool object contained a pool of names, but its only real purpose was to avoid the memory overhead of storing each name many times. Now, Saxon takes advantage of the NamePool to avoid storing references to Name objects on the tree at all: instead it stores a "namecode": an integer which can be used to identify the name within the NamePool.

A namecode has 4 bits unused, 8 bits representing the prefix, and 20 bits acting as a pointer to an entry in the namepool containing the local name and namespace URI. Two names are therefore equal if the namecodes are the same in the bottom 20 bits. The value in these 20 bits is also referred to as the fingerprint of the name.

All searching for objects by name is now done by comparing fingerprints; no string comparisons are involved. Fingerprints are used not only for matching names used in XPath expressions to refer to the source document, they are also used for all matching of names within a stylesheet, for example variable names, template names, mode names, key names, and decimal format names.

The name pool is also used for storing namespace declarations: each prefix/URI pair is allocated a namespace code, and all manipulation of namespace nodes in the tree is done using these integer codes.

A consequence of this is that all documents used in a transform must use the same NamePool. This has some implications on the Java API. With simple use of the API, you needn't worry about name pools, they will be taken care of automatically. However, if you are operating a continuously running service in which both source documents and stylesheets are cached in memory, you may need to exercise some care to specify the right NamePool when each document is built.

The model is further complicated by multi-threading. Rather than have synchronization problems with multiple threads updating the same NamePool, the NamePool used to build the stylesheet is copied (imported) into the NamePool used to build the source document, before parsing of the source document starts. When you use the transform() method to parse and transform an InputSource, this happens automatically. However, if you want to build the document yourself, and transform it using transformDocument() (which allows you to run more than one transformation on the same document), then you must manage the NamePool merging yourself. The system does include checks that the NamePools for the stylesheet and source document are compatible, though these are not completely foolproof.

The use of namecodes rather than String names has affected many internal interfaces, and some of these are interfaces that are also exposed externally. For example, the ParameterSet object which is used to pass parameters from a calling template to a called template can also be used to supply global parameters to the Transformer. The parameters in a parameter set are now identified by an integer fingerprint rather than a string name. You can get the integer namecode from the NamePool using the getFingerprint() method; alternatively use the TrAX method addParameter(), which still takes the name as a String.

The Emitter interface has also changed to use name codes; if you have written your own Emitter, the code will have to be modified.

Other changes

The classes and interfaces used in Saxon for manipulating collections of attributes now implement the SAX2 Attributes interface.

The standard XPath functions have been extensively revised. The main change, apart from tidying up the code, is that the functions are now responsible for evaluating their own arguments, which enables some optimisation, especially when the arguments are node-sets: they can now be evaluated using knowledge of the data type required. For example, the not() function now stops as soon as the first node in the argument node-set is found.

Some of the little-used methods on the NodeInfo interface have been moved as static methods to a separate helper class, com.icl.saxon.om.Navigator. This enables the code of these methods to be independent of the particular tree implementation.

The delayed evaluation of path expressions now works as follows: on the first two occasions that a path expression is evaluated, it navigates the source tree. On the third occasion, it saves the resulting node-set in memory. On subsequent uses, the result is retrieved from memory. This approach is designed to balance time against memory usage.

The optimisation of "//name" as "/descendant::name" (which is possible when there are no predicates) wasn't working in 5.5 (or for a while before that), causing an unnecessary sort. This has been corrected. In addition, the first time "//name" is used for a particular document, the results are now saved, and all subsequent uses of "//name" for the same document retrieve the results from memory. This means that the traditional assumption that "//name" is inefficient may no longer always be true.

A Sequencer class has been introduced for allocating globally-unique sequence numbers. There are two such sequences, one for document numbers, and one for node numbers. By default, two sequencers are created when Saxon is loaded, and remain in use until it is unloaded. However, it is now possible to reset the sequence numbering if required, either to prevent running out of numbers in a long-running server, or to ensure repeatability of the value of generate-id(). The result of generate-id() depends on the document number, and you can restart the sequence of document numbers by calling controller.setDocumentSequencer(new com.icl.saxon.om.Sequencer()). It is the caller's responsibility to ensure that this does not cause two documents that are in use at the same time to have the same number. The node sequence number is used when sorting nodes into document order, and when eliminating duplicates in a union operation. You can similarly allocate a new sequence using controller.setNodeSequencer().

Added an optimization for recursive processing of a node-set: the predicate "[position() > 1]" is now recognized and handled specially, allowing pipelined execution and reducing memory requirements.

Removed getAttributeValue(Name), replaced it with getAttributeValue(String uri, String localName). This is more efficient: in many cases it removes the need to construct the Name object and then take it apart. Attributes can also be found using the integer fingerprint of the name.

The Name class is no longer used for holding expanded names, it now serves merely as a container for a couple of static methods for name validation.

NameTest and its subclasses have been reorganised. There is a new class NodeTest which is a subclass of Pattern; it performs the test on node-type and node-name supporting a node-test in XPath. This test is context-free. As well as replacing the NameTest class, it also replaces NodeTypePattern and NamedNodePattern. The NodeTest is now used on a Step, and on an Axis, replacing the previous combination of a NameTest and a node type. These tests are also used in testing which nodes are candidates for whitespace stripping.

The interface between the Step and Axis classes and the expression parser has been much simplified.

Michael H. Kay
Saxonica Limited
22 June 2005