The &OM; Standard

List of Figures &OM; Movement This chapter is a historical account of &OM; and should be regarded as non-normative. &OM; is a standard for representing mathematical objects, allowing them to be exchanged between computer programs, stored in databases, or published on the worldwide web. While the original designers were mainly developers of computer algebra systems, it is now attracting has since attracted interest from other areas of scientific computation and from many publishers of electronic documents with a significant mathematical content. There is a strong relationship to the MathML recommendation MathML_2003 from the Worldwide Web Consortium, and a large overlap between the two developer communities. MathML deals principally with the presentation of mathematical objects, while &OM; is solely concerned with their semantic meaning or content. While MathML does have some limited facilities for dealing with content, it also allows semantic information encoded in &OM; to be embedded inside a MathML structure. Thus the two technologies may be seen as highly complementary.

History &OM; was originally developed through a series of workshops held in Zurich (1993 and 1996), Oxford (1994), Amsterdam (1995), Copenhagen (1995), Bath (1996), Dublin (1996), Nice (1997), Yorktown Heights (1997), Berlin (1998), and Tallahassee (1998). The participants in these workshops formed a global &OM; community which was coordinated by a Steering Committee and operated through electronic mailing groups and ad-hoc working parties. This loose arrangement has been formalised through the establishment of an &OM; Society. Up until the end of 1996 much of the work of the community was funded through a grant from the Human Capital and Mobility program of the European Union, and by the contributions of several institutions and individuals. A document outlining the objectives and basic design of &OM; was produced (later published as Abbott_Leeuwen_Strotmann_98). By the end of 1996 a simplified specification had been agreed upon and some prototype implementations had come about Dalmas_Gaetano_Watt_97. In 1996 a group of European participants in &OM; decided to bid for funding under the European Union's Fourth Framework Programme for strategic research in information technology. This bid was successful and the project started in late 1997. The principal aims of the project were to formalise &OM; as a standard and to develop it further through industrial applications; this process led to the OpenMath 1.0 and 1.1 standards which were endorsed at workshops in Tallahassee (November 1998) and Eindhoven (June 1999). In November 1998 the &OM; Society was established to coordinate all &OM; activities. The society is based in Helsinki, Finland and is coordinated by the executive committee whose members are elected by the society. The official web page of the society is http://www.openmath.org. In 2001 the European Union agreed to fund a Thematic Network under its Fifth Framework programme to coordinate further work on &OM; and MathML, and in particular to support a further series of workshops. This document is one outcome of that project, and seeks to update &OM; in the light of recent developments in XML and of the &OM; community's collective experience working with the old standard.

&OM; Society In November 1998 the &OM; Society has been established to coordinate all &OM; activities. The society is based in Helsinki, Finland and is steered by the executive committee whose members are elected by the society. The official web page of the society is http://www.openmath.org. The &OM; Society continues to provide long-term coordination of &OM; activities. Membership is open to anybody who is active in &OM;; for further details see the web-site at http://www.openmath.org/society/index.html.

Introduction to &OM; This chapter briefly introduces &OM; concepts and notions that are referred to in the rest of this document.

&OM; Architecture

The &OM; Architecture The architecture of &OM; is described in and summarizes the interactions among the different &OM; components. There are three layers of representation of a mathematical object OM_98. The first is a private layer that is the internal representation used by an application. The second is an abstract layer that is the representation as an &OM; object. Note that these two layers may, in some cases, be the same. The third is a communication layer that translates the &OM; object representation into a stream of bytes. An application dependent program manipulates the mathematical objects using its internal representation, it can convert them to &OM; objects and communicate them by using the byte stream representation of &OM; objects.

&OM; Objects and Encodings &OM; objects are representations of mathematical entities that can be communicated among various software applications in a meaningful way, that is, preserving their semantics. &OM; objects and encodings are described in detail in and . The standard endorses two encodings in &exml; and binary formats. At the time of writing, these are the encodings supported by most existing &OM; tools and applications, These are the encodings supported by the official &OM; libraries however they are not the only possible encodings of &OM; objects. Users who wish to define their own encoding using some other specific language (e.g. Lisp) may , are free to do so provided that there is an effective translation from this encoding to an official one a well-defined correspondence between the new encoding and the abstract model defined in .

Content Dictionaries Content Dictionaries (CDs) are used to assign informal and formal semantics to all symbols used in the &OM; objects. They define the symbols used to represent concepts arising in a particular area of mathematics. The Content Dictionaries are public, they represent the actual common knowledge among &OM; applications. Content Dictionaries fix the meaning of objects independently of the application. The application receiving the object may then recognize whether or not, according to the semantics of the symbols defined in the Content Dictionaries, the object can be transformed to the corresponding internal representation used by the application.

Additional Files Several additional files are related to Content Dictionaries. Signature files contain the signatures of symbols defined in some &OM; Content Dictionary and their format is endorsed by this standard. Furthermore, the standard fixes how to define a specific set of Content Dictionaries as a CDGroup. Auxiliary files that define presentation and rendering or that are used for manipulating and processing Content Dictionaries are not discussed by the standard.

Phrasebooks The conversion of an &OM; object to/from the internal representation in a software application is performed by an interface program called a Phrasebook. The translation is governed by the Content Dictionaries and the specifics of the application. It is envisioned that a software application dealing with a specific area of mathematics declares which Content Dictionaries it understands. As a consequence, it is expected that the Phrasebook of the application is able to translate &OM; objects built using symbols from these Content Dictionaries to/from the internal mathematical objects of the application. &OM; objects do not specify any computational behaviour, they merely represent mathematical expressions. Part of the &OM; philosophy is to leave it to the application to decide what it does with an object once it has received it. &OM; is not a query or programming language. Because of this, &OM; does not prescribe a way of forcing evaluation or simplification of objects like

2 + 3

\sin (π)

. Thus, the same object

2 + 3

could be transformed to

5

by a computer algebra system, or displayed as

2 + 3

by a typesetting tool.

&OM; Objects In this chapter we provide a self-contained description of &OM; objects. We first do so by means of an abstract grammar description () and then give a more informal description ().

Formal Definition of &OM; Objects &OM; represents mathematical objects as terms or as labelled trees that are called &OM; objects or &OM; expressions. The definition of an abstract &OM; object is then the following.

Basic &OM; objects The Basic &OM; Objects form the leaves of the &OM; Object tree. A Basic &OM; Object is of one of the following. (i) Integer. Integers in the mathematical sense, with no predefined range. They are infinite precision integers (also called bignums in computer algebra). (ii) ieee floating point number. Double precision floating-point numbers following the ieee 754-1985 standard ieee754_85. (iii) Character string. A Unicode Character string. This also corresponds to `characters' in &exml;. (iv) Bytearray. A sequence of bytes. (v) Symbol. A Symbol encodes two fields of information, a name and a Content Dictionary. Each is a sequence of characters matching a regular expression, as described below. A Symbol encodes three fields of information, a symbol name, a Content Dictionary name, and (optionally) a Content Dictionary base URI, The name of a symbol is a sequence of characters matching the regular expression described in . The Content Dictionary is the location of the definition of the symbol, consisting of a name (a sequence of characters matching the regular expression described in ) and, optionally, a unique prefix called a cdbase which is used to disambiguate multiple Content Dictionaries of the same name. There are other properties of the symbol that are not explicit in these fieleds but whose values may be obtained by inspecting the Content Dictionary specified. these include the symbol definition, formal properties and examples and, optionally, a Role which is a restriction on where the symbol may appear in an &OM; object. The possible roles are described in . (vi) Variable. A Variable consists of must have a name which is a sequence of characters matching a regular expression, as described in .

Derived &OM; Objects Derived &OM; objects are currently used as a way by which non-&OM; data is embedded inside an &OM; object. A derived &OM; object is built as follows: (i) If

A

is not an &OM; object, then

foreign (A)

is an &OM; foreign object. An &OM; foreign object may optionally have an encoding field which describes how its contents should be interpreted.

<phrase revisionflag="deleted">Compound</phrase>&OM; Objects &OM; objects are built recursively as follows. (i) Basic &OM; objects are &OM; objects. (Note that derived &OM; objects are not &OM; objects, but are used to construct &OM; objects as described below.) (ii) If

A_{1}

, …,

A_{n}

(n > 0)

are &OM; objects, then

application (A_{1}, \dots, A_{n})

is an &OM; application object. (iii) If

S_{1}, \dots, S_{n}

are &OM; symbols, and

A

A_{1}

, …,

A_{n}

(n > 0)

are &OM; objects, then

A

is an &OM; object, and

A_{1}

, …,

A_{n}

(n > 0)

are &OM; objects or &OM; derived objects, then

attribution (A, S_{1} A_{1}, \dots, S_{n} A_{n})

is an &OM; attribution object. and

A

is the object stripped of attributions.

S_{1}, \dots, S_{n}

are referred to as keys and

A_{1}

, …,

A_{n}

as their associated values. The operation of recursively applying stripping to the stripped object is called flattening of the attribution. When the stripped object after flattening is a variable, the attributed object is called attributed variable. If, after recursively applying stripping to remove attributions, the resulting un-attributed object is a variable, the original attributed object is called an attributed variable. (iv) If

B

and

C

are &OM; objects, and

v_{1}

\dots

v_{n}

(n \geq 0)

are &OM; variables or attributed variables, then

binding (B, v_{1}, \dots, v_{n}, C)

is an &OM; binding object. (v) If

S

is an &OM; symbol and

A_{1}

, …,

A_{n}

(n \geq 0)

are &OM; objects or &OM; derived objects, then

error (S, A_{1}, \dots, A_{n})

is an &OM; error object.

&OM; Symbol Roles We say that an &OM; symbol is used to construct an &OM; object if it is the first child of an &OM; application, binding or error object, or an even-indexed child of an &OM; attribution object (i.e. the key in a (key, value) pair). The role of an &OM; symbol is a restriction on how it may be used to construct a compound &OM; object and, in the case of the key in an attribution object, a clarification of how that attribution should be interpreted. Possible roles are: binder The symbol may appear as the first child of an &OM; binding object. attribution The symbol may be used as key in an &OM; attribution object, i.e. as the first element of a (key, value) pair, or in an equivalent context (for example to refer to the value of an attribution). This form of attribution may be ignored by an application, so should be used for information which does not change the meaning of the attributed &OM; object. semantic-attribution This is the same as attribution except that it modifies the meaning of the attributed &OM; object and thus cannot be ignored by an application. error The symbol can appear as the first child of an &OM; error object. application The symbol can appear as the first child of an &OM; application object. constant The symbol cannot be used to construct an &OM; compound object. A symbol cannot have more than one role and cannot be used to construct a compound &OM; object in a way which requires a different role (using the definition of construct given earlier in this section). This means that one cannot use a symbol which binds some variables to construct, say, an application object. However it does not prevent the use of that symbol as an argument in an application object (where by argument we mean a child with index greater than 1). If no role is indicated then the symbol can be used anywhere. Note that this is not the same as saying that the symbol's role is constant.

Further Description of &OM; Objects Informally, an &OM; object can be viewed as a tree and is also referred to as a term. The objects at the leaves of &OM; trees are called basic objects. The basic objects supported by &OM; are: IntegerArbitrary Precision integers. Float &OM; floats are ieee 754 Double precision floating-point numbers. Other types of floating point number may be encoded in &OM; by the use of suitable content dictionaries. Character stringsare sequences of characters. These characters come from the Unicode standard UNICODE. Bytearraysare sequences of bytes. There is no byte in &OM; as an object of its own. However, a single byte can of course be represented by a bytearray of length 1. The difference between strings and bytearrays is the following: a character string is a sequence of bytes with a fixed interpretation (as characters, Unicode texts may require several bytes to code one character), whereas a bytearray is an uninterpreted sequence of bytes with no intrinsic meaning. Bytearrays could be used inside &OM; errors to provide information to, for example, a debugger; they could also contain intermediate results of calculations, or `handles' into computations or databases. Symbols are uniquely defined by the Content Dictionary in which they occur and by a name. In definition in we have left this information implicit. However, it should be kept in mind that all symbols appearing in an &OM; object are defined in a Content Dictionary. The form of these definitions is explained in . Each symbol has no more than one definition in a Content Dictionary. Many Content Dictionaries may define differently a symbol with the same name (e.g. the symbol union is defined as associative-commutative set theoretic union in a Content Dictionary set1 but another Content Dictionary, multiset1 might define a symbol union as the union of multi-sets). The name of a symbol can only contain alphanumeric characters and underscores. More precisely, a symbol name matches the following regular expression:

[A-Za-z] [A-Za-z0-9_]*

Notice that these symbol names are case sensitive. &OM; recommends that symbol names should be no longer than 100 characters. Variablesare meant to denote parameters, variables or indeterminate (such as bound variables of function definitions, variables in summations and integrals, independent variables of derivatives). Plain variable names are restricted to use a subset of the printable ASCII characters. Formally the names must match the regular expression:

[A-Za-z0-9=+(),-./:?!#$%*;=@[]^_`{|}]+

Derived &OM; objects are constructed from non-&OM; data. They differ from bytearrays in that they can have any structure. Currently there is only one way of making a derived &OM; object. Foreignis used to import a non-&OM; object into an &OM; attribution. Examples of its use could be to annotate a formula with a visual or aural rendering, an animation etc. They may also appear in &OM; error objects, for example to allow an application to report an error in processing such an object. The four following constructs can be used to make compound &OM; objects out of basic or derived &OM; objects. Applicationconstructs an &OM; object from a sequence of one or more &OM; objects. The first argument of an application is referred to as its head while the remaining objects are called its arguments. An &OM; application object can be used to convey the mathematical notion of application of a function to a set of arguments. For instance, suppose that the &OM; symbol

\sin

is defined in a suitable Content Dictionary, for trigonometry then

application (\sin, x)

is the abstract &OM; object corresponding to

\sin (x)

. More generally, an &OM; application object can be used as a constructor to convey a mathematical object built from other objects such as a polynomial constructed from a set of monomials. Constructors build inhabitants of some symbolic type, for instance the type of rational numbers or the type of polynomials. The rational number, usually denoted as

1 / 2

, is represented by the &OM; application object

application (Rational, 1, 2)

. The symbol

Rational

must be defined, by a Content Dictionary, as a constructor symbol for the rational numbers.

The &OM; application and binding objects for <math><mi>sin</mi> <mo>(</mo><mi>x</mi> <mo>)</mo></math> and <math><mi>λ</mi> <mi>x</mi><mo>.</mo><mi>x</mi> <mo>+</mo> <mn>2</mn></math> in tree-like notation. Bindingobjects are constructed from an &OM; object, and from a sequence of zero or more variables followed by another &OM; object. The first &OM; object is the binder object. Arguments 2 to

n - 1

are always variables to be bound in the body which is the

n^{th}

argument object. It is allowed to have no bound variables, but the binder object and the body should be present. Binding can be used to express functions or logical statements. The function

λ x . x + 2

, in which the variable

x

is bound by

λ

, corresponds to a binding object having as binder the &OM; symbol

lambda

binding (lambda, x, application (plus, x, 2)) .

Phrasebooks are allowed to use

α

conversion in order to avoid clashes of variable names. Suppose an object

Ω

contains an occurrence of the object

binding (B, v, C)

. This object

binding (B, v, C)

can be replaced in

Ω

binding (B, z, C')

where

z

is a variable not occurring free in

C

and

C'

is obtained from

C

by replacing each free (i.e., not bound by any intermediate binding construct) occurrence of

v

z

. This operation preserves the semantics of the object

Ω

. In the above example, a phrasebook is thus allowed to transform the object to, e.g.

binding (lambda, v, binding (lambda, z, application (times, z, z))) .

binding (lambda, z, application (plus, z, 2)) .

Repeated occurrences of the same variable in a binding operator are allowed. An &OM; application should treat a binding with multiple occurrences of the same variable as equivalent to the binding in which all but the last occurrence of each variable is replaced by a new variable which does not occur free in the body of the binding.

binding (lambda, v, v, application (times, v, v))

is semantically equivalent to:

binding (lambda, v^{'}, v, application (times, v, v))

so that the resulting function is actually a constant in its first argument (

v^{'}

does not occur free in the body

application (times, v, v))

). Attributiondecorates an object with a sequence of one or more pairs made up of an &OM; symbol, the attribute, and an associated &OM; object, the value of the attribute. The value of the attribute can be an &OM; attribution object itself. As an example of this, consider the &OM; objects representing groups, automorphism groups, and group dimensions. It is then possible to attribute an &OM; object representing a group by its automorphism group, itself attributed by its dimension. &OM; objects can be attributed with &OM; foreign objects, which are containers for non-&OM; structures. For example a mathematical expression could be attributed with its spoken or visual rendering. Composition of attributions, as in

attribution (attribution (A, S_{1} A_{1}, \dots, S_{h} A_{h}), S_{h + 1} A_{h + 1}, \dots, S_{n} A_{n})

is semantically equivalent to a single attribution, that is

attribution (A, S_{1} A_{1}, \dots, S_{h} A_{h}, S_{h + 1} A_{h + 1}, \dots, S_{n} A_{n}) .

The operation that produces an object with a single layer of attribution is called flattening. Multiple attributes with the same name are allowed. While the order of the given attributes does not imply any notion of priority, potentially it could be significant. For instance, consider the case in which

S_{h} = S_{n}

(

h < n

) in the example above. Then, the object is to be interpreted as if the value

A_{n}

overwrites the value

A_{h}

. (&OM; however does not mandate that an application preserves the attributes or their order.) Attribution acts as either adornment annotation or as semantical annotation. When the key has role attribution, then replacement of the attributed object by the object itself is not harmful and preserves the semantics. When the key has role semantic-attribution then the attributed object is modified by the attribution and cannot be viewed as semantically equivalent to the stripped object. If the attribute lacks the role specification then attribution is acting as adornment annotation. Objects can be decorated in a multitude of ways. In OMD132b, typing of &OM; objects is expressed by using an attribution. An example of the use of an adornment attribution would be to indicate the colour in which an &OM; object should be displayed, for example

attribution (A, colour red)

. Note that both

A

and

red

are &OM; objects. An example of the use of a semantic attribution would be to indicate the type of an object. For example the object

attribution (A, type t)

represents the judgment stating that object

A

has type

t

. Note that both

A

and

t

are &OM; objects. Attribution can act as either annotation, in the sense of adornment, or as modifier. In the former case, replacement of the adorned object by the object itself is probably not harmful (preserves the semantics). In the latter case however, it may very well be. Therefore, attribution in general should by default be treated as a construct rather than as adornment. Only when the CD definitions of the attributes make it clear that they are adornments, can the attributed object be viewed as semantically equivalent to the stripped object. Erroris made up of an &OM; symbol and a sequence of zero or more &OM; objects. This object has no direct mathematical meaning. Errors occur as the result of some treatment on an &OM; object and are thus of real interest only when some sort of communication is taking place. Errors may occur inside other objects and also inside other errors. Error objects might consist only of a symbol as in the object:

error (S)

Names The names of symbols, variables and content dictionaries must conform to the production Name specified in the following grammar (which is identical to that for &exml; names in XML 1.1, xml_04). Informally speaking, a name is a sequence of Unicode UNICODE characters which begins with a letter and cannot contain certain punctuation and combining characters. The notation #x... represents the hexadecimal value of the encoding of a Unicode character. Some of the character values or code points in the following productions are currently unassigned, but this is likely to change in the future as Unicode evolves We note that in XML 1 the name production explicitly listed the characters that were allowed, so all the characters added in versions of Unicode after 2.0 (which amounted to tens of thousands of characters) were not allowed in names. .

Name $⟶$ NameStartChar (NameChar)* NameStartChar $⟶$ ":" | [A-Z] | "_" | [a-z] | [#xC0-#xD6] | [#xD8-#xF6] | [#xF8-#x2FF] | [#x370-#x37D] | [#x37F-#x1FFF] | [#x200C-#x200D] | [#x2070-#x218F] | [#x2C00-#x2FEF] | [#x3001-#xD7FF] | [#xF900-#xFDCF] | [#xFDF0-#xFFFD] | [#x10000-#xEFFFF] NameChar $⟶$ NameStartChar | "-" | "." | [0-9] | #xB7 | [#x0300-#x036F] | [#x203F-#x2040]

CD Base A cdbase must conform to the grammar for URIs described in IETF2396. Note that if non-ASCII characters are used in a CD or symbol name then when a URI for that symbol is constructed it will be necessary to map the non-ASCII characters to a sequence of octets. The precise mechanism for doing this depends on the URI scheme. Note on content dictionary names It is a common convention to store a Content Dictionary in a file of the same name, which can cause difficulties on many file systems. If this convention is to be followed then &OM; recommends that the name be restricted to the subset of the above grammar which is a legal POSIX POSIX filename, namely:

Name $⟶$ (PosixLetter | '_') (Char)* Char $⟶$ PosixLetter | Digit | '.' | '-' | '_' PosixLetter $⟶$ 'a' | 'b' | ... | 'z' | 'A' | 'B' | ... | 'Z'

Canonical URIs for Symbols To facilitate the use of &OM; within a URI-based framework (such as RDF rdf or OWL owl), we provide the following scheme for constructing a canonical URI for an &OM; Symbol:

URI = cdbase-value + '/' + cd-value + '#' + name-value

So for example the URI for the symbol with cdbase http://www.openmath.org/cd, cd transc1 and name sin is:

http://www.openmath.org/cd/transc1#sin

In particular, this now allows us to refer uniquely to an &OM; symbol from a MathML document MathML_2003: <mathml:csymbol xmlns:mathml="http://www.w3.org/1998/Math/MathML/" definitionURL="http://www.openmath.org/cd/transc1#sin"> <mo> sin </mo> </csymbol>

Summary &OM; supports basic objects like integers, symbols, floating-point numbers, character strings, bytearrays, and variables. &OM; compound objects are of four kinds: applications, bindings, errors, and attributions. &OM; objects may be attributed with non-&OM; objects via the use of foreign &OM; objects. &OM; objects have the expressive power to cover all areas of computational mathematics. Observe that an &OM; application object is viewed as a tree by software applications that do not understand Content Dictionaries, whereas a Phrasebook that understands the semantics of the symbols, as defined in the Content Dictionaries, should interpret the object as functional application, constructor, or binding accordingly. Thus, for example, for some applications, the &OM; object corresponding to

2 + 5

may result in a command that writes

7

&OM; Encodings In this chapter, two encodings are defined that map between &OM; objects and byte streams. These byte streams constitute a low level representation that can be easily exchanged between processes (via almost any communication method) or stored and retrieved from files. The first encoding uses ISO 646:1983 characters iso646_83 (also known as ascii characters) and is an &exml; application. Although the &exml; markup of the encoding uses only ascii characters, &OM; strings may use arbitrary Unicode/ISO 10646:1988 characters UNICODE. It can be used, for example, to send &OM; objects via e-mail, news, cut-and-paste, etc. The texts produced by this encoding can be part of &exml; documents. The first encoding is a character-based encoding in &exml; format. In previous versions of the &OM; Standard this encoding was a restricted subset of the full legal &exml; syntax. In this version, however, we have removed all these restrictions so that the earlier encoding is a strict subset of the existing one. The &exml; encoding can be used, for example, to send &OM; objects via e-mail, cut-and-paste, etc. and to embed &OM; objects in &exml; documents or to have &OM; objects processed by &exml;-aware applications. The second encoding is a binary encoding that is meant to be used when the compactness of the encoding is important (inter-process communications over a network is an example). Note that these two encodings are sufficiently different for auto-detection to be effective: an application reading the bytes can very easily determine which encoding is used.

The &exml; Encoding This encoding has been designed with two main goals in mind: to provide an encoding that uses common character sets (so that it can easily be included in most documents and transport protocols) and that is both readable and writable by a human. to provide an encoding that can be included (embedded) in &exml; documents or processed by &exml;-aware applications.

A <phrase revisionflag="deleted">Grammar</phrase><phrase>Schema</phrase> for the &exml; Encoding The &exml; encoding of an &OM; object is defined by the Relax NG schema RELAX given below. Relax NG has a number of advantages over the older XSD Schema format XSD, in particular it allows for tighter control of attributes and has a modular, extensible structure. Although we have made the &exml; form, which is given in , normative, it is generated from the compact syntax given below. It is also very easy to restrict the schema to allow a limited set of &OM; symbols as described in . Standard tools exist for generating a DTD or an XSD schema from a Relax NG Schema. Examples of such documents are given in and respectively. &omrnc; the &exml; encoding of an &OM; object is defined by the dtd given in Figure 4.1 below with the following additional rules not implied by the &exml; dtd. Comments are permitted only between elements, not within element character data. Processing Instructions are only allowed before the OMOBJ element. The content of an OMB element, is a valid base64-encoded text. The character data forming element content and attribute values matches the regular expressions of . DTD for the &OM; &exml; encoding of objects. ]]> ]]> ]]> ]]> ]]> ]]> ]]> ]]> ]]> ]]> ]]> In addition, if the &exml; document encoding the &OM; object is linearised into the &exml; concrete syntax, the following further constraints apply, which ensure that the encoding may be read by &OM; applications that may not include a full &exml; parser. The document should use utf-8 encoding. A <!DOCTYPE declaration should not be used. Character references should not be used. As <!DOCTYPE is not used, the only entity references that are allowed are the five predefined entity references: ' ('), " ("), < (<), > (>), & (&). The &exml; empty element form <|#8230;/> should always be used to encode elements such as omf which are specified in the dtd as being empty. It should never be used for elements that may sometimes be empty, such as omstr. Such a linearisation of an &exml; encoded &OM; Object would match the match the character based grammar given in . The notation used in this section and in should be quite straightforward (+ meaning one or more, ? meaning zero or one, and | meaning or). The start symbol of the grammar is start, space stands for the space character, cr for the carriage return character, nl for the line feed character and tab for the horizontal tabulation character. Grammar for the &exml; encoding of &OM; objects. S

⟶

(space | tab | cr | nl)+ integer

⟶

(- S?)? [&digits;]+ (S [&digits;]+)* | (- S?)? x S? [&exadigits;]+ (S [&exadigits;]+)* cdname

⟶

[&lcalpha;][&lcalpha;&digits;_]* symbname

⟶

[&ucalpha;&lcalpha;][&ucalpha;&lcalpha;&digits;_]* fpdec

⟶

(-?)([&digits;]+)?(.[&digits;]+)?(e(&sign;?)[&digits;]+)? fphex

⟶

[&digits;ABCDEF]+ varname

⟶

([&ucalpha;&lcalpha;&digits;&varnamechar;])+ base64

⟶

([&ucalpha;&lcalpha;&digits; +/=] | S)+ char

⟶

&exml; Character Data symbnameatt

⟶

name S? = S? (" symbname " | ' symbname ') cdnameatt

⟶

cd S? = S? (" cdname " | ' cdname ') varnameatt

⟶

name S? = S? (" varname " | ' varname ') fpdecatt

⟶

dec S? = S? (" fpdec " | ' fpdec ') fphexatt

⟶

hex S? = S? (" fphex " | ' fphex ') PI

⟶

<? char ?> comment

⟶

<!-&zsp;- char -&zsp;-> SC

⟶

S+ | (comment S)+ start

⟶

(SC | PI)* <OMOBJ S?> S? object S? </OMOBJ S?> symbol

⟶

<OMS [[(S symbnameatt) (S cdnameatt) ]] S? /> variable

⟶

<OMV varnameatt S? /> | <OMATTRx S?> SC? omatp SC? variable SC? </OMATTR S?> omatp

⟶

<OMATP S?> SC? attrs SC? </#1 S?> object

⟶

⟶

symbol S? object | symbol S? object S? attrs objects

⟶

SC? | object SC? objects variables

⟶

SC? | variable SC? variables Note: This schema specifies names as being of the xsd:NCName type. At the time of writing, W3C Schema types are defined in terms of XML 1 xml_98. This limits the characters allowed in a name to a subset of the characters available in Unicode 2.0, which is far more restrictive than the definition for an &OM; name given in . It is expected that W3C Schema types will be augmented to match the new XML 1.1 recommendation xml_04, but for portability reasons applications should avoid using the new XML 1.1 name characters unless they are absolutely required. The XML 1.1 specification has a useful appendix giving advice on good strategies to use when naming identifiers.

<phrase revisionflag="added">Informal</phrase> description of the <phrase revisionflag="deleted">Grammar</phrase><phrase revisionflag="added">&exml; Encoding</phrase> An encoded &OM; object is placed inside an OMOBJ element. This element can contain the elements (and integers) described above. It can take an optional version (&exml;) attribute which indicates to which version of the &OM; standard it conforms. In previous versions of this standard this attribute did not exist, so any &OM; object without such an attribute must conform to version 1 (or equivalently 1.1) of the &OM; standard. Objects which conform to the description given in this document should have version="2.0". We briefly discuss the &exml; encoding for each type of &OM; object starting from the basic objects. Integers are encoded using the OMI element around the sequence of their digits in base 10 or 16 (most significant digit first). White space may be inserted between the characters of the integer representation, this will be ignored. After ignoring white space, integers written in base 10 match the regular expression -?[0-9]+. Integers written in base 16 match -?x[0-9A-F]+. The integer 10 can be thus encoded as <OMI> 10 </OMI> or as <OMI> xA </OMI> but neither <OMI> +10 </OMI> nor <OMI> +xA </OMI> can be used. The negative integer

-120

can be encoded as either as decimal <OMI> -120 </OMI> or as hexadecimal <OMI> -x78 </OMI>. Symbolsare encoded using the OMS element. This element has two three (&exml;) attributes cd, name, and cdbase. The value of cd is the name of the Content Dictionary in which the symbol is defined and the value of name is the name of the symbol. The optional cdbase attribute is a URI that can be used to disambiguate between two content dictionaries with the same name. If a symbol does not have an explicit cdbase attribute, then it inherits its cdbase from the first ancestor in the &exml; tree with one, should such an element exist. In this document we have tended to omit the cdbase for clarity. The name of the Content Dictionary is compulsory, but a future revision of the &OM; standard might introduce a defaulting mechanism. For example:

<OMS cd="transc" name="sin"/>

<OMS cdbase="http://www.openmath.org/cd" cd="transc1" name="sin"/>

is the encoding of the symbol named sin in the Content Dictionary named transc1, which is part of the collection maintained by the &OM; Society. As described in , the three attributes of the OMS can be used to build a URI reference for the symbol, for use in contexts where URI-based referencing mechanisms are used. For example the URI for the above symbol is http://www.openmath.org/cd/transc1#sin. Note that the role attribute described in is contained in the Content Dictionary and is not part of the encoding of a symbol, also the cdbase attribute need not be explicit on each OMS as it is inherited from any ancestor element. Variablesare encoded using the OMV element, with only one (&exml;) attribute, name, whose value is the variable name. The variable name is a subset of the printable ascii set of characters. In particular, neither spaces nor double-quote " are allowed in variable names. For instance, the encoding of the object representing the variable

x

is: <OMV name="x"/> Floating-point numbersare encoded using the OMF element that has either the (&exml;) attribute dec or the (&exml;) attribute hex. The two (&exml;) attributes cannot be present simultaneously. The value of dec is the floating-point number expressed in base 10, using the common syntax:

(-?)([0-9]+)?("."[0-9]+)?([eE](-?)[0-9]+)?. or one of the special values: INF, -INF or NaN.

The value of hex is a base 16 representation of the 64 bits of the ieee Double. Thus the number represents mantissa, exponent, and sign from lowest to highest bits using a least significant byte ordering. This consists of a string of 16 digits 0-9, A-F. For example, both <OMF dec="1.0e-10"/> and <OMF hex="3DDB7CDFD9D7BDBB"/> are valid representations of the floating point number

1 \times 10^{-10}

. The symbols INF, -INF and NaN represent positive and negative infinity, and not a number as defined in ieee754_85. Note that while infinities have a unique representation, it is possible for NaNs to contain extra information about how they were generated and if this informations is to be preserved then the hexadecimal representation must be used. For example <OMF hex="FFF8000000000000"/> and <OMF hex="FFF8000000000001"/> are both hexadecimal representations of NaNs. Character stringsare encoded using the OMSTR element. Its content is a Unicode text (The default encoding is utf-8utf8, although &exml; encoded &OM; may be embedded in a containing &exml; document that specifies alternative encoding in the &exml; declaration. Note that as always in &exml; the characters < and & need to be represented by the entity references < and & respectively. Bytearraysare encoded using the OMB element. Its content is a sequence of characters that is a base64 encoding of the data. The base64 encoding is defined in rfc 1521 rfc1521 2045 rfc2045. Basically, it represents an arbitrary sequence of octets using 64 digits (A through Z, a through z, 0 through 9, + and /, in order of increasing value). Three octets are represented as four digits (the = character is used for padding at the end of the data). All line breaks and carriage return, space, form feed and horizontal tabulation characters are ignored. The reader is referred to rfc1521 rfc2045 for more detailed information. In detail the encoding of an &OM; object is described below. Applicationsare encoded using the OMA element. The application whose head is the &OM; object

e_{0}

and whose arguments are the &OM; objects

e_{1}

, …,

e_{n}

is encoded as <OMA>

C_{0}

C_{1}

…

C_{n}

</OMA> where

C_{i}

is the encoding of

e_{i}

. For example,

application (\sin, x)

is encoded as: ]]> provided that the symbol sin is defined to be a function symbol in a Content Dictionary named transc1. Bindingis encoded using the OMBIND element. The binding by the &OM; object

b

of the &OM; variables

x_{1}

x_{2}

\dots

x_{n}

in the object

c

is encoded as <OMBIND>

B

X_{1}

\dots

X_{n}

</OMBVAR>

C

</OMBIND> where

B

C

, and

X_{i}

are the encodings of

b

c

and

x_{i}

, respectively. For instance the encoding of

binding (lambda, x, application (\sin, x))

is: ]]> Binders are defined in Content Dictionaries, in particular, the symbol lambda is defined in the Content Dictionary fns1 for functions over functions. Attributionsare encoded using the OMATTR element. If the &OM; object

e

is attributed with (

s_{1}

e_{1}

), …, (

s_{n}

e_{n}

) pairs (where

s_{i}

are the attributes), it is encoded as <OMATTR> <OMATP>

S_{1}

C_{1}

…

S_{n}

C_{n}

</OMATP>

E

</OMATTR> where

S_{i}

is the encoding of the symbol

s_{i}

C_{i}

of the object

e_{i}

and

E

is the encoding of

e

. Examples are the use of attribution to decorate a group by its automorphism group: [..group-encoding..] [..group-encoding..] ]]> or to express the type of a variable: ]]> A special use of attributions is to associate non-&OM; data with an &OM; object. This is done using the OMFOREIGN element. The children of this element must be well-formed &exml;. For example the attribution of the &OM; object

\sin (x)

with its representation in Presentation MathML is:

\sin (x)

]]> Of course not everything has a natural XML encoding in this way and often the contents of a OMFOREIGN will just be data or some kind of encoded string. For example the attribution of the previous object with its LaTeX representation could be achieved as follows: \sin(x) ]]> For a discussion on the use of the encoding attribute see . Errors are encoded using the OME element. The error whose symbol is

s

and whose arguments are the &OM; objects or &OM; derived objects

e_{1}

, …,

e_{n}

is encoded as <OME>

C_{s}

C_{1}

…

C_{n}

</OME> where

C_{s}

is the encoding of

s

and

C_{i}

the encoding of

e_{i}

. If an aritherror Content Dictionary contained a DivisionByZero symbol, then the object

error (DivisionByZero, application (divide, x, 0))

would be encoded as follows: 0 ]]> If a mathml Content Dictionary contained an unhandled_csymbol symbol, then an &OM; to MathML translator might return an error such as: Ai ]]> Note that it is possible to embed fragments of valid &OM; inside an OMFOREIGN element but that it cannot contain invalid &OM;. In addition, the arguments to an OMERROR must be well-formed &exml;. If an application wishes to signal that the &OM; it has received is invalid or is not well-formed then the offending data must be encoded as a string. For example: <OMA> <OMS name="cos" cd="transc1"> <OMV name="v"> </OMA> ]]> Note that the `<' and `>' characters have been escaped as is usual in an &exml; document. References &OM; integers, floating point numbers, character strings, bytearrays, applications, binding, attributions can also be encoded as an empty OMR element with an href attribute whose value is the value of a URI referencing an id attribute of an &OM; object of that type. The &OM; element represented by this OMR reference is a copy of the &OM; element referenced href attribute. Note that this copy is structurally equal, but not identical to the element referenced. For instance, the &OM; object

application (f, application (f, application (f, a, a), application (f, a, a)), application (f, application (f, a, a), application (f, a, a)))

can be encoded in the &exml; encoding as either one of the &exml; encodings given in (and some intermediate versions as well).

Shared vs. unshared representations ]]>

Some Notes on References We say that an &OM; element dominates all its children and all elements they dominate. An OMR element dominates its target, i.e. the element that carries the id attribute pointed to by the xref attribute. For instance in the representation in , the OMA element with id="t1" and also the second OMR dominate the OMA element with id="t11".

An Acyclicity Constraint The occurrences of the OMR element must obey the following global acyclicity constraint: An &OM; element may not dominate itself. Consider for instance the following (illegal) &exml; representation 1 1 ]]> Here, the OMA element with id="foo" dominates its third child, which dominates the OMR element, which dominates its target: the element with id="foo". So by transitivity, this element dominates itself, and by the acyclicity constraint, it is not the &exml; representation of an &OM; element. Even though it could be given the interpretation of the continued fraction

\frac{1}{1 + \frac{1}{1 + \frac{1}{...}}}

this would correspond to an infinite tree of applications, which is not admitted by the structure of &OM; objects described in . Note that the acyclicity constraints is not restricted to such simple cases, as the example in shows.

Sharing between &OM; objects (A cycle of order <math><mn>2</mn></math>. 1 1 ]]> Here, the OMA with id="bar" dominates its third child, the OMR with xref="baz", which dominates its target OMA with id="baz", which in turn dominates its third child, the OMR with xref="bar", this finally dominates its target, the original OMA element with id="bar". So this pair of &OM; objects violates the acyclicity constraint and is not the &exml; representation of an &OM; object.

Sharing and Bound Variables Note that the OMR element is a syntactic referencing mechanism: an OMR element stands for the exact &exml; element it points to. In particular, referencing does not interact with binding in a semantically intuitive way, since it allows for variable capture. Consider for instance the following &exml; representation: ]]> it represents the &OM; object

binding (λ, X, application (f, binding (λ, X, application (g, X)), application (g, X))))

which has two sub-terms of the form

application (g, X)

, one with id="orig" (the one explicitly represented) and one with id="copy", represented by the OMR element. In the original, the variable

X

is bound by the outer OMBIND element, and in the copy, the variable

X

is bound by the inner OMBIND element. We say that the inner OMBIND has captured the variable

X

. It is well-known that variable capture does not conserve semantics. For instance, we could use

α

-conversion to rename the inner occurrence of

X

into, say,

Y

arriving at the (same) object

binding (λ, X, application (f, binding (λ, Y, application (g, Y)), application (g, X))))

Using references that capture variables in this way can easily lead to representation errors, and is not recommended.

Embedding &OM; in &exml; Documents The above encoding of &exml; encoded &OM; specifies the grammar to be used in files that encode a single &OM; object, and specifies the character streams that a conforming &OM; application should be able to accept or produce. When embedding &exml; encoded &OM; objects into a larger &exml; document one may wish, or need, to use other &exml; features. For example use of extra &exml; attributes to specify &exml; Namespaces xmlns or xml:lang attributes to specify the language used in strings xml_04. Also, the encoding used in the larger document may not be utf-8. In particular, if &OM; is used with applications that use the &exml; Namespace Recommendation xmlns then they should ensure that &OM; elements are in the namespace http://www.openmath.org/OpenMath. This is most conveniently achieved by adding the namespace declaration xmlns="http://www.openmath.org/OpenMath" as an attribute to each OMOBJ element in the document. If such &exml; features are used then the &exml; application controlling the document must, if passing the &OM; fragment to an &OM; application, remove any such extra attributes and must ensure that the fragment is encoded according to the grammar specified above.

The Binary Encoding The binary encoding was essentially designed to be more compact than the &exml; encodings, so that it can be more efficient if large amounts of data are involved. For the current encoding, we tried to keep the right balance between compactness, speed of encoding and decoding and simplicity (to allow a simple specification and easy implementations).

A Grammar for the Binary Encoding

Grammar of the binary encoding of &OM; objects. start

⟶

[24] object [25] | [24+64] [

m

] [

n

] object [25] object

⟶

⟶

[1] [_] | [1+64] [

n

] id:

n

[_]

|

[1+32] [_] | [1+128] {_} | [1+64+128] {

n

} id:

n

{_} | [1+32+128] {_} | [2] [

n

] [_] digits:

n

| [2+64] [

n

] [

m

] [_] digits:

n

id:

m

| [2+32] [

n

] [_] digits:

n

| [2+128] {

n

} [_] digits:

n

| [2+64+128] {

n

} {

n

} [_] digits:

n

id:

n

| [2+32+128] {

n

} [_] digits:

n

float

⟶

[3] {_}{_} | [3+64] [

n

] id:

n

{_}{_} variable

⟶

[5] [

n

] varname:

n

| [5+64] [

n

] [

m

] varname:

n

id:

m

| [5+128] {

n

} varname:

n

| [5+64+128] {

n

} {

m

} varname:

n

id:

m

symbol

⟶

[8] [

n

] [

m

] cdname:

n

symbname:

m

| [8+64] [

n

] [

m

] [

k

] cdname:

n

symbname:

m

id:

k

| [8+128] {

n

} {

m

} cdname:

n

symbname:

m

| [8+64+128] {

n

} {

m

} {

k

} cdname:

n

symbname:

m

id:

k

string

⟶

[6] [

n

] chars bytes:

n

| [6+64] [

n

] chars bytes:

n

|

[6+32] [

n

] bytes:

n

| [6+128] {

n

} chars bytes:

n

| [6+64+128] {

n

} {

m

} chars bytes:

n

id:

m

| [6+32+128] {

n

} bytes:

n

| [7] [

n

] chars bytes:

2 n

| [7+64] [

n

] [

m

] chars bytes:

n

id:

m

| [7+32] [

n

] bytes:

2 n

| [7+128] {

n

} chars bytes:

2 n

| [7+64+128] {

n

} {

m

} chars bytes:

2 n

id:

m

| [7+32+128] {

n

} bytes:

2 n

bytearray

⟶

[4] [

n

] bytes:

n

| [4+64] [

n

] [

m

] bytes:

n

id:

m

|

[4+32] [

n

] bytes:

n

| [4+128] {

n

} bytes:

n

| [4+64+128] {

n

} {

m

} bytes:

n

id:

m

| [4+32+128] {

n

} bytes:

n

cdbase

⟶

[9] [

n

] uri:

n

object | [9+128] {

n

} uri:

n

object foreign

⟶

[12] [

n

] [

m

] bytes:

n

bytes:

m

| [12+64] [

n

] [

m

] [

k

] bytes:

n

bytes:

m

id:

k

|

[12+32] [

n

] [

m

] bytes:

n

bytes:

m

| [12+128] {

n

} {

m

} bytes:

n

bytes:

m

| [12+64+128] {

n

} {

m

} {

k

} bytes:

n

bytes:

m

id:

k

| [12+32+128] {

n

} {

m

} bytes:

n

bytes:

m

construct

⟶

[16] object objects [17] | [16+64] {

m

} id:

m

object objects [17] | [22] symbol objects [23] | [22+64] {

m

} id:

m

symbol objects [23] | [18] attrpairs object [19] | [18+64] {

m

} id:

m

attrpairs object [19] | [26] object bvars object [27] | [26+64] {

m

} id:

m

object bvars object [27] attrpairs

⟶

[20] pairs [21] | [20+64] {

m

} id:

m

pairs [21] pairs

⟶

symbol object | symbol object pairs bvars

⟶

[28] vars [29] | [28+64] {

m

} id:

m

vars [29] vars

⟶

attrvar | attrvar vars attrvar

⟶

variable | [18] attrpairs attrvar [19] | [18+64] {

m

} id:

m

attrpairs attrvar [19] objects

⟶

object objects internal_reference

⟶

[30] [_] | [30+128] {_} external_reference

⟶

[31] [

n

] uri:

n

| [31+128] {

n

} uri:

n

gives a grammar for the binary encoding (start is the start symbol).. The following conventions are used in this section: [

n

] denotes a byte whose value is the integer

n

(

n

can range from 0 to 255), {

m

} denotes four bytes representing the (unsigned) integer

m

in network byte order, [_] denotes an arbitrary byte, {_} denotes an arbitrary sequence of four bytes. name:

n

denotes a sequence of

n

bytes named name. name:2

n

denotes a sequence of

2 n

bytes. start is the start symbol of the grammar. xxxx:

n

, where xxxx is one of symbname, cdname, varname, uri, id, digits, or bytes denotes a sequence of

n

bytes that conforms to the constraints on xxxx strings. For instance, for symbname, varname, or cdname this is the regular expression described in , for uri it is the grammar for URIs in IETF2396.

Description of the Grammar An &OM; object is encoded as a sequence of bytes starting with the begin object tag (value 24 values 24 and 88) and ending with the end object tag (value 25). These are similar to the <OMOBJ> and </OMOBJ> tags of the &exml; encoding. Objects with start token [88] have two additional bytes

m

and

n

that characterize the version (

m . n

) of the encoding directly after the start token. This is similar to <OMOBJ version="m.n"> The encoding of each kind of &OM; object begins with a tag that is a single byte, holding a token identifier that describes the kind of object and two flags, the long flag and the shared flag. The identifier is stored in the first 6 bits (1 to 6). The long flag is the eighth bit and the shared flag is the seventh bit. If the long flag is set, this signifies that the names, strings, and data fields in the encoded &OM; object are longer than 255 bytes or characters. The sharing flag indicates that the encoded object may be shared in another (part of an) object somewhere else (see ). Note that if the sharing flag is set (in the right column of the grammar in , then the encoding includes a representation of an identifier that serves as the target of a reference (internal with token identifier 30 or external with token identifier 31). The concept of structure sharing in &OM; encodings and in particular the sharing bit in the binary encoding has been introduced in &OM; 2 (see section for details). The binary encoding in &OM; 2 leaves the tokens with sharing flag 0 unchanged to ensure &OM; 1 compatibility. To make use of functionality like the version attribute on the &OM; object introduced in &OM; 2, the tokens with sharing flag 1 should be used. To facilitate the streaming of &OM; objects, some basic objects (integers, strings, bytearrays, and foreign objects) have variant token identifiers with the fifth bit set. The idea behind this is that these basic objects can be split into packets. If the fifth bit is not set, this packet is the final packet of the basic object. If the bit is set, then more packets of the basic object will follow directly after this one. Note that all packets making up a basic object must have the same token identifier (up to the fifth bit). In we have represented an integer that is split up into three packets. Here is a description of the binary encodings of every kind of &OM; object: Integersare encoded depending on how large they are. There are four possible formats. Integers between -128 and 127 are encoded as the small integer tags (token identifier 1) followed by a single byte that is the value of the integer (interpreted as a signed character). For example 16 is encoded as 0x01 0x10. Integers between

{-2}^{31}