AMPS Data Types

Each value in AMPS is assigned a data type when the message type module parses the value. AMPS operators and functions attempt to convert values into compatible types, based on the type of operation. For example, the * operator (multiplication) will attempt to convert all values to numeric values, while the CONCAT function (string concatenation) will attempt to convert all values to strings. In effect, a value in AMPS can be transparently treated as any type to which it can be meaningfully converted.

Internally, AMPS uses the data types in the table below. As mentioned above, the message type module is responsible for assigning the type of a value from an incoming message as part of the parsing process. For some types, such as JSON, XML, FIX and NVFIX, the parser infers the type of the value from the field. For other types, such as MessagePack, BFLAT, Google Protocol Buffers or BSON, the message itself contains information about the type of the field.

As mentioned above, the AMPS expression language does not limit the value to the type assigned by the message type module. Instead, a value in AMPS can be used in any context.

For example, given the following JSON document:

{"a":1,"b":"47"}

The values of /a and /b can be used as either string values or numeric values. AMPS will automatically convert these values as necessary, and AMPS considers the string or numeric representation to be equally correct and valid.

The following table lists the data types in the AMPS expression language:

Type	Description	Untyped Message Examples
NULL	Unknown, untyped value (SQL-92 semantics)	[no field provided] NVFIX: a=<SOH> JSON: {"a":null} XML: <a/>
Boolean	True (1) or false (0)	JSON: {"e":true}
Integer	Signed 64-bit integer or unsigned 64-bit integer for values > LONG_MAX	NVFIX: b=24 JSON: {"b":24} XML: <b>24</b>
Floating Point Number	64-bit floating point number	NVFIX: c=24.0 JSON: {"c":24.0} XML: <c>24.0</c>
String	Arbitrary sequence of bytes of a specific length An empty string is considered to be NULL	NVFIX: d=Grilled cheese sandwich<SOH> JSON: {"d":"Grilled cheese sandwich"} XML: <d>Grilled cheese sandwich</d>

Numeric Types and Literals in AMPS Expressions

Numeric values in AMPS are always typed as either integers or floating point values. All numeric types that are less than or equal to the LONG_MAX limit in AMPS are signed, otherwise, the numeric type is unsigned. AMPS message types convert the original numeric types (or original representation for message types that do not have typed values) into the internal AMPS type system for the purposes of expression evaluation.

Within expressions, integer values are all numerals, with no decimal point, and can have a value in the same range as a 64-bit integer. For example:

42
149
-273
18446744073709551610

Within expressions, all numerals with a decimal point are floating-point numbers. AMPS interprets these numerals as double-precision floating point values. For example:

3.1415926535
98.6
-273.0

or, in scientific notation:

31.4e-1
6.022E23
2.998e8

AMPS automatically converts strings that contain numeric values to numbers when strings are used with an operator, function or comparison that expects a numeric value.

Type Promotion for Numeric Types

AMPS uses the following rules for type promotion when evaluating numeric expressions:

1. If any of the values in the expression is NaN, the result is NaN.

2. Otherwise, if any of the values in the expression is floating point, the result is floating point.

3. Otherwise, all of the values in the expression are integers, and the result is an integer.

Notice that, for division in particular, the results returned are affected by the type of the values. For example, the expression 1 / 5 evaluates to 0 since the result is interpreted as an integer. In comparison, the expression 1.0 / 5 evaluates to 0.2 since the result is interpreted as a floating point value.

When a function or operator that expects a numeric type is provided with a string, AMPS will attempt to convert string values to numeric types as necessary. When converting string values, AMPS recognizes the same numeric formats in message data as are supported in the AMPS expression language (see Numeric Types and Literals). If the string is in an unrecognized format, AMPS converts the string as NaN.

String Literals in AMPS Expressions

When creating expressions for AMPS, string literals are indicated with single or double quotes. For example:

/FIXML/Order/Instrmt/@Sym = 'IBM'

AMPS supports the following escape sequences within string literals:

Escape Sequence	Definition
\a	Alert
\b	Backspace
	Horizontal tab
	Newline
\f	Form feed
	Carriage return
\xHH	Hexadecimal digit where H is (0..9,a..f,A..F)
\OOO	Octal Digit (0..7)

Additionally, any character which follows a backslash will be treated as a literal character.

AMPS string operations have no restrictions on character set, and correctly handle embedded NULL characters (\x00) and characters outside of the 7-bit ASCII range. AMPS string operations are not unicode-aware.

NULL, NaN and IS NULL

XPath expressions are considered to be NULL when they evaluate to an empty or nonexistent field reference. NULL values follow SQL-92 semantics.

This means that comparisons with NULL are never true (in other words, even if /a is NULL, /a != NULL is false and /a == NULL is also false).

In numeric expressions where the operands or results are not a valid number, the XPath expression evaluates to NaN (not a number). The rules for applying the AND and OR operators against NULL and NaN values are outlined in the tables below:

Operand1		Operand2	Result
TRUE	(AND)	NULL	NULL
FALSE	(AND)	NULL	FALSE
NULL	(AND)	NULL	NULL
NULL	(AND)	TRUE	NULL
NULL	(AND)	FALSE	NULL

Operand1		Operand2	Result
TRUE	(OR)	NULL	TRUE
FALSE	(OR)	NULL	NULL
NULL	(OR)	NULL	NULL
NULL	(OR)	TRUE	NULL
NULL	(OR)	FALSE	NULL

Likewise, direct comparisons with NULL are not ever true (so, if /b is NULL, /b == NULL does not produce a true value, and neither does /b != NULL). AMPS, like SQL-92, provides an IS NULL predicate for testing whether a value is NULL, and an IS NOT NULL predicate for testing whether a value is not NULL.

There also exists an IS NAN predicate for checking that a value is NaN (not a number.)

To reliably check for existence of a NULL value, you must use the IS NULL predicate such as the filter: /optionalField IS NULL

To reliably check that a value is not NULL, you must use the IS NOT NULL predicate or negate the value of an IS NULL test: /optionalField IS NOT NULL and NOT /optionalField IS NULL are equivalent.

AMPS also provides a COALESCE() function that accepts a set of values and returns the first value that is not NULL. For example, given the following filter expression:

COALESCE(/userCategory,
         /employeeCategory,
         /vendorCategory,
         'restricted') != 'restricted'

AMPS will return the first value that is not NULL, and compare that value to the constant string 'restricted'. Notice that, to make the intent of the filter clear, this example provides a constant value for AMPS to return from the COALESCE if all of the field values are NULL.

The COALESCE function, like other functions in AMPS, is not array-aware. This means that when one of the XPath expressions provided to COALESCE specifies an array in the original message, AMPS provides the first item in the array to the COALESCE function. See Working With Arrays for details.

Compound Types in AMPS

Many messaging applications are designed for high performance and use a simplified message structure. For applications that use compound types, AMPS includes the ability to parse and filter on the contents of nested data structures.

For performance, AMPS parses nested data structures into a set of values. As with single-valued (or scalar) values, the AMPS expression language refers to a parsed set of values that is common to all message types rather than the underlying data.

The AMPS message types treat compound data types as a set of paths with corresponding scalar values. A field that only contains other fields is represented as a step in the path to the primitive values that it contains.

AMPS parses compound types as follows:

• Any field that contains a scalar value is represented as an identifier/value pair.

• Any field that contains other fields is represented as a step in the path to that value.

• Multiple values with identical paths are represented as an array. For more information on arrays in the AMPS expression language, see Working With Arrays.

The following JSON document is a simple example.

{"outer": {"middle": { "inner": 5  } } }

With this document, AMPS produces the following parsed value:

Path	Value
/outer/middle/inner	5

In the parsed representation, the outer and middle fields contain no data of their own. They serve only as containers for the inner field which contains data.

Notice that the intermediate paths do not have an explicit scalar value.

With a more complex document the parsed representation continues to follow the same principles, as shown in the following example.

{"outer" :
   {
      "array"    : ["a1", "a2", "a3"],
      "compound" : { "A" : "middle-A",
                     "B" : "middle-B",
                     "C" :
                           [ {"C1":"first-C1", "D1":"first-D1"},
                             {"C1":"second-C1","D1":"second-D1"} ]
                   }
   }
 }

The representation of the above message in the AMPS expression language would typically be as follows:

Path	Value	Notes
/outer/array	['a1', 'a2', 'a3]	Elements in the array can be referred to directly with subscript notation. For example: /outer/array[0] is 'a1'.
/outer/compound/A	'middle-A'
/outer/compound/B	'middle-B'
/outer/compound/C/C1	['first-C1', 'second-C1']	Elements in the array can be referred to directly with subscript notation. For example: /outer/compound/C/C1[0] is 'first-C1'.
/outer/compound/C/D1	['first-D1', 'second-D1']	Elements in the array can be referred to directly with subscript notation. For example: /outer/compound/C/D1[0] is 'first-D1'.

As with the first example, fields that do not directly contain a value do not have an explicit scalar value. Values with the same identifier are represented as an array of values with that identifier.