LogoLogo
AMPS Server Documentation 5.3.4
AMPS Server Documentation 5.3.4
  • Welcome to AMPS 5.3.4
  • Introduction to AMPS
    • Overview of AMPS
    • Getting Started With AMPS
      • Installing AMPS
      • Starting AMPS
      • JSON Messages - A Quick Primer
      • spark: the AMPS command-line client
      • Evaluating AMPS on Windows or MacOS
      • Galvanometer and RESTful Statistics
    • AMPS Basics: Subscribe and Publish to Topics
    • State of the World (SOW): The Message Database
      • When Should I Store a Topic in the SOW?
      • How Does the SOW Work?
      • Configuration
      • Queries
      • Atomic Query and Subscribe
      • Advanced Messaging and the SOW
    • Record and Replay Messages with the AMPS Transaction Log
    • Message Queues
    • Scenario and Feature Reference
      • Recovery Strategies
    • Getting Support
    • Advanced Topics
    • Next Steps
  • AMPS Evaluation Guide
    • Introduction
    • Evaluation and Development with AMPS
    • Tips on Measuring Performance
    • Next Steps
  • AMPS User Guide
    • Introduction
      • Product Overview
      • Requirements
      • Organization of this Guide
        • Documentation Conventions
      • Technical Support
    • Installing and Starting AMPS
      • Installing AMPS
      • Starting AMPS
      • Production Configuration
    • Subscribe and Publish
      • Topics
      • Filtering Subscriptions by Content
      • Conflated Subscriptions
      • Replacing Subscriptions
      • Messages in AMPS
      • Message Ordering
      • Retrieving Part of a Message
    • AMPS Expressions
      • Syntax
      • Identifiers
      • AMPS Data Types
      • Grouping and Order of Evaluation
      • Logical Operators
      • Arithmetic Operators
      • Comparison Operators
      • LIKE Operator
      • Conditional Operators
      • Working with Arrays
      • Regular Expressions
      • Performance Considerations
    • AMPS Functions
      • AMPS Function Overview
      • String Comparison Functions
      • Concatenating Strings
      • Managing String Case
      • Replacing Text in Strings
      • String Manipulation Functions
      • Date and Time Functions
      • Array Reduce Functions
      • Geospatial Functions
      • Numeric Functions
      • CRC Functions
      • Message Functions
      • Client Functions
      • Coalesce Function
      • AMPS Information Functions
      • Typed Value Creation
      • Constructing Fields
      • Aggregate Functions
    • State of the World (SOW) Topics
      • How Does the SOW Work?
      • Using the State of the World
      • Understanding SOW Keys
      • Indexing SOW Topics
      • Programmatically Deleting Records from the Topic State
      • SOW Maintenance
        • Creating a Maintenance Schedule for a Topic
        • Setting Per-Message Lifetime
      • Storing Multiple Logical Topics in One Physical Topic
    • Querying the State of the World (SOW)
      • Overview of SOW Queries
      • Query and Subscribe
      • Historical SOW Topic Queries
      • Managing Result Sets
      • Batching Query Results
    • Out-of-Focus Messages (OOF)
    • State of the World Message Enrichment
    • Incremental Message Updates
      • Using Delta Publish
      • Understanding Delta Publish
      • Delta Publish Support
    • Receiving Only Updated Fields
      • Using Delta Subscribe
      • Identifying Changed Records
      • Conflated Subscriptions and Delta Subscribe
      • Select List and Delta Subscribe
      • Options for Delta Subscribe
    • Conflated Topics
    • Aggregation and Analytics
      • Understanding Views
      • Defining Views and Aggregations
      • Constructing Field Contents
      • Best Practices for Views
      • View Examples
      • Aggregated Subscriptions
    • Record and Replay Messages
      • Using the Transaction Log and Bookmark Subscriptions
      • Understanding Message Persistence
      • Configuring a Transaction Log
      • Replaying Messages with Bookmark Subscription
      • Managing Journal Files
      • Using amps-grep to Search the Journal
    • Message Queues
      • Getting Started with AMPS Queues
      • Understanding AMPS Queuing
      • Advanced Messaging and Queues
      • Replacing Queue Subscriptions
      • Handling Unprocessed Messages
      • Advanced Queue Configuration
      • Queue Subscriptions Compared to Bookmark Replays
    • Message Types
      • Default Message Types
      • BFlat Messages
      • MessagePack Messages
      • Composite Messages
      • Protobuf Message Types
      • Struct Message Types
    • Command Acknowledgment
      • Requesting Acknowledgments
      • Receiving Acknowledgments
      • Bookmark Subscriptions and Completed Acknowledgments
      • Bookmark Subscriptions and Persisted Acknowledgments
      • Acknowledgment Conflation and Publish Acknowledgements
    • Transports
      • Client Connections
      • Replication Connections
      • Transport Filters
    • Running AMPS as a Linux Service
      • Installing the Service
      • Configuring the Service
      • Managing the Service
      • Uninstalling the Service
    • Logging
      • Configuring Logging
      • Log Message Format
      • Message Levels
      • Message Categories
      • Logging to a File
      • Logging to a Compressed File
      • Logging to Syslog
      • Logging to the Console
      • Looking up Errors with ampserr
    • Event Topics
      • Client Status Events
      • SOW Statistics Events
      • Persisting Event Topics
    • Utilities
      • Command-Line Basic Client
      • Dump clients.ack File
      • Dump journal File
      • Dump queues.ack File
      • Dump SOW File
      • Dump Journal Topic Index File
      • Find Bookmark or Transaction ID in Transaction Log
      • Find Information in Error Log or Transaction Log
      • Identify Type of AMPS File
      • List/Explain Error Codes
      • Query Statistics Database
      • Statistics Database Report
      • Storage Performance Testing
      • Submit Minidump to 60East
      • Obsolete Utility: Upgrade File Formats
    • Monitoring AMPS
      • Statistics Collection
        • Time Range Selection
        • Output Formatting
      • Galvanometer
      • Configuring Monitoring
    • Automating AMPS with Actions
    • Replicating Messages Between Instances
      • Replication Basics
      • Configuring Replication
      • Replication Configuration Validation
      • Replication Resynchronization
      • Replication Compression
      • Destination Server Failover
      • Two-Way Replication
      • PassThrough Replication
      • Guarantees on Ordering
      • Replication Security
      • Understanding Replication Message Routing
      • Replicated Queues
      • Replication Best Practices
    • Highly Available AMPS Installations
      • Overview of High Availability
        • Example: Pair of Instances for Failover
        • Example: Regional Distribution
        • Example: Regional Distribution with HA
        • Example: Hub and Spoke / Expandable Mesh
      • Details of High Availability
      • Slow Client Management and Capacity Limits
      • Message Ordering Considerations
    • Operation and Deployment
      • Capacity Planning
      • Linux OS Settings
      • Upgrading AMPS
      • Using AMPS with a Proxy
      • Operations Best Practices
    • Securing AMPS
      • Authentication
      • Entitlement
      • Providing an Identity for Outbound Connections
      • Protecting Data in Transit Using TLS/SSL
    • Troubleshooting AMPS
      • Planning for Troubleshooting
      • Diagnostic Utilities
      • Finding Information in the Log
      • Reading Replication Log Messages
      • Troubleshooting Disconnected Clients
      • Troubleshooting Regular Expression Subscriptions
    • AMPS Distribution Layout
    • Optionally-Loaded Modules
      • Optional Functions
        • Legacy Messaging Functions
        • Special-Purpose Functions
      • Optional SOW Key Generator
        • Chaining Key Generator
      • Optional Authentication/Entitlements Modules
        • RESTful Authentication and Entitlements
        • Multimethod Authentication Module
        • Simple Access Entitlements Module
      • Optional Authenticator Modules
        • Multimethod Authenticator
        • Command Execution Authenticator
    • AMPS Statistics
    • File Format Versions
  • AMPS Configuration Guide
    • AMPS Configuration Basics
      • Getting Started With AMPS Configuration
      • Units, Intervals, and Environment Variables
      • Working With Configuration Files
      • Including External Files
    • Instance Level Configuration
    • Admin Server and Statistics
    • Modules
    • Message Types
    • Transports
    • Logging
    • State of the World (SOW)
      • SOW/Topic
      • SOW/*Queue
      • SOW/ConflatedTopic
      • SOW/View
    • Replication
      • Replication Validation
    • Transaction Log
    • Authentication
    • Entitlement
    • Actions
      • Configuration for Actions
      • Choosing When an Action Runs
        • On a Schedule
        • On AMPS Startup or Shutdown
        • On a Linux Signal
        • On a REST Request
        • On Minidump Creation
        • On Client Connect or Disconnect
        • On Client Logon
        • On Client Offline Message Buffering
        • On Subscribe or Unsubscribe
        • On Incoming Replication Connections
        • On Outgoing Replication Connections
        • On Message Published to AMPS
        • On Message Delivered to Subscriber
        • On Message Affinity
        • On SOW Message Expiration
        • On SOW Message Delete
        • On OOF Message
        • On Message Condition Timeout
        • On Message State Change
        • On a Custom Event
      • Choosing What an Action Does
        • Rotate Error/Event Log
        • Compress Files
        • Truncate Statistics
        • Manage Transaction Log Journal Files
        • Remove Files
        • Delete SOW Messages
        • Compact SOW Topic
        • Query SOW Topic
        • Manage Security
        • Enable or Disable Transports
        • Publish Message
        • Manage Replication Acknowledgment
        • Extract Values from a Message
        • Translate Data Within an Action
        • Increment Counter
        • Raise a Custom Event
        • Execute System Command
        • Manage Queue Transfers
        • Create Minidump
        • Shut Down AMPS
        • Debug Action Configuration
      • Conditionally Stopping an Action
        • Based on File System Capacity
        • Based on an Expression
      • Examples of Action Configuration
        • Archive Journals Once a Week
        • Archive Journals On RESTful Command
        • Record Expired Queue Messages to a Dead Letter Topic
        • Copy Messages that Exceed a Timeout to a Different Topic
        • Deactivate and Reactivate Security on Signals
        • Reset Entitlements for a Disconnected Client
        • Extract Values from a Published Message
        • Shut Down AMPS When a Filesystem Is Full
        • Increment a Counter and Echo a Message
    • Protocols
  • AMPS Monitoring Guide
    • Statistics Types
    • Table Reference
    • Administrative Actions
    • Host Statistics
      • cpu
      • disks
      • memory
      • name
      • network
    • AMPS Instance Statistics
      • api
      • clients
      • config.xml
      • config_path
      • conflated_topics
      • cpu
      • cwd
      • description
      • environment
      • lifetimes
      • logging
      • memory
      • message_types
      • name
      • name_hash
      • pid
      • processors
      • queues
      • queries
      • replication
      • sow
      • statistics
      • subscriptions
      • timestamp
      • transaction_log
      • transports
      • tuning
      • uptime
      • user_id
      • version
      • views
  • AMPS Command Reference
    • Commands to AMPS
      • logon
      • Publishing
        • publish
        • delta_publish
      • Subscribing to and Querying Topics
        • subscribe
        • sow
        • sow_and_subscribe
        • unsubscribe
        • delta_subscribe
        • sow_and_delta_subscribe
      • Removing Messages (SOW/Topic or Message Queue)
      • heartbeat
      • flush
    • Responses from AMPS
      • sow: Content from Server
      • publish: Content from Server
      • oof: Content from Server
      • ack: Status from Server
      • group_begin / group_end : Result Set Delimiters
    • Protocol Reference
      • AMPS Protocol
      • Legacy Protocols Reference
    • Command Cookbook
      • Cookbook: Delta Publish
      • Cookbook: Delta Subscribe
      • Cookbook: Publish
      • Cookbook: SOW
      • Cookbook: SOW and Delta Subscribe
      • Cookbook: SOW and Subscribe
      • Cookbook: SOW Delete
      • Cookbook: Subscribe
  • Deployment Checklist
    • Ensure Sufficient Capacity
    • Apply System and AMPS Configuration
    • Create Maintenance Plan
    • Create Monitoring Strategy
    • Create Patch and Upgrade Plan
    • Create and Test Support Process
    • Conclusion
  • AMPS Clients
    • Performance Tips and Best Practices
    • C++
    • C#/.NET
    • Java
    • JavaScript
    • Python
Powered by GitBook

Get Help

  • FAQ
  • Legacy Documentation
  • Support / Contact Us

Get AMPS

  • Evaluate
  • Develop

60East Resources

  • Website
  • Privacy Policy

Copyright 2013-2024 60East Technologies, Inc.

On this page
  • Numeric Types and Literals in AMPS Expressions
  • Type Promotion for Numeric Types
  • String Literals in AMPS Expressions
  • NULL, NaN and IS NULL
  • Compound Types in AMPS
Export as PDF
  1. AMPS User Guide
  2. AMPS Expressions

AMPS Data Types

Each value in AMPS is assigned a data type when the message type module parses the value. AMPS operators and functions attempt to convert values into compatible types, based on the type of operation. For example, the * operator (multiplication) will attempt to convert all values to numeric values, while the CONCAT function (string concatenation) will attempt to convert all values to strings. In effect, a value in AMPS can be transparently treated as any type to which it can be meaningfully converted.

Internally, AMPS uses the data types in the table below. As mentioned above, the message type module is responsible for assigning the type of a value from an incoming message as part of the parsing process. For some types, such as JSON, XML, FIX and NVFIX, the parser infers the type of the value from the field. For other types, such as MessagePack, BFLAT, Google Protocol Buffers or BSON, the message itself contains information about the type of the field.

As mentioned above, the AMPS expression language does not limit the value to the type assigned by the message type module. Instead, a value in AMPS can be used in any context.

For example, given the following JSON document:

{"a":1,"b":"47"}

The values of /a and /b can be used as either string values or numeric values. AMPS will automatically convert these values as necessary, and AMPS considers the string or numeric representation to be equally correct and valid.

The following table lists the data types in the AMPS expression language:

Type
Description
Untyped Message Examples

NULL

Unknown, untyped value (SQL-92 semantics)

[no field provided]

NVFIX: a=<SOH>

JSON: {"a":null}

XML: <a/>

Boolean

True (1) or false (0)

JSON: {"e":true}

Integer

Signed 64-bit integer or unsigned 64-bit integer for values > LONG_MAX

NVFIX: b=24

JSON: {"b":24}

XML: <b>24</b>

Floating Point Number

64-bit floating point number

NVFIX: c=24.0

JSON: {"c":24.0}

XML: <c>24.0</c>

String

Arbitrary sequence of bytes of a specific length

An empty string is considered to be NULL

NVFIX: d=Grilled cheese sandwich<SOH>

JSON: {"d":"Grilled cheese sandwich"}

XML: <d>Grilled cheese sandwich</d>

Numeric Types and Literals in AMPS Expressions

Numeric values in AMPS are always typed as either integers or floating point values. All numeric types that are less than or equal to the LONG_MAX limit in AMPS are signed, otherwise, the numeric type is unsigned. AMPS message types convert the original numeric types (or original representation for message types that do not have typed values) into the internal AMPS type system for the purposes of expression evaluation.

Within expressions, integer values are all numerals, with no decimal point, and can have a value in the same range as a 64-bit integer. For example:

42
149
-273
18446744073709551610

Within expressions, all numerals with a decimal point are floating-point numbers. AMPS interprets these numerals as double-precision floating point values. For example:

3.1415926535
98.6
-273.0

or, in scientific notation:

31.4e-1
6.022E23
2.998e8

AMPS automatically converts strings that contain numeric values to numbers when strings are used with an operator, function or comparison that expects a numeric value.

Type Promotion for Numeric Types

AMPS uses the following rules for type promotion when evaluating numeric expressions:

  1. If any of the values in the expression is NaN, the result is NaN.

  2. Otherwise, if any of the values in the expression is floating point, the result is floating point.

  3. Otherwise, all of the values in the expression are integers, and the result is an integer.

Notice that, for division in particular, the results returned are affected by the type of the values. For example, the expression 1 / 5 evaluates to 0 since the result is interpreted as an integer. In comparison, the expression 1.0 / 5 evaluates to 0.2 since the result is interpreted as a floating point value.

String Literals in AMPS Expressions

When creating expressions for AMPS, string literals are indicated with single or double quotes. For example:

/FIXML/Order/Instrmt/@Sym = 'IBM'

AMPS supports the following escape sequences within string literals:

Escape Sequence
Definition

\a

Alert

\b

Backspace

Horizontal tab

Newline

\f

Form feed

Carriage return

\xHH

Hexadecimal digit where H is (0..9,a..f,A..F)

\OOO

Octal Digit (0..7)

Additionally, any character which follows a backslash will be treated as a literal character.

AMPS string operations have no restrictions on character set, and correctly handle embedded NULL characters (\x00) and characters outside of the 7-bit ASCII range. AMPS string operations are not unicode-aware.

NULL, NaN and IS NULL

XPath expressions are considered to be NULL when they evaluate to an empty or nonexistent field reference. NULL values follow SQL-92 semantics.

This means that comparisons with NULL are never true (in other words, even if /a is NULL, /a != NULL is false and /a == NULL is also false).

AMPS considers a zero-length string to be NULL.

In numeric expressions where the operands or results are not a valid number, the XPath expression evaluates to NaN (not a number). The rules for applying the AND and OR operators against NULL and NaN values are outlined in the tables below:

Operand1
Operand2
Result

TRUE

(AND)

NULL

NULL

FALSE

(AND)

NULL

FALSE

NULL

(AND)

NULL

NULL

NULL

(AND)

TRUE

NULL

NULL

(AND)

FALSE

NULL

Operand1
Operand2
Result

TRUE

(OR)

NULL

TRUE

FALSE

(OR)

NULL

NULL

NULL

(OR)

NULL

NULL

NULL

(OR)

TRUE

NULL

NULL

(OR)

FALSE

NULL

Likewise, direct comparisons with NULL are not ever true (so, if /b is NULL, /b == NULL does not produce a true value, and neither does /b != NULL). AMPS, like SQL-92, provides an IS NULL predicate for testing whether a value is NULL, and an IS NOT NULL predicate for testing whether a value is not NULL.

There also exists an IS NAN predicate for checking that a value is NaN (not a number.)

To reliably check for existence of a NULL value, you must use the IS NULL predicate such as the filter: /optionalField IS NULL

To reliably check that a value is not NULL, you must use the IS NOT NULL predicate or negate the value of an IS NULL test: /optionalField IS NOT NULL and NOT /optionalField IS NULL are equivalent.

AMPS also provides a COALESCE() function that accepts a set of values and returns the first value that is not NULL. For example, given the following filter expression:

COALESCE(/userCategory,
         /employeeCategory,
         /vendorCategory,
         'restricted') != 'restricted'

AMPS will return the first value that is not NULL, and compare that value to the constant string 'restricted'. Notice that, to make the intent of the filter clear, this example provides a constant value for AMPS to return from the COALESCE if all of the field values are NULL.

Compound Types in AMPS

Many messaging applications are designed for high performance and use a simplified message structure. For applications that use compound types, AMPS includes the ability to parse and filter on the contents of nested data structures.

For performance, AMPS parses nested data structures into a set of values. As with single-valued (or scalar) values, the AMPS expression language refers to a parsed set of values that is common to all message types rather than the underlying data.

The AMPS message types treat compound data types as a set of paths with corresponding scalar values. A field that only contains other fields is represented as a step in the path to the primitive values that it contains.

AMPS parses compound types as follows:

  • Any field that contains a scalar value is represented as an identifier/value pair.

  • Any field that contains other fields is represented as a step in the path to that value.

The following JSON document is a simple example.

{"outer": {"middle": { "inner": 5  } } }

With this document, AMPS produces the following parsed value:

Path
Value

/outer/middle/inner

5

In the parsed representation, the outer and middle fields contain no data of their own. They serve only as containers for the inner field which contains data.

Notice that the intermediate paths do not have an explicit scalar value.

With a more complex document the parsed representation continues to follow the same principles, as shown in the following example.

{"outer" :
   {
      "array"    : ["a1", "a2", "a3"],
      "compound" : { "A" : "middle-A",
                     "B" : "middle-B",
                     "C" :
                           [ {"C1":"first-C1", "D1":"first-D1"},
                             {"C1":"second-C1","D1":"second-D1"} ]
                   }
   }
 }

The representation of the above message in the AMPS expression language would typically be as follows:

Path
Value
Notes

/outer/array

['a1', 'a2', 'a3]

Elements in the array can be referred to directly with subscript notation.

For example: /outer/array[0] is 'a1'.

/outer/compound/A

'middle-A'

/outer/compound/B

'middle-B'

/outer/compound/C/C1

['first-C1', 'second-C1']

Elements in the array can be referred to directly with subscript notation.

For example: /outer/compound/C/C1[0] is 'first-C1'.

/outer/compound/C/D1

['first-D1', 'second-D1']

Elements in the array can be referred to directly with subscript notation.

For example: /outer/compound/C/D1[0] is 'first-D1'.

As with the first example, fields that do not directly contain a value do not have an explicit scalar value. Values with the same identifier are represented as an array of values with that identifier.

PreviousIdentifiersNextGrouping and Order of Evaluation

Last updated 4 months ago

When a function or operator that expects a numeric type is provided with a string, AMPS will attempt to convert string values to numeric types as necessary. When converting string values, AMPS recognizes the same numeric formats in message data as are supported in the AMPS expression language (see ). If the string is in an unrecognized format, AMPS converts the string as NaN.

The COALESCE function, like other functions in AMPS, is not array-aware. This means that when one of the XPath expressions provided to COALESCE specifies an array in the original message, AMPS provides the first item in the array to the COALESCE function. See for details.

Multiple values with identical paths are represented as an array. For more information on arrays in the AMPS expression language, see .

Working With Arrays
Working With Arrays
Numeric Types and Literals