Special-Purpose Functions

This module contains User-Defined Functions (UDF) that are a part of the AMPS distribution, but which serve specialized use cases and are not currently intended for use outside of those scenarios. The current implementations of these UDFs are considered to be experimental and are only supported for use in the intended context -- a general-purpose version of these features might behave somewhat differently.

In this release, the experimental UDF module contains one function for use in preprocessing or enrichment.

Preprocessing/Enrichment

The VALUE_LOOKUP function can be used to look up a value in another topic during preprocessing or enrichment of a SOW topic.

To use the function, you must first load the experimental UDF module and register the lookups that will be used by the module.

A lookup is declared using the Lookup option to the module. This option uses a semicolon-delimited list of parameters to define the lookup. All parameters are required.

Option	Definition
Name	The name to use for the lookup.
Parameter Count	The number of fields to use for the lookup.
Message Type	The message type of the topic to use for lookup.
Topic Name	The name of the topic to use for lookup.
Return Field	The field to return from the located message.
Lookup Fields	The fields to use to look up the message.

For example, to create a lookup named return-value-by-id, that takes a single parameter and uses that to match the /id field in the data topic of json type, returning the /value field of the matched record, you would provide this option when loading the module:

<Lookup>return-value-by-id;1;json;data;/value;/id</Lookup>

Likewise, to create a lookup named getNotes that uses a combination of /customerId and /orderId to return the /notes field of a matching record from the orders topic of nvfix type, you would provide this option when loading the module:

<Lookup>getNotes;2;nvfix;orders;/notes;/customerId;/orderId</Lookup>

To use the UDF, provide the name of the lookup and the values to lookup:

VALUE_LOOKUP("return-value-by-id", /thisId)

The above function call will take the /thisId field from the current message, use that to find a matching /id value in the data topic (of json type), and then return the /value field from a matching record.

The VALUE_LOOKUP function uses an exact match for the fields specified.

Function	Parameters	Description
VALUE_LOOKUP	name of lookup one or more values for lookup The number of values provided must match the definition for the lookup name.	Returns a value from the referenced topic as a string. If multiple records match the lookup, returns one of those records. If no records match the lookup, returns NULL.

To use this function, the module must be loaded and one or more lookup functions must be declared when the module is loaded, as described in this section. This function is not loaded by default.

Limitations and Usage

This UDF has the following limitations:

• 60East makes no particular performance guarantees for this UDF, but recommends that the topics used for lookup be relatively small topics with infrequently updated data.

• The UDF caches lookup results for performance. This means that the UDF will increase the memory footprint of the AMPS instance (typically by the size of the lookup values + the size of the return values + 32 bytes per record indexed in the target topic + a small amount of overhead per defined lookup).

• The lookup returns the value retrieved as an AMPS string. As with all AMPS values, if the value of the string can be coerced to a number, it can be used as a numeric value.

• If a lookup results in multiple matches, the UDF will return values for one of the matching records, but does not guarantee which result will be returned.

• The lookup function should only be used in preprocessing and enrichment in cases where it is not possible to use a view instead. The function cannot be used in view definitions, since it cannot guarantee that data is available during view recovery. The function cannot be used for a SOW topic with transient durability and a recovery point other than now, since the function cannot guarantee that data is available during SOW recovery.

• Values provided are cached for performance and the cache is updated asynchronously. This means that a publish to the source topic immediately followed by a call to the UDF may or may not produce the newly published value.

• AMPS considers the lookup function non-deterministic, since the value returned depends on the state of a message stored in a SOW topic, rather than the message being currently evaluated.

Loading the Module

The experimental functions module is shipped in the AMPS lib directory, and is named libamps_udf_experimental.so. To use the module, you load it as shown below, providing any options necessary for the functions contained in the module.

<Modules>
  <Module>
    <Name>experimental-udf</Name>
    <Library>libamps_udf_experimental.so</Library>
    <Options>
      <Lookup>value-by-id;1;json;source-sow;/value;/id</Lookup>
      <Lookup>customer-id-by-order-id;1;nvfix;orders;/id;/customerId</Lookup>
    </Options>
  </Module>
</Modules>