RESTful Authentication and Entitlements

The AMPS distribution includes a module that provides authentication and entitlement via an external Web Service. For some installations, this module provides a convenient way to integrate with an existing authentication and entitlement infrastructure without creating an entitlement plugin.

In this release, the HTTP authentication module is provided with AMPS, but is not loaded by default. This module is an optional extension to the AMPS product, and while it is included with the AMPS distribution, the module must be explicitly loaded, enabled, and configured.

When using this module, AMPS requests permissions documents from an external service using http or https. The request to retrieve the permissions document includes the credentials provided with the client logon. If the request succeeds, the module considers the user to have successfully authenticated to AMPS. If the request to retrieve the permissions document fails, the module considers the user to have failed authentication. When authentication succeeds, the contents of the document returned specify the permissions that the module grants to the user.

The web service module expects that the web service endpoint will follow RESTful (HTTP) semantics. For example, the module uses standard HTTP headers for authentication and expects that if the credentials provided are not valid, the endpoint will return an HTTP 403 status code (or equivalent).

When to Use the Web Service Module

The AMPS Web Service Authentication and Entitlement module can be a good option when:

• The site does not have an existing authentication and entitlements infrastructure for AMPS.

• It is more feasible to develop and test a standalone web service than to develop a server plugin for AMPS.

• Applications need to integrate with an existing authentication and entitlement system that offers limited Linux or C/C++ support.

• An application that will use another authentication scheme in production needs an easy way to test changes to entitlements and entitlement scenarios that are difficult to replicate in the production system.

Permissions Document Format

This section describes the format of the permissions documents used by the Web Service Authentication and Entitlement module.

All documents are in JSON format, and consist of a set of permissions. The document is not required to contain the user name: this is intentional, and allows systems to easily provide identical permissions for all users in a group without having to create unique documents.

The entitlement document expresses each permission as a field of a JSON document. The following is an example of a permissions document:

{
  "logon": true,
  "replication-logon" : false,
  "topic": [
            { "topic": "test",
              "read": "/priority = 1",
              "write": false },
            { "topic": ".*",
              "read": true,
              "write": true }
           ],
  "admin": [
            { "topic": "^/amps/instance/.*",
              "read": true,
              "write": false },
            { "topic": ".*",
              "read": false,
              "write": false }
           ]
}

The Web Authentication Module processes the entitlements in document order. Going through the document in order, this set of entitlements specifies the following permissions:

• This user has permission to log on to AMPS, as set by the logon field.

• This user does not have permission to make a replication connection to AMPS. A replication connection that uses these credentials will be refused.

• This user has read permissions to the topic test for messages that match the filter /priority = 1. This user does not have write permissions to the topic.

• The user has read and write permissions to every other topic in the instance without content restrictions.

• The user has read permissions to the administrative interface for information under the /amps/instance path.

• The user has no other permissions to the administrative interface.

The Web Service Authentication and Entitlement Module also allows fine-grained control of the topics that a given user is allowed to publish to via replication with the replicated-topics configuration element. The following permissions document shows sample permissions for a replication connection:

{
  "replication-logon": true,
  "logon": false,
  "replicated-topics":["^/orders/NYC/.*",
                       "/events/P1"]
}

This document specifies that the user can only log on to AMPS via replication connections. The user has permission to replicate messages to the topic /events/P1 (using an exact match) and topics that begin with /orders/NYC, as specified by the regular expression ^/orders/NYC/.*. The user has no other permissions. In particular, the user cannot log on from an AMPS client, and could not publish to or subscribe to any topics even if logon was changed to true without an explicit topic permission.

The structure of the permissions document is as follows:

Client Transport Permissions

These fields control access for applications to connect, including the SQL tab of the Galvanometer (which is an AMPS client application).

Field	Value
logon	Specifies permission for an application to log on to AMPS. When this field is present, the value of the field must be a boolean true or false.
topic	Controls access to topics within AMPS. When this field is present, the value of the field must be a permission list, as described below. If this field is not present, the user cannot publish to or subscribe to any topics.

Admin Transport Permissions

This field controls access to the administrative interface and statistics, including Galvanometer statistics.

Field	Value
admin	Controls access to the administrative interface. When this field is present, the value of the field must be a permission list, as described below. If this field is not present, the user has no access to the administrative interface. Notice that, because the Admin console uses HTTP, there is no equivalent of the logon or replication-logon permission for the Admin transport.

Replication Transport Permissions

These fields control permission to replicate messages to this instance of AMPS.

Field	Value
replication-logon	Controls permission for a replication connection to log on to AMPS. When this field is present, the value of this field must be a boolean true or false.
replicated-topics	An array containing the topics that this user can replicate to. When this field is present, the value of the field must be an array of strings that specify topic names or regular expressions. For example, the following entry allows this user to replicate ONLY to topics that begin with /orders/NYC "replicated-topics":["^/orders/NYC/.*"] If this field is not present in the permissions document, the user cannot replicate to any topics.

Connection Properties

This field allows you to set the user name for this connection to a value different than that used for logon.

Field	Value
user_name	Specifies that the module should set the authenticated user name of the connection to the provided value. This option is ignored when the module is in EntitlementsOnly mode. If this field is not present in the returned permissions document, the authenticated user name is set to the value provided with the logon request from the client.

Permissions Lists

Permissions lists within this document are arrays of entries. Each entry in the array is a JSON object with the following format:

Field	Value
topic	The name of the topic this permission definition applies to. This name can be either a literal value, or a regular expression to use to match topic names. A name is interpreted as a regular expression if it contains any characters used in regular expression matching (for example; ^, $, *, . and so on). Regular expression matching provides full support for PCRE regular expressions.
read	Defines the read permission for the topic. The value of this field can be either true, false or an AMPS filter. When the value is true, the module grants read permission to the topic with no restrictions. When the value is false, the module denies read permission. When the value is a filter, the module grants read permission to the topic only for those messages that match the filter.
write	Defines the write permission for the topic. The value of this field can be either true, false or an AMPS filter. When the value is true, the module grants write permission to the topic with no restrictions. When the value is false, the module denies write permission. When the value is a filter, the module grants write permission to the topic only for those messages that match the filter.
select	Defines the entitlement select list for this topic. When this value is present, the module applies the select list provided to the topic entitlement. Notice that this element should include only the select list specifier and not the enclosing brackets for the select list. For example, a valid select element might be: "select":"-/,+/id,+/home/range"

Configuring AMPS to use Web Service Authentication and Entitlements

The web service authentication and entitlement module is included in the AMPS distribution, but is not loaded by default. To load the module in AMPS, add the following configuration item to the Modules block of the AMPS configuration file:

<Modules>
    ...

    <Module>
        <Name>web-entitlements</Name>
        <Library>libamps_http_entitlement.so</Library>

        <!-- You may specify options here, or where the module is used.-->
    </Module>

    ...
</Modules>

Options for the module may be set when the module is loaded, when the module is used for Authentication or Entitlement, or in both places. Options set when the module is loaded are inherited as the default values for all uses of the module in the instance. Options specified in an Authentication or Entitlement block override options set when the module is loaded.

For Authentication, the module supports the following options:

Option	Description
ResourceURI	The URI to request when a user logs into AMPS using this module. This option is required. For this option, AMPS substitutes the placeholder {{USER_NAME}} with the name of the user being authenticated. For example, here are two possible values for the ResourceURI: http://cred-server:8080/admin/group_policy.json http://cred-server:8080/{{USER_NAME}}.json In the first case, AMPS requests a document with the current user name of the user logging on substituted for the {{USER_NAME}} component of the URI. In the second case, the document is requested with the credentials of the user connecting, but AMPS requests the same document for each user. There is no default for this parameter.
CredentialStore	Identifier for the entitlement store where the retrieved permission sets will be stored. When used for authentication, this is the name of the entitlement cache that the module stores parsed information into until AMPS requests the information. In most cases, there is no need to set this parameter. When multiple transports use different ResourceURI values, but those transports are expected to maintain the same information, specifying a CredentialStore value can speed the logon process and reduce the number of copies of the parsed permissions document in memory. Likewise, if a module must ensure that permission sets are kept separate (so that, for example, an internal user named 'johndoe' and a web user named 'johndoe' have separate credentials), explicitly setting a CredentialStore value can make it easier to verify that the permission sets are completely separate. Default: The literal value of the ResourceURI parameter.
ConnectionTimeout	The maximum amount of time to wait for a connection to the web server for each request, in milliseconds. If a connection is not made within the specified time, the module stops the connection attempt and the request fails. Default: 2000
RequestTimeout	The maximum amount of time to wait for the web server to return a permissions document on each request, in milliseconds. If the server does not return a permissions document within the specified time, the module closes the connection and the request fails. Increasing this timeout above the default value may produce potential stuck thread warnings if the web service is slow to respond. Default: 5000
RetryCount	Sets the number of times to retry the request if retrieving the authentication document fails for any reason. Default: 0 (only try once)
HTTPHeader	Sets a header to add to the HTTP request. The configuration can specify any number of HTTPHeader elements, and the module will provide each of the specified headers with the authentication request. There is no default for this option. If no HTTPHeader option is included, the module provides a standard set of headers. This option supports variable replacement during an authentication request, as described in the following table. This option supports expansion of the AMPS_USER_NAME (and the USER_NAME backward compatibility form) when the module is in EntitlementsOnly mode.
EntitlementTimeout	Optionally, sets the amount of time to consider an entitlements document for a given user to be valid. After this timeout period, AMPS will check each entitlements document returned for that user to see if the entitlements have changed. If an authentication request returns a different entitlements document, the module will reset permissions for all connections currently connected with that username (which will disconnect those connections and clear the entitlement cache). When this option is not provided, the module retains permissions until all connections with the current user name are disconnected, then removes cached permissions in the module and clears the AMPS entitlement cache. This option is not supported in EntitlementsOnly mode. This option accepts a number of milliseconds or an AMPS interval. The granularity for the EntitlementTimeout is a second. Fractions of a second, or values less than one second, may be rounded or truncated.
ReuseConnections	Optionally, sets the module to reuse connections to the entitlement service when possible. By default, the module creates a new connection for each entitlement request. When set to enabled, the module will reuse HTTP connections when possible. This option is only supported within the Modules definition, and is set for all uses of the module within the configuration. The option will be ignored if it is set within the Authentication or Entitlement blocks. Default: disabled
ServerAcceptsEmptyAuthId	By default, the module will refuse authentication requests that do not contain an auth ID without allowing the request to reach the server. Instead, the module returns an authentication failure to AMPS immediately. This is designed to protect the authentication web service from requests that cannot succeed. In some environments, the web service providing authentication will authenticate a user based entirely on the authentication token provided as the password, and will return the authentication ID for AMPS to use in the permissions document. When this option is set to true, the module will submit authentication requests to the service even when no authentication ID is provided on the logon request from the AMPS client. An authentication ID is still required for AMPS to be able to track permissions. When this option is set to true, the authentication service must set the auth ID in the permissions document when no auth ID is provided in the permissions request. Otherwise, the authentication request will fail. Default: false

The following tokens are expanded in the HTTPHeader element of an authentication request:

Authentication Header Token	Expansion
AMPS_CLIENT_NAME	Client name provided in the logon request.
AMPS_CONNECTION_NAME	The connection name for the connection requesting logon, as assigned by the AMPS server (based on the transport name).
AMPS_CORRELATION_ID	The correlation ID provided on the logon request.
AMPS_MESSAGE_TYPE	The message type of the logon request.
AMPS_PASSWORD	The password provided with the logon request.
AMPS_REMOTE_ADDRESS	Remote address from which the logon request was made.
AMPS_USER_NAME	User name for the request.
CORRELATION_ID	The correlation ID provided on the logon request. Legacy compatibility token
USER_NAME	User name for the request. Legacy compatibility token

For Entitlement blocks, the module requires one of the following two options. These options are used to specify the entitlements to use for the context.

Option	Description
ResourceURI	Identifier for the credential store to use for this entitlement context. This is a synonym for the CredentialStore parameter. The module accepts this synonym to make it easier to verify that the Entitlement context and the Authentication context use the same values. There is no default for this parameter. Either the ResourceURI or CredentialStore must be provided. If both are provided, the module uses the value of the CredentialStore.
CredentialStore	Identifier for the entitlement cache. When used for entitlement, this is the name of the entitlement cache that AMPS uses to look up the information. There is no default for this parameter. Either the ResourceURI or CredentialStore must be provided. If both are provided, the module uses the value of the CredentialStore.

For example, the following configuration loads the module and sets default values that are used for client transports. The module is also used for the admin interface, but that interface uses a separate credential store. Notice that, since the HTTPHeader options are set when the module is loaded, the custom headers are provided everywhere the module is used, regardless of the ResourceURI or CredentialStore values.

<AMPSConfig>
    <Modules>
        ...

        <Module>
            <Name>web-entitlements</Name>
            <Library>libamps_http_entitlement.so</Library>

            <!-- Sets defaults for all uses of the module -->
            <Options>
                <ResourceURI>http://permissions-server:8080/{{USER_NAME}}.json</ResourceURI>
                <HTTPHeader>x-tracking-id: {{CORRELATION_ID}}</HTTPHeader>
                <HTTPHeader>x-origin: AMPS</HTTPHeader>
            </Options>
        </Module>

        ...
    </Modules>

    <!-- Use the web-entitlements module with the default values for all authentication and entitlement
        unless a transport or the admin interface explicitly sets different values. -->
    <Authentication>
        <Module>web-entitlements</Module>
    </Authentication>
    <Entitlement>
        <Module>web-entitlements</Module>
    </Entitlement>

    <Admin>
        <InetAddr>localhost:8085</InetAddr>
        <!-- enable Basic authentication; see the Monitoring chapter for other options -->
        <WWWAuthenticate>Basic realm="AMPS Admin"</WWWAuthenticate>
        <!-- Use the web-entitlements module, but set a different URI and credential store for the admin
            interface. -->
        <Authentication>
            <Module>web-entitlements</Module>
            <Options>
                <CredentialStore>AdminCreds</CredentialStore>
                <ResourceURI>http://permissions-server:8080/admin/{{USER_NAME}}.json</ResourceURI>
            </Options>
        </Authentication>
        <Entitlement>
            <Module>web-entitlements</Module>
            <Options>
                <ResourceURI>http://permissions-server:8080/admin/{{USER_NAME}}.json</ResourceURI>
                <CredentialStore>AdminCreds</CredentialStore>
            </Options>
        </Entitlement>
    </Admin>

    <!-- All of these transports use the Authentication and Entitlement set at the instance level. -->
    <Transports>
        <Transport>
            <Name>json-tcp</Name>
            <Type>tcp</Type>
            <InetAddr>9007</InetAddr>
            <MessageType>json</MessageType>
            <Protocol>amps</Protocol>
        </Transport>
        <Transport>
            <Name>any-tcp</Name>
            <Type>tcp</Type>
            <InetAddr>9090</InetAddr>
            <Protocol>amps</Protocol>
        </Transport>
    </Transports>
</AMPSConfig>

Using HTTPS for Authentication and Entitlement Requests

The Web Service Authentication and Entitlement Module optionally supports https entitlement requests. When the ResourceURI uses https as the scheme, the module will attempt to use https to connect to the web service.

By default, the module attempts to verify the identity of the remote web service, which requires a key file containing the key for the certificate authority that signed the certificate for the remote web service.

AMPS does not require that the certificate and key provided for outgoing https requests be the same certificates used for incoming SSL connections to AMPS. However, if you have configured AMPS to accept SSL connections from AMPS clients, the certificates you use for those connections are often suitable for outgoing web authentication module connections, and the same certificates can be provided in both sections of the configuration file.

Option	Description
Certificate	The certificate to use for the AMPS connection to the web service. When configured, this certificate can be provided to the web service if the web service requests client authentication for the connection. There is no default for this parameter.
Key	The key file to use for the AMPS connection to the web service. When configured, this key can be used for client authentication if the web service requests client authentication for the connection. There is no default for this parameter.
CAKey	The certificate authority key. When configured, this key can be used for the AMPS server to verify the identity of the web service. There is no default for this parameter. The CAKey must be provided when AllowUnverfiedPeer is set to false, the default.
AllowUnverifiedPeer	Specifies whether AMPS requires the web service to identify itself. When this is set to false, the default, AMPS requires that the web service provides a certificate that can be verified with the configured CAKey. Default: false
AllowSelfSigned	Specifies whether AMPS will accept self-signed certificates for https connections. Default: false

The following configuration file shows a common way to configure the module for testing purposes when working with a server that accepts https connections. While the outgoing connection will use SSL, the module will not provide certificates to the server, or request that the server verify its identity.

<Module>
    <Name>web-entitlements</Name>
    <Library>libamps_http_entitlement.so</Library>
    <Options>
        <ResourceURI>https://permissions-server:443/{{USER_NAME}}.json</ResourceURI>
        <AllowUnverifiedPeer>true</AllowUnverifiedPeer>
        <AllowSelfSigned>true</AllowSelfSigned>
    </Options>
</Module>

The following configuration file demonstrates how to configure the module to verify the identity of the remote server, provide verification of the AMPS server identity to that server, and require that all certificates be signed by a certificate authority.

<Module>
    <Name>web-entitlements</Name>
    <Library>libamps_http_entitlement.so</Library>
    <Options>
        <ResourceURI>https://permissions-server:443/{{USER_NAME}}.json</ResourceURI>
        <Certificate>/etc/security/amps-cert.pem</Certificate>
        <Key>/etc/security/amps-key.pem</Key>
        <CAKey>/etc/security/ca.pem</CAKey>
    </Options>
</Module>

Permissions Management and Request Flow

This section describes the request flow for the Web Service Authentication and Entitlement Module when used in the default mode.

Notice that authentication and entitlement are two separate steps. The module obtains the set of permissions during the authentication step, and then provides responses to AMPS entitlement requests during the entitlement step. What this means is that, in the default mode, a user must have authenticated using the module for an entitlement request to be allowed.

Authentication Step

1. Logon request received from the client.

2. Module requests an entitlement document (via GET) from a specified Web Service. The credentials in the logon request are provided as the credentials for the GET. The module supports both Basic and Digest authentication to the Web Service, as described in RFC 7616 (HTTP digest access authentication) and RFC 7617 (The 'Basic' HTTP authentication scheme).

3. If AMPS cannot retrieve the entitlement document using the provided credentials, AMPS returns a failure for the logon request.

4. If the Web Service Authentication and Entitlement module already has a parsed entitlement document for this user in the CredentialStore for this request, the module simply returns success for the authentication request.

5. Otherwise, the module parses the entitlement document and stores the entitlements in the CredentialStore. If the module can't successfully parse the document, it returns a failure for the authentication request.

Entitlement Step

1. The module looks up the user name in the CredentialStore for the request. If the module has no stored entitlements for the user, the module denies the request.

2. The module looks for a matching entitlement. Entitlements for a user are searched in exactly the same order in which they appear in the document. The module uses the first entitlement that matches the request. If no entitlements match, the module denies the request.

3. The module checks the entitlement to see if the entitlement grants access to the user or disallows access to the user. If the entitlement disallows access, the module denies the request.

4. The module allows the request and applies any filter specified in the matching entitlement.

Entitlement Reset

The module caches the set of entitlements for a user while that user is connected. As described in the previous section, the module parses the returned permissions document if the module does not have an existing set of entitlements for that user or if the EntitlementTimeout has expired and the document has changed.

The module provides two mechanisms for automatically resetting the entitlement cache and updating permissions for the user:

1. The module provides "reset cache on disconnect" entitlement behavior when in authentication and entitlement mode. The module resets both the AMPS entitlement cache and the information on the parsed permissions document for a given user when all of the connections for that user are closed. In practical terms, this means that changing the entitlement document returned by the web service has no immediate effect on the permission set that AMPS enforces. AMPS will continue to use the permissions in the document returned from the first logon request for that user until all connections for that user have been closed. When used in authentication and entitlement mode, this functionality is always active; when the last connection for a given user disconnects, the AMPS entitlement cache and the module entitlements are reset.

   Notice that this also means that if AMPS entitlements are reset for a user (for example, using the amps-action-do-reset-entitlement action), this will also clear the module cache, since all connections for that user will be closed. Likewise, a user can always update permissions by completely logging out of AMPS and then logging back in.

2. Optionally, the module can set an EntitlementTimeout. When this is set, a change to the user permissions after the timeout period will reset any current connections for that user, clear the AMPS entitlement cache, and replace existing permissions with the contents of the permissions document returned. Notice that AMPS only has access to the entitlements document when a user is authenticated. AMPS does not retain the logon credentials for a user, and only retrieves documents from the web service during logon. This means that a change in entitlements can only be checked during the logon process for the same user name.

This functionality is not available when the module is in EntitlementOnly mode, as described below. When used in EntitlementOnly mode, the module does not track connection or disconnection, so entitlements are cached in the module for the lifetime of the instance.

Entitlement Only Mode

Starting in AMPS 5.3.1, the HTTP authentication and entitlements module supports a mode where another module can be used for authentication, and this module can be used for entitlements. To enable this mode, include the following configuration item in the Modules block that loads the module:

<Options>
  <!-- other options here -->
  <EntitlementOnly/>
</Options>

When an EntitlementOnly element is provided, the module cannot be used to authenticate users. Instead, the module makes an unauthenticated HTTP request the first time that permissions are requested for a given user ID. The permissions document returned is cached, and used for subsequent requests.

When using the module in this mode, the module may not be used in an Authentication block. The module configuration or Entitlement block configuration must provide a ResourceURI, and may provide the other options normally accepted in the Authentication block (for example, HTTPHeader, RetryCount, and so on).

In this mode, the module does not see connect and logon requests. Therefore, the module does not provide automatic entitlement cache reset in this mode and the module caches the permissions for a given user in a given credential store for the lifetime of the instance.

Entitlement Only Request Flow

This section describes the request flow when the module is operating in entitlement only mode. This is very similar to the flow in the default mode, except that all of the processing happens as a part of the first entitlement request for a given user ID, and no credential information is available for the request to the HTTP server.

1. Entitlement request is received from AMPS.

2. The module looks up the user name in the CredentialStore for the request. If the module has stored entitlements for the user, the module proceeds to step 6.

3. Module requests an entitlement document (via GET) from a specified Web Service. No credentials are provided on this request.

4. If the module cannot retrieve the entitlement document, the module returns a failure for the entitlement request.

5. The module parses the returned permissions document. If an error occurs while parsing the permissions document, the module returns a failure for the entitlement request. Otherwise, the module loads the parsed permissions into the CredentialStore.

6. The module uses the CredentialStore to look for a matching entitlement. Entitlements for a user are searched in exactly the same order in which they appear in the document. The module uses the first entitlement that matches the request. If no entitlements match, the module denies the request.

7. The module checks the entitlement to see if the entitlement grants access to the user or disallows access to the user. If the entitlement disallows access, the module denies the request.

8. If the module allows the request, any filters specified in the matching entitlement are applied.