Package org.opensaml.storage.impl.memcached

# Class MemcachedStorageService

java.lang.Object
net.shibboleth.shared.component.AbstractInitializableComponent
net.shibboleth.shared.component.AbstractIdentifiedInitializableComponent
net.shibboleth.shared.component.AbstractIdentifiableInitializableComponent
org.opensaml.storage.impl.memcached.MemcachedStorageService

All Implemented Interfaces:
  `Component`, `DestructableComponent`, `IdentifiableComponent`, `IdentifiedComponent`, `InitializableComponent`, `StorageService`

public class MemcachedStorageService extends AbstractIdentifiableInitializableComponent implements StorageService
Memcached storage service. The implementation of context names is based on the implementation of simulated namespaces discussed on the Memcached project site:

https://code.google.com/p/memcached/wiki/NewProgrammingTricks#Namespacing

This storage service supports arbitrary-length context names and keys despite the 250-byte limit on memcached keys. Keys whose length is greater than 250 bytes are hashed using the SHA-512 algorithm and hex encoded to produce a 128-character key that is stored in memcached. Collisions are avoided irrespective of hashing by using the memcached add operation on all create operations which guarantees that an entry is created if and only if a key of the same value does not already exist. Note that context names and keys are assumed to have single-byte encodings in UTF-8 (i.e. ASCII characters) such that key lengths are equal to their size in bytes. Hashed keys naturally meet this requirement.

An optional context key-tracking feature is available to support `updateContextExpiration(String, Long)`. Key tracking is disabled by default, but can be enabled by setting the `enableContextKeyTracking` parameter in the `MemcachedStorageService(net.spy.memcached.MemcachedClient, int, boolean)` constructor. While there is modest performance impact for create and delete operations, the feature limits the number of keys per context. With the default 1M memcached slab size, in the worst case 4180 keys are permitted per context. In many if not most situations the value is easily double that. The limitation can be overcome by increasing the slab size, which decreases overall cache memory consumption efficiency. When key tracking is disabled, there is no limit on the number of keys per context other than overall cache capacity.

Limitations and requirements

1. The memcached binary protocol is strong recommended for efficiency and full versioning support. In particular, `deleteWithVersion(long, String, String)` and `deleteWithVersion(long, Object)` will throw runtime errors under the ASCII protocol.
2. Memcached server 1.4.14 or later MUST be used with binary protocol for proper handling of cache entry expiration values. See the 1.4.14 release notes for details.

## Field Summary

Fields
Modifier and Type
Field
Description
`protected static final String`
`CTX_KEY_DELETED_SUFFIX`
Key suffix for entry that contains a list of deleted context keys.
`private static final String`
`CTX_KEY_LIST_DELIMITER`
Delimiter of items in the context key list.
`protected static final String`
`CTX_KEY_LIST_SUFFIX`
Key suffix for entry that contains a list of context keys.
`private final org.slf4j.Logger`
`logger`
Logger instance.
`private static final int`
`MAX_KEY_LENGTH`
Maximum length in bytes of memcached keys.
`private final net.spy.memcached.MemcachedClient`
`memcacheClient`
Memcached client instance.
`private int`
`operationTimeout`
Memcached asynchronous operation timeout in seconds.
`private MemcachedStorageCapabilities`
`storageCapabilities`
Invariant storage capabilities.
`private final net.spy.memcached.transcoders.Transcoder<MemcachedStorageRecord<?>>`
`storageRecordTranscoder`
Handles conversion of `MemcachedStorageRecord` to bytes and vice versa.
`private final net.spy.memcached.transcoders.Transcoder<String>`
`stringTranscoder`
Handles conversion of strings to bytes and vice versa.
`private boolean`
`trackContextKeys`
Flag that controls context key tracking.

## Constructor Summary

Constructors
Constructor
Description
`MemcachedStorageService(net.spy.memcached.MemcachedClient client, int timeout)`
Creates a new instance.
`MemcachedStorageService(net.spy.memcached.MemcachedClient client, int timeout, boolean enableContextKeyTracking)`
Creates a new instance with optional context key tracking.

## Method Summary

Modifier and Type
Method
Description
`boolean`
`create(Object value)`
Creates a new record in the store using an annotated object as the source.
`boolean`
`create(String context, String key, String value, Long expiration)`
Creates a new record in the store with an expiration.
`<T> boolean`
`create(String context, String key, T value, StorageSerializer<T> serializer, Long expiration)`
Creates a new record in the store with an expiration, using a custom serialization process for an arbitrary object.
`protected String`
`createNamespace(String context)`
Creates a cache-wide unique namespace for the given context name.
`boolean`
`delete(Object value)`
Deletes an existing record from the store, using an annotated object as the source.
`boolean`
`delete(String context, String key)`
Deletes an existing record from the store.
`void`
`deleteContext(String context)`
Forcibly removes all records in a given context along with any associated resources devoted to maintaining the context.
`boolean`
`deleteWithVersion(long version, Object value)`
Deletes an existing record from the store, using an annotated object as the source, if it currently has a specified version.
`boolean`
`deleteWithVersion(long version, String context, String key)`
Deletes an existing record from the store if it currently has a specified version.
`protected void`
`doDestroy()`
`StorageCapabilities`
`getCapabilities()`
Returns the capabilities of the underlying store.
`private <T> T`
`handleAsyncResult(net.spy.memcached.internal.OperationFuture<T> result)`
Handle async result.
`protected String`
`lookupNamespace(String context)`
Looks up the namespace for the given context name in the cache.
`private String`
`memcachedKey(String... parts)`
Creates a memcached key from one or more parts.
`Object`
`read(Object value)`
Returns an existing record from the store, if one exists, and uses it to update the annotated fields of a target object.
`<T> StorageRecord<T>`
`read(String context, String key)`
Returns an existing record from the store, if one exists.
`<T> Pair<Long,StorageRecord<T>>`
`read(String context, String key, long version)`
Returns an existing record from the store, along with its version.
`void`
`reap(String context)`
Manually trigger a cleanup of expired records.
`void`
`setCapabilities(MemcachedStorageCapabilities capabilities)`
Sets the storage capabilities.
`boolean`
`update(Object value)`
Updates an existing record in the store, using an annotated object as the source.
`boolean`
`update(String context, String key, String value, Long expiration)`
Updates an existing record in the store.
`<T> boolean`
`update(String context, String key, T value, StorageSerializer<T> serializer, Long expiration)`
Updates an existing record in the store using a custom serialization strategy.
`void`
`updateContextExpiration(String context, Long expiration)`
Updates the expiration time of all records in the context.
`private boolean`
`updateContextKeyList(String suffix, String namespace, String key)`
Update context key list.
`boolean`
`updateExpiration(Object value)`
Updates expiration of an existing record in the store, using an annotated object as the source.
`boolean`
`updateExpiration(String context, String key, Long expiration)`
Updates expiration of an existing record in the store.
`Long`
`updateWithVersion(long version, Object value)`
Updates an existing record in the store, if a version matches, using an annotated object as the source.
`Long`
`updateWithVersion(long version, String context, String key, String value, Long expiration)`
Updates an existing record in the store, if a version matches.
`<T> Long`
`updateWithVersion(long version, String context, String key, T value, StorageSerializer<T> serializer, Long expiration)`
Updates an existing record in the store, if a version matches, using a custom serialization strategy.

### Methods inherited from class net.shibboleth.shared.component.AbstractIdentifiableInitializableComponent

`setId`

### Methods inherited from class net.shibboleth.shared.component.AbstractIdentifiedInitializableComponent

`doInitialize, ensureId, getId, ifDestroyedThrowDestroyedComponentException, ifInitializedThrowUnmodifiabledComponentException, ifNotInitializedThrowUninitializedComponentException`

### Methods inherited from class net.shibboleth.shared.component.AbstractInitializableComponent

`checkComponentActive, checkSetterPreconditions, destroy, initialize, isDestroyed, isInitialized`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Methods inherited from interface net.shibboleth.shared.component.IdentifiedComponent

`getId`

## Field Details

### CTX_KEY_LIST_SUFFIX

@Nonnull @NotEmpty protected static final String CTX_KEY_LIST_SUFFIX
Key suffix for entry that contains a list of context keys.

See Also:
  - Constant Field Values

### CTX_KEY_DELETED_SUFFIX

@Nonnull @NotEmpty protected static final String CTX_KEY_DELETED_SUFFIX
Key suffix for entry that contains a list of deleted context keys.

See Also:
  - Constant Field Values

### CTX_KEY_LIST_DELIMITER

@Nonnull @NotEmpty private static final String CTX_KEY_LIST_DELIMITER
Delimiter of items in the context key list.

See Also:
  - Constant Field Values

### MAX_KEY_LENGTH

private static final int MAX_KEY_LENGTH
Maximum length in bytes of memcached keys.

See Also:
  - Constant Field Values

### logger

@Nonnull private final org.slf4j.Logger logger
Logger instance.

### storageRecordTranscoder

@Nonnull private final net.spy.memcached.transcoders.Transcoder<MemcachedStorageRecord<?>> storageRecordTranscoder
Handles conversion of `MemcachedStorageRecord` to bytes and vice versa.

### stringTranscoder

@Nonnull private final net.spy.memcached.transcoders.Transcoder<String> stringTranscoder
Handles conversion of strings to bytes and vice versa.

### storageCapabilities

@Nonnull private MemcachedStorageCapabilities storageCapabilities
Invariant storage capabilities.

### memcacheClient

@Nonnull private final net.spy.memcached.MemcachedClient memcacheClient
Memcached client instance.

### operationTimeout

@Positive private int operationTimeout
Memcached asynchronous operation timeout in seconds.

### trackContextKeys

private boolean trackContextKeys
Flag that controls context key tracking.

## Constructor Details

### MemcachedStorageService

public MemcachedStorageService(@Nonnull net.spy.memcached.MemcachedClient client, @Positive int timeout)
Creates a new instance.

Parameters:
  `client` - Memcached client object. The client MUST be configured to use the binary memcached protocol, i.e. `BinaryConnectionFactory`, in order for `deleteWithVersion(long, String, String)` and `deleteWithVersion(long, Object)` to work correctly. The binary protocol is recommended for efficiency as well.
  `timeout` - Memcached operation timeout in seconds.

### MemcachedStorageService

public MemcachedStorageService(@Nonnull net.spy.memcached.MemcachedClient client, @Positive int timeout, boolean enableContextKeyTracking)
Creates a new instance with optional context key tracking.

Parameters:
  `client` - Memcached client object. The client MUST be configured to use the binary memcached protocol, i.e. `BinaryConnectionFactory`, in order for `deleteWithVersion(long, String, String)` and `deleteWithVersion(long, Object)` to work correctly. The binary protocol is recommended for efficiency as well.
  `timeout` - Memcached operation timeout in seconds.
  `enableContextKeyTracking` - True to enable context key tracking, false otherwise. NOTE this flag must be set to `true` in order for `updateContextExpiration(String, Long)` to work. If that capability is not needed, the flag should be set to `false` for better performance. The feature is disabled by default.

## Method Details

### getCapabilities

@Nonnull public StorageCapabilities getCapabilities()
Returns the capabilities of the underlying store.

Specified by:
  `getCapabilities` in interface `StorageService`
Returns:
  interface to access the service's capabilities

### setCapabilities

public void setCapabilities(@Nonnull MemcachedStorageCapabilities capabilities)
Sets the storage capabilities. This method should be used when the default 1M slab size is changed; the `MemcachedStorageCapabilities.getValueSize()` should be set equal to the chosen slab size.

Parameters:
  `capabilities` - Memcached storage capabilities.

### create

public boolean create(@Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key, @Nonnull @NotEmpty String value, @Nullable @Positive Long expiration) throws IOException
Creates a new record in the store with an expiration.

Specified by:
  `create` in interface `StorageService`
Parameters:
  `context` - a storage context label
  `key` - a key unique to context
  `value` - value to store
  `expiration` - expiration for record, or null
Returns:
  true iff record was inserted, false iff a duplicate was found
Throws:
  `IOException` - if fatal errors occur in the insertion process

### create

public <T> boolean create(@Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key, @Nonnull T value, @Nonnull StorageSerializer<T> serializer, @Nullable @Positive Long expiration) throws IOException
Creates a new record in the store with an expiration, using a custom serialization process for an arbitrary object.

Specified by:
  `create` in interface `StorageService`
Type Parameters:
  `T` - type of record
Parameters:
  `context` - a storage context label
  `key` - a key unique to context
  `value` - object to store
  `serializer` - custom serializer for the object
  `expiration` - expiration for record, or null
Returns:
  true iff record was inserted, false iff a duplicate was found
Throws:
  `IOException` - if fatal errors occur in the insertion process

### create

public boolean create(@Nonnull Object value) throws IOException
Creates a new record in the store using an annotated object as the source.

The individual parameters for the creation are extracted from the object using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.

Specified by:
  `create` in interface `StorageService`
Parameters:
  `value` - object to store
Returns:
  true iff record was inserted, false iff a duplicate was found
Throws:
  `IOException` - if fatal errors occur in the insertion process

### read

public <T> StorageRecord<T> read(@Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key) throws IOException
Returns an existing record from the store, if one exists.

Specified by:
  `read` in interface `StorageService`
Type Parameters:
  `T` - type of record
Parameters:
  `context` - a storage context label
  `key` - a key unique to context
Returns:
  the record read back, if present, or null
Throws:
  `IOException` - if errors occur in the read process

### read

public Object read(@Nonnull Object value) throws IOException
Returns an existing record from the store, if one exists, and uses it to update the annotated fields of a target object.

The context and key to look up are obtained from the target object, and the value and expiration are written back, using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.

Specified by:
  `read` in interface `StorageService`
Parameters:
  `value` - object to look up and populate
Returns:
  the updated object passed into the method, or null if no record was found
Throws:
  `IOException` - if errors occur in the read process

### read

@Nonnull public <T> Pair<Long,StorageRecord<T>> read(@Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key, @Positive long version) throws IOException
Returns an existing record from the store, along with its version.

The first member of the pair returned will contain the version of the record in the store, or will be null if no record exists. The second member will contain the record read back. If null, the record either didn't exist (if the first member was also null) or the record was the same version as that supplied by the caller.

Specified by:
  `read` in interface `StorageService`
Type Parameters:
  `T` - type of record
Parameters:
  `context` - a storage context label
  `key` - a key unique to context
  `version` - only return record if newer than supplied version
Returns:
  a pair consisting of the version of the record read back, if any, and the record itself
Throws:
  `IOException` - if errors occur in the read process

### update

public boolean update(@Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key, @Nonnull @NotEmpty String value, @Nullable @Positive Long expiration) throws IOException
Updates an existing record in the store.

Specified by:
  `update` in interface `StorageService`
Parameters:
  `context` - a storage context label
  `key` - a key unique to context
  `value` - updated value
  `expiration` - expiration for record, or null
Returns:
  true if the update succeeded, false if the record does not exist
Throws:
  `IOException` - if errors occur in the update process

### update

public <T> boolean update(@Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key, @Nonnull T value, @Nonnull StorageSerializer<T> serializer, @Nullable @Positive Long expiration) throws IOException
Updates an existing record in the store using a custom serialization strategy.

Specified by:
  `update` in interface `StorageService`
Type Parameters:
  `T` - type of record
Parameters:
  `context` - a storage context label
  `key` - a key unique to context
  `value` - updated value
  `serializer` - custom serializer
  `expiration` - expiration for record, or null
Returns:
  true if the update succeeded, false if the record does not exist
Throws:
  `IOException` - if errors occur in the update process

### update

public boolean update(@Nonnull Object value) throws IOException
Updates an existing record in the store, using an annotated object as the source.

The individual parameters for the update are extracted from the object using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.

Specified by:
  `update` in interface `StorageService`
Parameters:
  `value` - object to update from
Returns:
  true if the update succeeded, false if the record does not exist
Throws:
  `IOException` - if errors occur in the update process

### updateWithVersion

public Long updateWithVersion(@Positive long version, @Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key, @Nonnull @NotEmpty String value, @Nullable @Positive Long expiration) throws IOException, VersionMismatchException
Updates an existing record in the store, if a version matches.

Specified by:
  `updateWithVersion` in interface `StorageService`
Parameters:
  `version` - only update if the current version matches this value
  `context` - a storage context label
  `key` - a key unique to context
  `value` - updated value
  `expiration` - expiration for record, or null
Returns:
  the version of the record after update, null if no record exists
Throws:
  `IOException` - if errors occur in the update process
  `VersionMismatchException` - if the record has already been updated to a newer version

### updateWithVersion

@Nullable public <T> Long updateWithVersion(@Positive long version, @Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key, @Nonnull T value, @Nonnull StorageSerializer<T> serializer, @Nullable @Positive Long expiration) throws IOException, VersionMismatchException
Updates an existing record in the store, if a version matches, using a custom serialization strategy.

Specified by:
  `updateWithVersion` in interface `StorageService`
Type Parameters:
  `T` - type of record
Parameters:
  `version` - only update if the current version matches this value
  `context` - a storage context label
  `key` - a key unique to context
  `value` - updated value
  `serializer` - custom serializer
  `expiration` - expiration for record, or null
Returns:
  the version of the record after update, null if no record exists
Throws:
  `IOException` - if errors occur in the update process
  `VersionMismatchException` - if the record has already been updated to a newer version

### updateWithVersion

@Nullable public Long updateWithVersion(@Positive long version, @Nonnull Object value) throws IOException, VersionMismatchException
Updates an existing record in the store, if a version matches, using an annotated object as the source.

The individual parameters for the update are extracted from the object using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.

Specified by:
  `updateWithVersion` in interface `StorageService`
Parameters:
  `version` - only update if the current version matches this value
  `value` - object to update from
Returns:
  the version of the record after update, null if no record exists
Throws:
  `IOException` - if errors occur in the update process
  `VersionMismatchException` - if the record has already been updated to a newer version

### updateExpiration

public boolean updateExpiration(@Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key, @Nullable @Positive Long expiration) throws IOException
Updates expiration of an existing record in the store.

Specified by:
  `updateExpiration` in interface `StorageService`
Parameters:
  `context` - a storage context label
  `key` - a key unique to context
  `expiration` - expiration for record, or null
Returns:
  true if the update succeeded, false if the record does not exist
Throws:
  `IOException` - if errors occur in the update process

### updateExpiration

public boolean updateExpiration(@Nonnull Object value) throws IOException
Updates expiration of an existing record in the store, using an annotated object as the source.

The individual parameters for the update are extracted from the object using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.

Specified by:
  `updateExpiration` in interface `StorageService`
Parameters:
  `value` - object to update from
Returns:
  true if the update succeeded, false if the record does not exist
Throws:
  `IOException` - if errors occur in the update process

### delete

public boolean delete(@Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key) throws IOException
Deletes an existing record from the store.

Specified by:
  `delete` in interface `StorageService`
Parameters:
  `context` - a storage context label
  `key` - a key unique to context
Returns:
  true iff the record existed and was deleted
Throws:
  `IOException` - if errors occur in the deletion process

### delete

public boolean delete(@Nonnull Object value) throws IOException
Deletes an existing record from the store, using an annotated object as the source.

The individual parameters for the deletion are extracted from the object using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.

Specified by:
  `delete` in interface `StorageService`
Parameters:
  `value` - object to delete
Returns:
  true iff the record existed and was deleted
Throws:
  `IOException` - if errors occur in the deletion process

### deleteWithVersion

public boolean deleteWithVersion(@Positive long version, @Nonnull @NotEmpty String context, @Nonnull @NotEmpty String key) throws IOException, VersionMismatchException
Deletes an existing record from the store if it currently has a specified version.

Specified by:
  `deleteWithVersion` in interface `StorageService`
Parameters:
  `version` - record version to delete
  `context` - a storage context label
  `key` - a key unique to context
Returns:
  true iff the record existed and was deleted
Throws:
  `IOException` - if errors occur in the deletion process
  `VersionMismatchException` - if the record has already been updated to a newer version

### deleteWithVersion

public boolean deleteWithVersion(@Positive long version, @Nonnull Object value) throws IOException, VersionMismatchException
Deletes an existing record from the store, using an annotated object as the source, if it currently has a specified version.

The individual parameters for the deletion are extracted from the object using the annotations in the org.opensaml.storage.annotation package. If any are missing, or a field inaccessible, a runtime exception of some kind will occur.

Specified by:
  `deleteWithVersion` in interface `StorageService`
Parameters:
  `version` - record version to delete
  `value` - object to delete
Returns:
  true iff the record existed and was deleted
Throws:
  `IOException` - if errors occur in the deletion process
  `VersionMismatchException` - if the record has already been updated to a newer version

### reap

public void reap(@Nonnull @NotEmpty String context) throws IOException
Manually trigger a cleanup of expired records. The method MAY return without guaranteeing that cleanup has already occurred.

Specified by:
  `reap` in interface `StorageService`
Parameters:
  `context` - a storage context label
Throws:
  `IOException` - if errors occur in the cleanup process

### updateContextExpiration

public void updateContextExpiration(@Nonnull @NotEmpty String context, @Nullable Long expiration) throws IOException
Updates the expiration time of all records in the context.

Specified by:
  `updateContextExpiration` in interface `StorageService`
Parameters:
  `context` - a storage context label
  `expiration` - a new expiration timestamp, or null
Throws:
  `IOException` - if errors occur in the cleanup process

### deleteContext

public void deleteContext(@Nonnull @NotEmpty String context) throws IOException
Forcibly removes all records in a given context along with any associated resources devoted to maintaining the context.

Specified by:
  `deleteContext` in interface `StorageService`
Parameters:
  `context` - a storage context label
Throws:
  `IOException` - if errors occur in the cleanup process

### doDestroy

protected void doDestroy()

Overrides:
  `doDestroy` in class `AbstractInitializableComponent`

### lookupNamespace

@Nullable protected String lookupNamespace(@Nonnull @NotEmpty String context) throws IOException
Looks up the namespace for the given context name in the cache.

Parameters:
  `context` - Context name.
Returns:
  Corresponding namespace for given context or null if no namespace exists for context.
Throws:
  `IOException` - On memcached operation errors.

### createNamespace

@Nonnull protected String createNamespace(@Nonnull @NotEmpty String context) throws IOException
Creates a cache-wide unique namespace for the given context name. The context-namespace mapping is stored in the cache.

Parameters:
  `context` - Context name.
Returns:
  Namespace name for given context.
Throws:
  `IOException` - On memcached operation errors.

### memcachedKey

private String memcachedKey(@Nonnull String... parts)
Creates a memcached key from one or more parts.

Parameters:
  `parts` - Key parts (i.e. namespace, local name)
Returns:
  Key comprised of 250 characters or less.

### handleAsyncResult

private <T> T handleAsyncResult(@Nonnull net.spy.memcached.internal.OperationFuture<T> result) throws IOException
Handle async result.

Type Parameters:
  `T` - type of result
Parameters:
  `result` - the result
Returns:
  the result
Throws:
  `IOException` - if an error occurs

### updateContextKeyList

private boolean updateContextKeyList(@Nonnull String suffix, @Nonnull String namespace, @Nonnull String key) throws IOException
Update context key list.

Parameters:
  `suffix` - the suffix
  `namespace` - the namespace
  `key` - the storage key
Returns:
  whether the update was a success
Throws:
  `IOException` - if an error occurs