Data Types¶

Traditionally all data stored in Riak was an opaque binary type. Then in version 1.4 came the introduction of a counter, the first Convergent Data Type supported in Riak. In Riak 2.0, several additional Data Types were introduced. Riak “knows” about these data types, and conflicting writes to them will converge automatically without presenting sibling values to the user.

Here is the list of current Data Types:

Counter increments or decrements integer values

Set allows you to store multiple distinct opaque binary values against a key

Map is a nested, recursive struct, or associative array. Think of it as a container for composing ad hoc data structures from multiple Data Types. Inside a map you may store sets, counters, flags, registers, and even other maps

Register stores binaries accoring to last-write-wins logic within Map

Flag is similar to a boolean and also must be within Map

All Data Types must be stored in buckets bearing a BucketType that sets the datatype property to one of "counter", "set", or "map". Note that the bucket must have the allow_mult property set to true.

These Data Types are stored just like RiakObjects, so size constraints that apply to normal Riak values apply to Riak Data Types too.

An in-depth discussion of Data Types, also known as CRDTs, can be found at Data Types.

Examples of using Data Types can be found at Using Data Types.

Sending Operations¶

Riak Data Types provide a further departure from Riak’s usual operation, in that the API is operation-based. Rather than fetching the data structure, reconciling conflicts, mutating the result, and writing it back, you instead tell Riak what operations to perform on the Data Type. Here are some example operations:

increment a Counter by 10

add 'joe' to a Set

remove the Set field called 'friends' from a Map

enable the prepay Flag in a Map

Datatypes can be fetched and created just like RiakObject instances, using RiakBucket.get and RiakBucket.new, except that the bucket must belong to a bucket-type that has a valid datatype property. If we have a bucket-type named “social-graph” that has the datatype “set”, we would fetch a Set like so:

graph = client.bucket_type('social-graph')
graph.datatype  # => 'set'
myfollowers = graph.bucket('followers').get('seancribbs')
# => a Set datatype

Once we have a datatype, we can stage operations against it and then send those operations to Riak:

myfollowers.add('javajolt')
myfollowers.discard('roach')
myfollowers.update()

While this looks in code very similar to manipulating RiakObject instances, only mutations are enqueued locally, not the new value.

Context and Observed-Remove¶

In order for Riak Data Types to behave well, you must have an opaque context received from a read when you:

disable a Flag (set it to false)

remove a field from a Map

remove an element from a Set

The basic rule is “you cannot remove something you haven’t seen”, and the context tells Riak what you’ve actually seen, similar to the Vector clock on RiakObject. The Python client handles opaque contexts for you transparently as long as you fetch before performing one of these actions.

Datatype abstract class¶

class riak.datatypes.Datatype(bucket=None, key=None, value=None, context=None)¶

Base class for all convergent datatype wrappers. You will not use this class directly, but it does define some methods are common to all datatype wrappers.

value¶

The pure, immutable value of this datatype, as a Python value, which is unique for each datatype.

NB: Do not use this property to mutate data, as it will not have any effect. Use the methods of the individual type to affect changes. This value is guaranteed to be independent of any internal data representation.

context¶

The opaque context for this type, if it was previously fetched.

Return type:	str

modified¶

Whether this datatype has staged local modifications.

Return type:	bool

Persistence methods¶

Datatype.reload(**params)¶

Reloads the datatype from Riak.

Parameters:

r (integer, string, None) – the read quorum
pr (integer, string, None) – the primary read quorum
basic_quorum (bool) – whether to use the “basic quorum” policy for not-founds
notfound_ok (bool) – whether to treat not-found responses as successful
timeout (int) – a timeout value in milliseconds
include_context (bool) – whether to return the opaque context as well as the value, which is useful for removal operations on sets and maps

Return type:

Datatype

Datatype.update(**params)¶

Sends locally staged mutations to Riak.

Parameters:

w (integer) – W-value, wait for this many partitions to respond before returning to client.
dw (integer) – DW-value, wait for this many partitions to confirm the write before returning to client.
pw (integer) – PW-value, require this many primary partitions to be available before performing the put
return_body (bool) – if the newly stored object should be retrieved, defaults to True
include_context (bool) – whether to return the new opaque context when return_body is True
timeout (int) – a timeout value in milliseconds

Return type:

a subclass of Datatype

Datatype.store(**params)¶: This is an alias for update().

Datatype.delete(**params)¶: Deletes the datatype from Riak. See RiakClient.delete() for options.

Datatype.clear()¶: Removes all locally staged mutations.

Counter¶

class riak.datatypes.Counter(bucket=None, key=None, value=None, context=None)¶: A convergent datatype that represents a counter which can be incremented or decremented. This type can stand on its own or be embedded within a Map.

Counter.value¶

The current value of the counter.

Return type:	int

Counter.increment(amount=1)¶

Increments the counter by one or the given amount.

Parameters:	amount (int) – the amount to increment the counter

Counter.decrement(amount=1)¶

Decrements the counter by one or the given amount.

Parameters:	amount (int) – the amount to decrement the counter

Set¶

class riak.datatypes.Set(bucket=None, key=None, value=None, context=None)¶

A convergent datatype representing a Set with observed-remove semantics. Currently strings are the only supported value type. Example:

myset.add('barista')
myset.add('roaster')
myset.add('brewer')

Likewise they can simply be removed:

myset.discard('barista')

This datatype also implements the Set ABC, meaning it supports len(), in, and iteration.

Set.value¶

An immutable copy of the current value of the set.

Return type:	frozenset

Set.add(element)¶

Adds an element to the set.

Parameters:	element (str) – the element to add

Set.discard(element)¶

Removes an element from the set.

Parameters:	element (str) – the element to remove

Map¶

class riak.datatypes.Map(bucket=None, key=None, value=None, context=None)¶

A convergent datatype that acts as a key-value datastructure. Keys are pairs of (name, datatype) where name is a string and datatype is the datatype name. Values are other convergent datatypes, represented by any concrete type in this module.

You cannot set values in the map directly (it does not implement __setitem__), but you may add new empty values or access non-existing values directly via bracket syntax. If a key is not in the original value of the map when accessed, fetching the key will cause its associated value to be created.:

map[('name', 'register')]

Keys and their associated values may be deleted from the map as you would in a dict:

del map[('emails', 'set')]

Convenience accessors exist that partition the map’s keys by datatype and implement the Mapping behavior as well as supporting deletion:

map.sets['emails']
map.registers['name']
del map.counters['likes']

Map.value¶

Returns a copy of the original map’s value. Nested values are pure Python values as returned by Datatype.value from the nested types.

Return type:	dict

Map.counters¶

Filters keys in the map to only those of counter types. Example:

map.counters['views'].increment()
del map.counters['points']

Map.flags¶

Filters keys in the map to only those of flag types. Example:

map.flags['confirmed'].enable()
del map.flags['attending']

Map.maps¶

Filters keys in the map to only those of map types. Example:

map.maps['emails'].registers['home'].set("user@example.com")
del map.maps['spam']

Map.registers¶

Filters keys in the map to only those of register types. Example:

map.registers['username'].set_value("riak-user")
del map.registers['access_key']

Map.sets¶

Filters keys in the map to only those of set types. Example:

map.sets['friends'].add("brett")
del map.sets['favorites']

Map-only datatypes¶

Two of the new Data Types may only be embedded in Map objects (in addition to Map itself):

Register¶

class riak.datatypes.Register(bucket=None, key=None, value=None, context=None)¶: A convergent datatype that represents an opaque string that is set with last-write-wins semantics, and may only be embedded in Map instances.

Register.value¶

Returns a copy of the original value of the register.

Return type:	str

Register.assign(new_value)¶

Assigns a new value to the register.

Parameters:	new_value (str) – the new value for the register

Flag¶

class riak.datatypes.Flag(bucket=None, key=None, value=None, context=None)¶: A convergent datatype that represents a boolean value that can be enabled or disabled, and may only be embedded in Map instances.

Flag.value¶

The current value of the flag.

Return type:	bool, None

Flag.enable()¶: Turns the flag on, effectively setting its value to True.

Flag.disable()¶: Turns the flag off, effectively setting its value to False.