NOSQL DATABASES

VISWANADH. MNV

1
 UNIT-3 Key-Value Databases
Introduction
● A key-value store is like a simple hash table.
● It is used when all access to the database is done through a primary key.
● Imagine a table with two columns: ID and NAME.
● The ID column is the key.
● The NAME column stores the value.
● In a relational database (RDBMS), the NAME column stores data as a String.
● You can provide an ID and a VALUE to store in the database.
● If the ID already exists, the value is updated.
● If the ID does not exist, a new entry is created.
● Here's how the terminology compares in Oracle and Riak.

Figure
3.1 What Is a Key-Value Store

● Definition and Basics

○ Simplest NoSQL data store that stores data in the form of key-value pairs
to use from an API perspective.
○ Operations: get a value by key, put a value for a key, delete a key.
○ Values are blobs; the data store doesn’t care about the content.
○ The application must understand the stored data.
● Performance and Scalability
○ Uses primary-key access, leading to great performance.
○ Easily scalable.
● Popular Key-Value Databases
○ Redis (also known as a Data Structure server)
○ Riak

2
 ○ Amazon DynamoDB
○ Berkeley DB
○ HamsterDB
○ Memcached and its variants
● Differences between Amazon DynamoDB, Redis and Riak

Amazon Redis Riak

DynamoDB

Type Key-Value Store Key-Value Store Key-Value Store

and Document
Store

Open Source No Yes Yes

Consistency Eventual Strong Eventual Eventual

Consistency Consistency Consistency

Data Key-Value Pairs Key-Value Pairs Key-Value Pairs

and Data
Structures(lists,
Hashes, strings,
sets and sorted
sets)

In Memory No Yes No
Capabilities

Popularity Popular with AWS Most Popular Less Popular

Installation AWS Cloud only Local and Cloud Local and Cloud

speed Fast Very Fast Fast

Redis Features

● Data Structure Support

○ Stores various data structures such as strings, lists, sets, hashes, and
sorted sets.
○ Flexible storage options enable diverse use cases and efficient data
manipulation.
● Advanced Operations

3
 ○ Supports operations like range queries, set differences, unions, and
intersections.
○ Example: You can perform set operations like finding common elements
between two sets.

SADD set1 "a" "b" "c"

SADD set2 "b" "c" "d"
SINTER set1 set2 # Output: "b" "c"

● Atomic Operations
○ Ensures data integrity by performing atomic operations, which means
operations are completed without interference from other operations.
○ Example: Incrementing a counter is done atomically to prevent race
conditions.
INCR counter # Increments counter by 1
● Pub/Sub Messaging
○ Redis supports publish/subscribe messaging, allowing applications to
communicate in real-time.
○ Example: A chat application where users can subscribe to channels and
receive messages instantly.
SUBSCRIBE channel1 # Subscribes to a channel
PUBLISH channel1 "Hello World" # Publishes a message to
the channel

● Persistence Options
○ Offers different levels of persistence to balance between performance
and data durability.
○ Options include snapshotting (RDB) and append-only file (AOF)
persistence.
SAVE # Manually trigger a snapshot
CONFIG SET appendonly yes # Enable AOF persistence

● Replication and High Availability

○ Redis supports master-slave replication to ensure high availability and
data redundancy.
○ Example: Setting up replicas to ensure data is not lost in case of a master
failure.
SLAVEOF master_host master_port # Configures the server
as a replica
● In-Memory Data Store
○ As an in-memory data store, Redis provides extremely fast read and write
operations.

4
 ○ Suitable for caching, real-time analytics, and other high-performance
applications.
● Expiration and Eviction
○ Allows setting expiration times for keys to manage memory efficiently.
○ Example: Setting a time-to-live (TTL) for session data.
SET session:12345 data EX 3600 # Expires in 1 hour

3.2 Key-Value Store Features

● Compared with RDBMS Key-Value Stores do not have all features of RDBMS and
to use them the application architecture should be adjusted to use key-value
stores effectively.
● Key Features to Consider
○ Consistency
○ Transactions
○ Query features
○ Data structure
○ Scaling

3.3 Consistency

● Consistency in Redis primarily revolves around ensuring that operations on data

maintain expected behavior and integrity, especially in distributed environments.
● Consistency in Redis is crucial for maintaining data integrity and ensuring
predictable behavior across distributed environments. Redis provides
mechanisms like optimistic writes, eventual consistency, conflict resolution
strategies, and transactional support to handle various consistency requirements
effectively. These features make Redis suitable for applications requiring both
high performance and data consistency guarantees.

1. Operations on Single Keys

● Redis ensures consistency for operations like GET, SET, and DELETE on individual
keys.
SET key1 "value1"
GET key1
DEL key1

5
 ● Example: Performing read and write operations on a single key guarantees
consistent results within a Redis instance.

2. Optimistic Writes
● Redis supports optimistic writes by allowing clients to perform conditional
operations.
SET key1 "value1"
WATCH key1
value = GET key1
MULTI
SET key1 "value2"
EXEC
● Example: Using WATCH, MULTI, and EXEC commands to perform optimistic
locking to ensure consistency in concurrent operations.

3. Eventual Consistency
● Redis employs an eventually consistent model across distributed nodes.
SET key1 "value1"
● Example: After a SET operation, eventual consistency ensures that the value of
`key1` will eventually propagate and be consistent across all nodes in a Redis
Cluster.

4. Handling Update Conflicts

● Redis handles update conflicts by allowing the latest write to win or returning
multiple values for resolution.
SET key1 "value1"
SET key1 "value2"
● Example: Depending on configuration, Redis can resolve conflicts by overwriting
older values or returning all versions (siblings) for the application to resolve.

5. Configurable Consistency Settings

● Redis allows configuration of consistency parameters like replication and
quorum settings.
CONFIG SET min-replicas-to-write 2

6
 ● Example: Setting `min-replicas-to-write` ensures that a write operation is
considered successful only after being replicated to a minimum number of
nodes, ensuring data consistency in distributed setups.

6. Transaction Consistency
● Redis transactions ensure atomicity of grouped operations.
MULTI
SET key1 "value1"
SET key2 "value2"
EXEC
● Example: Using MULTI and EXEC commands to execute a series of operations
atomically, maintaining consistency across multiple keys.

3.4 Transactions
Transactions in Redis allow multiple commands to be executed atomically, ensuring
consistency and integrity of data operations. Here are some key aspects of transactions
in Redis along with examples:

1. Basic Transaction Usage

● Transactions in Redis are initiated using the `MULTI` command, followed by
queuing up multiple commands, and finally executing them atomically using
`EXEC`.
MULTI
SET key1 "value1"
SET key2 "value2"
EXEC
● Example: Both `SET` commands are executed atomically as part of the
transaction. If any command fails, all changes are rolled back.

2. Transactional Integrity
● Redis ensures that all commands in a transaction either succeed or fail together,
maintaining data integrity.
MULTI
INCR counter
LPUSH mylist "value1"
EXEC

7
 ● Example: Incrementing a counter and pushing a value into a list are executed as
a single atomic operation. If any command fails (e.g., due to a key not existing),
the entire transaction fails and no changes are applied.
3. Watch and Optimistic Locking
● Redis supports optimistic locking with the `WATCH` command, allowing
transactions to be aborted if monitored keys are modified by other clients.
WATCH key1
GET key1
MULTI
SET key1 "new_value"
EXEC
● Example: Before starting a transaction, `key1` is watched. If its value changes
between `WATCH` and `EXEC`, the transaction is aborted to maintain data
consistency.

4. Handling Errors and Rollback

● Redis transactions handle errors gracefully, rolling back changes if any command
within the transaction fails or encounters an error.
MULTI
SET key1 "value1"
INCR key2 # Error: key2 is not a string
EXEC
● Example: The second command (`INCR`) fails because `key2` is not initialized as a
string. As a result, the entire transaction fails, and no changes are made to
`key1`.

5. Transactional Guarantees
● Redis transactions guarantee that all commands are processed sequentially and
atomically, providing isolation for concurrent clients.
MULTI
SET key1 "value1"
GET key1
EXEC
● Example: `GET key1` within the transaction retrieves `"value1"` consistently,
ensuring that the state of `key1` is unchanged during the transaction.

8
3.5 Query Features
Redis, being a key-value store, primarily supports querying data by keys. However, it
offers additional features and capabilities that enhance querying and data retrieval
beyond basic key operations. Here’s an exploration of Redis query features along with
examples:

1. Basic Key Operations

● Redis supports basic operations like `GET`, `SET`, and `DEL` for retrieving, storing,
and deleting data by keys.

SET key1 "value1"

GET key1
DEL key1

2. Pattern Matching with KEYS

● The `KEYS` command allows pattern matching against keys.
SET user:1:name "John Doe"
SET user:2:name "Jane Doe"
KEYS user:*:name
● Example: Retrieves all keys matching the pattern `user:*:name`, returning
`user:1:name` and `user:2:name`.

3. SCAN for Iterating Over Keys

● `SCAN` command provides an iterative approach to retrieve keys matching a
pattern.
SCAN 0 MATCH user:*:name
● Example: Returns keys matching `user:*:name` in an iterative manner, useful for
large datasets.

4. Sorted Sets and Range Queries

● Redis sorted sets (`ZADD`, `ZRANGE`, etc.) enable range queries based on scores.
ZADD leaderboard 1000 "player1"
ZADD leaderboard 950 "player2"
ZRANGE leaderboard 0 -1 WITHSCORES
● Example: Retrieves the leaderboard with scores for all players, sorted by score.

9
5. Sets and Set Operations
● Redis sets (`SADD`, `SMEMBERS`, etc.) allow set operations like union,
intersection, and difference.
SADD set1 "member1"
SADD set2 "member2"
SINTER set1 set2
● Example: Computes the intersection of `set1` and `set2`, returning common
members.

6. Hashes and Field Queries

● Redis hashes (`HSET`, `HGET`, etc.) allow storing and retrieving multiple fields per
key.
HSET user:1 name "John Doe"
HSET user:1 age 30
HGETALL user:1
● Example: Stores user information and retrieves all fields (`name` and `age`) for
`user:1`.

7. Full-Text Search with Redisearch

● Redisearch is a Redis module that provides full-text search capabilities.
FT.CREATE idx:documents ON HASH PREFIX 1 doc: SCHEMA title TEXT
content TEXT
FT.SEARCH idx:documents "redis search" LIMIT 0 10
● Example: Creates an index on `title` and `content` fields of documents and
performs a full-text search query.

3.6 Structure of Data

Redis allows flexibility in how data is structured and stored, supporting various data
types and operations tailored to different use cases.

1. String Values

● Basic Key-Value Storage: Stores a simple string value associated with a key.

SET key1 "Hello, Redis!"

10
 GET key1

2. Lists

● Ordered Collections: Ideal for maintaining sequences of data.

LPUSH mylist "item1"

LPUSH mylist "item2"

LRANGE mylist 0 -1

● Example: Storing logs or message queues where order matters.

3. Sets

● Unordered Collections: Stores unique values.

SADD myset "member1"

SADD myset "member2"

SMEMBERS myset

● Example: Tracking unique visitors to a website.

4. Sorted Sets

● Ordered by Score: Allows range queries and ranking operations.

ZADD leaderboard 1000 "player1"

ZADD leaderboard 950 "player2"

ZRANGE leaderboard 0 -1 WITHSCORES

● Example: Leaderboards in gaming applications.

5. Hashes

● Key-Value Maps: Stores fields and their values within a single key.

HSET user:1 name "John Doe"

HSET user:1 age 30

11
 HGETALL user:1

● Example: Storing user profiles with multiple attributes.

6. Bitmaps

● Bitwise Operations: Efficiently stores and manipulates bits.

SETBIT mybitmap 0 1

GETBIT mybitmap 0

● Example: Tracking user activity or preferences using bits.

7. HyperLogLogs

● Probabilistic Data Structure: Estimates cardinality of unique items.

PFADD visitors "user1"

PFADD visitors "user2"

PFCOUNT visitors

● Example: Counting unique visitors to a website.

8. Streams

● Append-Only Logs: Stores immutable sequences of data entries.

XADD mystream * sensor_id 1001 temperature 25.5

XADD mystream * sensor_id 1002 temperature 26.0

XRANGE mystream - +

● Example: Logging real-time events or messages.

3.7 Scaling
● Scaling is a critical feature of key-value stores like Redis, enabling them to handle
large amounts of data and high traffic loads efficiently.
● Scaling is a fundamental feature of key-value stores like Redis, providing the
ability to handle increasing amounts of data and user traffic efficiently. By

12
 leveraging horizontal and vertical scaling, sharding, replication, and load
balancing, Redis can ensure high performance, availability, and reliability for
various applications. Here's an expanded explanation with examples:

1. Horizontal Scaling
● Definition: Horizontal scaling, also known as scaling out, involves adding more
servers or nodes to distribute the load. This method enhances the system's
capacity to handle more operations by dividing the workload among multiple
machines.
● Example: In Redis, horizontal scaling can be achieved through clustering. A Redis
cluster spreads the dataset across multiple nodes, allowing for parallel
processing of read and write requests.
● Setting up a Redis Cluster:
redis-server --cluster-enabled yes --cluster-config-file nodes.conf
--cluster-node-timeout 5000 --appendonly yes

2 Vertical Scaling
● Definition: Vertical scaling, or scaling up, involves increasing the resources (CPU,
RAM, storage) of an existing server. While this can improve performance, it has
physical and economic limits.
● Example: Upgrading a Redis server to a machine with more RAM allows it to
store more data in memory, thereby increasing its capacity to handle larger
datasets without increasing latency.

3 Sharding
● Definition: Sharding is a technique used to distribute data across multiple
databases or nodes. Each shard holds a portion of the dataset, determined by a
shard key.
● Example: In Redis, sharding can be implemented by using consistent hashing or
custom sharding logic. The data is distributed across different shards based on
the hash of the key.

13
4 Replication
● Definition: Replication involves copying data from one Redis server (the master)
to one or more Redis servers (the replicas). This enhances data availability and
fault tolerance.
● Example: In Redis, replication is straightforward to set up. The replicas can be
used to offload read traffic from the master, improving read scalability.
● Setting Up Replication:
# Master server
redis-server --port 6379

# Replica server
redis-server --port 6380
redis-cli -p 6380 SLAVEOF 127.0.0.1 6379

Advantages of Scaling in Redis

● Improved Performance: Distributing data and load across multiple nodes
reduces latency and increases throughput.
● High Availability: Replication and load balancing ensure the system remains
available even if some nodes fail.
● Flexibility: Both vertical and horizontal scaling options allow Redis to adapt to
different requirements and workloads.
● Cost Efficiency: Horizontal scaling can be more cost-effective than upgrading to
more powerful hardware, especially in cloud environments.

3.8 Suitable Use Cases

Key-value stores are well-suited for specific use cases where their simplicity,
performance, and scalability provide significant advantages. Here are some scenarios
where key-value stores like Redis excel, along with examples:

1. Storing Session Information

● Use Case: Web applications often need to manage session data for each user,
storing information such as login status, preferences, and temporary user data.

14
 ● Example: Storing session information in Redis using a key-value pair where the
key is the session ID (`session:id`) and the value is a serialized JSON object
containing user preferences and session details.

SET session:12345 '{"user_id": "123", "username": "john_doe", "preferences":

{...}}'

Example: Storing session information for a user during their web session.

● Storing Session Data

SET session:12345
'{"userId":"12345","loginTime":"2024-07-06T10:00:00Z","preferences":{"them
e":"dark"}}' EX 3600

● `session:12345` is the key representing the session ID.

● The value is a JSON string containing user details and preferences.
● EX 3600` sets the session to expire after 1 hour.
● Retrieving Session Data

GET session:12345

Figure 2.2. Change the key design to segment the data in a single bucket.

2. User Profiles, Preferences

● Use Case: Applications frequently store user profiles and preferences, which can
be accessed and updated quickly.
● Example: Storing user profile details and preferences in Redis using hashes for
efficient retrieval and updates.
HSET user:12345 name "John Doe"
HSET user:12345 email "john@example.com"

15
 HSET user:12345 preferences '{"theme": "dark", "language": "en"}'

Example: Storing user-specific settings and preferences.

● Storing User Preferences

HMSET user:12345:preferences language "en" timezone "UTC" theme

"dark"

● `user:12345:preferences` is the key for storing preferences for user ID 12345.

● Each field represents a different preference.
● Retrieving User Preferences

HGETALL user:12345:preferences

3. Shopping Cart Data

● Use Case: E-commerce platforms need to maintain persistent shopping cart

data across user sessions and devices.
● Example: Storing shopping cart details for a user.
● Storing Shopping Cart Data

HSET cart:12345 product:567

'{"productId":"567","productName":"Laptop","quantity":1}'
HSET cart:12345 product:890
'{"productId":"890","productName":"Mouse","quantity":2}'

● `cart:12345` is the key for the user's cart.

● Each field represents a product in the cart, storing details in JSON format.
● Retrieving Shopping Cart Data

HGETALL cart:12345

16
 Figure 2.1. Storing all the data in a single bucket

4. Caching

● Use Case: Key-value stores are widely used for caching frequently accessed data
to improve application performance and reduce latency.
● Example: Caching database query results or computed values in Redis to avoid
repeated expensive operations.
SET cache:user_profile:12345 '{"name": "John Doe", "age": 30, "address":
"123 Main St"}'

5. Configuration Management

● Use Case: Storing and managing application configurations that need to be

quickly accessed and updated.
● Example: Storing configuration settings in Redis using simple key-value pairs.
SET config:smtp_server "smtp.example.com"
SET config:api_key "abcdef123456"

6. Real-Time Analytics

● Use Case: Logging and analyzing real-time events or metrics where quick data
retrieval is critical.
● Example: Storing event data in Redis sorted sets or lists for efficient aggregation
and analysis.

17
 ZADD analytics:page_views 1 "page1"
ZINCRBY analytics:page_views 1 "page1"

Advantages of Key-Value Stores

● Performance: Key-value stores offer fast read and write operations due to their
simple data structure and in-memory storage.
● Scalability: They can scale horizontally by adding more nodes, distributing data
across multiple servers.
● Flexibility: Support for various data types (strings, hashes, sets, sorted sets)
allows diverse use cases beyond simple key-value storage.

3.9 When Not to Use Key-Value Stores

While Redis offers powerful features for specific use cases, there are scenarios where a
key-value store may not be the ideal solution due to its inherent limitations. Here are
some situations where you might reconsider using Redis:

1. Relationships among Data

● Issue: Key-value stores like Redis do not inherently support relationships

between different sets of data or complex data models.
● Example: If you need to model relationships such as foreign keys or join
operations between multiple datasets, Redis may not provide the necessary
capabilities. For instance, if you have user data linked to their orders and need to
perform queries involving both entities, Redis's data model limitations could
make such operations inefficient or impractical.

2. Multi-Operation Transactions

● Issue: Redis transactions are designed to be atomic for operations on a single

key or multiple keys within the same hash slot.
● Example: If your application requires multi-key transactions where all operations
must succeed or fail together, Redis's transactional support may not be suitable.
For instance, if you need to ensure that transferring money between accounts
involves deducting from one account and adding to another atomically, Redis
may not handle this natively across different keys.

18
3. Querying by Data

● Issue: Key-value stores like Redis primarily allow querying by keys rather than by
the data stored within the values.
● Example: If you frequently need to query data based on attributes stored within
the values (e.g., finding all users with a specific age range or searching
documents by content), Redis's lack of secondary indexing or querying
capabilities could lead to inefficient solutions. Although the Redisearch module
provides full-text search capabilities, it's an add-on and not part of the core Redis
capabilities.

4. Operations by Sets

● Issue: Redis operations are typically limited to single keys or small sets of keys
managed by the same client-side logic.
● Example: If your application requires complex set operations across multiple
keys, such as set intersections, unions, or differences involving large datasets,
Redis's lack of native support for operations spanning multiple keys could result
in performance bottlenecks. While Redis supports sets and set operations,
managing large distributed sets or complex operations may require additional
client-side logic.

Disadvantages of Key-Value Stores

1. No Built-in Data Validation:

● Key-value stores do not inherently validate the data being stored. The application
must handle all data validation, which can lead to inconsistencies if not managed
properly.

2. Limited Data Modeling Capabilities:

● The simple key-value model is not suitable for complex data models.
Representing intricate data relationships or hierarchical data structures can be
cumbersome and inefficient.

19
3. Lack of Standardization:

● There is no standardized query language for key-value stores. Each key-value

database has its own APIs and commands, making it challenging to switch
between different systems.

4. Difficulty with Data Migration:

● Migrating data from a key-value store to another type of database or even to a

different key-value store can be complex and time-consuming due to the lack of
a standardized data format.

5. Operational Complexity for Large Data Sets:

● Managing backups, restorations, and failovers can be more complex for

key-value stores, especially as the data size grows. This can require specialized
knowledge and tools to handle effectively.

20