Cloud Computing Assignment-2 DynamoDB Team Members: K.Varun Chandra C.Vishnu Vardhan V.Saicharan D.Harsha Vikhyath M.Vijaya Pratap Reddy DynamoDB Introduction: Amazon DynamoDB is a cloud-native NoSQL primarily key-value database. Let's define each of those terms. DynamoDB is cloud-native in that it does not run on-premises or even in a hybrid cloud; it only runs on Amazon Web Services (AWS). This enables it to scale as needed without requiring a customer's capital investment in hardware. It also has attributes common to other cloud-native applications, such as elastic infrastructure deployment (meaning that AWS will provision more servers in the background as you request additional capacity) DynamoDB is NoSQL in that it does not support ANSI Structured Query Language (SQL). Instead, it uses a proprietary API based on JavaScript Object Notation (JSON). This API is generally not called directly by user developers, but invoked through AWS Software Developer Kits (SDKs) for DynamoDB written in various programming languages (C++, Go, Java, JavaScript, Microsoft NET, Node js, PHP, Python and Ruby) DynamoDB is primarily a key-value store in the sense that its data model consists of key-value pairs in a schemaless, very large, non-relational table of rows (records). It does not support relational database management systems (ROBMS) methods to join tables through foreign keys. It can also support a document store data model using JavaScript Object Notation (JSON).DynamoDB's NoSQL design is oriented towards simplicity and scalability, which appeal to developers and devops teams respectively. It can be used for a wide variety of semistructured data-driven applications prevalent in modern and emerging use cases beyond traditional databases, from the Internet of Things (loT) to social apps or massive multiplayer games, With its broad programming language support, it is easy for developers to get started and to create very sophisticated applications using DynamoDB.What is a DynamoDB Database? Outside of Amazon employees, the world doesn't know much about the exact nature of this database. There is a development version known as DynamoDB Local usod to run on. developer laptops written in Java, but the cloud-native database architecture is proprietary closed-source. While we cannot describe exactly what DynamoDB is, we can describe how you interact with it When you set up DynamoDB on AWS, you do not provision specific servers or allocate set amounts of disk. Instead, you provision throughput — you define the database based on provisioned capacity — how many transactions and how many kilobytes of traffic you wish to ‘support per second. Users specify a service level of read capacity units (RCUs) and write ‘capacity units (WCUs). ynamod8 (garegates) Data Sources ynamods 33 (Static Fie) Dashboards As stated above, users generally do not directly make DynamoDB API calls. Instead, they will integrate an AWS SDK into their application, which will handle the back-end communications. with the server. DynamoDB data modeling needs to be denormalized. For developers used to working with both SQL and NoSQL databases, the process of rethinking their data model is nontrivial, but also not insurmountable.DynamoDB Architecture a amazon ppstore i tan &— = a ‘ep sin oe sagen ey So ioe eee Recuesis Amazon iat we 2e5 = Tables: Data in DynamoDB is organized into tables. A table is a collection of items, and each item is a collection of attributes. Tables are created based on your application's needs, and they can range from small and simple to large and complex. Items: Items are the basic unit of data in DynamoDB. Each item is a set of attributes, Where each attribute has a name and a value. Items in a table can have different attributes, and each item does not need to have the same set of attributes. Attributes: Attributes are the fundamental data elements in DynamoDB. Each item consists of one or more attributes, where each attribute has a name and a value. Attributes can include scalars like strings, numbers, or binary data, as well as complex data types like lists or maps.Partition Key and Sort Key: Each table must have a primary key, which can be composed of one or two attributes: the partition key and the sort key. The partition key determines the partition in which the item will be stored, and the sort key (if used) determines the order of items with the same partition key. Partitions: DynamoDB partitions data across multiple servers to ensure scalability. Each partition holds a distinct range of partition key values. The goal is to distribute the workload evenly across partitions, allowing for efficient read and write operations. Replication: DynamoDB automatically replicates data across multiple Availability Zones (AZs) within a region to provide fault tolerance and high availability. Each write is, committed to the primary partition, and the changes are asynchronously replicated to replica partitions in other AZs. Global Tables: DynamoDB supports Global Tables, which allow you to replicate your table data across multiple AWS regions. This feature enables you to build globally istributed applications with low-latency access to data Indexes: DynamoDB supports secondary indexes to provide more querying flexibility. Global Secondary Indexes (GS!) and Local Secondary Indexes (LSI) allow you to query the data based on attributes other than the primary key. GSis are not bound by the same partition and sort key limitations as the main table. ‘Streams: DynamoDB Streams capture and stream item-level modifications in a table. This can be used for various purposes, including triggering AWS Lambda functions, replicating data to other tables, or maintaining an audit trail AWS Management Console, SDKs, and APIs: Developers can interact with DynamoDB through the AWS Management Console, SDKs in various programming languages (such as Java, Python, Nodes, etc.), and APIs. This allows for programmatic access and integration with applications.When to use DynamoDB? Amazon DynamoDB * Amazon DynamoDB is most useful when you need to rapidly prototype and deploy a key-value store database that can seamlessly scale to multiple gigabytes or terabytes of information — what are often referred to as “Big Data” applications. Because of its ‘emphasis on scalability and high availability DynamoDB is also appropriate for “always on" use cases with high volume transactional requests (reads and writes). © DynamoDB is inappropriate for extremely large data sets (petabytes) with high frequency transactions where the cost of operating DynamoDB may make it prohibitive Itis also important to remember DynamoDB is a NoSQL database that uses its own proprietary JSON-based query API, so it should be used when data models do not require normalized data with JOINs across tables which are more appropriate for SQL RDBMS systems. DynamoDB Data Storage Format ‘To manage data, DynamoDB uses hashing and b-trees. While DynamoDB supports JSON, it only uses it as a transport format; JSON is not used as a storage format. Much of the exact implementation of DynamoDB's data storage format remains proprietary. Generally a user would not need to know about it, Data in DynamoDB is generally exported through streaming technologies or bulk downloaded into CSV files through ETL-type tools like AWS Glue. As a managed service, the exact nature of data on disk, much like the rest of the system specification (hardware, operating system, etc.), remains hidden from end users of DynamoDB.TableName: "Table_X" KeyConditionExpression: “LastName = : ExpressionAttributeValues: { ra = “smitl DDL(Data Definition Language) Operations: 1.Create Table: This operation is used to create a new table. It involves specifying the table name, primary key attributes (partition key and optional sort key), provisioned throughput settings, and any secondary indexes. Example: AWS CLI ‘Syntax: ‘aws dynamodb create-table —table-name MyTable ~attribute-definitions AttributeName=ID,AttributeType=N —key-schema AttributeName=ID, KeyType=HASH ~provisioned- throughput ReadCapacityUnits=5, WriteCapacityUnits=5 2.Update Table: You can use the UpdateTable operation to modify the pro. settings of an existing table. joned throughput Syntax: ws dynamodb update-table ~table-name MyTable --provisioned-throughput ReadCapacityUnits=10,WriteCapacityUnits=10 3.Delete Table: This operation is used to delete an existing table and all of its data, Syntax: aws dynamodb delete-table —table-name MyTable SELECT * FROM Orders IWIER JOIN Order Items ON Orders.Order_ID = Order_Itens Order ID IMVER JOIN Products ON Products.Product_ID = Order_Ttens.Preduct_ID INNER JOIN Inventories ON Products.Product_ID = Inventories Product_ID (ORDER BY Quantity on Hand DESCDML(Data Manipulation Language) Operations: 1.Put Item: The Putitem operation is used to create a new item or replace an existing item's attributes if it already exists. Syntax: ‘aws dynamodb put-item —table-name MyTable —item PID": (°N*: "1", "Name": ("S": John") 2.Get Item: The Getltem operation retrieves a single item from a table based on the primary key. Syntax: ‘aws dynamodb get-item -table-name MyTable ~key ‘{"ID 7 3.Update Item: The Updateltem operation modifies the attributes of an existing item. Syntax: aws dynamodb update-item -table-name MyTable —key '{"ID": {"N" "SET Name = :newName" ~expression-attribute-values ‘{":newName" 1"}}' -update-expression lane" 4.Delete Item: The Deleteltem operation removes a single item from a table. Syntax: ‘aws dynamodb delete item —table-name MyTable --key {"ID": {°N' “ny 5.Query: The Query operation is used to retrieve items from a table based on the specified criteria, typically with the help of secondary indexes. Syntax: aws dynamodb query ~-table-name MyTable --key-condition-expressior —expression-attribute-values CNS ty "1D 6.Scan: The Scan operation reads every item in a table, making it useful for retrieving all items or filtering based on non-indexed attributes. Syntax: ws dynamodb scan ~table-name MyTable -fiter-expression "Name ~expression-attribute-valuesStorage Techniques of DynamoDB: * Partitioning: DynamoDB partitions your table's data across multiple physical partitions based on the partition key. Each partition is capable of handling a certain amount of read and write capacity. Effective partitioning is crucial for achieving high throughput and avoiding hot partitions (partitions that receive a disproportionate amount of traffic). * Partition Key: The partition key is the primary determinant of how data is distributed across partitions. A well-chosen partition key evenly distributes read and write requests across the entire table, Unevenly distributed keys can result in unevenly distributed traffic, leading to hot partitions and potential throttling * Sort Key: If your table includes a sort key, the combination of the partition key and sort key determines the item's physical storage within the partition. This enables efficient querying and sorting of data based on specific criteria. * Provisioned Throughput: DynamoDB uses provisioned throughput to manage the capacity for read and write operations. Provisioned throughput is specified in terms of read capacity units (RCUs) and write capacity units (WCUs), You can adjust these values based on your application's needs. * Auto Scaling: DynamoDB provides Auto Scaling, which automatically adjusts the provisioned throughput for your table in response to changing traffic patterns. This feature helps ensure that your table can handle varying workloads without manual intervention, © DynamoDB Accelerator (DAX): DynamoDB Accelerator (DAX) is a fully managed, highly available, in-memory cache for DynamoDB that helps improve response times for read-intensive workloads. DAX reduces the need to read from the DynamoDB table directly for frequently accessed data. * Global Tables: DynamoDB Global Tables replicate your table's data across multiple AWS regions. This provides low-latency access to the data for users in different geographical locations. Each region has its own set of partitions, and changes are asynchronously replicated across regions. © On-Demand Capacity Mode: In addition to provisioned throughput, DynamoDB offers ‘On-Demand Capacity Mode, where you pay per request instead of pre-provisioning capacity. This mode is suitable for applications with unpredictable workloads. * Backup and Restore: DynamoDB provides backup and restore functionality, allowing you to create full backups of your table data. This feature can be used for data archivit ‘compliance, and recovery purposes. 9.© Point-in-Time Recovery (PITR): PITR enables continuous backups of your DynamoDB table, allowing you to restore your table to any point in time within a specified retention period. This helps protect against accidental writes or deletions. Advantages of DynamoDB Managed Service: DynamoDB is a fully managed service, meaning AWS takes care of the administrative tasks such as hardware provisioning, setup, and configuration. This allows developers to focus on building applications rather than managing infrastructure. Scalability: DynamoDB provides seamless and automatic scalability. It can handle varying workloads by adjusting the read and write capacities, and it can scale both up and down without any downtime. High Performance: DynamoDB offers low-latency access to data, making it suitable for applications that require fast and responsive access to data. Its architecture is designed for high-performance read and write operations. NoSQL and Flexible Schema: As a NoSQL database, DynamoDB allows for flexible schema design. You can add or remove attributes without affecting other items in the table, making it adaptable to changing application requirements. © Built-in Security: DynamoDB provides features such as encryption at rest and in transit, fine-grained access control using AWS Identity and Access Management (IAM), and integration with AWS Key Management Service (KMS) for managing encryption keys. High Availability and Fault Tolerance: DynamoDB replicates data across multiple Availability Zones within a region, providing high availability and fault tolerance. In the event of a failure, the service automatically redirects requests to healthy servers. Global Tables: DynamoDB supports Global Tables, allowing you to replicate your table data across multiple AWS regions. This enables low-latency access for users in different geographic locations. Backups and Point-in-Time Recovery: DynamoDB offers backup and restore functionality, as well as Point-in-Time Recovery (PITR). This helps protect against accidental data loss or corruption and provides options for data recovery. DynamoDB Accelerator (DAX): DAX is an in-memory caching service that can be used with DynamoDB to further improve read performance by reducing the need to read from the underlying table.* Serverless Integration: DynamoDB integrates seamlessly with AWS Lambda, allowing you to build serverless applications where the database scales automatically based on demand. Disadvantages of DynamoDB ‘* Cost: While DynamoDB provides a serverless and scalable model, it can become ‘expensive for workloads with high and unpredictable traffic. Provisioned throughput and additional features such as DAX may contribute to costs. © Complexity for Complex Queries: DynamoD8 is optimized for simple key-value access pattems. Performing complex queries that involve multiple attributes or require sorting can be challenging compared to relational databases. * Consistency Models: DynamoDB offers two consistency models: Eventually Consistent Reads and Strongly Consistent Reads. Depending on your application's requirements, you may need to carefully choose the appropriate consistency level ted Indexing: While DynamoDB supports secondary indexes (Global and Local), they have limitations, and not all types of queries are supported. This can lead to challenges when designing complex data access patterns. * Data Size Limitations: Individual items in DynamoDB are limited to 400 KB. If your items frequently approach or exceed this limit, you may need to consider alternative strategies, such as using S3 for large objects. * Learning Curve: Developers familiar with traditional relational databases may experience a learning curve when transitioning to DynamoDB, as it has a different data model and query language. © Limited Transactions: While DynamoDB supports transactions within a single partition key, cross-partition transactions are not supported. This limitation may impact the design of certain applications. * Local Development Challenges: Developing and testing locally with DynamoDB may not fully replicate the performance characteristics and features available in the AWS cloud environment.Conclusion In conclusion, Amazon DynamoDB is a powerful and fully managed NoSQL database service provided by Amazon Web Services (AWS). It offers several advantages, making it a popular choice for a wide range of applications. DynamoDB excels in providing seamless scalability, high performance, and built-in security features. Its managed nature relieves developers from infrastructure management tasks, allowing them to focus on application development. The flexible schema design, support for various data types, and automatic scaling make DynamoDB suitable for dynamic and evolving data models. The service's ability to replicate data across multiple Availability Zones and AWS regions ensures high availabilty and fault tolerance, supporting global applications. However, DynamoDB is not without its challenges. Developers need to be mindful of the cost implications, especially for workloads with unpredictable traffic patterns. While DynamoDB supports complex queries, it may not be as intuitive as traditional relational databases for cerlain use cases. Learning to design tables effectively, considering partition keys, and understanding the limitations of secondary indexes are crucial for optimizing performance. Despite these considerations, DynamoDB remains a versatile and scalable solution for a variety of applications, ranging from small projects to large-scale, globally distributed systems. When used appropriately, DynamoDB can offer a robust and reliable foundation for building modem, cloud-native applications. As with any database solution, careful consideration of the specific requirements and trade-offs is essential for making informed decisions in the context of your application architecture and goals