Unit - 3
Unit - 3
Unit - 3
Introduction to No SQL databases – MongoDB system overview – Basic querying with MongoDB
shell – Request body parsing in Express – NodeJS Mongo DB connection – Adding and retrieving
data to MongoDB from NodeJS – Handling SQL databases from NodeJS – Handling Cookies in
NodeJS –Handling User Authentication with NodeJS
Introduction to NoSQL:
NoSQL is a type of database management system (DBMS) that is designed to handle and store large
volumes of unstructured and semi-structured data. Unlike traditional relational databases that use tables
with pre-defined schemas to store data, NoSQL databases use flexible data models that can adapt to
changes in data structures and are capable of scaling horizontally to handle growing amounts of data.
The term NoSQL originally referred to “non-SQL” or “non-relational” databases, but the term has since
evolved to mean “not only SQL,” as NoSQL databases have expanded to include a wide range of
different database architectures and data models.
Document databases: These databases store data as semi-structured documents, such as JSON or
XML, and can be queried using document-oriented query languages.
Key-value stores: These databases store data as key-value pairs, and are optimized for simple and fast
read/write operations.
Column-family stores: These databases store data as column families, which are sets of columns
that are treated as a single entity. They are optimized for fast and efficient querying of large
amounts of data.
Graph databases: These databases store data as nodes and edges, and are designed to handle
complex relationships between data.
NoSQL databases are often used in applications where there is a high volume of data that needs to be
processed and analyzed in real-time, such as social media analytics, e-commerce, and gaming. They can
also be used for other applications, such as content management systems, document management, and
customer relationship management.
However, NoSQL databases may not be suitable for all applications, as they may not provide the same
level of data consistency and transactional guarantees as traditional relational databases. It is important
to carefully evaluate the specific needs of an application when choosing a database management system.
NoSQL originally referring to non SQL or non relational is a database that provides a mechanism for
storage and retrieval of data. This data is modeled in means other than the tabular relations used in
relational databases. Such databases came into existence in the late 1960s, but did not obtain the
NoSQL moniker until a surge of popularity in the early twenty-first century. NoSQL databases are used
in real- time web applications and big data and their use are increasing over time.
NoSQL systems are also sometimes called Not only SQL to emphasize the fact that they may support
SQL-like query languages. A NoSQL database includes simplicity of design, simpler horizontal
scaling to clusters of machines and finer control over availability. The data structures used by NoSQL
databases are different from those used by default in relational databases which makes some operations faster
in NoSQL. The suitability of a given NoSQL database depends on the problem it should solve.
NoSQL databases, also known as “not only SQL” databases, are a new type of database management
system that have gained popularity in recent years. Unlike traditional relational databases, NoSQL
databases are designed to handle large amounts of unstructured or semi-structured data, and they can
accommodate dynamic changes to the data model. This makes NoSQL databases a good fit for modern web
applications, real-time analytics, and big data processing.
Data structures used by NoSQL databases are sometimes also viewed as more flexible than relational
database tables. Many NoSQL stores compromise consistency in favor of availability, speed and
partition tolerance. Barriers to the greater adoption of NoSQL stores include the use of low-level query
languages, lack of standardized interfaces, and huge previous investments in existing relational databases.
Most NoSQL stores lack true ACID(Atomicity, Consistency, Isolation, Durability) transactions but a
few databases, such as MarkLogic, Aerospike, FairCom c-treeACE, Google Spanner (though technically
a NewSQL database), Symas LMDB, and OrientDB have made them central to their designs.
Most NoSQL databases offer a concept of eventual consistency in which database changes are
propagated to all nodes so queries for data might not return updated data immediately or might result
in
reading data that is not accurate which is a problem known as stale reads. Also some NoSQL
systems may exhibit lost writes and other forms of data loss. Some NoSQL systems provide concepts
such as write-ahead logging to avoid data loss.
One simple example of a NoSQL database is a document database. In a document database, data is
stored in documents rather than tables. Each document can contain a different set of fields, making it
easy to accommodate changing data requirements
For example, “Take, for instance, a database that holds data regarding employees.”. In a relational
database, this information might be stored in tables, with one table for employee information and
another table for department information. In a document database, each employee would be stored
as a separate document, with all of their information contained within the document.
NoSQL databases are a relatively new type of database management system that have gained
popularity in recent years due to their scalability and flexibility. They are designed to handle large
amounts of unstructured or semi-structured data and can handle dynamic changes to the data model.
This makes NoSQL databases a good fit for modern web applications, real-time analytics, and big data
processing.
Distributed and high availability: NoSQL databases are often designed to be highly available and to
automatically handle node failures and data replication across multiple nodes in a database cluster.
Flexibility: NoSQL databases allow developers to store and retrieve data in a flexible and
dynamic manner, with support for multiple data types and changing data structures.
Performance: NoSQL databases are optimized for high performance and can handle a high volume
of reads and writes, making them suitable for big data and real-time applications. Advantages of
NoSQL: There are many advantages of working with NoSQL databases such as MongoDB and
Cassandra. The main advantages are high scalability and high availability.
High scalability : NoSQL databases use sharding for horizontal scaling. Partitioning of data and
placing it on multiple machines in such a way that the order of the data is preserved is sharding.
Vertical scaling means adding more resources to the existing machine whereas horizontal scaling
means adding more machines to handle the data. Vertical scaling is not that easy to implement but
horizontal scaling is easy to implement. Examples of horizontal scaling databases are MongoDB,
Cassandra, etc. NoSQL can handle a huge amount of data because of scalability, as the data grows
NoSQL scale itself to handle that data in an efficient manner.
Flexibility: NoSQL databases are designed to handle unstructured or semi-structured data, which
means that they can accommodate dynamic changes to the data model. This makes NoSQL
databases a good fit for applications that need to handle changing data requirements.
High availability : Auto replication feature in NoSQL databases makes it highly available because
in case of any failure data replicates itself to the previous consistent state.
Scalability: NoSQL databases are highly scalable, which means that they can handle large amounts
of data and traffic with ease. This makes them a good fit for applications that need to handle large
amounts of data or traffic
Performance: NoSQL databases are designed to handle large amounts of data and traffic, which
means that they can offer improved performance compared to traditional relational databases.
Cost-effectiveness: NoSQL databases are often more cost-effective than traditional relational
databases, as they are typically less complex and do not require expensive hardware or software.
Lack of standardization : There are many different types of NoSQL databases, each with its own
unique strengths and weaknesses. This lack of standardization can make it difficult to choose the right
database for a specific application
Lack of ACID compliance : NoSQL databases are not fully ACID-compliant, which means that they
do not guarantee the consistency, integrity, and durability of data. This can be a drawback for
applications that require strong data consistency guarantees.
Narrow focus : NoSQL databases have a very narrow focus as it is mainly designed for storage but
it provides very little functionality. Relational databases are a better choice in the field of
Transaction Management than NoSQL.
Open-source : NoSQL is open-source database. There is no reliable standard for NoSQL yet. In other
words, two database systems are likely to be unequal.
Lack of support for complex queries : NoSQL databases are not designed to handle complex queries,
which means that they are not a good fit for applications that require complex data analysis or
reporting.
Lack of maturity : NoSQL databases are relatively new and lack the maturity of traditional
relational databases. This can make them less reliable and less secure than traditional databases.
Management challenge : The purpose of big data tools is to make the management of a large
amount of data as simple as possible. But it is not so easy. Data management in NoSQL is much
more complex than in a relational database. NoSQL, in particular, has a reputation for being
challenging to install and even more hectic to manage on a daily basis.
GUI is not available : GUI mode tools to access the database are not flexibly available in the market.
Backup : Backup is a great weak point for some NoSQL databases like MongoDB. MongoDB has no
approach for the backup of data in a consistent manner.
Large document size : Some database systems like MongoDB and CouchDB store data in JSON
format. This means that documents are quite large (BigData, network bandwidth, speed), and
having descriptive key names actually hurts since they increase the document size.
Types of NoSQL database: Types of NoSQL databases and the name of the databases system that falls
in that category are:
1. Graph Databases: Examples – Amazon Neptune, Neo4j
2. Key value store: Examples – Memcached, Redis, Coherence
3. Tabular: Examples – Hbase, Big Table, Accumulo
4. Document-based: Examples – MongoDB, CouchDB, Cloudant
The data is growing continuously and you need to scale the database regularly to handle the data.
MongoDB: An introduction:
MongoDB, the most popular NoSQL database, is an open-source document-oriented database. The term
‘NoSQL’ means ‘non-relational’. It means that MongoDB isn’t based on the table-like relational
database structure but provides an altogether different mechanism for storage and retrieval of data. This
format of storage is called BSON ( similar to JSON format).
A simple MongoDB document Structure:
{
title: ‘Geeksforgeeks’,
by: ‘Harshit Gupta’,
url: ‘https://www.geeksforgeeks.org’,
type: ‘NoSQL’
}
SQL databases store data in tabular format. This data is stored in a predefined data model which is not
very much flexible for today’s real-world highly growing applications. Modern applications are more
networked, social and interactive than ever. Applications are storing more and more data and are
accessing it at higher rates.
Relational Database Management System(RDBMS) is not the correct choice when it comes to
handling big data by the virtue of their design since they are not horizontally scalable. If the
database runs on a single server, then it will reach a scaling limit. NoSQL databases are more scalable
and provide superior performance. MongoDB is such a NoSQL database that scales by adding more and
more servers and increases productivity with its flexible document model.
RDBMS vs MongoDB:
RDBMS has a typical schema design that shows number of tables and the relationship between these
tables whereas MongoDB is document-oriented. There is no concept of schema or relationship.
Complex transactions are not supported in MongoDB because complex join operations are not
available.
MongoDB allows a highly flexible and scalable document structure. For example, one data
document of a collection in MongoDB can have two fields whereas the other document in the same
collection can have four.
MongoDB is faster as compared to RDBMS due to efficient indexing and storage techniques.
There are a few terms that are related in both databases. What’s called Table in RDBMS is called a
Collection in MongoDB. Similarly, a Row is called a Document and a Column is called a Field.
MongoDB provides a default ‘_id’ (if not provided explicitly) which is a 12-byte hexadecimal number
that assures the uniqueness of every document. It is similar to the Primary key in RDBMS.
Features of MongoDB:
Document Oriented: MongoDB stores the main subject in the minimal number of documents and not
by breaking it up into multiple relational structures like RDBMS. For example, it stores all the
information of a computer in a single document called Computer and not in distinct relational
structures
like CPU, RAM, Hard disk, etc.
Indexing: Without indexing, a database would have to scan every document of a collection to select
those that match the query which would be inefficient. So, for efficient searching Indexing is a must
and MongoDB uses it to process huge volumes of data in very less time.
Scalability: MongoDB scales horizontally using sharding (partitioning data across various servers).
Data is partitioned into data chunks using the shard key, and these data chunks are evenly distributed
across shards that reside across many physical servers. Also, new machines can be added to a
running database.
Replication and High Availability: MongoDB increases the data availability with multiple copies
of data on different servers. By providing redundancy, it protects the database from hardware
failures. If one server goes down, the data can be retrieved easily from other active servers which
also
had the data stored on them.
Aggregation: Aggregation operations process data records and return the computed results. It is
similar to the GROUPBY clause in SQL. A few aggregation expressions are sum, avg, min, max, etc
Unstable Schema: Adding a new column in RDBMS is hard whereas MongoDB is schema- less.
Adding a new field does not effect old documents and will be very easy.
Distributed data Since multiple copies of data are stored across different servers, recovery of data is
instant and safe even if there is a hardware failure.
Syntax
way. Example
Syntax
>db.COLLECTION_NAME.find().pretty()
Example
Following example retrieves all the documents from the collection named mycol and arranges them in
an easy-to-read format.
> db.mycol.find().pretty()
{
“_id” : ObjectId(“5dd4e2cc0821d3b44607534c”),
“title” : “MongoDB Overview”,
“description” : “MongoDB is no SQL database”,
“by” : “tutorials point”,
“url” : “http://www.tutorialspoint.com”,
“tags” : [
“mongodb”,
“database”,
“NoSQL”
],
“likes” : 100}
{
“_id” : ObjectId(“5dd4e2cc0821d3b44607534d”),
“title” : “NoSQL Database”,
“description” : “NoSQL database doesn’t have tables”,
“by” : “tutorials point”,
“url” : “http://www.tutorialspoint.com”,
“tags” : [
“mongodb”,
“database”,
“NoSQL”
],
“likes” : 20,
“comments” : [
{
“user” : “user1”,
“message” : “My first comment”,
“dateCreated” : ISODate(“2013-12-09T21:05:00Z”),
“like” : 0
}
]
}
The findOne() method
Apart from the find() method, there is findOne() method, that returns only one
document. Syntax
>db.COLLECTIONNAME.findOne()
Example
Express body-parser is an npm module used to process data sent in an HTTP request body. It provides
four express middleware for parsing JSON, Text, URL-encoded, and raw data sets over an HTTP
request body. Before the target controller receives an incoming request, these middleware routines
handle it.
Body-parser is the Node.js body-parsing middleware. It is responsible for parsing the incoming request
bodies in a middleware before you handle it.
Installation of body-parser module:
You can visit the link to Install the body-parser module. You can install this package by using this
command.
Npm install body-parser
After installing body-parser you can check your body-parser version in the command prompt using the
command.
Npm version body-parser
Project Structure:
Filename:
SampleForm.ejs: The below code should be in the SampleForm.ejs
<!DOCTYPE html>
<html>
<head>
<title>Body-Parser Module Demo</title>
</head>
<body>
<h1>Demo Form</h1>
index.js
const bodyparser = require(‘body-parser’)
const express = require(“express”)
const path = require(‘path’)
const app = express()
// Body-parser middleware
app.use(bodyparser.urlencoded({ exte
nded: true }))
app.use(bodyparser.json())
But if we do not use this body-parser middle, then while parsing, an error will occur as shown below:
So this is how you can use the body-parser module for parsing incoming request bodies in middleware
before you handle it.
MongoDB is a NoSQL database used to store large amounts of data without any traditional relational
database table. Instead of rows & columns, MongoDB used collections & documents to store data. A
collections consist of a set of documents & a document consists of key-value pairs which are the basic
unit of data in MongoDB.
Make sure that MongoDB installs on your pc.
To connect a Node.js application to MongoDB, we have to use a library called Mongoose.
<!DOCTYPE html>
<html lang=”en”>
<head>
<meta charset=”UTF-8”>
<meta http-equiv=”X-UA-Compatible” content=”IE=edge”>
<meta name=”viewport” content=
“width=device-width, initial-scale=1.0”>
<title>Document</title>
</head>
<body>
<form action=”/contact” method=”post”>
<input type=”text” placeholder=”Email” name=”email”>
<input type=”text” placeholder=”Query” name=”query”>
<button type=”submit”>Submit</button>
</form>
</body>
</html>
mongoose.connect(“mongodb://localhost:27017/newCollection”, {
useNewUrlParser: true,
useUnifiedTopology: true
});
const contactSchema =
{ email: String,
query: String,
};
app.use(bodyParser.urlencoded({ exte
nded: true
}));
app.listen(3000, function(){
console.log(“App is running on Port 3000”);
});
MongoDB, the most popular NoSQL database, is an open-source document-oriented database. The term
‘NoSQL’ means ‘non-relational’. It means that MongoDB isn’t based on the table-like relational
database structure but provides an altogether different mechanism for storage and retrieval of data. This
format of storage is called BSON ( similar to JSON format). Refer (this) article.
MongoDB module: This module of Node.js is used for connecting the MongoDB database as well as
used for manipulating the collections and databases in MongoDB. The mongodb.connect() method is
used for connecting the MongoDB database which is running on a particular server on your machine.
(Refer to this article). We can also use promises, in this method in resolve the object contains all the
methods and properties required for collection manipulation and in reject the error occurs during
connection.
Installing module:
node install mongodb
Project Structure:
// Connect to collection
const collection = connect
.collection(“GFGcollections”);
const MongoClient =
require(“mongodb”); const url =
‘mongodb://localhost:27017/’;
const databasename = “GFG”; // Database
name MongoClient.connect(url).then((client) =>
{
// Connect to collection
const collection = connect
.collection(“GFGcollections”);
collection.find({}).toArray().then((ans) => {
console.log(ans);
});
}).catch((err) => {
OUTPUT:
Node.js MongoDB Insert:
Insert Into Collection
The first parameter of the insertOne() method is an object containing the name(s) and value(s) of each
field in the document you want to insert.
It also takes a callback function where you can work with any errors, or the result of the insertion:
1 document inserted
Insert Multiple Documents
To insert multiple documents into a collection in MongoDB, we use the insertMany() method.
The first parameter of the insertMany() method is an array of objects, containing the data you want to
insert.
It also takes a callback function where you can work with any errors, or the result of the insertion:
Mysql: An open source Relational Database Management System (RDBMS) that uses Structured
Query Language (SQL). It is the most popular language for adding, accessing and managing content in
a database. Here we will use the Mysql as a database for our node application. It can be
downloaded from here.
Node -v: It will show node version in our system.
Mysql -v: It will show Mysql version in our system.
Till now we have installed Node and Mysql successfully to our system. Modules/packages need to be
installed for connecting node application to Mysql:
Mysql: NodeJs driver for mysql
STEP-1: Create a separate folder for this task and come inside this folder with terminal or
command prompt.
STEP-2: Now, we will generate a package.json file so that all dependencies can be listed there for
future reference. For knowing more about package.json click here For generating package.json run the
following command in terminal of the project folder:
npm init -y
Now we have our package.json in our project folder as shown in the below screenshot:
Let’s understand the code flow of the server.js file in the project folder: Line 2: By using this line
of code we are importing the mysql module.
After that we will setup basic express app by writing following code in our app.js file in root directory .
let express = require(‘express’);
//setup express app
let app = express()
//basic route for homepage
app.get(‘/’, (req, res)=>{
res.send(‘welcome to express app’);
});
So until now we have successfully set up our express app now let’s start with cookies.
For cookies first, we need to import the module in our app.js file and use it like other middlewares.
app.use(cookieParser());
So if we restart our server and make a get request to the route: localhost:3000/getuser before setting the
cookies it is as follows :
After making a request to localhost:3000/setuser it will add user data to cookie and gives output as
follows :
Now if we again make a request to localhost:3000/getuser as this route is iterating user data from
cookies using req.cookies so output will be as follows :
Destroy the cookies :
We can destroy cookies using following code :
res.clearCookie(cookieName);
How to check user authentication in GET method using Node.js ?
There are so many authentication methods like web token authentication, cookies based
authentication, and many more. In this article, we will discuss one of the simplest authentication
methods using express.js during handling clients get a request in node.js with the help of the HTTP
headers.
Approach: HTTP protocols used various types of headers for authentication the client we will use
the WWW-Authenticate header. HTTP WWW-Authenticate header is a response-type header, and
it serves as a support for various authentication mechanisms which are important to control access to
pages and other resources as well.
Explanation of the authentication:
When the request header of the client does not contain WWW-Authenticate header servers response
header set
the header is res.setHeader(“WWW-Authenticate”, ‘Basic’) and set status code 401 and after this, a
pop will
appear on the client-side for valid authentication.
Authentication Form:
}
console.log(authheader)
})
app.listen(3000,()=>{
console.log(“Server is starting”)
})
Run index.js using the following command:
node index.js
Output:
● Open any browser with http://localhost:3000 location in a private window(in order to avoid a
saved password and username). A pop will occur near the address bar. Fill in the username and
password that are mention in the code.
● If the entered username and password match the mention, then location index.html will render
on the browser.
Explanation: The first middleware is used for checking the authentication of the client when the server
start and the client enter the localhost address. Initially req.headers.authorization is undefined and
next() callback function return 401 status code unauthorized access to the browser. The client fills the
credentials and the credentials encrypted in base64 format. After that, it decrypts the base64 format data
that contains username and password, then after checking the username and password is correct, the
next() method calls the next middleware that is mention below the authentication middleware, otherwise
the authentication form pop again and again.
Request Header Details: