[go: up one dir, main page]
Scribd
Upload
85 views23 pages

Why Is Big Data Processing Different?

The document discusses the requirements for processing big data. It notes that big data systems must (1) support big data operations like splitting large volumes of data and accessing it quickly, (2) handle fault tolerance through data replication and recovery, and (3) enable adding more racks to scale horizontally. The systems must also (4) be optimized and extensible to handle different data types, and (5) enable both streaming and batch processing of data.

Uploaded by

Kosuru ratnasai
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
85 views23 pages

Why Is Big Data Processing Different?

The document discusses the requirements for processing big data. It notes that big data systems must (1) support big data operations like splitting large volumes of data and accessing it quickly, (2) handle fault tolerance through data replication and recovery, and (3) enable adding more racks to scale horizontally. The systems must also (4) be optimized and extensible to handle different data types, and (5) enable both streaming and batch processing of data.

Uploaded by

Kosuru ratnasai
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 23

Why is Big Data

Processing Different?
After this video you will be able to..
• Summarize the requirements of programming
models for big data and why you should care
about them

• Explain how the challenges of big data related


to its variety, volume and velocity affects its
processing
Requirements for Big
Data Systems
A Big Data System for
an Online Game
Processing
node
…
…
…
…

Batch
Processing
Scalability

Complexity
Network

Data-parallel
Rack

scalability

Data
1 2 3 4 5

Compute
2 5
Rack

1
3 4
Programming Model = abstractions
Runtime Libraries Programming Languages

Data
1 2 3 4 5

Compute
2 5
Rack

1
3 4
Requirements for Big Data Systems
1. Support Big Data Operations

Split volumes of data


1. Support Big Data Operations

Split volumes of data

Access data fast


1. Support Big Data Operations

Split volumes of data

Access data fast

Distribute computations to nodes


2. Handle Fault Tolerance

Replicate data partitions

Recover files when needed


3. Enable Adding More Racks

Data 3

1 2 3 4 5

Compute
3
2 5

Rack
Rack

1
3 4 3
4. Optimized and extensible
for many data types

Document Table

Graph
Key-value

Multimedia Stream
5. Enable both streaming
and batch processing

Low latency processing


of streaming data

Accurate processing
of all available data
Volume Scalable batch
processing

Velocity Stream processing

Variety Extensible data storage,


access and integration

You might also like