See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/308281665

Analyzing Google File System and Hadoop Distributed File System

Article in Research Journal of Information Technology · March 2016

DOI: 10.3923/rjit.2016.66.74

CITATIONS READS

7 2,306

1 author:

Nader Gemayel
Notre Dame University
2 PUBLICATIONS 33 CITATIONS

SEE PROFILE

All content following this page was uploaded by Nader Gemayel on 30 July 2020.

The user has requested enhancement of the downloaded file.

 OPEN ACCESS Research Journal of Information Technology
ISSN 1815-7432
DOI: 10.3923/rjit.2016.66.74

Research Article
Analyzing Google File System and Hadoop Distributed File System
Nader Gemayel

Department of Compuer Science, Notre Dame University-Louaize, P.O. Box 72, Zouk Mosbeh, Zouk Mikayel, Lebanon
Control message File name, chunk index GFS master server
Application
Data message
GFS client File name space
Chunk location
Root /root/dir 1
Chunk 5f 3s
Dir 1
Dir 2 Chunk 7f 2a
Chunk data Chunk location byte range
Chunk 8d3a

File A Chunk 2wsx

File B
Chunk 1d33
Server state
GFS chunk server A
Chunk 5w21
DB
LINUX f ile system
Instructions Dir 3 Chunk 9ytr

GFS chunk server B Dir 4 ...

DB LINUX f ile system File A
File B
GFS chunk server X File n
DB LINUX f ile system
 Business intelligence, analytic layer,
Data warehouse decision support systems

Data connection

Management
Zookeeper Chukwa

Data access
Pig Hive Avro

Data processing
MapReduce framework

Data storage
Hadoop Distributed File
System (HDFS) HBase
C

C
C

C
C

C
 C

C
C
C
C

C
C

C
C

C C

C
C
C
 Control message
Data message

Application Request primary replica

chunkserver’s location GFS master server
1
File name space
GFS client Send location 2 /root/dir 1
Root
3 Dir 1 Chunk 5f3s
4
8
Send write data to replicas’ buffer

Send write Dir 2 Chunk 7f2a

order Confirm: Rite f inished
Chunk 8d3a
Chunkserver A Chunk File A Chunk 2wsx
File B Chunk 1d33

Report back: Write complete

Buffer Block A 5 Write
Chunk 5w21
6 Dir 3 Chunk 9ytr
Write data
Dir 4 ...
Chunkserver B Chunk File A
7 File B
Buffer Block A File n

Write data

Chunkserver X Chunk

Buffer Block A

Control message
Data message
1 Permission to write “Hello.txt”
HDFS client NameNode
Write block A to DataNode: 1, 7, 9 2
3 Rack awareness:
Ac
k:

Re

Rack 1
B

ad
lo

Core switch Data node 1

y?
ck
A
re

Rack 2
ce
iv

Data node 7
ed

Data node 9
7
Ye
s:
Re

Switch 1 Switch 2

ed
eiv
ad

File: Hello.txt
y?

rec
Block A Block B Block C Data node 1 Data node 7

A
ck
Yes, ready? Block A
5 Write block A Blo
Block A
k:
Ac

4 Yes, ready? 6
Data node 9
Block A

Rack 1 Rack 2

C C
C

C
View publication stats