MAP REb0CE fRAMEWoRK

Mahleduu dhat
oMables naaowe dealablity aeroas hundeds
tadboj cust.
1aenmNapRedue"heles to tiro sbanate and
Aldinck taok tlat todop rgan fuform.
Nheh tates a did
onet itnsth t a dae,
date and doun
we inolindual elemnane boteh
jnla tubles CReylveietia). oilbut
hediee job thd takes the
Le othu job ond combins thee dot
a map as nbut dct As the queme
into a malle the Aeoluce o

the jb.
mas
aloay tegemed agta
have been diocuool in
it
MaReduee:- he tuo bagst
Advadage ae
slnudag Malkeduce
Mabfeduee, ie fob do

Rocesainýt-Ju uode
Lfoallel mutiple nwdei and each
diided ameng dmettaneoulyt
dinide and Congnn
MapReduee baod on dha dala
whieh helbeJo to beeios
 a hemenduo amsut

Dala ock
Slave A
A

Slaves
Slaye
Rate Blek
MAS TeR

Stave c
Slare9
1. Thaolitial Aphoachi 2 MabRaduc Aten
Move dat to the
it

2ta dat to proeoaing

ality:- Maviuyktom. it,
tollediu thasttnal But a dat e geut
amot af dat
and became
and beanme huge bningig age
he ihicutl
beeame

dat to coaty
dzttnioat ten uformanee
>Precegtahe tinne as the dat i bocd
beeoms the bottlenek
by a dingl nt ohiet
and
’ Maste Msele cn get over-burdeneal
ail
Theefen, Mableduee Koabtue the isaus by bringing
He brocosing nt to the data. he data i'distbit

dota hearding on
Cn it Shi allese he follbuiag
adiaalage:
it to he da.
time o keduesd as al. the voela

yfuy noele gat a bat o the data t þacias

AMd thieoe, thew io ne chance
nd a aecle gotting
Ovebusdened.
Mableduee egangsfes eal atedat -

an boca kelabyle
alatt blueineas itibued tile sen
dsu stone da the ttadoop
CHDPS). eaai aees to multie
flayiblity: Hadoop enades
L multible typu date.
AAea data amd minimm
lwh paallel poaing cnd
L Speed:
Maosie amounto a data.
Jaua, Ct aud yton:
bagnge, ineluding
 detil

a Map Rudue dplcatioui

De,elohing heduce, dhiu ntion
Wite mah, dataatt
"Jet w a dnmall ubt

otlem it failo debug it

and iy
Run on full dataat topls.
haiop dekaggg
to tine the buyemamea dy the brogam.
MARREDUCE IN A 7VUTSHEL
Under
Sease

-atit sNOILINOJ Map

Tkeemmedk
tie HReduce
Goo le
HDES
MAP
REDVCE Apache
Hadoof]
LFEATURES
ArogaW
Sumnagatioh
PATTERNS
g-Invled
Inde
DESIGN
Lage seale
Cafieatbr
bitibuted Mlodol
Remmendaliord
Sot
Jualee
gToü, Selction
 - HOPS so a

CHDP) wolo
hardae Astibuted fle sisten iw mind:
dhetadsos dhe
duýad shauld stee bge

aud

built hettie dataatd

balte aning the tne to Jead
latiney
botant tham the
tan buly
Bistisg fles on the ystem fle at a
WRLR, end Medying a
abbinded to the vey
fe
Harahuae; tadeot hardweo
um on kaadwau.

o higi but the teneepecttdl to eontinue

in minclin) with hal
jnoking ettiy. keejig tolvant pcdoitred to be
desleyed o Uosoot harclwee
(onest al HDES:
deta Boka- he data bleeke au the
daa thd an be sead or witk
delautt igoa he HD3 blck io j2Rbo
he
black aige is changable. Uhe dat e diidd
ctol atered in thu
uo maller than the lbok
the daa Age
t well not euby thu whble bleok. hen.
Vame Nad ate Nocle

Dati
Stocgkefott

-Kead |aingle
Whate Strai
dsleal Oat
ient
Strage.
Stuetuad
vaiaus
OVERVIEW Of HDES. Soes

" NemeNede:- he NameAeolo o te the Maot ml

Centalle n the managu
dtone he data
Jot it stoe dhe metadala o al iles
 Jhese noles ae ihe the wcokenoda
Yhese stDac theaota loeks. Qata Nada
nyomalisu about the data blocbs
okoct otel they sbe ete) to tho ame cle
(he a
iot of date
yeiodáally.
OmnnBdidy harawase.

kebléalióu:- HDAS upieate dat blel o bide

jau. Namewad' io Reabonaible yor maimtaining
nauing
sebluatis gact andeplatd the elut
aclequately aas
sihgiby hy chckaun
at blbtk level.
olat
Shell:- Hadsos vido alleust ons to
Hadert tadeo} dhell, whveh
Jniesae caled vaiaus conaoo.
cdieitaie, and bmleesus,

name nsde,or n

Ahaitabety CHA) achcteeloe duo NaeNodlo n

"lonmalved HA Jnvoe rug Jhanad storage
tuie standby contiguaton, withe
m do enose thisystene hemqno available, ee
 and Nodlei
Jh HDrS, he Keoeure Mamagernkitecle,
ae hey toponento YARG w
whh
mamaemaitand
tadsop.
dcheduling
"Reoewsce Manage:
he Resawee NManags Ja the ceal auttty
thel mmgu and deheoluls esoes acrcy
cuse Shedue
t cowib tus man omfonint
Agpliatünaauat
’he sehedur hnoiblear aeting
onlau
AUnning aphlatns baod
do vaxiauo ohotiainb, and aehauling sleie
esQsee linik, restonaible Jor
-The Abblcalbns hanagr io
submitino, neguiating thu fat
aceubting yjob
fmiding the
Mazt, and then
Abþlcala ith dhe neclosauy Je
contin theabbliatou taok
o a

in yARN Oehiteetut, telbane

to enaue faut
High trailibutiy mod

1ofenoiblef maugng ucu

- The Node Mauagu cluti.
nthe Angle node w the Hadoot

utligaliew
alaw (CPU,
bligKode menmoy, duh, nitwek) o
 etg good haatth to Ahe
aud eeutig tagks on dhe moole.
mauag and
MAole
’ he. Naole mauag commieat uit Reouce
esaltiate
coctiinesavd hebeta back hi tatlo athe'
cOtaines aud noole health
Aa0fonaible for launchig and
Maitoong ijeeycle au keguitd
by Keaawset Managea.
Coutaim- Roprte aw alleegld cacunce
th shet Keoece Menag manag dehela and
AeADUes aeieas the clustiy while bade Managu

disbibutd poig
mngnt,
and Challengug

HOPS:
eftinas: he Balsasdl that tees
1sCoot
tage oto Auo, bicauee HoR&

dae.
 dloitd b oletiet faulti and aumatully Ales

t Podabuty: HDE portable aersse all hudua

folasle
biatfonu bd t onjalible with deeal ae
windle lin ad Macls,
inelading Acceost- HDRS io built fo hial
5.6- Seaning Sete aces to cteanl
dada
dae
Chalenjes
milioeonh

oith tDfs.
werk well
o
Siee mmsnode heldu fle
& Lot ad Small Files: limit o the o.
ment, the
dystem netadati w govend by Iu
' ile n a fole dyiem b
the amnode.
memeny
fla modifiatbnoi
8 Muttje, wiba, anbitay
Can be niten by a cigle
iles i HDES a away at the enl ythe
wAitees, Na~te dubpotfo multble wwita
madiyiatin at
at arbihay fd iu
 .Bock i Blech Abstatibu
File ige
dloocuaol
HORs Block1 bl0ek an
HDPs has dat
Jv breris tokes.
sde as wel ao
the
eoeh
elust dehuude pow
cabacity d entredtt tha willbe dleed
the sige 4 uohoe
ky Hoßs.
DeS the difault block ig
hH aye an be wentgud
Houener , th bleck
aetediug te thu Rugueement
the adnin
ae finesiged
wheh
’The data bleck, blcku, ae the Jndamen
magtanoes he whote
to steigdatd
tal uit
dividud nto Bne
o
HoPs theeloks ae
blk, and
Oh meu custena nodofer
the
diotibitd aro aalleel oeoeing
fedundanyand effieint at
hi abcastowhel hetieveal withi
dtoage,heblioaton,and
 TANoDEI DA
Metada &3 slestxt
3 B
4,2
usa\Ekstxt
A
Namelede
ATANGDE BLecks NAME FLE
metadate. the
the eae la&mB Jno
ins mamensde
dii d has that blech
Bleka Data
28 i28
Ms
BaBiB,
B Ba B
eaeh
on
Diris 8MB I2
IGigaBye PIEE INPU T
the AgeMaw
tho agMO, io 1Gigabyle
l024 trample,
= Po
terante.
he MB
thew
 Unit: 4
NOSL 9ATA BAS ES -
The tem NO SQL Aatabaes' ateu to

a
mechamiom tor stonage
data Awwaa owat otha
and
tha tabula
setieval
selatons MMpolel. Med en helatioval databaa
datab doentue tabies jo
dtiigdata.
el to stoe big data and
t io genally
Seal-kin neb applatiw
Flavible dehemer e. no pvedefieol or igid
wohena.
laay to dcale
t areiobd jais nd ds
Tyse NaSQL tabaoe
NosgL databao mainly catgeriged ino
au tp Key-alue Pai, Colwmn-oniied,
(afh Based

Nodtt Relatitmahjo
msle Jéxample enambe)
JCodotD8, Kasandua) Bramle
Neot ntogd,
Sow Patabe Bntenik Gr.
 NOSÌL
databdola a
are brimmanly|
datbaos Reldtia t. NOL
| S9L DBnson
callec NDn-Relate
DB2
called R onal or dibutd
Aatabas 2NosoL databaA c
be doeumntbaaed,
table-baaed atab y valebaaed
key
babed databaos
HorjentallyScalable
Saaasie
B. Veucaysealable
Rerible Sahens
hedetined 4
fxed or
t.dehema
.uilablo ou
5-Not table fo ieuanehial dal
tiesancheal data
Zéaplesi-ORacleMO, 6 stage
6ecples-Megobe,
6ewe,and bochDe,
Ke-Value-Reoli ymamle
MibLcsat a9L Collumn-Baod-HRao Caosande
|botge'sg lGtaph-NeoteAmago Meftine
Sca
a tabags?uom
Ned fo NasgL Fwnbbe
Requi
NasgL,dathag ae neumstnces t-
Jhe
Jblemend in follawing
es neeo to be
whn a huge amet
2stored and, atiea.
dat yeu sthud
biten the
Qheseetonchip onatttwta
time and
chang ae keguid
he clata suttet & not
Constainti and Join
a datbto level..
 he olaa i conttnously amd
do beale the tp hand
heed Data
thedata
Jncenhang fona JseN-Jovaseibt
o a bepula obndoree NY SQL
Atoniented datibaoe rogam. t eule TeON
dooument uwith obtinal idehemas, making
dewnent
ke
gaatin yer cutah
neauneaaie and
t
pB stoes oat in
"paLotabaals hengo
meaning flo
Cble, JeoNlihe doeumetdemetandl
fm domento to
Bn veny chamged ve tme his
sthutue canbe
ds

KeyFatures: alenible,
hiented late ntd
MongoDB Hawe daa
L0eLUmat
Lleumot
ISON-ike doumento that
allsu dvclatou ts
stuctasJhis teibltty
aeheal Aolalionahips
abunt comblor Aier
hie
seale-oit
oalabtb-MonoDB isallagad to
mee sve to
iontall bu adoing seangcan
deloynet dAlo horigontal
jebo date cnd taftic.
*ndle lage Volumes