Cloud Computing Lab Manual

Installation of Opennebula
Aim:
To install opennebula in ubuntu operating system for creating virtualization in cloud.
Procedure:
Step 1. Installation of opennenula in the Frontend
1.1. Install the repo
1. Open Terminal (ctrl+alt+t) or from dashdoard type terminal
2. Here # indirectly tells to work on root.
$ indirectly tells to work on normal user
Add the OpenNebula repository: # - root user
# wget -q -O- http://downloads.opennebula.org/repo/Ubuntu/repo.key | apt-key add # echo "deb http://downloads.opennebula.org/repo/4.12/Ubuntu/14.04/ stable
opennebula" \
> /etc/apt/sources.list.d/opennebula.list
1.2. Install the required packages
# apt-get update
# apt-get install opennebula opennebula-sunstone nfs-kernel-server
1.3. Configure and Start the services
There are two main processes that must be started, the main OpenNebula daemon: oned, and
the graphical user interface: sunstone.
Sunstone listens only in the loopback interface by default for security reasons. To change it
edit
# gedit /etc/one/sunstone-server.conf
and change :host: 127.0.0.1 to :host: 0.0.0.0.
Now we must restart Sunstone:
# /etc/init.d/opennebula-sunstone restart
1.4. Configure SSH Public Key
OpenNebula will need SSH for passwordless from any node (including the frontend) to any

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

other node.
To do so run the following commands:
# su - oneadmin
$ cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys
Add the following snippet to ~/.ssh/config so it doesnt prompt to add the keys to the
known_hosts file:
$ cat<< EOT > ~/.ssh/config#Type the below commands
Host *
StrictHostKeyChecking no
UserKnownHostsFile /dev/null
EOT
$ chmod 600 ~/.ssh/config
Step 3. Basic Usage
The default password for the oneadmin user can be found in ~/.one/one_auth which is
randomly generated on every installation.
$ nano~/.one/one_auth

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Open mozilla firefox

localhost:9869
Enter Username : oneadmin
Password
: from ~/.one/one_auth (file)

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Result :
Thus, opennebula has been installed successfully.

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Find procedure to run the virtual machine of different configuration. Check

how many virtual Machines can be utilized at particular time
Aim:

To find the procedure to run the virtual machine of different configuration and to
Check how many virtual can be create.
root@linux:$ /etc/init.d/opennebula-sunstone restart
Procedures:
Step 1: Check the processor virtualization in boot settings.
Step 2: Execute all the commands in root user if the command start with #, and one
admin user if command start with $.
a. Moving to roo user in terminal
linux@linux:~$ sudo bash
[sudo] password for linux:
Enter the password.
Step 3: Checking the virtualization support in terminal from root user.
root@linux:~# grep -E 'svm|vmx' /proc/cpuinfo

Step 4: Checking the Kernel Virtual Machine location and availability

root@linux:~# ls -l /dev/kvm
crw-rw----+ 1 root kvm 10, 232 Jul 1 04:09 /dev/kvm
Step 5: Setting up the opennenubla dependencies and it's packages by downloading the
following using cammand.
root@linux:~# dpkg -l opennebula-common ruby-opennebula opennebula opennebulanode opennebula-sunstone opennebula-tools opennebula-gate opennebula-flow
libopennebula-java

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Have to install the missing packages with the following commands in terminal.
root@linux:~# sudo apt-get install opennebula-node
root@linux:~# sudo apt-get install opennebula-gate
root@linux:~# sudo apt-get install opennebula-flow
root@linux:~# sudo apt-get install libopennebula-java
Now check the dependencies by giving the cammand in terminal where all packages shows as
installed
root@linux:~# dpkg -l opennebula-common ruby-opennebula opennebula
opennebula-node opennebula-sunstone opennebula-tools opennebula-gate
opennebula-flow libopennebula-java

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Step 6: Checking the opennebula and it's services status.

root@linux:~# service opennebula status
* one is running

## package name

root@linux:~# service opennebula-sunstone status ### Web interface name sunstone

* sunstone-server is running
root@linux:~#service nfs-kernel-server status
nfsd running
root@linux:~#service opennebula restart
* Restarting OpenNebula cloud one
oned and scheduler stopped
[ OK ]
root@linux:~# service opennebula-sunstone restart
* Restarting Sunstone Web interface sunstone-server
sunstone-server stopped
VNC proxy started
sunstone-server started
[ OK ]
root@linux:~#service nfs-kernel-server restart
* Stopping NFS kernel daemon
* Unexporting directories for NFS kernel daemon...
* Exporting directories for NFS kernel daemon...
* Starting NFS kernel daemon
Step 7: Setting up the physical bridge interface(br0)
root@linux:~# ifconfig

Dept of ISE,Dr.AIT Bangalore-560056

[ OK ]
[ OK ]
[ OK ]

Cloud Computing Lab Manual

Step 8: Changing the network interface and bridge configuration manually. The
network configuration in the ubuntu is stored under /etc/network/interfaces
root@linux:~# gedit /etc/network/interfaces
It has only few lines.
# interfaces(5) file used by ifup(8) and ifdown(8)
auto lo
iface lo inet loopback
Now we have to change the br0.cfg file manually which is located at
/etc/network/interfaces.d/br0.cfg
root@linux:~# gedit /etc/network/interfaces.d/br0.cfg
And paste the following lines there.
#auto br0
iface br0 inet static
address 192.168.0.10
network 192.168.0.0
netmask 255.255.255.0
broadcast 192.168.0.255
gateway 192.168.0.1
bridge_ports em1
bridge_fd 9
bridge_hello 2
bridge_maxage 12
bridge_stp off
Now move the configuration file to the interfaces.
root@linux:~# cat /etc/network/interfaces.d/br0.cfg >> /etc/network/interfaces
Now, open and view the interface file.
root@linux:~# gedit /etc/network/interfaces
(or)
root@linux:~# cat /etc/network/interfaces
# interfaces(5) file used by ifup(8) and ifdown(8)
auto lo
iface lo inet loopback
#auto br0
iface br0 inet static
address 192.168.0.10
network 192.168.0.0
netmask 255.255.255.0
broadcast 192.168.0.255
gateway 192.168.0.1
bridge_ports em1

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

bridge_fd 9
bridge_hello 2
bridge_maxage 12
bridge_stp off
Step 9: Change the settings at /etc/network/interfaces as
# interfaces(5) file used by ifup(8) and ifdown(8)
auto lo
iface lo inet loopback
auto br0
iface br0 inet static
address 192.168.0.28
network 192.168.0.0
netmask 255.255.255.0
broadcast 192.168.0.255
gateway 192.168.0.1
bridge_ports p5p1
bridge_fd 9
bridge_hello 2
bridge_maxage 12
bridge_stp off
Step 10: Up the changes of bridge through
## Getting interface which changed for bridge bro by ifup br0
root@linux:~# ifup br0

#######error

Waiting for br0 to get ready (MAXWAIT is 20 seconds).

root@linux:~# ifconfig

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Step 11: Moving to Oneadmin user and doing configuration changes.

### Open opennebula in webinterface
root@linux:~# su - oneadmin
oneadmin@linux:~$ pwd
/var/lib/one ########default home dir for opennebula
Step 12: Checking the default configurations of opennebula.
oneadmin@linux:~$ oneuser list
oneadmin@linux:~$ oneuser show 0
## 0 represents the id of oneuser. Here usind id only we can do all activities like delete, show,
status and etc.,
oneadmin@linux:~$ onegroup list
oneadmin@linux:~$ onehost list

Step 13: Creating the kvm for localhost. It helps to create image, template and template
instantiate to work.
oneadmin@linux:~$ onehost create localhost -i kvm -v kvm -n dummy
ID: 0
#### - I -information driver v- virtual driver n network driver
oneadmin@linux:~$ onehost list

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

oneadmin@linux:~$ onehost show 0

oneadmin@linux:~$ ls -l
total 312
-rw-rw-r-- 1 oneadmin oneadmin 3339 Jul 1 05:04 config
drwxr-xr-x 5 oneadmin oneadmin 4096 Jul 1 04:12 datastores
-rw-r--r-- 1 dovenull nova
93 May 28 16:07 mynetwork.one
-rw-r--r-- 1 oneadmin oneadmin 287744 Jul 1 05:51 one.db
drwxr-xr-x 9 oneadmin oneadmin 4096 Jun 14 01:55 remotes
drwxrwxr-x 2 oneadmin oneadmin 4096 Jun 14 16:52 sunstone_vnc_tokens
drwxr-xr-x 2 oneadmin oneadmin 4096 Nov 26 2015 vms
Step 14: Create and Modifying the mynetwork.one at /var/lib/onemynetwork.one
oneadmin@linux:~$ sudo bash
root@linux:~# cd /var/lib/one

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

root@linux:/var/lib/one# gedit mynetwork.one

NAME = "private"
BRIDGE = br0
AR = [

# paste the below lines.

TYPE = IP4,
IP = 192.168.0.141,
SIZE = 5

]
root@linux:/var/lib/one# sudo - oneadmin
oneadmin@linux:~$ cat mynetwork.one
NAME = "private"
BRIDGE = br0
AR = [

TYPE = IP4,
IP = 192.168.0.141,
SIZE = 5

]
Step 15: Creating the virtualnetwork as onevnet and viewing it's properties.
oneadmin@linux:~$ onevnet create mynetwork.one
oneadmin@linux:~$ onevnet list
oneadmin@linux:~$ onevnet show 0

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Step 16: Installing Virtual Machines, before that check list of oneimage, onetemplate
and onevm.
oneadmin@linux:~$ oneimage list
ID USER
GROUP NAME
DATASTORE

SIZE TYPE PER STAT RVMS

oneadmin@linux:~$ onetemplate list

oneadmin@linux:~$ onevmlist

Step 17: Updating the .ssh for passwordless handshaking with oneadmin web service.
oneadmin@linux:~$ cat ~/.ssh/id_rsa.pub
## You can see the key, copy that fully and paste it in by visiting localhost:9869/
Click oneadmin and choose configuration and deselect the public ssh and finally paste it.

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Step 18: Creating Oneimage, onetemplate and one vm.

Move to the datastores folder.
oneadmin@linux:~$ cd datastores
oneadmin@linux:~/datastores$
Creating oneimage
oneadmin@linux:~/datastores$ oneimage create --name "ttylinux" path
"/home/linux/Downloads/source/ttylinux.img" --driver raw --datastore default
Creating One Template:
oneadmin@linux:~/datastores$ onetemplate create --name "ttylinux" --cpu 1 --vcpu 1 -memory 512 --arch x86_64 --disk "ttylinux" --nic "private" --vnc ssh
Instantiating OneVm (oneemplate)
oneadmin@linux:~/datastores$ onetemplate instantiate "ttylinux"

The above image before creating the vm. Refresh and check once the above commands are
executed.

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Step 19: Opening the VM through opennebula.

Click the corner computer symbol icon, it will and ask the username and password. By default
the username is root and password is password.

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Through terminal you can access the vm by

oneadmin@linux:~/datastores$ sshroot@192.168.0.141
and give passwrod

Step 20: Similarly you can create as much vm your machine supports and can access
only 5vm at a time since we limited our ip range upto 5 in mynetwork.one
You can install ubuntu, centos and etc.,

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Change the unbold data in the below cammand and install for various vm size.
Creating One Template:
oneadmin@linux:~/datastores$ onetemplate create --name "ttylinux" --cpu1 --vcpu 1 -memory 512 --arch x86_64 --disk "ttylinux" --nic "private" --vnc ssh
Instantiating OneVm (oneemplate)
oneadmin@linux:~/datastores$ onetemplate instantiate "ttylinux"

Result:
Thus the procedure for running the virtual machine of different configuration has
been successfully implemented in opennebula.

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Find procedure to attach virtual block to the virtual machine and check
whether it holds the data even after the release of the virtual machine.
Aim:
To Find the procedure for attaching virtual block to the virtual machine and check whether it
holds the data even after the release of the virtual machine.
Procedure:
Step 1: Create the Oneimage, onetemplate and onevm through the commands
Creating oneimage
oneadmin@linux:~/datastores$ oneimage create --name "ttylinux" path
"/home/linux/Downloads/source/ttylinux.img" --driver raw --datastore default
Creating One Template:
oneadmin@linux:~/datastores$ onetemplate create --name "ttylinux" --cpu 1 --vcpu 1 -memory 512 --arch x86_64 --disk "ttylinux" --nic "private" --vnc ssh
Instantiating OneVm (oneemplate)
oneadmin@linux:~/datastores$ onetemplate instantiate "ttylinux"
Creating oneimage
oneadmin@linux:~/datastores$ oneimage create --name "Ubuntu" path
"/home/linux/Downloads/source/tubuntu1404-5.0.1.qcow2c" --driver qcow2 -datastore default
Creating One Template:
oneadmin@linux:~/datastores$ onetemplate create --name "ubuntu1" --cpu 1 --vcpu 1
--memory 1024 --arch x86_64 --disk "Ubuntu" --nic "private" --vnc ssh
Instantiating OneVm (oneemplate)
oneadmin@linux:~/datastores$ onetemplate instantiate "ubuntu1"
Step 2: Power off Virtual os
oneadmin@ubuntu:~/datastores$ onevm poweroff 1
oneadmin@ubuntu:~/datastores$ onevm poweroff 2
oneadmin@ubuntu:~/datastores$ onevm list
Step 3: Starting service
oneadmin@ubuntu:~/datastores$ onevm resume 0
Step 4: Deleting created VM
After power off do the operations
oneadmin@ubuntu:~/datastores$ onevm delete 1
oneadmin@ubuntu:~/datastores$ onevm list
oneadmin@ubuntu:~/datastores$ onevm delete 2
oneadmin@ubuntu:~/datastores$ onevm list

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

ID USER GROUP NAME

STAT UCPU UMEM HOST
TIME
0 oneadmin oneadmin CentOS-6.5 Virt runn 0.0 512M localhost 0d 01h43
3 oneadmin oneadmin ttylinux Virtua runn 9.5 256M localhost 0d 00h05
Step 5: Deleting image, template and vm
Rollback the operations
For deleting the template
oneadmin@ubuntu:~/datastores$ onetemplate delete 1
For deleting the image
oneadmin@ubuntu:~/datastores$ oneimage delete 1
Step 6: Deploying, undeploying, disable and enabling the services of onehost.
oneadmin@ubuntu:~/datastores$ onevm undeploy 0,3,4,5
oneadmin@ubuntu:~/datastores$ onevm list
ID USER GROUP NAME
STAT UCPU UMEM HOST
TIME
0 oneadmin oneadmin CentOS-6.5 Virt unde 0.0 0K
0d 02h11
3 oneadmin oneadmin ttylinux Virtua shut 11.0 256M localhost 0d 00h34
4 oneadmin oneadmin Debian 7 Virtua unde 0.0 0K
0d 00h23
5 oneadmin oneadmin Ubuntu 14.04 Vi unde 0.0 0K
0d 00h21
oneadmin@ubuntu:~/datastores$ onehost list
ID NAME
CLUSTER RVM ALLOCATED_CPU ALLOCATED_MEM STAT
0 localhost
0
0 / 400 (0%) 0K / 3.7G (0%) on
oneadmin@ubuntu:~/datastores$ onehost disable 0
oneadmin@ubuntu:~/datastores$ onehost list
ID NAME
CLUSTER RVM ALLOCATED_CPU
0 localhost
0
- off

ALLOCATED_MEM STAT

oneadmin@ubuntu:~/datastores$ onehost enable 0

oneadmin@ubuntu:~/datastores$ onehost list
ID NAME
CLUSTER RVM ALLOCATED_CPU ALLOCATED_MEM STAT
0 localhost
0
0 / 400 (0%) 0K / 3.7G (0%) on
Step 7: Password generation for root user (Ubuntu)
For Passwrod Generation through root user-guide
[root@localhost ~]# ls -al
[root@localhost ~]# cd .ssh/
[root@localhost .ssh]# cat authorized_keys ssh-rsa
AAAAB3NzaC1yc2EAAAADAQABAAABAQCpZ7VExltM+8w36OsQZdzBsINiIRTBqU6934vS2wI
RZvhjzT4RO6QS314gG3K0ghFk4cVAlS8ykMMjqW11G0LtIIqMYaKUYOG4oWfiB2hkeQoGGJCP
hjMzsz3RKXkOsn/bzgo2iYXldiCTVLaj5d+c8ZxXHIErCK0K3AM2JYoeN/iR88nP6h8vCdJwaahp
cysggpKyHTAsJ+TBaXFl3TGhVH9W0AAw6qM/OA2+FNKqCnR+b57KI7fXeBBVc/MckJfjI5PQX
m+ZDrKa2LtFV9L5f71VvOmc8YWIBmDfZ2Bx/FcHuCEphq7Sh8WLNrLuqNW+Kf9lRcr33DBY
IROm9w2B root@gcc-server

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Result:
Thus the procedure to attach virtual block to the virtual machine and check whether it holds
the data even after the release of the virtual machine has been successfully implemented.

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Install a C compiler in the virtual machine and execute a sample program.

Aim :
To install a C compiler in the virtual machine and execute a sample program.
Procedure:
step1:
Install the centos or ubuntu in the opennebula as per previous commands.
Step 2:
Login into the VM of installed OS.
Step 3:
If it is ubuntu then, for gcc installation
$ sudo add-apt-repository ppa:ubuntu-toolchain-r/test
$ sudo apt-get update
$ sudo apt-get install gcc-6 gcc-6-base
Step 4:
Write a sample program like
Welcome.cpp
#include<iostream.h>
using namespace std;
int main()
{
cout<<Hello world;
return 0;
}
Step 5:
First we need to compile and link our program. Assuming the source code is saved in a file
welcome.cpp, we can do that using GNU C++ compiler g++, for example
g++ -Wall -o welcome welcome.cpp
And output can be executed by ./welcome

Result:
Thus the GCC compiler has been successfully installed and executed a sample program.

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Show the virtual machine migration based on the certain condition from
one node to the other
Aim:
To show the virtual machine migration based on the certain condition from one node
to the other.
Procedure:
Step 1: Open Opennebula service from root user and view in localhost:9869
root@linux:$ /etc/init.d/opennebula-sunstone restart
Step 2: Create oneimage, onetemplate and onevm as like earlier
Creating oneimage
oneadmin@linux:~/datastores$ oneimage create --name "Ubuntu" path
"/home/linux/Downloads/source/tubuntu1404-5.0.1.qcow2c" --driver qcow2 --datastore
default
Creating One Template:
oneadmin@linux:~/datastores$ onetemplate create --name "ubuntu1" --cpu 1 --vcpu 1 -memory 1024 --arch x86_64 --disk "Ubuntu" --nic "private" --vnc ssh
Instantiating OneVm (oneemplate)
oneadmin@linux:~/datastores$ onetemplate instantiate "ubuntu1"
Step 3: To perform a migration. We use onevm command with VM id as VID = 0 to
host02(HID=1)
oneadmin@linux:~/datastores$ onevm migrate --live 0 1
This will move the VM from host01 to host02. The onevm list shows something like the
following
oneadmin@linux:~/datastores$ onevm list
ID
USER
GROUP
NAME
0
oneadmin
oneadmin
one-0

STAT CPU
runn 0

MEM HOSTNAME TIME

0k
host02 00:00:48

Result :
Thus the virtual machine migration based on the certain condition from one node to
the other has been executed successfully.

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Find procedure to set up the one node Hadoop cluster

Aim:
To find the procedure for setting up the one hadoop cluster in the linux platform.
Procedures:
Step 1:
Download the latest sun java and apache hadoop from the official website.
Step 2:
To install Java and Hadoop follow the belo lines
######## 1. Install Java #############################################
a. Extract the Downloaded java tar.gz file in Downloads / Documents folder
b. Open Terminal by pressing ctrl+alt+t
c. In Terminal, type
$gedit ~/.bashrc
d. At the bottom paste the following lines by changing the path alone
#--insert JAVA_HOME
JAVA_HOME= /opt/jdk1.8.0_05
#--in PATH variable just append at the end of the line
PATH=$PATH:$JAVA_HOME/bin
#--Append JAVA_HOME at end of the export statement
export PATH JAVA_HOME

e. Save the configuring by giving command as

$ source ~/.bashrc
f. Check java has been successfully installed by typing
$java -version

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Step 3. Install ssh for passwordless authentication

For passwordless authentication we need to do certain changes by following the
below procedure and we need internet connection.
In Terminal: copy and the paste the below lines
$ sudo apt-get update
### It will ask your root password. Give it

$ sudo apt-get install openssh-server

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

$ sshlocalhost
### It also will ask ask root password

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

$ ssh-keygen(Don't mention any path during key generation)

$ ssh-copy-id -ilocalhost

Step 4. Installation Procedure of Hadoop

As like java, extract hadoop tar.gz file also and do the changes in bashrc file by copy
and paste the following line

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

a. Extract hadoop at java located folder itself (Downloads or Documents)

b. $ gedit ~/.bashrc
paste the following lines below java path (change the path)
#--insert HADOOP_PREFIX
HADOOP_PREFIX=/opt/hadoop-2.7.0
#--in PATH variable just append at the end of the line
PATH=$PATH:$HADOOP_PREFIX/bin
#--Append HADOOP_PREFIX at end of the export statement
export PATH JAVA_HOME HADOOP_PREFIX
c. save it by typing the below command in terminal
$ source ~/.bashrc
d. To check the installed path of Hadoop. Type the command
$ echo $HADOOP_PREFIX
e. Command is to get into the hadoop directory is
$ cd $HADOOP_PREFIX
f. To check the installed hadoop version
$bin/hadoop version

Step 5. Modifying the Hadoop configuration files

Do the things as like we did before using terminal
(i) cd $HADOOP_PREFIX/etc/hadoop
$ gedit hadoop-env.sh
(paste the java and hadoop path as the first two lines)
export JAVA_HOME=/usr/local/jdk1.8.0_05
export HADOOP_PREFIX=/opt/hadoop-2.7.0

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

(ii) Modify the core-site.xml

$ gedit core-site.xml
Paste the line within <configuration></configuration>
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>

(iii) Modify the hdfs-site.xml

$ gedit hdfs-site.xml
Paste the configuration file
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
(iv) modify the mapred-site.xml
$ cpmapred-site.xml.template mapred-site.xml
$ gedit mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

</property>
</configuration>
(v) Modiy yarn-site.xml
>gedit yarn-site.xml
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>

Step 7. Formatting the HDFS file-system via the NameNode

$ cd $HADOOP_PREFIX
$ bin/hadoopnamenode -format

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

after formatting, start the services

7. Starting the services.
$sbin/start-dfs.sh#######it will start services by taking some time and it will ask
permission give yes.
$sbin/start-yarn.sh
else
$ sbin/start-all.sh
to check running services
>jps
3200 DataNode
12563 Jps
4036 ResourceManager
4172 NodeManager
5158NameNode
3685 SecondaryNameNode

Step 9. Stopping Services

>sbin/stop-dfs.sh
>sbin/stop-yarn.sh
(or)
>sbin/stop-all.sh
Once you start the services after stopped means it shows only 4 services.
>jps
12563 Jps

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

4036 ResourceManager
4172 NodeManager
3685 SecondaryNameNode
Step 10. only four services will run. To start datanode and name node we have to add some
lines in hdfs- site.xml
In Terminal
$ cd $HADOOP_PREFIX/etc/hadoop
$gedit hdfs-site.xml
(Paste the below lines)
<property>
<name>dfs.namenode.name.dir</name>
<value>/opt/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>/opt/data</value>
</property>

Next do these procedures, for creating permanent log storage to namenode and datanode.
$ sudomkdir /opt/name
$ sudomkdir /opt/data
$ ls /opt/
$ ls -l /opt/ ##To change the directory from root user to admin user
$sudo chown vrscet:vrscet -R /opt (root should be replaced by your system
username)
$ ls -l /opt/

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Step 11. Format the namenode

$ cd $HADOOP_PREFIX
$ bin/hadoopnamenode -format
Step 12. Start services
$sbin/start-dfs.sh
$sbin/start-yarn.sh
$jps
3200 DataNode
12563 Jps
4036 ResourceManager
4172 NodeManager
5158 NameNode
3685 SecondaryNameNode
Step 13. To view in Browser (Open Firefox and enter the below address)
localhost:50070
localhost:8088

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Result:
Thus single node hadoop cluster has been successfully created.

Dept of ISE,Dr.AIT Bangalore-560056

Cloud Computing Lab Manual

Write a program to use the API's of Hadoop to interact with it

Aim:
To write a program for using the API's on hadoop and as well as to interact with it.
Procedure:
Step 1:
Start the hadoop services by giving the following command in terminal
$ sbin/start-all.sh
$ jps
Step 2:
Open web browser and open

localhost:50070
localhost:8088

Step 3:
Creating folder in web interface (HDFS) from terminal.
$ bin/hadoop fs -mkdir /bala
Wait until the command executes.
Step 4:
Open the localhost:50070
Utilities --> Browse the filesytem.
An folder has been created which we had given in terminal
bin/hadoop ----> represents the location of hdfs
fs
---> file system
-mkdir
------> create a folder
/
------> root in hdfs
bala
----> folder name

Step 5: Loading the data into the folder we created in hdfs

$bin/hadoop fs -copyFromLocal /home/bala/Pictures /bala2
Open web browser and under utilities, browse the filesytem and check whether the content is
moved

Dept of ISE,Dr.AIT Bangalore-560056