Hadoop File System Management Project
Task 1: Verify Hadoop Daemons and Version
1. Check the running Hadoop daemons:
o Ensure that all Hadoop daemons, such as Name Node, Data
Node, Resource Manager etc. are running properly by lis ng
all Java processes related to Hadoop.
2. Check Hadoop Version:
o Print the Hadoop version to con rm you are using the correct
version for this project.
Task 2: HDFS Setup and Basic Commands
3. List the contents of the HDFS root directory:
o Before you begin, list all the contents of the root directory in
HDFS to see what’s available.
4. Create a new directory in HDFS:
o You need to create a directory named /project_data in the
root of HDFS, where all project-related les will be stored.
Task 3: File Upload and Retrieval
5. Upload les to HDFS:
fi
fi
fi
ti
o You have two local les project.txt and data.csv located in /
home/cloudera/. Upload these les to the newly created /
project_data directory in HDFS.
6. Verify le upload:
o A er uploading, list the contents of the /project_data
directory to con rm that the les have been successfully
uploaded.
7. View le content in HDFS:
o You need to verify the content of the project.txt le in HDFS to
ensure it was uploaded correctly. Display the content of the
le.
Task 4: HDFS Storage and Space Management
8. Check HDFS storage usage:
o Check how much space is used and available in the HDFS le
system a er the uploads.
9. Count les and directories:
o Count the number of directories, les, and the total bytes in
the /project_data directory.
Task 5: File Management and Cleanup
10.Move a le within HDFS:
o You need to organize the les by moving project.txt to a new
subdirectory named /project_data/text_ les.
Task 6: File Retrieval from HDFS
fi
ft
fi
fi
fi
fi
ft
fi
fi
fi
fi
fi
fi
fi
fi
fi
13. Download a le from HDFS to local:
o download the project.txt le to your local desktop (/home/
cloudera/Desktop) for backup.
fi
fi
