HDFS Operations and Commands

As a file system HDFS works as a storage component in hadoop to serve all read and write requests.

The HDFS operations  includes copy from local file system to hdfs, copy from hdfs to local file system, checking the list of available files / directories on hdfs, reading any file content on hdfs, deleting a file and so on.

You can perform these operations with the help of commands as described below:

To copy any file from local file system to hdfs use the command –copyFromLocal or -put with two parameters i.e. file location on local file system followed by desired location on hdfs.

hadoop fs -copyFromLocal /local_path /hdfs_path

Here we are uploading Documents/dataMR/bigdatafile-dihub.txt file from our local file system into hdfs.

To download / copy the data from hdfs to local file system the corresponding command will be -copyToLocal with hdfs path and local file system path parameters given accordingly.

You can read the content of any file on hdfs with the help of cat command. We can also pass the results to any other Linux command using pipe operation. If we wish to check the content of the file “bigdatafile-dihub.txt” which was uploaded to hdfs in previous command, we can check

hdfs dfs -cat /bigdatafile-dihub.txt

This file is very big, so if we just want to check the way in which we can access the content of files from hdfs, we can run this command to show only 10 entries for which we need to pipe it with head command as shown

hdfs dfs -cat /bigdatafile-dihub.txt | head -10

To delete any file from hdfs give the command -rm with file path and hdfs prefix to the rm command as shown in the given screenshot.

hdfs dfs -rm /bigdatafile-dihub.txt