Lab – Hadoop Distributed File System

In this lab we will do some more hands-on work with hdfs.

HDFS works one level above the local file system. Hadoop hdfs commands are like Linux commands (ls) with – (-ls) before the command.

Similar to the text file you can also upload any binary file (for e.g. image file) to the hdfs.

To do this exercise the sample binary file (hdfs-binary-dihub.JPG) is available in Documents/dataMR directory. You can use -put and -copyFromLocal commands interchangeably.

1. Use -put command to upload the provided binary file to hdfs.

hdfs dfs -put hdfs-binary-dihub.JPG /

2. To check the path of hadoop jars and default library modules use hadoop classpath

3. To delete trash entries from hdfs you can run the command hdfs dfs -expunge

4. To get help on hadoop commands you can run hdfs dfs -help

5. To get help on any specific command you can run -usage option with command name, for example hdfs dfs –usage -copyFromLocal

6. You can explore few more available options with hdfs dfs -help command as shown:

7. You can explore few more options available with hdfs dfsadmin available command.