Make sure to subscribe to our newsletter and be the first to know the news.
Related Posts
Make sure to subscribe to our newsletter and be the first to know the news.
HDFS Commands are similar to what we see in Linux commands. This blog we are going to concentrate on commands related to Creating and Listing Directories and Files in HDFS
Here is the list of the commands
| List Files and Directories | |
|---|---|
hadoop fs -ls /user |
List all the folders under the given directory, in this case all the files and directories under user folder will be displayed |
hadoop fs -ls -d /user |
List all the folders under the given directory, in this case all the files and directories under user folder will be displayed with details |
| hadoop fs -ls -h /user | Format file sizes in a human-readable fashion (eg 64.0m instead of 67108864). |
| hadoop fs -ls -R /user | Recursively list all files in hadoop directory and all subdirectories in hadoop directory. |
| hadoop fs -ls /user/logs* | List all the files matching the pattern. In this case, it will list all the files inside hadoop directory which starts with ‘logs’. |
| File Creation and Management | |
|---|---|
| hadoop fs -mkdir /user/codewithz | Creates a directory by name name codewithz under /user in HDFS |
| hadoop fs -rmdir /user/codewithz | Removes the directory from the specified directory |
| hadoop fs -cp /user/codewithz/log.txt /user/codewithz/new | Copies the file from source to destination. For instance here log.txt is copied from /codewithz folder to /new folder |
| hadoop fs -cp -p /user/codewithz/log.txt /user/codewithz/new | Copies the file from source to destination. For instance here log.txt is copied from /codewithz folder to /new folder -p preserves access and modification times, ownership and the mode. |
| hadoop fs -cp -f /user/codewithz/log.txt /user/codewithz/new | Copies the file from source to destination. For instance here log.txt is copied from /codewithz folder to /new folder -f overwrites the destination if it already exists. |
| hadoop fs -mv /user/codewithz/log.txt /user/codewithz/new1 | Moves the file from source to destination. For instance here log.txt is moves from /codewithz folder to /new1 folder |
| hadoop fs -rm /user/codewithz/log.txt | Deletes the file |
| hadoop fs -rm r /user/codewithz | Deletes all the file from the specified directory and any content under it recursively |