Snapshots, quotas and trash

The file that was deleted on purpose, by mistake

Snapshots, quotas and trash

Words you'll meet in this lesson

From our lab (Hadoop 3.5.0)

Protection at three levels

Which protection for which problem?

Commands at a glance

Delete, restore, limit, recycle

All commands ran on the namenode container of the lab cluster; the full script is in “Run it on your PC”.

1. Make a folder snapshottable and take a snapshot

   hdfs dfsadmin -allowSnapshot /user/hadoop/results
   time hdfs dfs -createSnapshot /user/hadoop/results before_results

   Allowing snapshot on /user/hadoop/results succeeded Created snapshot /user/hadoop/results/.snapshot/before_results real 0m1.727s
   The NameNode only records the snapshot; no blocks are copied, so a folder with a million files snaps just as fast. The 1.7 s is the Java client starting.

2. Delete for good, then restore

   hdfs dfs -rm -skipTrash /user/hadoop/results/2026/sem1.csv
   hdfs dfs -ls /user/hadoop/results/.snapshot/before_results/2026
   hdfs dfs -cp /user/hadoop/results/.snapshot/before_results/2026/sem1.csv /user/hadoop/results/2026/
   hdfs dfs -cat /user/hadoop/results/2026/sem1.csv

   Deleted /user/hadoop/results/2026/sem1.csv -rw-r--r-- 3 hadoop supergroup 25 2026-10-21 16:22 /user/hadoop/results/.snapshot/before_results/2026/sem1.csv roll,marks 101,78 102,91
   Even -skipTrash couldn't destroy the data: the snapshot still referenced the file's blocks. Restoring is a normal copy out of the read-only .snapshot view.

3. What changed between two snapshots?

   printf '103,66\n' | hdfs dfs -appendToFile - /user/hadoop/results/2026/sem1.csv
   hdfs dfs -createSnapshot /user/hadoop/results after_append
   hdfs snapshotDiff /user/hadoop/results before_results after_append

   Difference between snapshot before_results and snapshot after_append under directory /user/hadoop/results: M ./2026 + ./2026/sem1.csv - ./2026/sem1.csv
   The folder was modified (M); the old sem1.csv was deleted (−) and a new one created (+), because our restore made a new file with the same name. DistCp uses exactly these diffs to copy only changes to a backup cluster.

4. Snapshots protect themselves

   hdfs dfs -rm -r -skipTrash /user/hadoop/results/.snapshot/before_results
   hdfs dfs -rm -r -skipTrash /user/hadoop/results
   hdfs lsSnapshottableDir

   rm: Modification on a read-only snapshot is disallowed rm: The directory /user/hadoop/results cannot be deleted since /user/hadoop/results is snapshottable and already has snapshots drwxr-xr-x 0 hadoop supergroup 0 2026-10-21 16:22 2 65536 /user/hadoop/results
   Nobody can edit a snapshot, and a folder with snapshots can't be removed. lsSnapshottableDir shows 2 snapshots and the limit of 65,536. To free space, an admin must explicitly -deleteSnapshot.

5. Name quota: count the folder too

   hdfs dfsadmin -setQuota 3 /user/hadoop/quota_lab/names
   for i in 1 2 3; do echo x | hdfs dfs -put - /user/hadoop/quota_lab/names/f$i.txt; done

   put f1 ok put f2 ok put: The NameSpace quota (directories and files) of directory /user/hadoop/quota_lab/names is exceeded: quota=3 file count=4
   The folder itself is one of the 3 names, so only 2 files fit. Name quotas protect the NameNode's memory from tree explosions.

6. Space quota: the 384 MB surprise

   hdfs dfsadmin -setSpaceQuota 10m /user/hadoop/quota_lab/space
   echo x | hdfs dfs -put - /user/hadoop/quota_lab/space/tiny.txt
   hdfs dfsadmin -setSpaceQuota 1g /user/hadoop/quota_lab/space
   hdfs dfs -put /tmp/canteen_orders.csv /user/hadoop/quota_lab/space/
   hdfs dfs -count -q -h -v /user/hadoop/quota_lab/names /user/hadoop/quota_lab/space

   put: The DiskSpace quota of /user/hadoop/quota_lab/space is exceeded: quota = 10485760 B = 10 MB but diskspace consumed = 402653184 B = 384 MB put ok QUOTA REM_QUOTA SPACE_QUOTA REM_SPACE_QUOTA DIR_COUNT FILE_COUNT CONTENT_SIZE PATHNAME 3 0 none inf 1 2 4 /user/hadoop/quota_lab/names none inf 1 G 1010.6 M 1 1 4.5 M /user/hadoop/quota_lab/space
   A 2-byte file failed a 10 MB quota: when a block is allocated, HDFS reserves a full block (128 MB) × replication (3) = 384 MB until the block is finished. With 1 GB, the 4.5 MB file fits and uses 13.4 MB of quota (3 replicas), leaving 1010.6 M.

7. Trash: move, restore, expunge

   hdfs getconf -confKey fs.trash.interval
   hdfs dfs -D fs.trash.interval=1440 -rm /user/hadoop/quota_lab/names/f1.txt
   hdfs dfs -mv /user/hadoop/.Trash/Current/user/hadoop/quota_lab/names/f1.txt /user/hadoop/quota_lab/names/
   hdfs dfs -D fs.trash.interval=1440 -rm /user/hadoop/quota_lab/names/f2.txt
   hdfs dfs -D fs.trash.interval=1440 -expunge -immediate

   0 Moved: 'hdfs://namenode/user/hadoop/quota_lab/names/f1.txt' to trash at: hdfs://namenode/user/hadoop/.Trash/Current/user/hadoop/quota_lab/names/f1.txt -rw-r--r-- 3 hadoop supergroup 2 2026-10-21 16:22 /user/hadoop/quota_lab/names/f1.txt -rw-r--r-- 3 hadoop supergroup 2 2026-10-21 16:22 /user/hadoop/quota_lab/names/f2.txt Moved: 'hdfs://namenode/user/hadoop/quota_lab/names/f2.txt' to trash at: hdfs://namenode/user/hadoop/.Trash/Current/user/hadoop/quota_lab/names/f2.txt Created trash checkpoint: /user/hadoop/.Trash/261021162255 Deleted trash checkpoint: /user/hadoop/.Trash/261021162255
   With trash on, -rm is a rename into .Trash/Current, so restoring is another rename. -expunge rolls Current into a timestamped checkpoint (yyMMddHHmmss) and deletes expired checkpoints; -immediate deletes them now.

How a snapshot keeps a deleted file alive

Six steps inside the NameNode: the live tree, a snapshot, a delete, and a restore.

Protection at work

Simplified; organisations differ in tools and policies.

A copy-on-write snapshot in Python

The live folder is a dictionary of file names to block IDs. A snapshot records nothing at first; when a file is deleted or replaced, the old entry is saved into the snapshot's difference list. Blocks are freed only when nothing refers to them.

live = {"sem1.csv": "blkA", "sem2.csv": "blkB"}
snapshots = {}

def create_snapshot(name):
    snapshots[name] = {}                          # empty: nothing copied

def delete(fname):
    for diff in snapshots.values():
        diff.setdefault(fname, live[fname])       # remember the old version
    del live[fname]

def view(name):
    return {**live, **snapshots[name]}            # live tree + saved differences

def referenced_blocks():
    return set(live.values()) | {b for d in snapshots.values() for b in d.values()}

create_snapshot("before_results")
delete("sem1.csv")
print("live            :", live)
print("snapshot view   :", view("before_results"))
print("blocks kept     :", sorted(referenced_blocks()))

• snapshots[name] = {} # empty: nothing copiedCreating a snapshot is instant because it copies nothing. Our real snapshot took 1.7 s, almost all Java start-up.
• diff.setdefault(fname, live[fname]) # remember the old versionCopy-on-write: only when something changes is the old version recorded, and only once per snapshot.
• return set(live.values()) | {b for d in snapshots.values() for b in d.values()}blkA is still referenced by the snapshot, so it isn't deleted. That is why snapshots can hold disk space after files are deleted.

Run it on your PC: snapshots, quotas and trash

Run this against the Module 5 lab cluster. Copy canteen_orders.csv into the namenode first (docker cp canteen_orders.csv lab5-namenode-1:/tmp/). The script cleans up its own folders from earlier runs, so you can run it again.

#!/usr/bin/env bash
# protect_lab.sh: snapshots, quotas and trash on the lab cluster.
NN="docker exec lab5-namenode-1"
run() { echo "\$ $1"; $NN bash -c "$1" 2>&1 | grep -v -E "^\s*$|WARN|^\s+at |INFO  (Configuration|deprecation)"; echo; }
D=/user/hadoop/results
run "for s in before_results after_append; do hdfs dfs -deleteSnapshot $D \$s > /dev/null 2>&1; done; hdfs dfsadmin -disallowSnapshot $D > /dev/null 2>&1; hdfs dfs -rm -r -f -skipTrash $D /user/hadoop/quota_lab > /dev/null; hdfs dfs -mkdir -p $D/2026 && printf 'roll,marks\n101,78\n102,91\n' | hdfs dfs -put - $D/2026/sem1.csv"
echo "=== SNAPSHOTS"
run "hdfs dfsadmin -allowSnapshot $D"
run "time hdfs dfs -createSnapshot $D before_results"
run "hdfs dfs -rm -skipTrash $D/2026/sem1.csv; hdfs dfs -ls $D/2026"
run "hdfs dfs -ls $D/.snapshot/before_results/2026"
run "hdfs dfs -cp $D/.snapshot/before_results/2026/sem1.csv $D/2026/ && hdfs dfs -cat $D/2026/sem1.csv"
run "printf '103,66\n' | hdfs dfs -appendToFile - $D/2026/sem1.csv; hdfs dfs -createSnapshot $D after_append > /dev/null; hdfs snapshotDiff $D before_results after_append"
run "hdfs dfs -rm -r -skipTrash $D/.snapshot/before_results"
run "hdfs dfs -rm -r -skipTrash $D"
run "hdfs lsSnapshottableDir"
echo "=== QUOTAS"
Q=/user/hadoop/quota_lab
run "hdfs dfs -mkdir -p $Q/names $Q/space && hdfs dfsadmin -setQuota 3 $Q/names && hdfs dfsadmin -setSpaceQuota 10m $Q/space"
run "for i in 1 2 3; do echo x | hdfs dfs -put - $Q/names/f\$i.txt && echo put f\$i ok; done 2>&1 | grep -E '^put'"
run "echo x | hdfs dfs -put - $Q/space/tiny.txt 2>&1 | grep -E '^put'"
run "hdfs dfsadmin -setSpaceQuota 1g $Q/space && hdfs dfs -put /tmp/canteen_orders.csv $Q/space/ && echo put ok"
run "hdfs dfs -count -q -h -v $Q/names $Q/space"
echo "=== TRASH"
run "hdfs getconf -confKey fs.trash.interval"
run "hdfs dfs -D fs.trash.interval=1440 -rm $Q/names/f1.txt"
run "hdfs dfs -ls -R /user/hadoop/.Trash | grep f1.txt"
run "hdfs dfs -mv /user/hadoop/.Trash/Current$Q/names/f1.txt $Q/names/ && hdfs dfs -ls $Q/names"
run "hdfs dfs -D fs.trash.interval=1440 -rm $Q/names/f2.txt"
run "hdfs dfs -D fs.trash.interval=1440 -expunge -immediate 2>&1 | grep -oE '(Created|Deleted) trash checkpoint: .*'; hdfs dfs -ls /user/hadoop/.Trash"

# Run:  bash protect_lab.sh

Checking a space quota while writing

Follow the NameNode's quota check as a client writes a small file into a folder with a 10 MB space quota.

Worked examples: real problems, solved and traced in memory

Three protection calculations: the space snapshots really cost, a sensible quota, and a snapshot schedule.

20 TB · ×3 · +25%

setSpaceQuota 75t

50 TB · 2%/day · 7 days · ×3

about 7 TB of old data, 21 TB raw

days 1–30, Sundays every 7 days back from 30

keep: [9, 16, 23, 24, 25, 26, 27, 28, 29, 30]
delete: 20 snapshots

Check yourself

Quick revision

Interview and viva questions

Practice problems

Protection exercises using the real messages and numbers from the lab.

folder, snapshot, file

/user/hadoop/results/.snapshot/before_results/2026/sem1.csv

d, s, f = "/user/hadoop/results", "before_results", "2026/sem1.csv"
print(f"{d}/.snapshot/{s}/{f}")

1024 M − 1010.6 M

used 13.4 M raw -> 4.5 M of data

used = 1024 - 1010.6
print(f"used {used:.1f} M raw -> {used / 3:.1f} M of data")

128 MB × 3

402653184 bytes

print(128 * 1024 * 1024 * 3, "bytes")

3 lines

Counter({'M': 1, '+': 1, '-': 1})

from collections import Counter
out = """M	./2026
+	./2026/sem1.csv
-	./2026/sem1.csv"""
print(Counter(line.split()[0] for line in out.splitlines()))

600 × 20 MB

names: 601 of 1000 | raw: 35.2 GB of 30 GB

files, size_mb, rep = 600, 20, 3
names_needed = files + 1
raw_gb = files * size_mb * rep / 1024
print("names:", names_needed, "of 1000 | raw:", round(raw_gb, 1), "GB of 30 GB")

yyMMddHHmmss + 1440 min

created 2026-10-21 16:22:55 -> deleted after 2026-10-22 16:22:55

from datetime import datetime, timedelta
t = datetime.strptime("261021162255", "%y%m%d%H%M%S")
print("created", t, "-> deleted after", t + timedelta(minutes=1440))

24 other teams × ₹1.2 lakh × 6/8 day

Rs 21.6 lakh

print(f"Rs {24 * 1.2 * 6 / 8:.1f} lakh")

live {A,C}, s1 {A,B}, s2 {B,C,D}

delete s1 frees: []
then s2 frees  : ['B', 'D']

live, s1, s2 = {"A", "C"}, {"A", "B"}, {"B", "C", "D"}
before = live | s1 | s2
print("delete s1 frees:", sorted(before - (live | s2)))
print("then s2 frees  :", sorted((live | s2) - live))

Real-world scenario questions (asked in company interviews)

Data protection questions come up in every Hadoop administration and data platform interview.