MyPhatshala

Cluster Playground

A small Hadoop cluster that runs in your browser. Upload files and watch HDFS split and replicate them, break nodes and racks, fail over the NameNode, and run a MapReduce job with and without data locality. Nothing is installed and nothing can break for real.

Apache Hadoop⏱ 15 minFree · no sign-up

The full cluster

Three racks of three DataNodes, a client (edge node), two NameNodes (active and standby) and a ResourceManager. Everything you can do in the lessons is switched on here. Use the checklist as a guided tour, or just experiment.

What each part of the picture means

Client (edge node)Where users run hdfs and hadoop jar commands. Not a DataNode, so the first replica of an upload goes to any node.
NameNodeThe HDFS master. Knows every file, its blocks and where each replica is (the metadata table on the right).
Standby NameNodeShown when High Availability is on. Takes over automatically if the active one fails.
ResourceManagerThe YARN master. Hands out containers for map and reduce tasks.
Core switch / ToR switchThe network. Packets moving between racks pass through the core switch.
DataNode boxA worker node. Coloured squares are block replicas (number = block of the file). The two bars at the bottom are task containers.
Pink dotsHeartbeats from DataNodes to the NameNode, every 3 seconds (shown at 1× speed).
Green / amber / red barsMap tasks that are node-local / rack-local / off-rack. Violet bars are reducers.

Defaults follow real Hadoop: 128 MB blocks, replication 3, rack-aware placement, heartbeat every 3 seconds, a DataNode declared dead after 10 minutes 30 seconds. Transfer and task times are simplified, and some things that take minutes in a real cluster (such as noticing a lost task) happen immediately here.

Lessons that use the playground