Final test, cheat sheet and exam questions: Big Data Fundamentals

1 The story

One last session

Riya’s semester exam in Big Data is next week. Kabir has just received his full-time offer from TiffinGo, and the first thing he does is call Riya to help her revise. Arjun Sir joins them for one last session.

Riya

Sir, there’s so much. 5 Vs, CAP, Spark, Kafka, Parquet, DPDP… where do I even start?

Arjun Sir

With the map, Riya. Every lesson answers one of four questions: what is big data, how do we store and spread it, which tools process it, and how do we keep it trustworthy and useful. If you can place each topic on that map, you can write any answer.

Kabir

In my interviews, they never asked definitions alone. Always “why” and “what if”.

Arjun Sir

Exams reward the same thing in long answers: a definition, a diagram, an example and a trade-off. Today you get a one-page cheat sheet, the 2-mark and 5-mark answer patterns, a 50-mark practice paper you can regenerate on your PC, and a final test across the whole course.

Riya

And if I score well on the final test?

Arjun Sir

Then you’re ready, for the exam and for Kabir’s kind of interviews.

2 Definition

Big Data Fundamentals in one page

Formal definition (write this in exams)

Big Data Fundamentals covers the characteristics of data sets too large, fast or varied for a single traditional system (volume, velocity, variety, veracity, value); the storage formats and databases suited to them; the distributed-systems principles of scaling out, partitioning, replication and consistency; the ecosystem of processing and streaming tools (Hadoop, Spark, Kafka, lakehouse table formats, orchestration, cloud services); the practices of data quality, governance, security and privacy; and the roles and skills of data professionals, applied in an end-to-end pipeline.

In simple words

Big data is data too big, fast or messy for one computer. We spread it across many machines, store it in efficient formats, process it with tools like Spark and Kafka, keep it correct, safe and lawful, and turn it into useful answers.

The course on one page

Module	Lessons	Must remember
1. Introduction	4	5 Vs · one computer is not enough · distributed basics · history
2. Data and storage	4	data types · file formats · storage · NoSQL
3. Distributed systems	4	scale out · partitioning · replication · CAP
4. The ecosystem	7	Hadoop · Spark · Kafka · lakehouse · pipelines · cloud · India scale
5. Governance	1	quality · governance · security · DPDP
6. Careers	3	roles · roadmap · resume · interviews
7. Capstone	2	mini pipeline · final test

Revision at three levels

Pick your level. New to this? Start at Beginner and climb.

Beginner Remember the core

Big data: 5 Vs; why one machine fails (disk speed, memory, single point of failure); scale out.
Data: structured, semi-structured, unstructured; CSV, JSON, Avro, Parquet, ORC; files, blocks, objects; NoSQL types.
Systems: partitioning, replication (factor 3), CAP, consistency levels.
Tools: HDFS, YARN, MapReduce, Hive, Spark, Kafka, lakehouse; Airflow; AWS/Azure/GCP equivalents.

Intermediate Explain and compare

Row vs columnar; lake vs warehouse vs lakehouse; batch vs streaming; ETL vs ELT; MapReduce vs Spark.
Scale up vs scale out; leader-follower vs leaderless replication; strong vs eventual consistency.
Mod-N hashing vs consistent hashing; provisioned vs serverless cloud.
Six data quality dimensions; governance components; DPDP rights and duties.

Advanced Apply and design

Estimate load: totals → per second → peak; storage with replication and retention.
Design a pipeline: requirements, ingestion, bronze/silver/gold, orchestration, quality, idempotency, cost and governance.
Diagnose: skew, small files, wrong numbers (lineage), duplicates (idempotency), cost spikes (scans, idle clusters).
Case studies: UPI, IRCTC Tatkal, live cricket streaming, sale days.

Exam tip: A full-marks long answer has five parts: definition (1–2 lines), diagram (labelled), explanation (points, not paragraphs), example (preferably Indian and with numbers) and advantages/limitations or a comparison table. Underline key terms.

3 Types and ways to use it

Cheat sheet: numbers and comparisons

Cheat sheet: numbers and units

Unit or fact	Value	Use it for
1 KB → MB → GB → TB → PB	× 1,000 each step (decimal); 1,024 in binary units	Storage estimates
1 lakh / 1 crore	1,00,000 / 1,00,00,000 (100 lakh)	Indian figures in answers
Seconds in a day	86,400	Daily totals → per second
HDFS default block / replication	128 MB / 3 copies	Hadoop questions
Replicas for quorum	R + W > N (e.g. N=3, W=2, R=2)	Consistency questions
UPI (Sept 2026)	24.07 billion transactions ≈ 9,300 per second on average	Velocity and scale examples
IRCTC peak	37,410 tickets in one minute (Aug 2025)	Peak load examples
DPDP breach report	Detailed report to the Board within 72 hours	Privacy questions

Cheat sheet: comparisons examiners love

Compare	First	Second
Scale up vs scale out	Bigger machine; simple; hard ceiling; single point of failure	More machines; near-linear growth; needs distribution and fault tolerance
Row vs columnar	CSV, Avro: good for writing whole records	Parquet, ORC: good for reading few columns of many rows
Lake vs warehouse	Raw files, any type, cheap, schema-on-read	Modelled tables, SQL, fast, schema-on-write
Batch vs streaming	Bounded data, scheduled, minutes to hours	Unbounded events, continuous, milliseconds to seconds
MapReduce vs Spark	Disk between every step, Java-heavy	In-memory DAG execution, SQL/Python APIs, much faster for iterative work
CP vs AP	Refuse or delay to stay consistent (payments)	Answer with possibly stale data (feeds, carts)
ETL vs ELT	Transform before loading	Load raw, transform inside the warehouse/lakehouse

Before the exam, redraw five diagrams from memory: HDFS (NameNode, DataNodes, replicas), Spark (driver, executors, stages), Kafka (producers, partitions, consumer groups), consistent hashing ring and bronze-silver-gold pipeline. Diagrams earn marks quickly.

4 Step by step

A revision plan that works

Map before memorising

Write the seven modules on one page and, under each, the lessons and three key words from memory. Fill gaps from the cheat sheet.
A mental map makes every answer easier to place and connect.
Practise recall, not rereading
```
day1, day3, day7 = "learn", "recall", "recall again"
for d, task in [(1, day1), (3, day3), (7, day7)]:
    print(f"day {d}: {task}")
```
day 1: learn day 3: recall day 7: recall again
Testing yourself after a gap (day 1, day 3, day 7) builds memory far better than rereading notes.
Use answer patterns

2 marks: definition + one example or point. 5 marks: definition, diagram, 3–4 points, example. 10 marks: add comparison table, case study, advantages and limitations.
Patterns stop you writing too much for small questions and too little for big ones.
Take a timed paper

Generate a paper with the program in “Run it on your PC”, write answers in 2 hours, then check against the lessons.
Timing matters: about 22 minutes per 10 marks leaves 10 minutes to revise.
Fix weak areas

Note every question you struggled with, revisit that lesson’s Revision and FAQ sections, and redo its quiz.
Targeted revision on weak lessons gives the biggest improvement per hour.
Final test

Take the 25-question final test on this page. Aim for 20 or more; review explanations for every mistake.
It samples every module, so your score shows where you stand overall.

5 See it happen

A 5-mark answer: what Riya writes, and how it is marked

Step through the six frames. Left: Riya’s answer sheet. Right: the examiner’s marking scheme.

What Riya writes, and how it is marked 6 steps

What you see

Behind the screen

6 Real-world use

Real-time scenarios and exam question banks

Three live scenarios: the course map in four questions, an estimation question solved for full marks, and an advanced case answered with every module. Then exam question banks with model answers.

Real-time scenario 1

The whole course in four questions

Every lesson answers one of four questions. Watch the modules light up as each question is answered; use this map as the skeleton of your revision.

Course map 5 steps

Animated example

Real-time scenario 2

An estimation question, solved for full marks

Numbers questions reward clear steps and stated assumptions more than exact answers. Watch one solved line by line, using UPI-scale figures.

Estimation 5 steps

Animated example

Real-time scenario 3 · advanced

One 10-mark case, every module

The best long answers pull ideas from across the course. Watch a railway booking case answered module by module, the structure that earns top marks.

Integrated case answer 5 steps

Animated example

Exam question bank

Answer each aloud or on paper before opening the model answer. Long answers should follow the five-part pattern: definition, diagram, explanation, example, limitations.

2-mark questions with model answers

Define big data.

Data sets whose volume, velocity or variety exceed the capacity of traditional single-machine systems to store and process them in acceptable time, requiring distributed storage and processing.

What is veracity?

The quality and trustworthiness of data: accuracy, completeness and consistency. Example: duplicate or fake transactions reduce veracity.

Differentiate structured and semi-structured data.

Structured data has a fixed schema of rows and columns (a bank table); semi-structured data has tags or keys but a flexible schema (JSON, XML).

What is a NameNode?

The HDFS master that stores the file system metadata (file names, block locations) and coordinates DataNodes, which store the actual blocks.

What is schema-on-read?

The schema is applied when data is read rather than when it is written, as in data lakes; allows storing raw data first.

What is a partition key?

The column whose value decides which partition or shard a row belongs to, for example customer_id or date.

What is eventual consistency?

If no new updates occur, all replicas eventually converge to the same value; reads may briefly return stale data.

What is a DAG in Spark or Airflow?

A directed acyclic graph of steps with dependencies and no cycles; Spark uses it for stages of computation, Airflow for task order.

What is a Kafka offset?

The sequential position of a message within a partition, used by consumers to track what they have read.

What is the lakehouse?

An architecture combining low-cost lake storage with warehouse features (ACID transactions, schemas, fast SQL) using table formats such as Delta Lake, Iceberg or Hudi.

What is egress in cloud billing?

Charges for data transferred out of a cloud region or out of the cloud, often a hidden cost.

Who is a data fiduciary under the DPDP Act?

The person or organisation that determines the purpose and means of processing personal data.

5-mark questions with answer outlines

Explain the 5 Vs of big data with examples.

Define each V in one line with an Indian example: volume (UPI billions a month), velocity (live cricket concurrency), variety (text, images, logs), veracity (fake reviews, duplicates), value (fraud detection, recommendations). Add a small diagram of the five Vs.

Why is one computer not enough for big data?

Disk read speed limits (hours to scan TBs), memory limits, CPU limits, single point of failure, cost of very large machines. Solution: split data and work across many machines (parallelism) with replication. Include a calculation, e.g. 10 TB at 200 MB/s ≈ 14 hours vs 100 machines ≈ 8 minutes.

Compare CSV, JSON, Avro, Parquet and ORC.

Table with: row/columnar, schema, compression, splittable, best use. CSV/JSON human-readable but large; Avro row-based with schema, good for streaming and Kafka; Parquet/ORC columnar, compressed, best for analytics.

Explain replication strategies.

Leader-follower (writes to leader, followers copy; sync vs async), multi-leader, leaderless with quorums (R + W > N). Benefits: availability, durability, read scaling; issues: lag, conflicts, failover. Diagram.

Explain Spark architecture.

Driver, cluster manager, executors; jobs, stages, tasks; lazy evaluation and DAG; narrow vs wide transformations and shuffles; in-memory processing. Diagram with driver and executors.

Explain Kafka architecture.

Producers, brokers, topics, partitions, offsets, replication, consumer groups; ordering per partition; retention; use cases like UPI events or order tracking. Diagram.

Explain cloud service models for big data with AWS/Azure/GCP examples.

IaaS, managed PaaS, serverless, SaaS; examples: S3/ADLS/GCS, EMR/Databricks/Dataproc, Redshift/Synapse/BigQuery; benefits and concerns (cost, lock-in, residency).

Explain the DPDP Act, 2023 from a data engineer’s point of view.

Fiduciary and principal; consent and notice; purpose limitation; rights (access, correction, erasure); security safeguards; breach notification within 72 hours to the Board; penalties up to ₹250 crore; engineering impacts: PII tagging, masking, deletion workflows, audit logs.

10-mark questions to practise

Design a data pipeline for a food-delivery company’s daily revenue dashboard (use your capstone).
Explain how a platform like IRCTC handles the Tatkal spike, with calculations (Lesson 4.7).
Describe the Hadoop ecosystem and compare it with Spark and the modern lakehouse.
Explain partitioning, replication and consistency in distributed databases with diagrams.
Explain data quality, governance and security for a big data platform, including the DPDP Act.

7 Run the code

Flashcards for the whole course

A small flashcard program picks terms from every module and shows their meanings. Then try the challenges: estimation drills, unit conversions, a spaced-repetition schedule, a self-scoring quiz and Indian number formatting.

import random
cards = {
    "Veracity": "quality and trustworthiness of data (Module 1)",
    "Parquet": "columnar, compressed file format for analytics (Module 2)",
    "Consistent hashing": "ring placement so adding a node moves ~1/N of keys (Module 3)",
    "Replication factor": "number of copies of each block or partition, often 3 (Module 3)",
    "Shuffle": "moving rows between executors so equal keys meet (Module 4)",
    "Consumer group": "consumers sharing a Kafka topic's partitions (Module 4)",
    "Idempotent": "re-running gives the same result (Modules 4 and 7)",
    "Data fiduciary": "organisation deciding why and how personal data is processed (Module 5)",
}
random.seed(7)
for term in random.sample(list(cards), 4):
    print(f"{term:20} -> {cards[term]}")

Consumer group -> consumers sharing a Kafka topic's partitions (Module 4) Parquet -> columnar, compressed file format for analytics (Module 2) Replication factor -> number of copies of each block or partition, often 3 (Module 3) Veracity -> quality and trustworthiness of data (Module 1)

random.seed(7)Change the seed to get a different set of cards.
for term in random.sample(list(cards), 4):Try to say the meaning before reading it.

Run it on your PC: generate a 50-mark practice paper

This program builds a practice paper from a bank of course questions: five 2-mark, four 5-mark and two 10-mark questions, 50 marks in all. Change SEED for a new paper, and set SHOW_KEY = True to print answer hints after you have attempted it. Save it as exam_paper.py.

A new practice paper every time you change the seed

"""Generate a 50-mark Big Data Fundamentals practice paper.
Change SEED for a new paper; set SHOW_KEY = True to print answer hints."""
import random

SEED = 2026
SHOW_KEY = False
BANK = {
    2: [("Define big data using the 5 Vs.", "Volume, velocity, variety, veracity, value; one line each"),
        ("What is horizontal scaling?", "Add more machines (scale out) vs a bigger machine"),
        ("Differentiate OLTP and OLAP.", "Transactions, row-wise, many small writes vs analytics, columnar, large scans"),
        ("What is a data lake?", "Cheap storage of raw files of any type, schema-on-read"),
        ("What is replication factor?", "Number of copies of each block or partition, e.g. 3"),
        ("Name two NoSQL database types with an example each.", "Key-value: Redis; document: MongoDB; wide-column: Cassandra; graph: Neo4j"),
        ("Why is Parquet faster than CSV for analytics?", "Columnar, compressed, column pruning and statistics"),
        ("What is a Kafka consumer group?", "Consumers sharing a topic's partitions; each message to one consumer in the group"),
        ("What is idempotency in pipelines?", "Re-running gives the same result; overwrite or MERGE"),
        ("State one right of a data principal under the DPDP Act.", "Access, correction, erasure, grievance redressal, nominate"),
        ("What is a shuffle in Spark?", "Moving rows across executors so equal keys meet; costly disk + network"),
        ("Expand HDFS and YARN.", "Hadoop Distributed File System; Yet Another Resource Negotiator")],
    5: [("Explain the CAP theorem with one CP and one AP example.", "Definitions, partition forces a choice, payments CP, feeds/counters AP"),
        ("Compare row and columnar file formats with examples.", "CSV/Avro vs Parquet/ORC; access pattern, compression, schema"),
        ("Explain partitioning and sharding. How does consistent hashing help?", "Split data by key; mod-N moves most keys; ring moves ~1/N"),
        ("Describe the Hadoop ecosystem components and their roles.", "HDFS, YARN, MapReduce, Hive, HBase, Sqoop/Flume, Oozie; Spark"),
        ("Compare data warehouse, data lake and lakehouse.", "Schema, cost, users, ACID, examples, when to use"),
        ("Explain batch vs stream processing with Indian examples.", "Daily settlement vs UPI fraud checks; latency, tools"),
        ("What are the six data quality dimensions? Give a check for each.", "Accuracy, completeness, consistency, timeliness, validity, uniqueness"),
        ("Explain the medallion architecture.", "Bronze raw, silver clean, gold business; quality and reprocessing")],
    10: [("Design a data pipeline for a food-delivery company's daily revenue dashboard.", "Requirements, estimates, ingest, bronze/silver/gold, Airflow, quality, idempotency, cost, governance"),
         ("Explain how a system like IRCTC Tatkal can handle a sudden spike. Include calculations.", "Peak TPS from published figures, waiting room, caching, rate limits, consistency for seats, data platform"),
         ("Explain big data on the cloud: services, cost meters and data residency.", "AWS/Azure/GCP map, serverless vs provisioned, FinOps, egress, RBI residency"),
         ("Describe replication and consistency in distributed databases with diagrams.", "Leader-follower, quorum R+W>N, failover, eventual vs strong consistency")],
}
PLAN = [(2, 5, "Section A: answer all (2 marks each)"),
        (5, 4, "Section B: answer all (5 marks each)"),
        (10, 2, "Section C: answer both (10 marks each)")]

rnd = random.Random(SEED)
total, qn = 0, 1
print(f"BIG DATA FUNDAMENTALS · PRACTICE PAPER (seed {SEED})   Time: 2 hours   Max marks: 50\n")
for marks, count, title in PLAN:
    print(title)
    for q, hint in rnd.sample(BANK[marks], count):
        print(f"  Q{qn}. {q}  [{marks}]")
        if SHOW_KEY:
            print(f"       key: {hint}")
        total += marks
        qn += 1
    print()
print(f"Total: {total} marks · {qn - 1} questions")
print(f"Suggested time: {(2 * 60 - 10) * 10 // total} min per 10 marks, plus 10 min to revise")

# In a terminal:  python exam_paper.py

BIG DATA FUNDAMENTALS · PRACTICE PAPER (seed 2026) Time: 2 hours Max marks: 50 Section A: answer all (2 marks each) Q1. What is horizontal scaling? [2] Q2. Name two NoSQL database types with an example each. [2] Q3. What is idempotency in pipelines? [2] Q4. State one right of a data principal under the DPDP Act. [2] Q5. Expand HDFS and YARN. [2] Section B: answer all (5 marks each) Q6. Describe the Hadoop ecosystem components and their roles. [5] Q7. Compare data warehouse, data lake and lakehouse. [5] Q8. What are the six data quality dimensions? Give a check for each. [5] Q9. Explain batch vs stream processing with Indian examples. [5] Section C: answer both (10 marks each) Q10. Describe replication and consistency in distributed databases with diagrams. [10] Q11. Explain big data on the cloud: services, cost meters and data residency. [10] Total: 50 marks · 11 questions Suggested time: 22 min per 10 marks, plus 10 min to revise

Recorded from a real run with Python 3.13 using seed 2026. Add your university’s past questions to the bank to make it even more useful. Write answers on paper in 2 hours, then compare with the lessons.

8 Line by line

Scoring a test, question by question

Four answers are compared with a key. Watch the score build up and the topics to revise being collected.

Worked examples: real problems, solved and traced in memory

Four exam-style calculations and comparisons from across the course. Each runs live and is traced.

Example 1BeginnerModule 1 · scale

How long to read 10 TB on one disk vs 100 machines?

Problem statement

A disk reads 200 MB/s. How long does one machine take to scan 10 TB, and how long do 100 machines take if the data is split evenly?

Input

10 TB, 200 MB/s, 100 machines

Expected output

1 machine: 13.9 hours
100 machines: 8.3 minutes

Approach

Convert 10 TB to MB (decimal: × 1,000,000).
Time = size ÷ speed; convert to hours.
Divide by 100 for parallel machines.

Solution: run it live

What happens in memory, step by step

Recorded from a real Python run: press Next to execute one line at a time. Watch variables change in each frame, and lists and dictionaries (numbered #1, #2 …) fill up as objects.

Example 2IntermediateModule 3 · partitioning

Keys moved when adding a server: mod-N vs consistent hashing

Problem statement

1,000 keys are placed with hash mod 4. A fifth server is added. Count how many keys change server with mod-N, and compare with the ideal for consistent hashing (about 1/5).

Input

1,000 keys, 4 → 5 servers

Expected output

mod-N moved: 800 (80%)
consistent hashing ideal: about 200

Approach

Server = key mod N before and after.
Count keys whose server changes.
Compare with 1,000 ÷ 5 = 200.

Solution: run it live

What happens in memory, step by step

Recorded from a real Python run: press Next to execute one line at a time. Watch variables change in each frame, and lists and dictionaries (numbered #1, #2 …) fill up as objects.

Example 3IntermediateModule 3 · quorums

Which quorum settings guarantee reading the latest write?

Problem statement

With N = 3 replicas, list every (W, R) pair from 1 to 3 and mark which satisfy R + W > N.

Input

N = 3

Expected output

W=1 R=1: eventual
W=1 R=2: eventual
W=1 R=3: strong
W=2 R=1: eventual
W=2 R=2: strong
W=2 R=3: strong
W=3 R=1: strong
W=3 R=2: strong
W=3 R=3: strong

Approach

Loop over W and R from 1 to 3.
Check R + W > N.
Print strong or eventual.

Solution: run it live

What happens in memory, step by step

Recorded from a real Python run: press Next to execute one line at a time. Watch variables change in each frame, and lists and dictionaries (numbered #1, #2 …) fill up as objects.

Example 4AdvancedModules 2 and 4 · cost

Scan cost: CSV vs partitioned Parquet

Problem statement

A 40 TB CSV table is queried 300 times a day on a pay-per-scan engine at ₹450 per TB (illustrative). With Parquet, the data shrinks to 8 TB; queries read 1 of 365 date partitions and 2 of 20 columns. Compare monthly costs.

Input

40 TB CSV vs 8 TB Parquet, pruning, 300 queries/day

Expected output

CSV                   :   40.0000 TB/query, Rs    162,000,000/month
Parquet + partitions  :    0.0022 TB/query, Rs          8,877/month
saving: 99.995%

Approach

CSV: every query scans 40 TB.
Parquet: 8 TB × (1/365) × (2/20) per query.
Monthly cost = TB per query × queries × 30 × price.

Solution: run it live

What happens in memory, step by step

Recorded from a real Python run: press Next to execute one line at a time. Watch variables change in each frame, and lists and dictionaries (numbered #1, #2 …) fill up as objects.

9 Quiz and revision

Check yourself

Quick revision

Lesson 7.2 in one screen

Four questions: what/where (Modules 1–2) · how to spread (3) · which tools (4) · how to trust (5); then careers (6–7).
Numbers: 86,400 s/day · 1 crore = 100 lakh · HDFS 128 MB × 3 · R + W > N · UPI ≈ 9,300 TPS avg (Sept 2026).
Comparisons: up vs out · row vs columnar · lake vs warehouse vs lakehouse · batch vs streaming · MapReduce vs Spark · CP vs AP · ETL vs ELT.
Five diagrams: HDFS · Spark · Kafka · consistent hashing ring · bronze-silver-gold.
Answer patterns: 2 marks = definition + example; 5 = + diagram + points; 10 = + table, case, limitations.
Estimation: totals → per second → peak (state the ratio) → storage × replicas × retention.
Practise with timed papers; revise on day 1, 3 and 7; fix weak lessons using their quizzes and FAQs.

Interview and viva questions

What are the most important topics in Big Data for exams?

The 5 Vs and the need for distributed systems; data types and file formats; HDFS, YARN and MapReduce; Spark architecture; NoSQL types and the CAP theorem; partitioning and replication; Kafka and stream processing; data lakes, warehouses and lakehouses; cloud services for big data; and data quality, governance and privacy. Practise diagrams for HDFS, Spark, Kafka and pipelines.

How do I write a 10-mark answer in a Big Data exam?

Use five parts: a precise definition, a labelled diagram, explanation in numbered points, a real example (preferably with numbers, such as UPI or IRCTC figures) and a comparison table or advantages and limitations. Underline key terms and keep each point short.

How should I revise Big Data Fundamentals in one week?

Day 1: course map and cheat sheets. Days 2–4: two modules a day, redrawing diagrams and taking each lesson’s quiz. Day 5: question banks and estimation practice. Day 6: a timed 50-mark practice paper. Day 7: weak areas and the final test.

Is Hadoop still important to learn?

Its ideas are: distributed storage with replication, moving computation to data, resource management and table metadata. Many exams still test HDFS, YARN and MapReduce, and many companies still run or migrate Hadoop clusters, but new projects mostly use Spark, cloud object storage and lakehouse formats.

What should I study after Big Data Fundamentals?

Go deeper in the Big Data Engineering track: SQL and data modelling, PySpark in practice, Kafka and stream processing, a cloud data platform, lakehouse formats and orchestration with Airflow. Extend your capstone at each step and follow the roadmap in Lesson 6.2.

Practice problems

Mixed problems from every module. The last two are business-style estimates.

P1Easy

Seconds in a week

Print the number of seconds in a week.

Input

7 days

Expected output

Show solution and explanation

print(7 * 86_400)

604800

How it works: Per-second thinking starts with time units.

P2Easy

Crore to lakh

Print how many lakh are in 5.9 crore.

Input

5.9 crore

Expected output

590.0 lakh

Show solution and explanation

print(round(5.9 * 100, 1), "lakh")

590.0 lakh

How it works: 1 crore = 100 lakh.

P3Easy

HDFS blocks

How many 128 MB blocks does a 1 GB file (1,024 MB) need? Print blocks and stored copies at replication 3.

Input

1,024 MB

Expected output

8 blocks, 24 block copies

Show solution and explanation

import math
b = math.ceil(1024 / 128)
print(b, "blocks,", b * 3, "block copies")

8 blocks, 24 block copies

How it works: Files are split into blocks; each block is replicated.

P4Medium

Quorum check

N=5, W=3, R=2. Print whether reads are guaranteed to see the latest write.

Input

N=5, W=3, R=2

Expected output

False

Show solution and explanation

print(3 + 2 > 5)

False

How it works: R + W must exceed N; 5 is not greater than 5.

P5Medium

Mod-N movement

Count how many of keys 0–999 change server going from 10 to 11 servers with mod-N.

Input

10 → 11

Expected output

Show solution and explanation

print(sum(1 for k in range(1000) if k % 10 != k % 11))

900

How it works: Most keys move, which is why consistent hashing exists.

P6Advanced

Peak TPS

80 crore transactions a day, peak 4× average. Print the design peak TPS.

Input

80 crore, 4×

Expected output

Show solution and explanation

print(round(80e7 / 86_400 * 4))

37037

How it works: Totals → average → peak.

P7Advanced

Parquet saving

CSV 6.76 MB vs Parquet 1.53 MB (Lesson 2.2 results). Print how many times smaller Parquet is.

Input

6.76 vs 1.53

Expected output

4.4 x smaller

Show solution and explanation

print(round(6.76 / 1.53, 1), "x smaller")

4.4 x smaller

How it works: Columnar compression is a recurring theme across the course.

P8Business problem

Yearly storage

500 GB of new data a day, 3 replicas, kept 2 years. Print PB.

Input

500 GB/day

Expected output

1.09 PB

Show solution and explanation

print(round(500 * 365 * 2 * 3 / 1e6, 2), "PB")

1.09 PB

How it works: Retention and replication multiply storage quickly.

P9Business problem

Cost per query

A query scans 22 GB at Rs 450 per TB (illustrative). Print the cost in rupees.

Input

22 GB

Expected output

9.9

Show solution and explanation

print(round(22 / 1000 * 450, 2))

9.9

How it works: Small per-query costs add up across thousands of queries.

Real-world scenario questions (asked in company interviews)

Exam, viva and next-step questions. Decide your answer first, then open the model answer.

Exam strategyFresher

Ten minutes left, two questions to go

In the exam you have 10 minutes left and one 5-mark and one 10-mark question unanswered.

The interviewer is checking: can you maximise marks under time pressure?

Show a strong answer

Attempt both in outline form: for each, a one-line definition, a quick labelled diagram and 3–4 bullet points with an example. Partial structured answers earn marks; a blank page earns none. Spend roughly 4 minutes on the 5-mark and 6 on the 10-mark question.

Avoid: perfecting one answer and leaving the other blank.

Concept checkFresher

“Is Hadoop dead?”

A viva examiner or interviewer asks whether Hadoop is still relevant.

The interviewer is checking: can you give a nuanced, current answer?

Show a strong answer

Hadoop MapReduce is rarely chosen for new projects; Spark and cloud services have replaced it. But Hadoop’s ideas live on everywhere: distributed storage with replication (HDFS → object storage), resource management (YARN → Kubernetes and cloud schedulers), Hive’s metastore and table concepts (→ catalogs and lakehouse formats). Many enterprises still run Hadoop clusters and are migrating them to cloud lakehouses.

Avoid: a simple yes or no.

Integration1–3 years

“Which single idea from this course matters most?”

A reflective viva or interview question.

The interviewer is checking: can you connect concepts?

Show a strong answer

A strong answer picks one idea and links it across the course, for example partitioning: it decides how data is spread across machines (Module 3), how Parquet folders are laid out and queries prune data (Modules 2 and 4), how Kafka scales and orders events (Module 4), how cloud scan costs fall (Module 4.6), how pipelines stay idempotent by overwriting a partition (capstone), and where skew comes from (interviews).

Avoid: listing tools without connecting them.

Next stepsFresher

“I finished the course. What now?”

You have completed Big Data Fundamentals.

The interviewer is checking: do you have a plan to keep growing?

Show a strong answer

Publish your capstone on GitHub with a README; extend it with PySpark, Delta Lake and Airflow; pick one cloud and build the same pipeline there; follow the roadmap from Lesson 6.2; practise SQL daily and use Lesson 6.3 for interviews. Then go deeper in the next courses of the Big Data Engineering track.

Avoid: stopping at the certificate of completion.

Final test, cheat sheet and exam questions: Big Data Fundamentals

One last session

Big Data Fundamentals in one page

The course on one page

Revision at three levels

Beginner Remember the core

Intermediate Explain and compare

Advanced Apply and design

Cheat sheet: numbers and comparisons

Cheat sheet: numbers and units

Cheat sheet: comparisons examiners love

A revision plan that works

Map before memorising

Practise recall, not rereading

Use answer patterns

Take a timed paper

Fix weak areas

Final test

A 5-mark answer: what Riya writes, and how it is marked

Real-time scenarios and exam question banks

The whole course in four questions

An estimation question, solved for full marks

One 10-mark case, every module

Exam question bank

2-mark questions with model answers

5-mark questions with answer outlines

10-mark questions to practise

Flashcards for the whole course

Run it on your PC: generate a 50-mark practice paper

A new practice paper every time you change the seed

Scoring a test, question by question

Worked examples: real problems, solved and traced in memory

How long to read 10 TB on one disk vs 100 machines?

Keys moved when adding a server: mod-N vs consistent hashing

Which quorum settings guarantee reading the latest write?

Scan cost: CSV vs partitioned Parquet

Check yourself

Quick revision

Lesson 7.2 in one screen

Interview and viva questions

Practice problems

Seconds in a week

Crore to lakh

HDFS blocks

Quorum check

Mod-N movement

Peak TPS

Parquet saving

Yearly storage

Cost per query

Real-world scenario questions (asked in company interviews)

Ten minutes left, two questions to go

“Is Hadoop dead?”

“Which single idea from this course matters most?”

“I finished the course. What now?”