ExecutorService and thread pools

1 The story

One thread per order? The server disagrees

Riya's canteen app goes viral during exam week. Her server creates a new thread for every incoming order. At lunchtime 5,000 orders arrive in a minute, the server runs out of memory, and nobody gets lunch.

Riya

Sir, threads were supposed to make it faster!

Arjun Sir

Threads cost memory and switching time. Thousands of them help nobody. Keep a few workers and a queue of orders: each worker finishes an order and takes the next.

Riya

Do I write that queue and the workers myself?

Arjun Sir

No. Java has it ready: ExecutorService. You submit tasks; it runs them on a pool and gives you a Future for each result.

2 Definition

What are ExecutorService and thread pools?

Formal definition (write this in exams)

An executor separates task submission from task execution. java.util.concurrent.ExecutorService is an executor that manages a thread pool: a set of reusable worker threads that take submitted tasks from a queue and run them. Tasks are Runnable (no result) or Callable<V> (returns a V and may throw checked exceptions). submit() returns a Future<V>, which represents the pending result: get() waits for it, isDone() checks it, and cancel() attempts to stop the task. A pool is ended with shutdown() (finish queued tasks, accept no new ones) or shutdownNow() (interrupt running tasks).

In simple words

A thread pool is a call centre: a fixed team of agents (threads) and a waiting line (queue). You don't hire a new agent for every call. You get a ticket (Future) to collect your answer later.

Words you'll meet in this lesson

ExecutorServiceRuns tasks on a pool; you never call new Thread.

Thread poolReusable worker threads plus a task queue.

Callable<V>A task whose call() returns a value.

Future<V>A handle to a result that may not be ready yet.

shutdown()No new tasks; finish the queued ones; then the workers end.

CompletableFutureA Future you can chain steps onto, without blocking.

Runnable vs Callable

	`Runnable`	`Callable<V>`
Method	`void run()`	`V call() throws Exception`
Returns a result	No	Yes
Checked exceptions	Must catch inside	Allowed; reach you via `get()`
Use with `new Thread`	Yes	No (needs an executor)
Lambda example	`() -> send(sms)`	`() -> fetchPrice(shop)`

Exam tip: “A thread pool reuses a fixed number of threads to execute many tasks, reducing thread-creation overhead and limiting resource use.” And: Future.get() blocks; a failed task's exception arrives wrapped in ExecutionException.

3 Types and ways to use it

Pools and how they reuse threads

Five jobs, two workers

import java.util.concurrent.*;

public class Main {
    public static void main(String[] args) throws InterruptedException {
        ExecutorService pool = Executors.newFixedThreadPool(2);
        for (int i = 1; i <= 5; i++) {
            int job = i;
            pool.submit(() -> System.out.println("job " + job + " on " + Thread.currentThread().getName()));
        }
        pool.shutdown();
        pool.awaitTermination(5, TimeUnit.SECONDS);
    }
}

job 1 on pool-1-thread-1 job 3 on pool-1-thread-1 job 4 on pool-1-thread-1 job 5 on pool-1-thread-1 job 2 on pool-1-thread-2

One real run. Your order of lines and which worker took which job will differ, but you'll only ever see two thread names.

Kinds of executors

Fixed pool

Exactly n workers. The default choice for steady server work.

newFixedThreadPool(n)

Single thread

One worker: tasks run one at a time, in submission order.

newSingleThreadExecutor()

Scheduled

Run after a delay or repeatedly: reminders, cleanups, reports.

newScheduledThreadPool(n)

Virtual threads

Java 21: very cheap threads, one per task, ideal for I/O-heavy work.

newVirtualThreadPerTaskExecutor()

Executors.newFixedThreadPool(n)          // n workers, unbounded queue: steady CPU/IO work
Executors.newCachedThreadPool()          // grows as needed, idle threads die: short bursts
Executors.newSingleThreadExecutor()      // one worker: tasks run strictly in order
Executors.newScheduledThreadPool(n)      // delays and repeats: schedule(task, 5, SECONDS)
Executors.newVirtualThreadPerTaskExecutor()  // Java 21: a cheap virtual thread per task

Always shut pools down. Pool threads are normal (non-daemon) threads. Forget shutdown() and your program never exits; in a server, forgetting it on short-lived pools leaks threads until memory runs out. Use try/finally, or in Java 21 a try-with-resources block (ExecutorService is AutoCloseable).

4 Step by step

The life of a submitted task

Create the pool
```
ExecutorService pool = Executors.newFixedThreadPool(4);
```
A ThreadPoolExecutor object with an empty queue. Worker threads are started as the first tasks arrive.
submit()
```
Future<Integer> f = pool.submit(() -> fetchPrice("BuyNow"));
```
The Callable is wrapped in a FutureTask. If a worker is free (or the pool may start one), it runs; otherwise it joins the queue. submit returns at once.
A worker runs it
The worker calls call(). The result (or the exception) is stored inside the FutureTask, which becomes done, and any thread waiting in get() is woken.
The worker loops
Instead of ending, the worker takes the next task from the queue, or waits for one (like the consumer in Lesson 12.5).
shutdown()
New submissions are rejected with RejectedExecutionException. Queued tasks still run. When the queue is empty, workers exit; awaitTermination lets you wait for that moment.

5 See it happen

Submit, queue, reuse: both views

Left: the code you write and the Futures you hold. Right: inside the pool, the task queue and two worker threads that never get replaced.

A thread pool at work 6 steps

What you see

Behind the screen

6 Real-world use

Pools power every Java server

Web servers, database connection handling, message consumers and schedulers are all built on executors.

Watch it with an example: a travel site searching three airlines

A flight search asks three airline APIs for prices at the same time, using a pool and Futures, and shows the cheapest. One airline is slow, so the search uses a timeout.

Fare search with Futures 5 steps

Animated example

1. Web request handling

Servers like Tomcat keep a pool of request threads (often a couple of hundred). Each request borrows a worker and returns it when the response is sent.

HTTP request→request pool→response

2. Fan-out calls

A product page calls price, stock, reviews and delivery-estimate services at once and combines the answers with Futures or CompletableFuture.

4 services→CompletableFuture.allOf→one page

3. Scheduled jobs

Send EMI reminders every morning, clear expired carts every 10 minutes, refresh exchange rates every hour.

every 10 min→scheduleAtFixedRate→carts cleaned

4. Virtual threads for I/O

Since Java 21, services that mostly wait on databases and APIs can run one cheap virtual thread per request, handling far more concurrent requests with simple blocking code.

10,000 requests→virtual threads→few OS threads

Sizes and timings are illustrative. Frameworks such as Spring configure and manage these executors for you, but they are the same ExecutorService underneath.

7 Run the code

Three shop prices, fetched in parallel

import java.util.*;
import java.util.concurrent.*;

public class Main {
    static int fetchPrice(String shop) throws InterruptedException {
        Thread.sleep(100);                    // pretend network call
        return switch (shop) {
            case "Shopkart" -> 18999;
            case "BuyNow" -> 18499;
            default -> 19250;
        };
    }

    public static void main(String[] args) throws Exception {
        List<String> shops = List.of("Shopkart", "BuyNow", "MegaMart");
        ExecutorService pool = Executors.newFixedThreadPool(3);
        List<Future<Integer>> prices = new ArrayList<>();
        for (String s : shops) {
            prices.add(pool.submit(() -> fetchPrice(s)));
        }
        for (int i = 0; i < shops.size(); i++) {
            System.out.println(shops.get(i) + ": Rs " + prices.get(i).get());
        }
        pool.shutdown();
    }
}

Shopkart: Rs 18999 BuyNow: Rs 18499 MegaMart: Rs 19250

Executors.newFixedThreadPool(3)Three workers, so all three calls can run at the same time.
pool.submit(() -> fetchPrice(s))A Callable: it returns the price, and may throw InterruptedException (allowed in call()).
prices.get(i).get()Read results in shop order; each get waits only if that price isn't ready.
pool.shutdown()Lets the JVM exit once the workers are idle.

8 Line by line

Two Callables, two Futures

Recorded from the real JVM, following main. Watch the pool, the Callable objects and the Future objects appear on the heap; the prices are computed on pool threads and come back through get().

Worked examples: real problems, solved and traced in memory

Three common executor patterns: a pool processing many small jobs, Callables with Futures collected by invokeAll, and a Future with a timeout. Results are always read in submission order, so the output never changes.

Example 1E-commerce · invoices

Ten invoices on a pool of three workers

Problem statement

At closing time a shop generates 10 invoices. Creating a thread for each is wasteful; use a fixed pool of 3. Count the invoices with an AtomicInteger, shut down properly, and print the total.

Input

invoice ids 1..10

Expected output

finished: true, invoices made: 10

Approach

Create Executors.newFixedThreadPool(3).
Submit 10 Runnables; each increments a shared AtomicInteger.
shutdown(), then awaitTermination to wait until all tasks have finished.

Solution: run it live

What happens in memory, step by step

Recorded from the real JVM: press Next to execute one line at a time. Watch method calls push and pop frames on the stack, and objects (numbered #1, #2 …) appear on the heap.

Example 2Analytics · dashboard

Three dashboard numbers with invokeAll

Problem statement

A sales dashboard needs today's orders, revenue and new customers. Each is a separate query (a Callable<Integer>). Run all three at once with invokeAll and print them in a fixed order.

Input

orders = 3 bills; revenue = 1200 + 850 + 400; new customers = 2

Expected output

orders: 3
revenue: 2450
new customers: 2

Approach

Write three Callables in a List.
invokeAll runs them and returns a list of Futures in the same order, once all are done.
Read each Future with get().

Solution: run it live

What happens in memory, step by step

Recorded from the real JVM: press Next to execute one line at a time. Watch method calls push and pop frames on the stack, and objects (numbered #1, #2 …) appear on the heap.

Example 3Banking · timeouts

A credit-score check with a deadline

Problem statement

A loan app asks a credit bureau for a score. The bureau sometimes takes too long. Wait at most 200 ms with get(timeout); on timeout, cancel the task and send the application for manual review.

Input

the bureau call takes 5 s (too slow)

Expected output

bureau too slow, cancelled: true -> manual review

Approach

Submit a Callable that sleeps 5 s, then returns 742.
get(200, TimeUnit.MILLISECONDS) throws TimeoutException.
In the catch: cancel(true) interrupts the worker; then shutdown().

Solution: run it live

What happens in memory, step by step

Recorded from the real JVM: press Next to execute one line at a time. Watch method calls push and pop frames on the stack, and objects (numbered #1, #2 …) appear on the heap.

9 Quiz and revision

Check yourself

Quick revision

Lesson 12.6 in one screen

A thread pool = reusable workers + a task queue. Create it with Executors.newFixedThreadPool(n) and friends.
submit(Runnable | Callable) returns a Future; execute(Runnable) returns nothing.
Callable<V> returns a value and may throw checked exceptions.
get() waits; get(t, unit) may throw TimeoutException; failures come as ExecutionException.
invokeAll runs a list of Callables and returns Futures in the same order.
Always shutdown(); awaitTermination waits; submitting afterwards throws RejectedExecutionException.
CompletableFuture chains async steps; Java 21 virtual threads make one-thread-per-task cheap for I/O work.

Interview and viva questions

Why use a thread pool instead of creating threads?

Creating and destroying threads is costly, and unlimited threads exhaust memory and waste CPU on switching. A pool reuses a bounded number of threads, queues extra work, and centralises lifecycle management (shutdown, naming, error handling).

Difference between submit() and execute()?

execute(Runnable) comes from Executor and returns nothing; an exception in the task goes to the thread's uncaught-exception handler. submit() accepts Runnable or Callable and returns a Future; exceptions are captured and rethrown by get() as ExecutionException.

Difference between shutdown() and shutdownNow()?

shutdown() stops accepting tasks but lets queued and running tasks finish. shutdownNow() also interrupts running tasks and returns the list of tasks that never started. Neither waits; use awaitTermination for that.

How do you choose the pool size?

For CPU-bound work, about the number of cores (availableProcessors()). For I/O-bound work, more threads, roughly cores × (1 + wait time / compute time), or virtual threads. Always measure under realistic load.

What is the difference between Future and CompletableFuture?

A Future can only be polled or waited on with get(). A CompletableFuture can also be completed manually and lets you attach callbacks (thenApply, thenCombine, exceptionally) that run when the result arrives, so you can compose async steps without blocking threads.

Practice problems

Use pools, Callables and Futures instead of raw threads. Results are read from Futures in a fixed order, so every output is the same each run. The last three are business tasks.

P1Easy

Submit a Callable

Submit a Callable that returns 6 × 7 to a single-thread pool and print the result.

Input

(none)

Expected output

answer: 42

Show solution and explanation

import java.util.concurrent.*;

public class Main {
    public static void main(String[] args) throws Exception {
        ExecutorService pool = Executors.newSingleThreadExecutor();
        Future<Integer> f = pool.submit(() -> 6 * 7);
        System.out.println("answer: " + f.get());
        pool.shutdown();
    }
}

answer: 42

How it works: a lambda that returns a value is treated as a Callable. get() waits for the result.

P2Easy

Shutdown and awaitTermination

Submit 5 Runnables that each add their number to an AtomicInteger, shut down, wait, and print the sum.

Input

1..5

Expected output

sum = 15

Show solution and explanation

import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;

public class Main {
    public static void main(String[] args) throws InterruptedException {
        ExecutorService pool = Executors.newFixedThreadPool(2);
        AtomicInteger sum = new AtomicInteger();
        for (int i = 1; i <= 5; i++) {
            int n = i;
            pool.submit(() -> sum.addAndGet(n));
        }
        pool.shutdown();
        pool.awaitTermination(1, TimeUnit.SECONDS);
        System.out.println("sum = " + sum.get());
    }
}

sum = 15

How it works: shutdown() alone doesn't wait. awaitTermination blocks until the queued tasks are finished (or the timeout passes).

P3Easy

Runnable Future returns null

Submit a Runnable and print what its Future's get() returns, and whether it's done.

Input

(none)

Expected output

cleaning cache
get() = null, done = true

Show solution and explanation

import java.util.concurrent.*;

public class Main {
    public static void main(String[] args) throws Exception {
        ExecutorService pool = Executors.newSingleThreadExecutor();
        Future<?> f = pool.submit(() -> System.out.println("cleaning cache"));
        System.out.println("get() = " + f.get() + ", done = " + f.isDone());
        pool.shutdown();
    }
}

cleaning cache get() = null, done = true

How it works: a Runnable has no result, so get() returns null. It's still useful: it waits for completion and rethrows failures.

P4Medium

Exceptions come back through get()

A task divides by zero. Catch the ExecutionException from get() and print its cause.

Input

(none)

Expected output

task failed: java.lang.ArithmeticException: / by zero

Show solution and explanation

import java.util.concurrent.*;

public class Main {
    public static void main(String[] args) throws InterruptedException {
        ExecutorService pool = Executors.newSingleThreadExecutor();
        Future<Integer> f = pool.submit(() -> 100 / 0);
        try {
            f.get();
        } catch (ExecutionException e) {
            System.out.println("task failed: " + e.getCause());
        }
        pool.shutdown();
    }
}

task failed: java.lang.ArithmeticException: / by zero

How it works: the exception happened on a worker thread. The pool catches it and stores it in the Future; get() rethrows it wrapped in ExecutionException. If you never call get(), the failure is silently lost.

P5Medium

Squares with invokeAll

Build Callables that square 2, 3, 4 and 5, run them with invokeAll, and print the results in order.

Input

2, 3, 4, 5

Expected output

4 9 16 25

Show solution and explanation

import java.util.*;
import java.util.concurrent.*;

public class Main {
    public static void main(String[] args) throws Exception {
        List<Callable<Integer>> jobs = new ArrayList<>();
        for (int n = 2; n <= 5; n++) {
            int x = n;
            jobs.add(() -> x * x);
        }
        ExecutorService pool = Executors.newFixedThreadPool(2);
        for (Future<Integer> f : pool.invokeAll(jobs)) {
            System.out.print(f.get() + " ");
        }
        System.out.println();
        pool.shutdown();
    }
}

4 9 16 25

How it works: invokeAll returns Futures in the same order as the tasks, however the workers scheduled them.

P6Medium

CompletableFuture pipeline

Compute a price of 400 asynchronously, add 18% GST with thenApply, and format it with another thenApply. Print with join().

Input

price 400

Expected output

Total with GST: Rs 472

Show solution and explanation

import java.util.concurrent.CompletableFuture;

public class Main {
    public static void main(String[] args) {
        String bill = CompletableFuture.supplyAsync(() -> 400)
                .thenApply(p -> p * 118 / 100)
                .thenApply(t -> "Total with GST: Rs " + t)
                .join();
        System.out.println(bill);
    }
}

Total with GST: Rs 472

How it works: CompletableFuture lets you chain steps that run when the previous one completes, without blocking in between. By default it uses the shared ForkJoinPool. join() is like get() but throws an unchecked exception.

P7Business problem

Parallel GST for store branches

Three branches have daily sales of 25000, 18000 and 32000. Compute each branch's 18% GST in a Callable on a pool, then print each and the total GST.

Input

25000, 18000, 32000

Expected output

Pune: GST Rs 4500
Nagpur: GST Rs 3240
Nashik: GST Rs 5760
total GST: Rs 13500

Show solution and explanation

import java.util.*;
import java.util.concurrent.*;

public class Main {
    public static void main(String[] args) throws Exception {
        String[] branch = {"Pune", "Nagpur", "Nashik"};
        int[] sales = {25000, 18000, 32000};
        ExecutorService pool = Executors.newFixedThreadPool(3);
        List<Future<Integer>> gst = new ArrayList<>();
        for (int s : sales) gst.add(pool.submit(() -> s * 18 / 100));
        int total = 0;
        for (int i = 0; i < branch.length; i++) {
            int g = gst.get(i).get();
            total += g;
            System.out.println(branch[i] + ": GST Rs " + g);
        }
        System.out.println("total GST: Rs " + total);
        pool.shutdown();
    }
}

Pune: GST Rs 4500 Nagpur: GST Rs 3240 Nashik: GST Rs 5760 total GST: Rs 13500

How it works: results are summed on main from the Futures, so no shared variable or lock is needed. Collecting results through Futures is usually simpler than sharing state between tasks.

P8Business problem

Email batch with a bounded pool

Send 8 promotional emails using a pool of 2 workers. Each task returns true if the address contains "@". Count successes and failures.

Input

8 addresses, 2 invalid

Expected output

sent 6, failed 2

Show solution and explanation

import java.util.*;
import java.util.concurrent.*;

public class Main {
    public static void main(String[] args) throws Exception {
        String[] to = {"a@x.in", "b@x.in", "bad-address", "c@x.in", "d@x.in", "e@x.in", "oops", "f@x.in"};
        ExecutorService pool = Executors.newFixedThreadPool(2);
        List<Callable<Boolean>> sends = new ArrayList<>();
        for (String addr : to) sends.add(() -> addr.contains("@"));
        int ok = 0;
        for (Future<Boolean> f : pool.invokeAll(sends)) if (f.get()) ok++;
        pool.shutdown();
        System.out.println("sent " + ok + ", failed " + (to.length - ok));
    }
}

sent 6, failed 2

How it works: the pool size caps how many emails are sent at once, protecting the mail server, however many addresses there are. This is why pools are preferred over one thread per task.

P9Business problem

Seat confirmation with timeout fallback

A train-booking app asks two seat servers. The first answers at once (seat "B4-32"), the second hangs. Wait up to 150 ms for each; print the answer or "no reply", and cancel the hanging task.

Input

server 1 fast, server 2 hangs

Expected output

seat: B4-32
no reply, cancelled

Show solution and explanation

import java.util.concurrent.*;

public class Main {
    public static void main(String[] args) throws Exception {
        ExecutorService pool = Executors.newFixedThreadPool(2);
        Future<String> s1 = pool.submit(() -> "B4-32");
        Future<String> s2 = pool.submit(() -> {
            Thread.sleep(10_000);
            return "S2-11";
        });
        for (Future<String> f : java.util.List.of(s1, s2)) {
            try {
                System.out.println("seat: " + f.get(150, TimeUnit.MILLISECONDS));
            } catch (TimeoutException e) {
                f.cancel(true);
                System.out.println("no reply, cancelled");
            }
        }
        pool.shutdown();
    }
}

seat: B4-32 no reply, cancelled

How it works: each server gets its own deadline. cancel(true) interrupts the stuck task so the worker is freed and shutdown() can finish.

Real-world scenario questions (asked in company interviews)

Executors are where most production concurrency lives, so interviewers probe how you size, protect and shut them down.

Backend · resource leakFresher

Thread count keeps climbing

Each time a report is requested, the code does Executors.newFixedThreadPool(4), submits the work and returns. After a day the service has 20,000 threads and crashes. What's wrong?

The interviewer is checking: pool lifecycle.

Show a strong answer

A new pool per request is never shut down, so its 4 threads live forever.
Create one shared pool at startup and reuse it; shut it down when the application stops.
If a short-lived pool is really needed, close it in finally (or try-with-resources in Java 21).

Avoid: calling System.gc(); live threads are never garbage collected.

Debugging · silent failureJunior

Refunds that silently never happen

Refund tasks are submitted with pool.submit(refundTask) and the Future is ignored. Some refunds never complete and nothing appears in the logs. Why, and how do you fix it?

The interviewer is checking: exception handling with submit().

Show a strong answer

submit captures exceptions inside the Future; if nobody calls get(), the failure is swallowed.
Catch and log inside the task (with refund id), or keep the Futures and check them, or use CompletableFuture with exceptionally/whenComplete.
Add retries and an alert for failed refunds.

Avoid: assuming “no error in the logs” means success.

Performance · sizingMid

Sizing pools for two kinds of work

An 8-core server does image compression (pure CPU) and calls a partner API (mostly waiting, ~200 ms). Both use one pool of 8 threads. API calls queue up and pages are slow. What would you change?

The interviewer is checking: CPU-bound vs I/O-bound sizing and isolation.

Show a strong answer

Separate the workloads into two pools, so slow API calls can't starve compression (bulkheading).
CPU pool ≈ 8 threads; I/O pool much larger (or virtual threads), since those threads mostly wait.
Add timeouts on API calls and bounded queues with a rejection policy.

Avoid: one giant pool of 500 threads for everything; CPU work would thrash.

Release · shutdownMid

Orders lost during deployment

During a deployment, the app is stopped while order-confirmation tasks are still in the executor's queue. Some customers never get confirmations. Design a graceful shutdown.

The interviewer is checking: shutdown(), awaitTermination() and durability.

Show a strong answer

On stop: stop taking new requests, call shutdown(), then awaitTermination(30, SECONDS); only then call shutdownNow() and log the tasks it returns.
For anything that must not be lost, store it durably first (database or message broker) and process from there, so a restart can resume.

Avoid: relying on an in-memory queue for work that must survive a restart.

ExecutorService and thread pools

One thread per order? The server disagrees

What are ExecutorService and thread pools?

Words you'll meet in this lesson

Runnable vs Callable

Pools and how they reuse threads

Five jobs, two workers

Kinds of executors

Fixed pool

Single thread

Scheduled

Virtual threads

The life of a submitted task

Create the pool

submit()

A worker runs it

The worker loops

shutdown()

Submit, queue, reuse: both views

Pools power every Java server

Watch it with an example: a travel site searching three airlines

1. Web request handling

2. Fan-out calls

3. Scheduled jobs

4. Virtual threads for I/O

Three shop prices, fetched in parallel

Two Callables, two Futures

Worked examples: real problems, solved and traced in memory

Ten invoices on a pool of three workers

Three dashboard numbers with invokeAll

A credit-score check with a deadline

Check yourself

Quick revision

Lesson 12.6 in one screen

Interview and viva questions

Practice problems

Submit a Callable

Shutdown and awaitTermination

Runnable Future returns null

Exceptions come back through get()

Squares with invokeAll

CompletableFuture pipeline

Parallel GST for store branches

Email batch with a bounded pool

Seat confirmation with timeout fallback

Real-world scenario questions (asked in company interviews)

Thread count keeps climbing

Refunds that silently never happen

Sizing pools for two kinds of work

Orders lost during deployment