The benchmark that proved the wrong thing
Riya wants to show her team that her new invoice calculator is fast. She times one call with System.nanoTime(): 3 milliseconds. Kabir times it in a loop and gets 0.4 milliseconds per call. Same code, same laptop, a seven-fold difference. They argue until Arjun Sir walks by.
Sir, who is right? Is my method 3 ms or 0.4 ms?
Both measurements are real, but they measure different things. The first time, your method ran in the interpreter. As it kept running, the JIT compiler turned it into optimised machine code, so later rounds were far faster.
So Java changes its own speed while it runs?
Exactly. Today you'll see how the JVM decides what to compile, what the compilers do, and how to measure performance without fooling yourself.
What is the JIT compiler?
A Just-In-Time (JIT) compiler translates frequently executed bytecode into native machine code at run time. In HotSpot, methods start in the interpreter; per-method invocation and loop (back-edge) counters detect hot code, which is compiled in the background. With tiered compilation (the default since Java 8), hot methods are first compiled by C1, a fast compiler that also collects profiling data, and the hottest are recompiled by C2, an optimising compiler that uses the profile for optimisations such as inlining, escape analysis and dead code elimination. If an assumption behind optimised code becomes invalid, the code is deoptimised and execution falls back to the interpreter.
A translator reads a speech line by line for the first few minutes. Once she notices the same phrases coming up again and again, she writes polished translations of those phrases and just reads them out. The speech gets faster the longer it goes on.
Words you'll meet in this lesson
JIT facts at a glance
| Starts with | The interpreter. No method is compiled before it has been used |
|---|---|
| Hot detection | Counters per method for calls and loop back-edges. Defaults (JDK 21): about 200 calls for C1 (Tier3InvocationThreshold), about 5,000 for C2 (Tier4InvocationThreshold), adjusted for loops and load |
| Tiers | 0 interpreter; 1–3 C1 (3 = with full profiling); 4 C2. Tiered compilation is on by default since Java 8 |
| Key C2 optimisations | Inlining (small methods under 35 bytes of bytecode, hot ones up to 325), escape analysis with scalar replacement, dead code elimination, loop optimisations |
| See it | java -XX:+PrintCompilation Main; compare -Xint (interpreter only) |
| Measure properly | Warm up, repeat, use the results, and prefer JMH (Java Microbenchmark Harness) |
Exam tip: javac does almost no optimisation: it turns source into bytecode. The big optimisations happen at run time, in the JIT. So “Java is interpreted” and “Java is compiled” are both half-truths: it is compiled to bytecode, interpreted at first, and JIT-compiled where it matters.
Two compilers, three big optimisations
C1 (client compiler)
Compiles quickly with basic optimisations, and in tier 3 adds counters that record which types and branches actually occur. Gets code faster early.
tiers 1–3C2 (server compiler)
Slower to compile, much better code. Uses C1's profile to make bold assumptions, such as “this call always goes to the same class”.
tier 4Inlining, escape analysis, dead code
Inline small hot methods; avoid allocating objects that never escape; delete code whose result is never used. Most other optimisations build on inlining.
C2 at workSeeing the JIT work: -XX:+PrintCompilation
This program times the same method six times (your numbers will differ):
public class Main {
static long invoiceTotal(int items) {
long total = 0;
for (int i = 1; i <= items; i++) {
total += i * 118L / 100; // each item's price with 18% GST
}
return total;
}
public static void main(String[] args) {
for (int round = 1; round <= 6; round++) {
long start = System.nanoTime();
long t = invoiceTotal(1_000_000);
long micros = (System.nanoTime() - start) / 1_000;
System.out.println("Round " + round + ": " + micros + " us (total " + t + ")");
}
}
}The same run with compilation logging, keeping only our method's lines (hundreds of JDK methods are compiled too):
java -XX:+PrintCompilation Main| Column | Example | Meaning |
|---|---|---|
| Time | 47 | Milliseconds since the JVM started |
| Compile id | 121 | A running number for each compilation task |
| Flags | % | % = on-stack replacement (a hot loop compiled while running); others: s synchronized, ! has exception handlers, n native |
| Tier | 3 / 4 | 3 = C1 with profiling, 4 = C2 |
| Method and size | Main::invoiceTotal @ 4 (30 bytes) | The method, the loop's bytecode index for OSR, and the bytecode size |
| Status | made not entrant | This compiled version is retired: replaced by a better tier, or deoptimised |
Naive benchmarks lie. Timing one call measures the interpreter and class loading. Timing a loop whose result is never used may measure nothing, because C2 can delete the work. GC pauses, CPU frequency changes and other processes add noise. For real measurements use JMH (Java Microbenchmark Harness), which handles warm-up, forks fresh JVMs, and consumes results so they can't be optimised away.
The journey of one hot method
Interpret and count
The interpreter executes bytecode one instruction at a time. Each call tototal += withGst(price); // called again and againwithGstincrements its invocation counter; each jump back to the top of a loop increments a back-edge counter.Hot: queue it for C1
After about 200 calls (default tiered thresholds, adjusted by the JVM), the method is queued for compilation.
Compilation happens on separate compiler threads, so your code keeps running in the interpreter meanwhile. A long loop inside a single call can be compiled on the spot and switched to mid-loop: on-stack replacement, shown as%.C1 code runs and gathers a profile
The tier 3 code is already much faster than the interpreter, and it records what really happens.
The profile notes which classes appear at each call site, which branches are taken and how often loops run. C2 relies on this to make speculative optimisations.Very hot: C2 optimises
C2 inlines small hot methods, keeps values in CPU registers, removes unneeded bounds checks and allocations (escape analysis), unrolls loops and deletes dead code. The new tier 4 code replaces the tier 3 version, which is “made not entrant”.// after inlining, the loop effectively becomes: total += price * 118 / 100;When assumptions break: deoptimise
Suppose C2 assumed a call always reached one class, and a new subclass shows up.
The JVM discards that compiled code and continues in the interpreter, then recompiles later with the new information. Your program's results never change; only its speed does.
Interpreter to C1 to C2: both views
Left: the program and real -XX:+PrintCompilation lines from the run above. Right: the tier a method is running in, and what the JVM is doing.
invoiceTotal gets fast 6 steps- Interpreter: the first call starts in the interpreter, which is simple but slow.
- OSR: the loop inside the very first call becomes hot, so C1 compiles it and switches to the compiled loop mid-call (
%). - C1: the whole method is compiled at tier 3. Round 1 finishes at 3186 microseconds.
- C2: tier 4 versions arrive (compile ids 120 and 121); rounds 3 and 4 are about three times faster than round 1.
- Retired: the old tier 3 code is “made not entrant” because the C2 version replaces it.
- Steady state: by round 5 the method is fully optimised: about 7 times faster than round 1. Your numbers will differ.
- In tier 0 the interpreter runs the method and updates its invocation and back-edge counters. A million loop iterations push the back-edge counter over the threshold quickly.
- The method's loop is queued for C1. A compiler thread builds it while the interpreter keeps going, then execution jumps into the compiled loop (on-stack replacement).
- Tier 3 code is fast and also profiles: which branches run, how many times loops turn, which classes appear at each call.
- Using that profile, C2 compiles tier 4 code with inlining, register allocation, escape analysis, dead code elimination and loop unrolling.
- The tier 3 version is marked not entrant. Deoptimisation works the same way if C2's assumptions later turn out wrong.
- The time per round falls as the code moves up the tiers. Benchmarks must skip this warm-up to measure steady-state speed.
The JIT in real systems
Most of the time the JIT just works. These are the situations where developers need to understand it.
Watch it with an example: a bank's month-end interest job speeding up
A batch job calls addInterest() for every account. Watch the JVM spot the hot method, compile it twice, and briefly step back when a new account type appears.
- Batch starts: the job runs in the interpreter, one bytecode at a time. The JVM counts every call to
addInterest. - Hot: around 200 calls (the default C1 threshold) the method is queued for compilation. A compiler thread works on it while the job keeps running.
- C1: quickly compiled tier 3 code takes over and records a profile: so far every account at this call site is a
SavingsAccount. - C2: after thousands of calls, C2 compiles tier 4 code that relies on that profile and inlines
addInterestand the getters it uses. The job reaches full speed. - Surprise: the first
FixedDepositaccount arrives. C2's assumption no longer holds, so that code is thrown away (deoptimised) and execution falls back for a while. - Recompiled: with the new profile, C2 compiles again, this time handling both types. Speed recovers, and the interest credited was correct at every moment: only speed ever changes.
1. Warming up before taking traffic
A freshly started instance is slower for its first seconds or minutes. Teams send synthetic requests to new instances, or use gradual traffic ramps, before they serve real users at full load.
2. Honest micro-benchmarks
Before replacing one algorithm or library with another, teams measure with JMH. It runs warm-up iterations, forks fresh JVMs, reports errors, and protects results from dead code elimination.
3. Clean code without a speed penalty
Getters, small helper methods and interfaces with one implementation are typically inlined by C2. Write readable code first; optimise only what a profiler shows is slow.
4. Allocation-heavy hot paths
Escape analysis removes some temporary objects, but not all (for example, objects stored in fields or passed to methods that aren't inlined). In very hot paths, teams reduce allocation, which also lowers GC pressure (Lesson 8.3).
5. Short-lived programs
Command-line tools and serverless functions may finish before the JIT has warmed up. For them, start-up techniques such as class data sharing, or ahead-of-time compiled native images (for example with GraalVM), can matter more than peak speed.
Examples are simplified. JIT behaviour varies between JVM versions and hardware; always confirm a performance idea with measurements on your own system.
Warm-up you can see
The same program as above. Its timings are machine-dependent: your numbers will differ, but the pattern (slow first rounds, fast later ones) should appear.
public class Main {
static long invoiceTotal(int items) {
long total = 0;
for (int i = 1; i <= items; i++) {
total += i * 118L / 100; // each item's price with 18% GST
}
return total;
}
public static void main(String[] args) {
for (int round = 1; round <= 6; round++) {
long start = System.nanoTime();
long t = invoiceTotal(1_000_000);
long micros = (System.nanoTime() - start) / 1_000;
System.out.println("Round " + round + ": " + micros + " us (total " + t + ")");
}
}
}System.nanoTime()A high-resolution timer for measuring durations (not the time of day).i * 118L / 100TheLmakes the multiplication uselong, so it can't overflowint.(total " + t + ")"Printing the result keeps the work alive; an unused result could be optimised away.Round 1vsRound 6Interpreter and C1 first, C2 later. Locally, tryjava -Xint Mainto see every round stay slow.
Work whose result nobody uses
Recorded from the real JVM, which runs in the interpreter while you step. Every call happens here; the notes explain which ones C2 would delete once the loop is hot.
Worked examples: real problems, solved and traced in memory
The JIT compiler is invisible in a debugger: while you step through code, the JVM runs it in the interpreter. These traces show the program exactly as the interpreter runs it, and the notes explain what the JIT would change once the same code becomes hot: which calls get inlined, and which objects never need to reach the heap.
A small method that will become hot
Problem statement
A cart adds 18% GST to each price (in paise, to avoid decimals) using a small helper withGst. A static counter shows how often the helper runs. In production it runs lakhs of times a minute. Trace one cart.
cart = (49900, 12000, 7500) paise
Total: 81892 paise, withGst called 3 times
Approach
- Keep money in
longpaise: integer maths, no rounding surprises. - Call the helper once per item. Each call pushes and pops a frame in the interpreter.
- The JVM counts calls per method. After a few hundred calls, C1 compiles it; after thousands, C2 compiles an optimised version, usually inlined into the loop.
Solution: run it live
What happens in memory, step by step
Recorded from the real JVM: press Next to execute one line at a time. Watch method calls push and pop frames on the stack, and objects (numbered #1, #2 …) appear on the heap.
Getters: calls in the interpreter, gone after inlining
Problem statement
Good design hides fields behind getters. A cart total calls getPrice() and getQty() for each item. Does all this calling cost anything in production? Trace it and read the notes on inlining.
items: (Rs 250 × 2), (Rs 99 × 3)
Cart total: Rs 797
Approach
- Store each item as an object with private fields and getters.
- Loop over the items and add price × quantity.
- In the trace every getter call pushes a frame. Once the loop is hot, the JIT inlines these tiny methods: it copies their body into the caller, so the compiled loop reads the fields directly.
Solution: run it live
What happens in memory, step by step
Recorded from the real JVM: press Next to execute one line at a time. Watch method calls push and pop frames on the stack, and objects (numbered #1, #2 …) appear on the heap.
Escape analysis: which object really needs the heap?
Problem statement
A payment service adds a Rs 5 fee (500 paise) using a temporary Money object, then records the charged amount in a ledger, which is a static list. Trace both methods and decide which Money object escapes its method.
amount = 10000 paise, fee = 500 paise
Charged 10500 paise, ledger size 1
Approach
addFeecreates a Money, reads one field, and returns a plainlong: the object never leaves the method.recordcreates a Money and stores it in a static list: the object escapes and must live on the heap.- C2's escape analysis can replace a non-escaping object with plain local values (scalar replacement), so in hot compiled code the first object may never be allocated at all.
Solution: run it live
What happens in memory, step by step
Recorded from the real JVM: press Next to execute one line at a time. Watch method calls push and pop frames on the stack, and objects (numbered #1, #2 …) appear on the heap.
Check yourself
Quick revision
Lesson 8.5 in one screen
- javac → bytecode; the JVM interprets first, then JIT-compiles hot code to machine code at run time.
- Hot = counters for calls and loop back-edges cross thresholds (about 200 for C1, about 5,000 for C2 by default).
- Tiers: 0 interpreter → 3 C1 with profiling → 4 C2 optimised. OSR compiles hot loops mid-call.
- C2: inlining, escape analysis (scalar replacement), dead code elimination, loop optimisations.
- Deoptimisation: compiled code is discarded when assumptions break (“made not entrant”).
- Warm-up makes early runs slow; measure steady state, use results, prefer JMH.
- Tools:
-XX:+PrintCompilation,-Xint,-XX:TieredStopAtLevel=1.
Interview and viva questions
What is the JIT compiler and why does Java use it?
It compiles frequently executed bytecode into native machine code while the program runs. Starting in the interpreter gives fast start-up and keeps bytecode portable; compiling only hot code focuses effort where it pays, and run-time profiling lets the JIT make optimisations an ahead-of-time compiler can't safely make, such as optimising for the classes and branches actually seen.
Explain tiered compilation.
Code starts at tier 0 in the interpreter. When it becomes hot it is compiled by C1, usually at tier 3, which produces decent code quickly and collects profiling data. If it stays hot, C2 recompiles it at tier 4 using that profile for aggressive optimisations. Tiered compilation has been the default since Java 8.
What is escape analysis?
An analysis in C2 that checks whether an object created in a method can be seen outside it (returned, stored in a field or a collection, passed to code that isn't inlined). If it doesn't escape, the JIT can replace the object with its fields as local values (scalar replacement) and avoid the heap allocation, and it can also remove unneeded locking on that object.
Why can a micro-benchmark written with System.nanoTime() be wrong?
It may measure warm-up (interpretation, class loading, compilation) instead of steady state; the JIT may remove work whose results are unused; GC pauses and other processes add noise; and one run proves little. JMH handles warm-up, multiple forks and iterations, and prevents dead code elimination with Blackholes.
What is deoptimisation?
C2 makes speculative assumptions, for example that a call site always sees one class. If that stops being true (say a new subclass is loaded), the JVM invalidates the compiled code, continues in the interpreter and may recompile later. It affects speed, never correctness.
Practice problems
You can't print the JIT's decisions from inside a normal program, so these problems practise the skills around it: spotting hot code, reading -XX:+PrintCompilation output, and measuring honestly. The last three are business tasks.
Which methods are hot?
Count how many times each of three methods runs while processing 500 orders, and flag those called more than 200 times (roughly where C1 compilation starts with default settings).
500 orders; every order is validated, every 4th gets a coupon, 1 report at the end
validate: 500 calls <- hot, will be compiled coupon : 125 calls report : 1 calls
Show solution and explanation
public class Main {
static int validateCalls, couponCalls, reportCalls;
static boolean validate(int amount) { validateCalls++; return amount > 0; }
static int coupon(int amount) { couponCalls++; return amount / 10; }
static void report() { reportCalls++; }
static void show(String name, int calls) {
System.out.println(name + ": " + calls + " calls" + (calls > 200 ? " <- hot, will be compiled" : ""));
}
public static void main(String[] args) {
for (int order = 1; order <= 500; order++) {
if (validate(order * 100) && order % 4 == 0) {
coupon(order * 100);
}
}
report();
show("validate", validateCalls);
show("coupon ", couponCalls);
show("report ", reportCalls);
}
}How it works: the JVM keeps counters like these for every method (and for loop back-edges). Methods that cross the thresholds are compiled, first by C1, later by C2 if they stay hot. report(), called once, is simply interpreted: compiling it would cost more than it saves. The 200 here is the default Tier3InvocationThreshold; the real decision also considers loops and compiler load.
Read one PrintCompilation line
Split a line of -XX:+PrintCompilation output into its parts and explain the tier.
" 36 9 4 Main::gst (6 bytes)"
time: 36 ms after start compile id: 9 tier: 4 = C2, fully optimised method: Main::gst bytecode: 6 bytes
Show solution and explanation
public class Main {
public static void main(String[] args) {
String line = " 36 9 4 Main::gst (6 bytes)";
String[] p = line.trim().split("\\s+");
String[] tierNames = {"interpreter", "C1, no profiling", "C1, light profiling",
"C1, full profiling", "C2, fully optimised"};
int tier = Integer.parseInt(p[2]);
System.out.println("time: " + p[0] + " ms after start");
System.out.println("compile id: " + p[1]);
System.out.println("tier: " + tier + " = " + tierNames[tier]);
System.out.println("method: " + p[3]);
System.out.println("bytecode: " + p[4].substring(1) + " bytes");
}
}How it works: the columns are a timestamp (ms since start), a compile id, optional flags (such as % for OSR), the tier (0 to 4), the method and its bytecode size. This real line says Main::gst, a 6-byte method, was compiled by C2 (tier 4) 36 ms after start-up. The regex \s+ splits on any run of spaces.
Classify compilation events
Given real lines from -XX:+PrintCompilation, count normal compilations, OSR compilations (marked %) and deoptimised code (“made not entrant”).
7 lines (see the code)
Normal compilations: 3 OSR compilations: 2 Old code discarded: 2
Show solution and explanation
public class Main {
public static void main(String[] args) {
String[] log = {
"35 8 3 Main::gst (6 bytes)",
"36 9 4 Main::gst (6 bytes)",
"36 8 3 Main::gst (6 bytes) made not entrant",
"38 10 % 3 Main::main @ 4 (36 bytes)",
"39 11 3 Main::main (36 bytes)",
"39 12 % 4 Main::main @ 4 (36 bytes)",
"42 10 % 3 Main::main @ 4 (36 bytes) made not entrant"
};
int normal = 0, osr = 0, discarded = 0;
for (String line : log) {
if (line.contains("made not entrant")) discarded++;
else if (line.contains(" % ")) osr++;
else normal++;
}
System.out.println("Normal compilations: " + normal);
System.out.println("OSR compilations: " + osr);
System.out.println("Old code discarded: " + discarded);
}
}How it works: OSR (on-stack replacement) compiles a long-running loop while the method is still inside it, switching to compiled code mid-loop; @ 4 is the bytecode index of the loop. “Made not entrant” means older code is retired, usually because a better tier replaced it (tier 3 → 4) or an assumption broke (deoptimisation). These lines come from the GST example in this lesson.
Keep the result, or the JIT may drop the work
A benchmark computes discounts in a loop. Make sure the work can't be optimised away by combining every result into a checksum that is printed at the end.
prices 1..1,00,000 (Rs), discount 10% above Rs 1000
Checksum: 499910400
Show solution and explanation
public class Main {
static int discount(int price) {
return price > 1000 ? price / 10 : 0;
}
public static void main(String[] args) {
long checksum = 0;
for (int price = 1; price <= 100_000; price++) {
checksum += discount(price); // result is used
}
System.out.println("Checksum: " + checksum);
}
}How it works: if the loop just called discount(price) and ignored the result, C2 could inline it, see it has no side effects, and delete it: a timing would then measure an empty loop. Using every result (and printing the total) keeps the computation real. JMH does the same thing with its Blackhole class.
Ignore the warm-up when reporting speed
A team timed 10 rounds of an invoice-generation job (microseconds, as measured on one machine). Report the naive average, the steady-state average after discarding 4 warm-up rounds, and how much slower the first round was.
rounds = {4551, 2228, 1217, 1198, 819, 557, 561, 549, 553, 560}Naive average: 1279.3 us Steady average: 599.8 us First round was 7.6x slower than steady state
Show solution and explanation
public class Main {
public static void main(String[] args) {
long[] rounds = {4551, 2228, 1217, 1198, 819, 557, 561, 549, 553, 560};
int warmup = 4;
long all = 0, steady = 0;
for (int i = 0; i < rounds.length; i++) {
all += rounds[i];
if (i >= warmup) steady += rounds[i];
}
double naive = (double) all / rounds.length;
double steadyAvg = (double) steady / (rounds.length - warmup);
System.out.printf("Naive average: %.1f us%n", naive);
System.out.printf("Steady average: %.1f us%n", steadyAvg);
System.out.printf("First round was %.1fx slower than steady state%n", rounds[0] / steadyAvg);
}
}How it works:
- The first rounds ran in the interpreter and C1 while C2 was still compiling, so they are much slower. The first six numbers are real measurements from this lesson's warm-up demo.
- Averaging everything mixes two different situations. Report steady-state numbers (and, separately, start-up time if that matters to your users).
- Tools like JMH run warm-up iterations automatically and repeat measurements in fresh JVMs.
Daily sales total: primitives in the hot loop
Sum 1,00,000 order amounts (in paise) twice: once with a long accumulator and once with a boxed Long. Show the totals agree, and explain why the hot loop should use the primitive.
amount of order i = (i % 500 + 1) × 100 paise
long total: Rs 25050000 Long total: Rs 25050000 Same? true
Show solution and explanation
public class Main {
public static void main(String[] args) {
long primitiveTotal = 0;
Long boxedTotal = 0L;
for (int i = 0; i < 100_000; i++) {
long amount = (i % 500 + 1) * 100L;
primitiveTotal += amount;
boxedTotal += amount; // unbox, add, box a new Long
}
System.out.println("long total: Rs " + primitiveTotal / 100);
System.out.println("Long total: Rs " + boxedTotal / 100);
System.out.println("Same? " + (primitiveTotal == boxedTotal));
}
}How it works:
- The results are identical, but
boxedTotal += amountcreates a newLongobject on most iterations (only small values are cached). - Escape analysis can sometimes remove such boxes in compiled code, but you can't rely on it; the primitive version never allocates.
- In hot paths (billing runs, analytics), prefer primitives and let the JIT do the rest.
Latency of the first requests after a deploy
After a deploy, an API's first 10 requests and the next 10 were timed (ms). Compare the averages and the worst request of each group to decide whether the service needs a warm-up step before taking traffic.
first = {180, 150, 96, 88, 61, 47, 40, 33, 30, 28}
later = {22, 21, 23, 20, 22, 21, 24, 22, 21, 23}First 10: average 75.3 ms, worst 180 ms Next 10: average 21.9 ms, worst 24 ms Needs warm-up before traffic? yes
Show solution and explanation
public class Main {
static void summary(String name, int[] ms) {
int sum = 0, worst = 0;
for (int t : ms) {
sum += t;
worst = Math.max(worst, t);
}
System.out.printf("%s average %.1f ms, worst %d ms%n", name, (double) sum / ms.length, worst);
}
public static void main(String[] args) {
int[] first = {180, 150, 96, 88, 61, 47, 40, 33, 30, 28};
int[] later = {22, 21, 23, 20, 22, 21, 24, 22, 21, 23};
summary("First 10:", first);
summary("Next 10: ", later);
System.out.println("Needs warm-up before traffic? " + (first[0] > 3 * later[0] ? "yes" : "no"));
}
}How it works:
- Right after start-up the code runs in the interpreter, classes are still loading and caches are cold, so the first requests are several times slower.
- Teams send synthetic warm-up requests before adding a new instance to the load balancer, or use readiness checks so real users never see the cold phase.
- The figures here are sample data for the exercise; measure your own service.
Real-world scenario questions (asked in company interviews)
Performance questions test whether you measure before you change things. Decide your answer first, then open the model answer.
“Getters are slow, use public fields”
During a code review for a loan-EMI service, a senior asks you to replace getters with public fields “because method calls are slow in hot loops”. How do you respond?
The interviewer is checking: do you know about inlining, and do you ask for measurements?
Show a strong answer
- Tiny methods like getters are among the first things C2 inlines once code is hot, so in steady state the call usually disappears and the field is read directly.
- Public fields would weaken encapsulation (anyone could set an invalid interest rate) for a gain that is probably zero.
- If there is a real concern, measure: a JMH benchmark or a profiler on the real service, not a guess.
Avoid: optimising by folklore.
Timeouts for two minutes after every deploy
After each rolling deploy of a checkout service, the first one to two minutes on each new instance show many slow requests and some timeouts, then everything is normal. CPU on the new instances is very high during that time. What is happening and what can you do?
The interviewer is checking: can you connect cold-start symptoms to class loading and JIT warm-up?
Show a strong answer
- A fresh JVM is loading classes and running code in the interpreter, while compiler threads compile hot paths. That burns CPU and makes early requests slow: classic warm-up.
- Send warm-up traffic (exercise the main endpoints) before the instance is marked ready, and ramp real traffic gradually.
- Make sure the container has enough CPU for compiler threads during start-up, and consider start-up features such as application class data sharing.
- Measure: compare latency per instance age to confirm the cause.
Avoid: disabling the JIT tiers or adding flags without measuring.
The benchmark that said the new parser is 50 times faster
A teammate times two CSV parsers by running each once on a small file with System.nanoTime(), old parser first, and reports that the new one is 50 times faster. You need to decide which parser the team adopts. What is wrong with the test and how would you redo it?
The interviewer is checking: do you know the classic benchmarking traps?
Show a strong answer
- The first parser paid for warm-up: class loading of shared libraries, interpretation, JIT compilation. The second benefited from the already-warm JVM. One run on a small file measures start-up, not parsing speed.
- Redo it with JMH: warm-up iterations, several measurement iterations, separate forks per parser, and consume the parsed results with a Blackhole.
- Use realistic input sizes (the real daily file), and also check memory allocation and correctness, not only time.
Avoid: trusting a single timing, or benchmarking both in the same JVM run in a fixed order.
Throughput dropped after adding a new payment type
A payment router calls processor.process(txn) in a hot loop. For months there was one Processor implementation. After adding a second and third (UPI and wallet), throughput of that loop dropped noticeably, although each implementation is simple. Explain a likely JIT-related reason and how you would confirm it.
The interviewer is checking: do you understand profile-based inlining and deoptimisation?
Show a strong answer
- With one implementation, C2 saw a single receiver type at that call site and inlined
processdirectly (monomorphic call). When new classes appeared, that assumption broke: the code was deoptimised and recompiled. - With two types C2 can still inline both behind a type check (bimorphic), but with three or more the site is usually megamorphic: a virtual call that isn't inlined, which also blocks optimisations such as escape analysis inside the loop.
- Confirm with
-XX:+PrintCompilation(look for “made not entrant” around the change) and diagnostic inlining output, or with a profiler; then decide whether it matters for the business SLA before redesigning (for example, grouping transactions by type).
Avoid: blaming the new classes' code without evidence; the change was in what the JIT can assume.