Over the Reloading Java Classes (RJC) article series, we’ve examined how class reloading happens, from objects and classloaders to application servers and solutions that reduce Turnaround. It’s been pretty in-depth, so we’d like to take a step back, move to more shallow ground, and take a look at why we would want class reloading to work in the first place. What are the real costs of missing class reloading on Java teams?
As we discover, this is a challenging question to answer — but we’ll give it a shot here, using the attempts of others to guide us.
Other Articles in the Reloading Java Classes Series
- RJC101: Objects, Classes and ClassLoaders
- RJC201: How do Classloader leaks happen?
- RJC301: Classloaders in Web Development — Tomcat, GlassFish, OSGi, Tapestry 5 and so on
- RJC401: HotSwap and JRebel — Behind the Scenes
- RJC501: How Much Does Turnaround Cost?
The Easily-Measurable Cost of Turnaround: Builds and Redeploys
Last year we ran two surveys asking developers how much time they spend building and redeploying their application. Although we’ve been talking with developers about this for a long time, and everyone understands that some amount of time is spent on builds and redeploys, it was still surprising to see the results – including actual numbers of minutes, days, and weeks spent on the process. For example, during an average hour of coding developers spend:
- An average of 6 minutes building the application [Source: The Build Tool Report].
- An average of 10.5 minutes redeploying the application [Source: The Java EE Container Redeploy & Restart Report].
To be fair, there are large deviations on both sides of these averages — the survey data we used only came from 600 and 1100 respondents respectively, and they used a variety of technologies which you can read about in the reports themselves. Some folks don’t spend as much as the averages, while others spend much more, and your project could be anywhere in that range.
Given that, the averages should apply pretty nicely to Java EE development as an industry (with the usual disclaimer of “lies, damn lies and statistics”). Let’s take a look at an awful scenario — where everyone in the industry is right on the average:
- Build (6 mins) + Redeploy (10.5 mins) = 16.5 minutes per hour of development or 27.5% of coding time
- Assuming that developers only code 5 hours per day x 5 days per week x 48 weeks a year = 1200 coding hours a year
- 1200 hours * 27.5% = 330 hours a year or 330/40 = 8.25 full-time work-weeks a year spent on turnaround, per developer
- Assuming $20/h average salary, this adds up to 330 * 20 = $6600 per developer per year [Source: conservative estimation backed by http://www.worldsalaries.org/computerprogrammer.shtml]
- Larry Ellison recently quoted the number of Java developers around the world at 9 million.
If you’d like to work out the time or cash spent annually by the industry on Turnaround, these are some numbers you can start with.
Context Switching
Programming is complicated. In fact it’s so complicated that most people are not capable of doing it. Those who are capable must mentally juggle a lot of context while writing code.. context including specifications, GUI considerations, relevant library calls, relevant application classes, methods and variables and so on.
This information is held in the the working (aka short-term) memory. Since the working memory is not meant for long term storage, switching away from a current task means that you will start losing the context surrounding the code, and then spend time trying to get back into it. The rest of the article will discuss how quickly context is lost, how long it takes to restore it, and what effect it has on the quality of your work.
How quickly does working memory degrade?
In 1959, an article titled, “Short-Term Retention of Individual Verbal Items” by L. Peterson and M. Peterson appeared in the Journal of Experimental Psychology [Source: http://www-test.unifr.ch/psycho/site/assets/files/Allg/ExU/Peterson_1959.pdf]. It showed that on average, after only 3 seconds, about half of our working memory is lost. The following graph plots the amount of working memory preserved after every subsequent 3 seconds:
After 15 seconds, less than 10% of the original memory is preserved. While these results may not be directly applicable to the programming context, it is clear that memory degradation occurs after seconds, not minutes. Translated, it means that short distractions cause dramatic losses in working memory.
How long it takes to restore context after an interruption?
Although there’s not a direct answer to that question, going through the relevant literature gives us some clues:
[…] the recovery time after a phone call is at least 15 minutes. [...] If more than 10 interrupts occur during a day, the time between the interrupts becomes too short to accomplish product development work.
To take a call you need to completely switch away from your working environment and focus on the conversation. Other interrupts are less disruptive than that:
Recovering from an email interrupt, and returning to work at the same work rate as before the interrupt — 64 seconds.
Instant message — 11 to 25 seconds.
FYI – These quotes are not based on software developers, but rather more generic office workers. Take that as you’d like. Since context recovery is a process, it takes some time to get back to maximum speed after an interrupt:
The trouble is, getting into “the zone” is not easy. When you try to measure it, it looks like it takes an average of 15 minutes to start working at maximum productivity. [...] The other trouble is that it’s so easy to get knocked out of the zone. Noise, phone calls, going out for lunch, having to drive 5 minutes to Starbucks for coffee, and interruptions by coworkers [...] all knock you out of the zone.
So what about development Turnaround — should that be considered an interrupt and how much does it add to the cost?
From experience, any pause will cause the working memory to start fading, and longer pauses will cause developers to multi-task (even if the other task is reading Slashdot). This will mean that the context surrounding a particular task is at least partially lost and needs to be restored. A longer pause means more memory degradation, increased likelihood of a task switch, and the assumption that recovery time is also longer. Just to illustrate, let’s assume that it takes 50% of the length of the pause to recover the context after getting back to work. In that case, the total cost of turnaround for a java developer dealing with average build and redeploy delays looks something like:
- 16.5 (mins per hour spent on Build & Redeploy phases) + 50% = 24.75 minutes per hour of development or 41.25% of coding time
- 1200 (coding hours per year) * 41.25% = 495 hours per year spent on Turnaround (divide that by 40 hours per week to get 12.375 full-time work-weeks a year spent on turnaround)
- Assuming a $20/h average salary this adds up to 495 * 20 = $9900 per developer per year
These numbers make a lot of assumptions. I’m not convinced that the average hourly wage of someone reading this blog is $20. I’m also not convinced that these numbers make an air-tight argument — but the point here is: the social cost of Turnaround is more than just the time spent on building, redeploying, & restarting.
Finally, I’m not convinced that developers keep all the code they wrote on a day full of distractions – I get the feeling that long turnaround times, distractions, and interruptions have a negative effect on code quality, and on the mental abilities of other developers. Though we couldn’t find data measuring this specifically, there is some indication that this impact is also not trivial, and I’d like to finish with it:
In a series of tests carried out by Dr Glenn Wilson, Reader in Personality at the Institute of Psychiatry, University of London, an average worker’s functioning IQ falls ten points when distracted by ringing telephones and incoming emails. This drop in IQ is more than double the four point drop seen following studies on the impact of smoking marijuana. Similarly, research on sleep deprivation suggests that an IQ drop of ten points is equal to missing an entire night of sleep. This IQ drop was even more significant in the men who took part in the tests.
Many programmers appear to be continually frustrated in attempts to work. They are plagued by noise and interruption, and pessimistic that the situation will ever be improved. The data recorded about actual interruptions supports the view that the so-called “work-day” is made up largely of frustration time.
What are your thoughts on all this?
Resources
- Concerning Interruptions – Stephen B. Jenkins, Institute for Aerospace Research, National Research Council of Canada
- Interruptions.net Literature — comprehensive index of published papers concerning interruptions
