Survey Results: Java EE Containers – Heaven or Hell?
Thursday, July 2nd, 2009Last week we asked a few questions regarding Java EE development, containers and redeploy and restart times. It was surprising to see how quickly people responded — I suppose it’s becoming a hot topic!
We had to draw the line for our analysis somewhere, so we took the first 700 responses and created the charts below. If there is a significant change in the number of responses, we’ll update these charts with new results. The 2 minute survey is still running here. Feel free to fill it out if you haven’t already.
If you’d like to analyze the results for yourself, we have provided a copy of the data. This data may be updated as new responses come in. Email addresses have been removed from this data set.
We hope this info provides interesting insights into the world of Java EE containers and development productivity! Since we’re not professional surveyors there may be some flaws in the data and in our interpretation — so we mention some of them below, and you’re welcome to analyze the data yourself :-)
“Most Often Used Java EE Containers”
We asked, “What container do you use on your largest current Java EE project?”, which is slightly different than the chart title. Unfortunately, we didn’t clarify the difference between containers used for development and production, so answers here may be mixed – we’ll do better next time.
As usual, there was a long tail of answers. Any answer that did not receive at least 7 votes (1% of the total votes in the survey) was not put on the pie chart. The omitted containers were:
- Tmaxsoft JEUS- 4
- SAP NetWeaver- 3
- Iona/Progress Artix- 1
- zeus 4.x- 1
- Sybase EAServer- 1
- Impala- 1
- Adobe JRun- 1
- Jonas 4.x- 1
- Assorted earlier versions of containers listed in the chart- 13
“How long is the average container redeploy + app restart?”
Actual question asked: “How long does it take to restart your container and redeploy your app?” Now, you would think that folks who answered “I never redeploy” in the next question would have had a challenge with this one, since, well… they never redeploy. Apparently they overcame that challenge. They may have answered randomly, or, if they are JavaRebel users, answered with the amount of time that it used to take to redeploy/restart, before they eliminated redeploys.
“How often do Java EE developers redeploy, per hour?”
Actual question asked: “In an hour of coding, how many times do you redeploy?” Unfortunately, we made an error here. When choosing how many times you redeploy, folks who know for a fact that they redeploy exactly seven (7) times per hour faced a dilemma. Hopefully they were able to overcome this and err on the side that felt more reasonable to them: 5-7 or 7-10.
For the folks who answered, “I never redeploy”, we asked some of them how they did this. Responses were:
- “I’m not the guy that does the redeploys”
- “We develop on embedded jetty & activemq & atomikos in debug mode instead of nearly unusable target OracleAS. So of course we need redeploy or restart jetty as usual, but not the OAS.”
- “I’m in the very early stages of a project so a lot of out time is spent coding and testing without redeploying – typically I redeploy 3-4 times per hour”
- “We use JavaRebel and it’s awesome” – which led me to believe that I should have asked people if they use JavaRebel right up front, since that may bias the numbers somewhat.
Java EE Container Specific Productivity Information
For the next few container-specific charts, there is the potential for a slight error when it comes to Jetty, Oracle, and Caucho’s containers, for the simple reason that there were not a lot of people in our survey reporting numbers for them. Jetty was used by only 27 respondents, Oracle 20, and Resin had 10. With numbers like these it’s easy to sway results. Of course, the 207 results from Tomcat users isn’t a HUGE number, but at least averages are more accurate. Solution: more respondents. The survey is still live, here.
How long does container X take to redeploy?
Does container X affect incremental development?
We thought that containers that redeploy quickly would lead to more incremental development – and this was generally true, but there were exceptions.
Time spent on redeploys, per hour, with container X
Calculated like this:
Time spent redeploying in an average hour = Average number of redeploys per hour of coding x Average length of redeploy (see “How long is the average redeploy” chart).
Percentage of development time spent redeploying with container X
Calculated like this:
Percentage of coding time spent redeploying = Time spent redeploying in an average hour of coding divided by 60 (mins)
“Annually, How much time do Java EE developers spend redeploying containers & restarting apps?” (in hours)
We wanted to estimate this number quite conservatively, so we took the amount of time spent redeploying in an average hour of coding and added that up over a year, based on these assumptions:
- A developer rarely actually codes for 8 full hours per day. We should account for non-coding days, meetings, coffee breaks, foosball, email, twitter, etc. We assumed an average of 4 hours of coding per day, 5 days per week.
- We also assumed that some people get (and use) holidays and vacations, and have other reasons why they don’t code everyday. These reasons would a) reduce the amount of full weeks in the year, and b) reduce the average # of hours of coding per day. To account for these, we based a year on 40 work weeks instead of 52. Those 12 weeks should account for almost anything else you can throw in here to reduce avg weekly/daily coding hours. We figure that if you’re not coding 4 hours per day, 5 days per week, for at least 40 weeks a year, then you probably didn’t answer this survey. If we’re off here, or this is not conservative enough, let us know.
It would be interesting to know what this costs, in terms of development team budget. You can estimate it for your team here.
Survey Round-up: We were impressed to receive over 700 responses so quickly and are curious to know what you think of the results! To start – is it acceptable that every developer spends an average of 81 to 319 hours per year redeploying?
Just as a note, JavaRebel eliminates the need to redeploy in most redeployment situations (see Container Support and the Comparison to HotSwap and Hot Redeploy). From these survey results, that would give your team an extra 2 to 8 weeks of development time per year, or you would cut development time on a project by 3.8% to 15.4%.
Is that worth $149? (see Pricing)
