Python 3 Adoption for Web Apps - New Relic blog
blog.newrelic.com/2014/01/21/python-3-adoptio...Posted Nov 16, 2014, 8:38 PM UTC
Posted Nov 16, 2014, 8:38 PM UTC
The implication of the post, whilst only vaguely stated, is that the number of deployments of a given version somehow reflects the current state of the uptake of Python 3 "This is not a good sign for Python 3 adoption". I'm calling bullshit on that, and calling bullshit on the vague implication of the blog post that deployments = takeup.
The version of Python used in deployed applications reflects history, not the present.
If we measured the past to understand the present then we could examine the vast deposits of coal underground and conclude that the earth must be thickly covered with forest today.
We are told that "Python 2.7 dominates.". What is true is that Python 2 dominated in the past and that's why there are so many deployed applications using it. But that is the past, not the present.
This unfortunate blog post spreads further the misinformation that Python 3 has no traction and that it is little used in the real world. This fuels the rubbish assertion that Python 3 is not being taken up at all.
If measuring the number of deployments in any way reflects current makeup of a technology then I'm willing to bet that Perl is absolutely huge in 2014. How about we see some stats from NewRelic showing number of deployed Perl applications, along with the suggestion that based on number of deployments Perl is not a declining technology but a hugely popular one.
The real question is not "what version of Python is used on all deployed Python systems", but rather "what version of Python are people using for projects currently in active development". That's a much harder thing to find real world information on.
I'm no statistician and I don't think the blogger in question is either. Examining history however does not tell us what the present looks like. NewRelic is doing no favours to the Python community with its vague pseudo statistical analysis.
[deleted]
I posted this because I was trying to come up with a way of measuring python 3 adoption. I thought to look at global package downloads, but decided that the data there would not really really measure adoption especially with packages that are 2 and 3 compatible. Then I thought to look at the number of setup.py classifiers with python 3 vs 2, but I didn't figure out a good way of quantifying this over time. I might have done a Google search for a given time frame for each of the classifiers, but searching for them at all seemed difficult. I stumbled upon this and thought it was probably the best measurement technique available for tracking python3 adoption.
The number obviously has some flaws. Python 3.3 as the only python3 version and having only had support for a few months means that there may have been many companies not yet able to put their python 3 apps onto NewRelic. I interpreted the number differently however and hopefully we can get some assistance in disambiguating the figures. I figured that the number was the proportion of active applications using each version. If the number is the total historical deployments, it would be more difficult to translate into interesting terms. Even looking at active number isn't great.
The number of requests routed through each version might be a better indicator of the market share of each version and perhaps Graham could find that number. That would hopefully even out the many wordpress like sites which use prepackaged software (which I assume have less version inertia than dissimilar codebases). Limiting to the paid tier may also help to improve the meaning of the number as small inactive sites are probably less likely to pay for the service (didn't know they offered a free tier before).
What are the units of your proposed real question?
what version of Python are people using for projects currently in active development
Is that developer-hours per python version? I could see that also being a very meaningful number but possibly much harder to approximate than requests / python version. Even distinguishing whether a line of code is python3 or python2 is difficult and many programs are designed now to be both.
My goal was not to show a perfect statistic, but instead to show how it might be measured and I'm glad that you brought up the measurement. Reviewing exactly what was measured and how large NewRelic is might help motivate a better number coupled with a confidence interval.
Looking at StackOverflow questions might also be interesting though I never investigated what was available. Sum historical data, as you have brought up, would be more difficult to draw inferences from.
I am on of those old stubborn 2.7 people. I just like it when things work, when I dont have to google for solutions and I can focus on what my client wants. I have an irrational fear that switching to 3 means changing that.
First step is understanding that it's irrational. Python 3 works.
Yes, a fear that porting existing code to Python 3 will introduce bugs is realistic. But you should not be afraid of starting new projects in Python 3.
Not big (yet, anyway), but as a partner of my app development company, I (and everyone on my team) uses Python 3 for all web apps and backend code.
I'm in the process of rewriting eventlet from the ground up (it's only < 6k sloc) for Python 3, building on top of pyuv. The performance numbers already look very good and so far it integrates perfectly with the popular Python 3 libraries that we use. I'll publish on github when it's ready.
I'm hoping this will help other people pick up Python 3 and show that Python 3 is a very solid, high performance platform for writing highly concurrent web apps.
Edit: I just have to get this out of my system: the eventlet code is hideous and looks very hackish. Doesn't help that they have very ugly usage of their "six" module. Getting rid of Python 2 support is a breath of fresh air - everything is so much cleaner in Python 3.
Cool project. I wonder how long it will be before six passes into history like from __future__ import with_statement. I don't have very much experience using six. Can you pos some of the ugly usage? They seem to be more focused on six.moves than I've seen before (mostly try: except import defines). I think I prefer six.moves. Do you have any clue why six.py seems to be included raw so often?
i was monitoring my python 3.4 uwsgi webapps with newrelic but lately I had problems with uwsgi hanging on launch so i removed new relic and now everything is fine.
I use python 2. To be honest, if my OS (Mac) had python 3 as /usr/bin/python, I'd use that.
The fact I have to explicitly say python3 makes me nervous, because I hate to deviate from the norm when it comes to platform.
virtualenv?
alias python=python3 should work. Pointing python to python3 outside of the shell though will cause issues.
Sad to see the statistics showing adoption that low.
What significant/important sites run Python 3 in production? Apparently python.org requires Python 3.3, but are there any other large or well known sites which also run on Python 3?
Sad to see people believing that the numbers after 3 months of python 3 support mean... well, anything at all. (Note the "It should be noted that we only started supporting Python 3 in September of 2013" and the "21 January 2014" dateline.)
I would be fascinated to see their data after a year, but this isn't it.
Let's try to summon Graham Dumpleton to rerun these numbers!
[deleted]
Ooops, didn't realise this was a blog post from January 2014.
Would be nice to see updated numbers.
My suspicion is that python3 is popular among scientists looking to prototype a model or do some quick data analysis--and that is primarily motivated by the responses to "who is using python3 in production?" threads--but that companies with existing python code bases are not actively porting. I don't have much information though some I'm not too confident in my assessment, though this blog entry backs my guess where it relates to websites.