ENTRIES TAGGED "Python"
Lightweight Django by example
The following comes to you from Julia Elman and Mark Lavin. Julia is a a hybrid designer/developer who has been working her brand of web skills since 2002; and Mark is the Development Director at Caktus Consulting Group in Carrboro, NC where he builds scalable web applications with Django. Together, they are working on Lightweight Django, a book due out later this year that explores bringing Django into modern web practices.
Despite Django’s popularity and maturity, some developers believe that it is an outdated web framework made primarily for “content-heavy” applications. Since the majority of modern web applications and services tend not to be rich in their content, this reputation leaves Django seeming like a less than optimal choice as a web framework.
Let’s take a moment to look at Django from the ground up and get a better idea of where the framework stands in today’s web development practices.
Plain and Simple Django
A web framework’s primary purpose is to help to generate the core architecture for an application and reuse it on other projects. Django was built on this foundation to rapidly create web applications. At its core, Django is primarily a Web Server Gateway Interface (WSGI) application framework that provides HTTP request utilities for extracting and returning meaningful HTTP responses. It handles various services with these utilities by generating things like URL routing, cookie handling, parsing form data and file uploads.
Also, when it comes to building those responses Django provides a dynamic template engine. Right out of the box, you are provided with a long list of filters and tags to create dynamic and extensible templates for a rich web application building experience.
By only using these specific pieces, you easily see how you can build a plain and simple micro-framework application inside a Django project.
We do know that there are some readers who may enjoy creating or adding their own utilities and libraries. We are not trying to take away from this experience, but show that using something like Django allows for fewer distractions. For example, instead of having to decide between Jinja2, Mako, Genshi, Cheetah, etc, you can simply use the existing template language while you focus on building out other parts. Fewer decisions up front make for a more enjoyable application building process.
Assertions, regression tests, and version control
Programming any non-trivial piece of software feels like rock climbing up the side of a mountain. The larger and more complex the software, the higher the peak.
You can’t make it to the top in one fell swoop, so you need to take careful steps, anchor your harnesses for safety, and set up camp to rest. Each time you start coding on your project, your sole goal is to make some progress up that mountain. You might struggle a bit to get set up at first, but once you get going, progress will be fast as you get the basic cases working. That’s the fun part; you’re in flow and slinging out dozens of lines of code at a time, climbing up that mountain step by steady step. You feel energized.
However, as you keep climbing, it will get harder and harder to write each subsequent line. When you run your program on larger data sets or with real user inputs, errors arise from rare edge cases that you didn’t plan for, and soon enough, that conceptually elegant design in your head gives way to a tangled mess of patches and bug fixes. Your software starts getting brittle and collapsing under its own weight.
Unlocking Scientific Data with Python
Most people working on complex software systems have had That Moment, when you throw up your hands and say “If only we could start from scratch!” Generally, it’s not possible. But every now and then, the chance comes along to build a really exciting project from the ground up.
In 2011, I had the chance to participate in just such a project: the acquisition, archiving and database systems which power a brand-new hypervelocity dust accelerator at the University of Colorado.
Recharging the Python standard library
It’s unfortunate that the official About Python page still describes Python’s standard library as having “batteries included.” Sure, some of those old standbys will keep your project going and going, but many of them are leaking acid all over the place. Guido Van Rossum, head developer of Python, has said “the stdlib offerings … are not very convenient and may not support popular idioms very well.” Five years ago, I always assumed the Python library contained the “best of breed” for all packages. These days, I tend to think the opposite.
To counteract this minor flaw, I keep a small “personal standard library.” I keep a pip requirements file listing all the packages I use in every project. A simple script automatically installs that file whenever I create a virtualenv for a new project. With the pip download cache enabled, this is a near-painless process.
An interview with Allen Downey, the author of Think Bayes
When Mike first discussed Allen Downey’s Think Bayes book project with me, I remember nodding a lot. As the data editor, I spend a lot of time thinking about the different people within our Strata audience and how we can provide what I refer to “bridge resources”. We need to know and understand the environments that our users are the most comfortable in and provide them with the appropriate bridges in order to learn a new technique, language, tool, or …even math. I’ve also been very clear that almost everyone will need to improve their math skills should they decide to pursue a career in data science. So when Mike mentioned that Allen’s approach was to teach math not using math…but using Python, I immediately indicated my support for the project. Once the book was written, I contacted Allen about an interview and he graciously took some time away from the start of the semester to answer a few questions about his approach, teaching, and writing.
How did the “Think” series come about? What led you to start the series?
Allen Downey: A lot of it comes from my experience teaching at Olin College. All of our students take a basic programming class in the first semester, and I discovered that I could use their programming skills as a pedagogic wedge. What I mean is if you know how to program, you can use that skill to learn everything else.
I started with Think Stats because statistics is an area that has really suffered from the mathematical approach. At a lot of colleges, students take a mathematical statistics class that really doesn’t prepare them to work with real data. By taking a computational approach I was able to explain things more clearly (at least I think so). And more importantly, the computational approach lets students dive in and work with real data right away.
At this point there are four books in the series and I’m working on the fifth. Think Python covers Python programming–it’s the prerequisite for all the other books. But once you’ve got basic Python skills, you can read the others in any order.
Testing, Python 3, and Dealing with Technical Debt
Nathan Yergler (@nyergler), Principal Engineer at Eventbrite, and I had a chance to talk Django at OSCON 2013. We talk about why Django is the go-to choice for Pythonistas and about the growing technical debt that each programmer has to deal with on Python projects and beyond.
Key highlights include:
- Django is mature and feature complete amidst many Python frameworks [Discussed at 0:15]
- Testing in Django leads to straightforward code that the next programmer can read as well as you can [Discussed at 1:02]
- Dare we discuss Django’s weaknesses like: Is Django too monolithic? [Discussed at 2:43]
- Django at long last supports Python 3! Check out Django 1.5 [Discussed at 4:06]
- Dealing with technical debt while programming [Discussed at 5:36]
You can view the full interview here:
Teaching Future Coders
Ever since PyCon 2013, the interest in the Young Coders class has been intensifying. Practically every Python conference since then has asked about doing one, and several have run their own. Classes outside of conferences have sprung up, as well, from one time workshops to after school clubs.
As more classes happen, more people have been asking about running their own. These classes do take quite a bit of effort to set up, but the payoff is enormous. Also, once you do one, doing subsequent ones gets easier and easier.
The O'Reilly Open Source Awards 2013
Over the years, OSCON has become a big conference. With over 3900 registered this year, it was hard not to look at the packed hallways and sessions and think what a huge crowd it is. The number of big-name companies participating–Microsoft, Google, Dell, and even General Motors–reinforce the popular refrain that open source has come a long way; it’s all mainstream now.
Which is as it should be. And it’s been a long haul. But thinking of open source in terms of numbers and size puts us in danger of forgetting the very thing that makes open source special, and that’s the individual contributor. So while open source software has indeed found a place in almost every organization that exists, it was made possible by the hard work of real people who saw the need for it, most of them volunteering in their spare time.
The O’Reilly Open Source Awards were created to recognize and thank these individuals. It’s a community-driven effort: nominations come in from the open source community (this year there were around 50) and then are judged by the previous year’s winners. It’s not intended to be political or a popularity contest, but honest appreciation for hard work that matters. Let’s look at this year’s winners.
OSCON 2013 Speaker Series
Automating the configuration management of your operating systems and the rollout of your applications is one of the most important things an administrator or developer can do to avoid surprises when updating services, scaling up, or recovering from failures. However, it’s often not enough. Some of the most common operations that happen in your datacenter (or cloud environment) involve large numbers of machines working together and humans to mediate those processes. While we have been able to remove a lot of human effort from configuration, there has been a lack of software able to handle these higher-level operations.
I used to work for a hosted web application company where the IT process for executing an application update involved locking six people in a room for sometimes 3-4 hours, each person pressing the right buttons at the right time. This process almost always had a glitch somewhere where someone forgot to run the right command or something wasn’t well tested beforehand. While some technical solutions were applied to handle configuration automation, nothing that could perform configuration could really accomplish that high level choreography on top as well. This is why I wrote Ansible.
Ansible is a configuration management, application deployment, and IT orchestration system. One of Ansible’s strong points is having a very simple, human readable language – it allows users very fine, precise control over what happens on what machines at what times.
To get started, create an inventory file, for instance, ~/ansible_hosts that defines what machines you are managing, and which machines are frequently organized into groups. Ansible can also pull inventory from multiple cloud sources, but an inventory file is a quick way to get started:
# add more webservers here
Now that you have defined what machines you are managing, you have to define what you are going to do on the remote machines.
Ansible calls this description of processes a “playbook,” and you don’t have to have just one, you could have different playbooks for different kinds of tasks.
Let’s look at an example for describing a rolling update process. This example is somewhat involved because it’s using haproxy, but haproxy is freely available. Ansible also includes modules for dealing with Netscalers and F5 load balancers, so this is just an example — ordinarily you would start more simply and work up to an example like this: