Bringing Programming Out of Its Temples

If everyone should learn to code, code needs to change

2013 may have seen a tipping point for the popularity of programming. Even people who aren’t programmers are suddenly arguing that it’s a good idea for everyone to learn programming. Computers are now everywhere, traveling with us, common in places where they would once have seemed an extravagance, and it seems obvious that we need to be able to do more with them.

Anil Dash looked beyond the basic vision of teaching everyone to code this week. Dash leads with President Obama’s encouragement of computer science as part of this week’s Hour of Code project, and his suggestions to young Americans:

“Don’t just play the latest video game. Make one.

Don’t just download the latest app. Help design it.

Don’t just play on your phone. Program it.

(It’s not just the President—it’s House Majority Leader Eric Cantor too.)

It is, of course, harder than that. I love Dash’s piece, which reflects on the value of code, the challenges of crossing tech culture boundaries, and the need for tech culture itself to support and make more valuable such efforts at democratizing coding. I shared it on Facebook, happily citing the part about:

we have to embrace the idea of blue collar coders. If we succeed in bringing all of these new young people to the tech industry, we’re going to necessarily undo the image of programming as the exclusive domain of a rarified coterie of high priests. Instead, we have to see it as a useful trade that is a meaningful path toward the middle class for those who are struggling to get there, akin to the blue collar trades that honorably served that purpose in the last century.

Then, I returned to my usual technology programming, and crashed into a story about the perils of “not an A-grade software developer… haven’t completed a formal Software Engineering or Computer Science degree. You have meddled with computers enough to be dangerous…” (It’s far from unique—just a good example I happen to have close at hand.)

Okay, then. I get that people on an XML list can be unhappy when people don’t follow all the rules XML claims to have established. I’m reminded of the years where people looked at me, puzzled that I thought JavaScript had a future as a programming language and not just a tinkering tool. I get that programmers have their hierarchies, and that some tasks demand formal engineering approaches and years of experience just as some tasks in the physical world demand that.

Over the last two decades, though, computers have shifted from being computational devices to being devices for communications. There are still complex situations in those communications, particularly computationally intensive issues like media compression and massive scale. Fewer and fewer communications issues, though, need the kind of skill set that used to be expected for programming. More and more projects need a deep understanding of tasks rather than a deep understanding of code.

However, many programmers seem stuck in a past landscape where the demands for code expertise were considerably higher. Programming languages were tightly bound to the internal structure of the computers, requiring humans to change the way they approached solving many kinds of problems. Resources were limited, and logical optimization the supreme directive. Programmers had to keep vast amounts of information about their program in their heads, because there was no easy place to keep track of it. Testing and debugging could make the invisible visible, but were themselves difficult skills to master.

Despite that resistance, I think Dash’s vision still points the way forward. As business becomes more and more dependent on moving bits from place to place and analyzing them, the cost of separating code creation from the people who use the code is going to become more and more visible. It is possible that programming processes will become more adept at collecting requirements for users and responding to them, or that programmers will find better ways to let their users customize processes, but it seems more likely to me that more people are going to end up coding.

We already have a model for that, though I’ve long enjoyed the gasps from programmers who don’t consider it real programming. Electronic spreadsheets started out with extremely simple logic but have grown more and more capable over the decades, connecting with more advanced code when needed. Spreadsheets provide all kinds of opportunities not just to process data, but to make the calculations and intermediate values observable. Chris Granger of Light Table has a brilliant talk about the challenges of programming not being very observable, but it strikes me that spreadsheets offer at least the possibility of opening the door. They’re also quite capable, if not necessarily efficient. As my friend Ben Rothfeld says, “Give me a spreadsheet wide enough and a monitor to put it on and I could tabulate the world.”

My father was not primarily a programmer. He modeled large business processes for generating investment returns into the future over ten year periods with custom programs. Then, he marveled that they ran so well on luggable Compaqs. Today, those calculations and iterations can run within a supporting spreadsheet including necessary formulas. It took long experience with the business processes to construct the earlier model and generate the calculation, and it takes continuing experience with the process to put in the right production adjustments and data to make sure the results continue to reflect reality. While it’s critically important to get the core logic right, the long-term maintenance and growth of such tools requires regular user feedback. Processes and productivity are altered periodically causing changes in the inputs and outputs of the spreadsheet as well as its results

The need for user connections to software goes beyond modeling, though. Part of what attracts me to flow-based programming is its division of labor between those creating components—largely calculations and transformations—and those assembling them. The first group of people still needs to know all the details of making code efficient, while the second group needs far less of that. Both groups still need to understand the underlying logic of the work, though.

Looking forward, I’m not sure that the spreadsheet grid is flexible enough to deal with data that isn’t tabular, and though spreadsheets are used constantly for communications, that isn’t quite what spreadsheets do. I can’t help thinking, though, that the long-term answer to democratizing code looks more like a spreadsheet than it does traditional text-based code.

Can we split the work this way? Can large numbers of programmers get used to the idea of creating specialty modules, the “hard part”, in ways that are meant for people who code but aren’t necessarily programmers? How much complaining will there be about shifts toward interfaces that make “everyone should code” a reality, and will that complaining make it impossible? Whole categories of programming may have to shift (or disappear) for democratization to work.

Related

Sign up for the O'Reilly Programming Newsletter to get weekly insight from industry insiders.
  • Eric Elliott

    I think it’s arrogant to think that only a small class of elite thinkers could be programmers. I was programming computer games at five years old (not very sophisticated ones, I admit), and I believe strongly that it’s a good age to start the next generation of coders. If a five year old with no introduction to math or logic can pick up a computer and learn to write a video game, anybody can learn to code.

    I think the key is education. We need to make programming easier to digest. Dish it out in bite-sized chunks, and make sure each one of those bites is delicious. There’s so much to learn that it’s easy to get overwhelmed — even for people who are already in the professional field. I think a lot of us are coming to grips with the idea that there is simply no way we can learn everything there is to know about programming. There will always be specialists writing parts of the program that we will never really understand — and that’s OK.

    In the end, it doesn’t matter whether you can personally account for every line of code that goes into your app. What matters is that your app does something useful or interesting — and there’s plenty of room in the world for everybody to contribute something useful and interesting.

    • FishingCactus

      Totally agree with you.

      We are ourselves developing a game about programming because we think that everyone should know a bit of programming. We don’t really teach players to code but instead, we want to teach them the logic behind programming. It’s more like optimize the way they think and make them understand what is coding.

  • Larry Garfield

    This reminds me of “The Future of Programming”: http://vimeo.com/m/71278954

    Really, everyone who’s job is to “write code” needs to see it. Twice.

  • Stephen Cameron

    In general I think the trend that you are describing is correct, but I am quite sure that considerations of risk should still be the main criteria when approaching this issue.

    http://www.forbes.com/sites/timworstall/2013/02/13/microsofts-excel-might-be-the-most-dangerous-software-on-the-planet/

    • Simon St.Laurent

      The Forbes piece is right that there are many problems with spreadsheets. While they _can_ make code more observable, they frequently don’t. Complexity can escalate rapidly, especially when cut-and-paste is your primary approach to modularity. The wide distribution of random spreadsheets (often with minor invisible tweaks even when they seem similar) across organizations creates further problems.

      Though there are programming practices that would help with that, and I can imagine interfaces to integrate them, yes, it’s difficult. Much of the problem is with the people who are supposed to know their field – in talking with my father, he made it absolutely clear that the results need to be compared to reality repeatedly to make sure the models still apply. That gets more difficult as the “results” get more meta, of course, but in theory these folks know what they’re doing…

      The other large drawback of spreadsheets in particular is that they tend to be “owned” by a person or a small group of person, and don’t integrate well into larger processes. Google Docs eases the sharing question but not so much the process question. I’m hoping flow-based programming and its cousins can help make this kind of coding fit into larger (and more observable) systems.

  • Matt Edgar

    I recently heard an IT director at a large healthcare trust bemoan that data in his organisation was scattered across hundreds of spreadsheets and Microsoft Access databases in departments and on individual desktops. His solution was one patient record to rule them all. But it struck me that every one of those spreadsheets and Access databases was an artefact of innovation on the care frontline. How much poorer would patient care be if all that embodied knowhow were trampled underfoot in the quest for some possibly illusory benefits from standardisation? There seem to be two challenges for software here: 1. to create tools that set each worker free to use her domain knowledge, her problem-solving skills and her creativity in the pursuit of her everyday purpose; and 2. to join up her solutions with those of other workers, near and far, to form more complex systems and perform bigger tasks collectively. For all the promise of “big data” you can’t have 2 without first nurturing 1.

  • gibarto

    When I was a kid, I learned BASIC – Beginner’s All-Purpose Symbolic Instruction Code, with an emphasis on Beginner’s. A lot of us did a lot with this language in the 80s and 90s – it was the only way to get much use out of a personal computer back then. Today, I still use what I learned for VBA with my Excel sheets, but people who didn’t grow up with BASIC are mystified by such things.

    Today, between DSLs and packages, there’s no reason we can’t create a new general-purpose language. I think Python is half-way there: O’Reilly sells a ton of books where you add a package to Python and the next thing you know you’re doing data mining, linguistic analysis, social media data analysis and more. How about a package where instead of the domain being gaming (pygame), science (scipy) or something like linguistics (NLTK), the package is designed for beginners to do a little bit of graphics, a little bit of data table analysis and a little bit of text processing, maybe simply by providing a simple, unified way to access the basic features of the more common packages?

    The most important element to learning to code, however, is not so that we can all become software designers, but so that we get some idea how computers process things and what they can and cannot do with data. I have worked with a lot of people who don’t understand why the magical computer can’t get information out of a database when such information is never put in or why it’s hard to simulate a process where a person with experience in the field guesstimates here and there before punching the numbers into a pocket calculator but has no formal process for deciding which numbers to punch in. At the same time, I see a lot of other people underestimate the possibilities for transforming data you do have if IT hasn’t already created a report for doing so. The biggest value of doing a little coding is not just that you can do basic tasks yourself, but that you can learn to think about what information you have and how it can be transformed before you ask a programmer to create something new. How many messes could be averted if only the people who love their Excel sheets had to save to CSV and do something with the data in R, as opposed to thinking that if you add another datasheet and paste some information into it, the eggheads can figure out how to get something useful out of it?

    • marty alain

      « How about a package where instead of the domain being gaming (pygame),
      science (scipy) or something like linguistics (NLTK), the package is
      designed for beginners to do a little bit of graphics, a little bit of
      data table analysis and a little bit of text processing, maybe simply by
      providing a simple, unified way to access the basic features of the
      more common packages? »
      I propose this kind of answer : http://epsilonwiki.free.fr/alphawiki_2/
      - a tool for coding a web page, from simple things to more complex ones, in a unique syntax, something like (a tiny) HyperCard/HyperTalk on Internet.
      Do you think it could be an answer to your question ?

      Alain Marty

      • gibarto

        My concern is with the phrase “a unique syntax”. The thing I like about Python is that while packages can simplify coding, you’re working in an environment where you can do the hard stuff later on in an identical environment so there’s room to grow in any direction. For your link, I’d be more comfortable if, e.g., a JavaScript package was letting you do some neat stuff with minimal effort but you were also writing barebones html and js at the level you could master to that point.