ENTRIES TAGGED "erlang"
Phil Dibowitz explains the challenges and the results they got with Chef
At OSCON, Phil Dibowitz reminded me how little I understand about large systems – as he puts it, really large systems, systems of systems, with some similarities but with different people controlling parts. His work at Facebook explores the challenges (and opportunities) of creating tools that work across a company’s many networks and computers.
If you deal with such challenges, he’s worth listening to as a model. If not, he’s worth listening to for a sense of just how different work at this scale can be, though much of what he accomplishes can be worthwhile at scales much smaller than the 17,000 servers he describes for Facebook at 26:42 in the session.
I talked with him in an interview:
and we’ve posted his OSCON session:
This summer, I’ve seen all kinds of programming approaches as I’ve bounced between the Web, XSLT, Erlang, and XML, with visits to many other environments. As I look through the cool new possibilities for interfaces, for scaling up and down, and for dealing with data, I keep seeing two basic patterns repeating: walking trees (of data or document structure), and handling events.
Every programming experience teaches
I’ve never formally trained to be a programmer, outside of occasional conference workshops and a week of XSL tutorials. In some ways, that’s terrible, because it’s taken me about thirty years to learn what some friends of mine appear to have learned in four. I’ve written some code that goes way beyond spaghetti, though fortunately the worst of it was probably when I was 15.
On the bright side, when I look past my many mistakes, I can see what I learned from a large number of various different experiences, and the pieces they helped me see. It’s a little easier to tell this story through the parts than it might be through a formal curriculum.
- My parents’ FORTRAN books
- I was reading computer books—dry ones—before I even got to play. I have vague memories about program structure, but mostly I learned that knowledge sticks better if it includes hands-on work, and not just a book.
- Sinclair ZX81
- 1K of memory! The sheer thrill of seeing my creations on screen was amazing. I had just enough logic to get things done, and leave myself puzzled. The Sinclair community seemed focused on making great small things. I learned simple logic in BASIC and that sometimes it takes a hack to get things done.
- Applesoft BASIC
- After Sinclair BASIC, Applesoft seemed vast. Much of what I did was transfer what I’d done on the Sinclair (itself a lesson in platform-shifting). As I settled, I started writing larger and larger programs, eventually forcing myself to restructure everything into subroutines…with global variables, of course.
- 6502 Assembly
- I knew there was more than BASIC. My early adventures with assembly language were mostly about graphics, and didn’t work all that well, but I picked up two key things: recursion and the importance of registers.
Software grows patient
The biggest change I’ve seen in the last few years of software development isn’t a new language, a new environment, or magical new algorithms. The biggest change is that programmers in many different arenas, working independently, have come to accept waiting.
Part of the joy of computers, a magic that grew and grew as computers and networks got faster and faster, was the confidence that making a change was immediate. Yes, it took time to get a message over a network or for a CPU to execute calls—but things happened.
The event loop has been around for a long time—computers have always had to wait for us slow humans. Transactions have provided a buffer against the possibility of simultaneous changes to the same data, and certainly slowed things down, but the time that buffering took was generally considered a cost, not a feature. Message queues have existed for a long time, but again, seemed bulletproof but expensive.
Over the last few years, these approaches have become more common and better understood. I suspect that there are two main drivers of this change:
As larger scale projects have become more common, the knowledge needed to use these approaches has become more widely available. The average project may be larger scale as well, but these techniques are appearing even in cases where I wouldn’t have thought them necessary. (Of course, I also like playing with Erlang in tiny single-user environments.)
I saw a great talk last week on IndexedDB. It wasn’t the data storage or the asynchronous API that grabbed my attention, but the conversation about promises and ways to make “wait for it” seem like a normal programming idiom. There are a lot of those conversations happening now, about many environments.
Should we make asynchronous seem normal with syntax sugar? Or should we flag it, call attention to it, and make sure programmers remember that their code is waiting?
A few iterations make even functional programming digestible
Programming gets easier and easier as you do more of it. Languages and logic become familiar, in ways that extend from looking up less and less all the way to muscle memory in your typing. The first few iterations in a new language are often difficult, as everything seems odd. Shifting not just to new languages but to new approaches makes practice even more important.
One of my worst features has been that I never liked to practice. I understood rehearsing, working together in a group, but practicing? Exercises, especially for the sake of exercise, were never my thing. That character flaw has hobbled my writing. I can come up with examples for books, focusing tightly on an issue I’m explaining. But then, really? I have to ask readers to go solve a particular (already solved) problem?
I’m not alone in this—too many writers have inflicted that attitude on their readers. Exploring sample code is not the same as writing your own code. Miguel de Icaza points out that:
When you finish a chapter in a modern computing book, there are no exercises to try. When you finish it, your choices are to either take a break by checking some blogs or keep marching in a quest to collect more facts on the next chapter.
During this process, while you amass a bunch of information, at some neurological level, you have not really mastered the subject, nor gained the skills that you wanted.
We can fix this.
Steve Vinoski on when to make the leap to functional programming.
Functional programming has a long and distinguished heritage of great work — that was only used by a small group of programmers. In a world dominated by individual computers running single processors, the extra cost of thinking functionally limited its appeal. Lately, as more projects require distributed systems that must always be available, functional programming approaches suddenly look a lot more appealing.
Steve Vinoski, an architect at Basho Technologies, has been working with distributed systems and complex projects for a long time, first as a tentative explorer and then leaping across to Erlang when it seemed right. Seventeen years as a columnist on C, C++, and functional languages have given him a unique viewpoint on how developers and companies are deciding whether and how to take the plunge.
Highlights from our recent interview include:
Stuart Sierra on why Clojure is catching on.
Stuart Sierra digs into Clojure: what it is, how it works, and why it's attracting Java developers.