Category Archives: programming

What systems are available for end-user programming.

Of course you should learn to code!

I just wanted to respond to a recent volley in the ongoing “should you learn to code?” blogging wars. Ciara Byrne writes in No–You Don’t Need To Learn To Code:

Imagine being harangued by a deluge of articles with headlines like “Why everyone should learn to knit.”  With all due respect to knitters, it would seem ridiculous. …  I’m a programmer. Do I think everyone should learn how to code? No. Here’s why.

She proceeds to argue that better than learning to code is learning to understand coders, to learn how everyday software works, and to learn something else entirely like yoga.

Since I am putting a ton of energy into creating a new introductory computer science course that doesn’t emphasize programming (“A First Byte of Computer Science“), one might think I agree with the sentiment of the article. I don’t. Here’s why.

I think the issue is: What do we mean by “learn to code”? Or, for that matter, “learn to knit”? I don’t know about you, but I was forced to do arts and crafts in school and summer camp as a kid. Why? Did they expect me to become an expert craftsman? To start a sweater business? No. I’m sure, in part, they were just trying to keep us busy. But, I bet part of it was that learning how to make things with your hands is a huge win in life. Even if you never make things professionally, it helps you understand how things are made, how they can be used most effectively, and what to do when they misbehave. And, sometimes, you need to cobble something together quickly because you have a need that is acute (like fixing my zipper when it got stuck right before a job interview) or short-lived (like creating a halloween costume with my daughter). I have drawn on my experiences as a once-a-summer knitter to bail myself out of many binds as a grown-up. 

To me, “learning to code” is more like “learning arts and crafts” and less like “learning to become a professional-grade knitter”. Casual coding is valuable for analogous reasons: it gives you an understanding of how software works and enables you able to do something when the need strikes.

Here’s another analogy: Imagine you move to Colombia for a year. Should you learn Spanish? No, some might say. Learning Spanish is extremely time consuming. There’s almost no chance you’ll become the craftsman Gabriel García Márquez is or even get rich as a Spanish journalist. But, that’s not the point, is it? By learning some Spanish, you’ll find it easier to get around. You’ll be able to let people know what you need without having to arrange for a translator for everything. And, you might discover your Spanish does help you get rich because it allows you to make valuable connections to the people around you so you know where the needs are and can jump in and bring to bear your own special talents. It’s not the Spanish that’s valuable per se but the ability to communicate. That’s what coding is. It lets you tell machines what you need them to do. It’s a means to an end and not an end unto itself. Yes, you should learn to code. We’re surrounded by machines that are here to help us and most of us can’t even tell them what we want.

Her argument that “Programming is not a get-rich-quick scheme” is a straw man. The article that she lampoons as “Why everyone should learn to knit” is actually entitled “Why every single one of you should learn a little code“. A little code. Take a moment to learn a powerful idea. It’s a small investment of time and it might be fun. It’s just a scarf, not a hundred years of solitude.

Wired’s Programmable World

Wired magazine has an article on what they call the “Programmable World”.  (Thanks for the pointer, Blase!).  I feel like that’s a pretty good description of the world I want to live in, so I read the article: .

Here’s a short reaction to it.  I liked it.  Here’s a (slightly) longer reaction.

The article asserts that the programmable world will arrive in three stages:

1. More and more devices get onto the network.

2. These devices work together to automate tasks.

3. The devices are organized into a programmable software platform that can run apps.

I suppose I agree with this sequence in terms of the underlying technology, but—as is often the case—the evolution of the technology is less interesting to end users than the evolution of the end user experience.  Here’s my take on what the transformation could look like to regular folks:

1. More and more devices become available with their own iphone/android apps.  That’s a safe and easy prediction because it’s already well under way.  I just bought a Jawbone Up (an activity monitor), for example, and it comes with a free iphone app for plotting daily activity and sleep cycles.

2. Cross-device interfaces like IFTTT or Smartthings (which I’ve written about in the past) will make it possible for end-users to specify simple programs that involve multiple devices.  I’m not as sure about this prediction.  End user programming may or may not catch on (although I deeply believe it should).  Regardless of how popular they will become, these cross-devices interfaces are already here.  In fact, I bought the Up because I have a rule that says “if a device is advertised as working with IFTTT, I must buy it.”  So, I have a little iFTTT program now that turns on colored lights (Phillips Hue, which also just got integrated with IFTTT) in my apartment if I reach my per-day step goal.  It’s not at all useful and it doesn’t work the way I want it to.  But, it’s cool that it’s becoming possible to knit these gizmos together at all.

3. We move from a sensor-action model to an event-activity model.  I’m not really seeing this stage in the real world yet, but the more I play with programmable devices, the more I think the whole enterprise flounders without it.  Here’s a concrete example.  The Wired article talks about  automating our daily tasks.  They mention that a computational system should be able to observe the rules you the user follow and then adopt them itself.  So, if you behave according to the rule “If  the sun hits your computer screen, then you lower a shade,” then the computer could do it for you and save you a step. That’s fine, but here’s the thing.  That’s not really the rule we follow.  It’s something more akin to “if I need to see the screen and I can’t, do something to make the screen visible.”  There is no sensor that can directly measure whether you can see the screen.  Such an event needs to be inferred from a set of observables like the brightness of the screen, the direction you are looking, how long it has been since the mouse was moved on the computer, the time of day, whether the shades have already been pulled, etc.  And the action of pulling the shades isn’t always the right response.  It might be more effective to brighten the screen, rotate it a little, or perhaps dim the lights in the room.  We need a computing infrastructure that (1) can integrate information across devices, (2) infer the desirability of possible interventions, and (3) orchestrate a coordinated response across devices.

From the end user’s perspective, this last step requires careful interaction with the devices.  They need to learn from you without distracting you.  The line between teaching, training, and programming begins to blur as we tell the computing systems what matters to us, how it can be measured, and what to do in response to undesirable situations.

Well, that’s the trajectory, anyway.  For now, the best bet is to focus on getting an infrastructure in place so we can start experimenting with different models of interaction.  We have lots of goals for this summer and I hope to be able to report on exciting and steady progress over the next few months!

Some things About SmartThings

It’s been a busy semester and I haven’t been able to find time to write.  And the longer I wait, the more I feel like the next post should be really good to justify the gap.  But, now it’s Spring Break and maybe I should just write something bad just to take the pressure off.  🙂  Here goes.

Yesterday, I was fortunate to have a visit from the former CTO of iRobot, the makers of the Roomba vacuum cleaning robot and perhaps history’s only profitable robotics company.  Tom Wagner stopped by mainly to talk about the Human-Robot Interaction initiative my colleagues at Brown and I are trying to sell.  But, since he was there, I took the opportunity to tell him a bit about the end-user programmable devices project.  Blase Ur participated from Pittsburgh via a robotic telepresence device called a Vgo.


We demoed several programmable devices—a fan, a blender, and two strings of colored lights—and asked Tom for feedback.  He was amazingly insightful and refreshingly forthright on a number of issues.  Unfortunately, many of the juiciest tidbits he shared were prefaced with “Don’t blog this, but…”.  Oh, well.  But, one excellent comment he made that got us thinking was “How does your project relate to SmartThings?”

Mmmm.  Well, I didn’t recognize the name at first, but SmartThings is a project that Vukosi Marivate brought to my attention as a great example of work relevant to end-user programming that was seeking funding through KickStarter.  I probably should have followed up right away when he first told me, but I always feel very much teased by KickStarter campaigns.  The pleas for funds are very compelling, but so many of the amazing ideas never materialize that it’s often a big letdown.

Briefly, SmartThings is a venture to create a networking infrastructure for programmable physical devices.  In the words of their homepage, “SmartThings adds intelligence to everyday things in your world, so that your life can be more awesome.”  Huh.  Awesome is good.  They’ve got over a million dollars in funding through KickStarter and three million in seed funding.  They’ve got over 30 people working on programmable devices.

Their vision is to have a platform so that developers can create apps that interface with physical devices.  They’ve got a cool hub that all the various devices in your home can talk to.


The hub acts as a relay station for messages between the devices and the SmartThings cloud server.  Then, you can create apps that communicate with the cloud to monitor and control the devices.  It’s even possible to reconfigure and automate the behavior of the devices.  In addition to their own custom devices, they are hard at work making other wifi devices compatible with their system.  All the devices I’ve bought for my apartment in the last few months appear in their preliminary list of compatible devices, which is very encouraging!

I think SmartThings is an exciting development, but it’s hard not to feel a little competitive with them.  They did a demo of their system January 2013 that features the creation of an app that monitors a switch to turn a light on and off.  The audience loved it.  But, our upod project demoed precisely that same functionality (and more!) in November 2011.  We had a cloud server, a switch, lights, and a programming interface.  We didn’t have dozens of people and millions of dollars.  We had a handful of undergraduates and a few thousand dollars for parts.

For me, the frustrating part is that nothing like SmartThing existed back in 2010 when we started.  If it had, we wouldn’t have put the energy into creating our infrastructure and would have used theirs.  But, we couldn’t use theirs because it didn’t exist.

Based on their demo, it appears that they have two programming interfaces for their system.  One is very very close to the When-do interface we built for upod (which is, itself, inspired by the if-this-then-that design).  In our experience, this interface style is extremely easy to use, but also quite limited in its ability to express anything beyond the most basic programming functionality.  Their second programming interface is essentially raw code—lots of keywords and curly braces and colons.  It provides much more complete functionality but at the expense that non computer scientists would find it impenetrable.

In the upod project, we experiments with a kind of middle ground, which is a fully functioning programming language that is friendly and easy to use.  Specifically, we extended Scratch with blocks for listening to and controlling devices.

Looking ahead, we think there’s a lot more that needs to happen to make device programming friendly for the end user.  In particular, real-world devices get their information from real-world sensors.  Whereas if-this-then-that has recipes that can be specified with absolute precision like “When a new book is added to Kindle Top 100 Free eBooks, send me an email”, real-world sensors are more vague and messy.  Consider a recipe that appears in a screen shot in SmartThings’ advertising video: “when it’s going to rain, check if Barkley is in the yard”.


smartthings1 (1)


It’s a perfectly reasonable recipe and exactly what you’d want to be able to say as an end user.  But, questions like whether it’s going to rain or if a dog is outside hide a tremendous amount of complexity about how information is gathered from the physical world.  I think it’s terribly unlikely that SmartThings has this functionality working in a way an end-user can leverage yet.  But, it’s a lovely and important research question.  And, it’s what I’m anxious to work on assuming SmartThings really has solved the infrastructure problem.  I can’t wait until summer!


Merry Chr-IFTTT-Mas

It’s Christmas morning and I awoke to find a single email in my inbox.


It’s from a service called IFTTT (“if this, then that”) and it is a friendly reminder that, if I want to make a blog post once a week, it’s that time again.  It even helpfully gives me a link to the blogging site, since I’m inclined to have to look that sort of thing up.

I really like the form of this reminder mainly because I made it myself.  It’s a gift from me to me, made possible by IFTTT.  And it’s just what I wanted!

If I were giving out an award for “best attempt so far at an end-user-programming system” award, IFTTT definitely gets a nomination.  It has three of the four attributes that I think are critical:

  1. Relevance.  It lets people control something that actually matters to them in their lives.  In this case, it’s primarily social networking sites.
  2. Simplicity.  The designers have bent over backwards to make creating programs (“recipes”) easy, friendly, and even a little fun.
  3. Universality.  Instead of each social networking site developing its own end-user programming interface, IFTTT is a single system that works for all of them.  That’s key because it means the investment of time and energy a user makes in learning about one site pays off when applying her knowledge to other sites.  It also means that the user can start to mix and match functionality across sites, which isn’t possible if each site’s interface is independent of the others.
  4. Expandibility.  Unfortunately, IFTTT falls short in this category.  The simple form all programs must take is not completely general and it doesn’t take long before a user can think of a perfectly reasonable program that cannot be constructed within the constraints of IFTTT.

I’ll give you a little tutorial to give you a sense of what IFTTT can do.  (It’s a free service, so consider this description an invitation to sign up and experiment.)  The central concept in IFTTT is that of a channel.  To a first approximation, a channel is essentially a web service that you subscribe to.  IFTTT supports 59 channels right now, with maybe one or two added each month.  (I’ve set up 23 of the channels so far, which often means I need to give IFTTT my passwords to these services.  Hmmm, we’ll need to discuss privacy and security issues another time!)


Most sites that are on my radar are included as channels, like youtube, Facebook, Google docs, and Gmail.  There are a bunch of channels that are on the periphery of my awareness that show up as well, things like bitly, Flickr, and Delicious.  Then there are a bunch that I’m utterly oblivious to and in many cases not even sure how to pronounce, like ffffound!, Svpply, Diigo, and Moped.  Aside from making me feel old and clueless, the presence of these channels doesn’t seem to cause any trouble.

An IFTTT recipe consists of filling in the “this” and the “that” in “if this, then that”.  A “this” is filled in with a “trigger” and a “that” is filled in with an “action”.  Each channel includes these two categories of functionality, capturing a piece of what you can do on the channel’s corresponding site.  I think an example will help make things more concrete.  My friend Tim was writing a personal blog for a healthcare technology site.  I didn’t want to miss any of his posts, so I made the following IFTTT recipe:


It checks the company’s blog feed for new posts that include the phrase “tim’s blog” (something that I had noticed was consistent to all of Tim’s articles).  When this trigger from the “Feed” channel fires, it results in an action from the “Email” channel that sends an email to me with a link to Tim’s writing.  This recipe successfully kept me informed of Tim’s posts and, I hope, will let me know if he starts writing again.

Other IFTTT recipes I use on a regular basis include letting me know (via something called Boxcar) whenever my daughter Molly posts a picture on Instagram, posting a status update for my Facebook friends whenever I put up a new video on youtube, and the latest addition that sends me an email reminder every Tuesday that it’s time to blog.  By and large, I’ve found it useful to be able to make these little programs to fine-tune my online life and then not have to think about them.

One recipe that didn’t quite work was my attempt to log temperatures during the day.   (I’m interested in the question of how reliable weather reports are and can never seem to find data that links predictions to the actual weather that happened!)  IFTTT has a clock channel with a trigger that can be made to fire every hour.  It has a weather channel that includes forecast information and current weather conditions.  And, it includes a Google Drive channel that has an action for creating a row at the bottom of a Google spreadsheet to log data.  These are exactly the pieces I need to collect my own weather data!  Unfortunately, IFTTT recipes are combinations of triggers and actions from two channels, not three.  There’s no way to put together the behavior I want.  I’ve contented myself with a recipe that logs the predicted high and low temperature each day at sunrise.


Ok, not completely contented.  In fact, I’m not contented at all.  I want a language that fixes this kind of limitation.  I’ll let you know when I’ve got it.

But, in the spirit of the day, let me just say that I’m a fan of IFTTT.  Not only have I found it interesting as a case study in end-user programming, but I’ve incorporated into my life and I’m glad I did.  Merry Chr-IFTT-mas, everyone!