Author Archives: mlittman

Of course you should learn to code!

I just wanted to respond to a recent volley in the ongoing “should you learn to code?” blogging wars. Ciara Byrne writes in No–You Don’t Need To Learn To Code:

Imagine being harangued by a deluge of articles with headlines like “Why everyone should learn to knit.”  With all due respect to knitters, it would seem ridiculous. …  I’m a programmer. Do I think everyone should learn how to code? No. Here’s why.

She proceeds to argue that better than learning to code is learning to understand coders, to learn how everyday software works, and to learn something else entirely like yoga.

Since I am putting a ton of energy into creating a new introductory computer science course that doesn’t emphasize programming (“A First Byte of Computer Science“), one might think I agree with the sentiment of the article. I don’t. Here’s why.

I think the issue is: What do we mean by “learn to code”? Or, for that matter, “learn to knit”? I don’t know about you, but I was forced to do arts and crafts in school and summer camp as a kid. Why? Did they expect me to become an expert craftsman? To start a sweater business? No. I’m sure, in part, they were just trying to keep us busy. But, I bet part of it was that learning how to make things with your hands is a huge win in life. Even if you never make things professionally, it helps you understand how things are made, how they can be used most effectively, and what to do when they misbehave. And, sometimes, you need to cobble something together quickly because you have a need that is acute (like fixing my zipper when it got stuck right before a job interview) or short-lived (like creating a halloween costume with my daughter). I have drawn on my experiences as a once-a-summer knitter to bail myself out of many binds as a grown-up. 

To me, “learning to code” is more like “learning arts and crafts” and less like “learning to become a professional-grade knitter”. Casual coding is valuable for analogous reasons: it gives you an understanding of how software works and enables you able to do something when the need strikes.

Here’s another analogy: Imagine you move to Colombia for a year. Should you learn Spanish? No, some might say. Learning Spanish is extremely time consuming. There’s almost no chance you’ll become the craftsman Gabriel García Márquez is or even get rich as a Spanish journalist. But, that’s not the point, is it? By learning some Spanish, you’ll find it easier to get around. You’ll be able to let people know what you need without having to arrange for a translator for everything. And, you might discover your Spanish does help you get rich because it allows you to make valuable connections to the people around you so you know where the needs are and can jump in and bring to bear your own special talents. It’s not the Spanish that’s valuable per se but the ability to communicate. That’s what coding is. It lets you tell machines what you need them to do. It’s a means to an end and not an end unto itself. Yes, you should learn to code. We’re surrounded by machines that are here to help us and most of us can’t even tell them what we want.

Her argument that “Programming is not a get-rich-quick scheme” is a straw man. The article that she lampoons as “Why everyone should learn to knit” is actually entitled “Why every single one of you should learn a little code“. A little code. Take a moment to learn a powerful idea. It’s a small investment of time and it might be fun. It’s just a scarf, not a hundred years of solitude.

Bathroom Humor

New York Times blogger Nick Bilton writes, in “Disruptions: As New Targets for Hackers, Your Car and Your House“, that researchers at the DefCon conference reported hacking into hybrid cars and aggressively influencing their behavior.  The blog says

The researchers completely disabled a driver’s ability to control a vehicle. No brakes. Distorted steering. All with a click of a button. 

Eek.  Even more vivid descriptions can be found on the Forbes.com site by a journalist who was driving a hacked car and experienced the effects first hand.  It’s definitely frightening, but strikes me as a little overblown.  The current round of hacks require that the hacker have a device physically connected to the diagnostic port on the car, so it’s not like any old baddy with a smartphone can crash your car.  On the other hand, wireless attacks on cars have been reported in the past.  Colleagues in my former department showed how to mess up a car by communicating with its wireless tire sensors from bluetooth transmitters by the side of the road.  So, while people are not reporting that any of these life-threatening interventions are happening in the real world, it should give one pause.

 

In addition to sounding the alarm about breaking into cars over the Internet, the article cites some risks of using Lockitron, an online door lock that I have on order, online lighting like the Hue I have, and Internet-controlled refrigerators.  There’s a real concern that Internet-enabled user-programmable devices of the kind I advocate could open us up to some serious malicious mischief if designers don’t keep security at the forefront of system design.

The line that convinced me it was worth making my first blog post in many weeks, however, was in the context of the INAX wireless toilet.  This state-of-the-art commode sports, not one, but four automatic features.  Here’s the description from their site:

  1. Automatic Lid & Heated Seat: When you approach the toilet, the lid opens and the heated seat is activated.
  2. Sound Module: When the lid automatically opens, music from the sound card will begin to play and the deodorizer will be activated.
  3. Automatic Flushing & Deodorizing: When you step away from the toilet, it will flush automatically.
  4. Self-Closing Lid: When you are finished, the lid closes automatically, the deodorizer deactivates and the air purifier will activate emitting ions to cleanse the air in the room surrounding the bowl.

Actually, that sounds kind of awesome.  Maybe not worth the $5600 price tag, though.  And it’s even more expensive if you want to control these features, as there is a $500 remote control center that they offer.  Happily, these features can also be controlled from your smartphone! Unhappily, the bluetooth security code built into the toilet is, by default, set to ‘0000’.  So, anyone with the free app can control any toilets in the vicinity.  Oh, ick.  Now, although I wasn’t able to verify it, the Times blog post claims that the company has issued a patch, and summarized the situation with this gem:

Yes, in the future, you will need to download security updates for your toilet.

That cracked me up.

Unfortunately, in the process of researching the topic for (I thought) a quick blog post, I ran across an article that envisions what happens when homes are only as secure as the passwords people use to secure their website access.  In short, it summarizes the various physical items that are being put online and pairs them with what we know about the history of attempts to secure websites, which hasn’t been stellar.  One of this posts concluding thoughts about the security situation for controllable devices:

The rush for vendors to compete in this extremely fast moving market will inevitably result in rushing aspects of the product design and we know very well from past incidents that security is one of the areas most frequently overlooked in favour of delivering features.

Ok, that makes me a bit paranoid.  As more stuff goes online, developers and end-users will get sloppy and opportunities for troublemakers to hurt people and property expand tremendously.  It seems kind of inevitable.

Although I never watched Battlestar Gallactica, I saw most of the pilot.  A critical plot point is that all the ships in the defense fleet have all their weapons disabled by a backdoor in the control system software installed by the enemy.  The only battlestar that survives the vicious attack is the one with the paranoid captain who refused to trick out his ship with the latest in software control.

Boy, I sure hope there’s another choice between refusing to build programmable systems and getting exterminated.  I would really like to have my programmable devices and my life. Perhaps there’s a role for universities to create a lightweight free open security package that could be offered to developers everywhere?  Maybe, if it’s easy enough, consumers will demand a certain minimal level of security in the products they buy?  Perchance such developments might help stave off the doomsday scenario?  Help?

Wired’s Programmable World

Wired magazine has an article on what they call the “Programmable World”.  (Thanks for the pointer, Blase!).  I feel like that’s a pretty good description of the world I want to live in, so I read the article: http://www.wired.com/gadgetlab/2013/05/internet-of-things/all .

Here’s a short reaction to it.  I liked it.  Here’s a (slightly) longer reaction.

The article asserts that the programmable world will arrive in three stages:

1. More and more devices get onto the network.

2. These devices work together to automate tasks.

3. The devices are organized into a programmable software platform that can run apps.

I suppose I agree with this sequence in terms of the underlying technology, but—as is often the case—the evolution of the technology is less interesting to end users than the evolution of the end user experience.  Here’s my take on what the transformation could look like to regular folks:

1. More and more devices become available with their own iphone/android apps.  That’s a safe and easy prediction because it’s already well under way.  I just bought a Jawbone Up (an activity monitor), for example, and it comes with a free iphone app for plotting daily activity and sleep cycles.

2. Cross-device interfaces like IFTTT or Smartthings (which I’ve written about in the past) will make it possible for end-users to specify simple programs that involve multiple devices.  I’m not as sure about this prediction.  End user programming may or may not catch on (although I deeply believe it should).  Regardless of how popular they will become, these cross-devices interfaces are already here.  In fact, I bought the Up because I have a rule that says “if a device is advertised as working with IFTTT, I must buy it.”  So, I have a little iFTTT program now that turns on colored lights (Phillips Hue, which also just got integrated with IFTTT) in my apartment if I reach my per-day step goal.  It’s not at all useful and it doesn’t work the way I want it to.  But, it’s cool that it’s becoming possible to knit these gizmos together at all.

3. We move from a sensor-action model to an event-activity model.  I’m not really seeing this stage in the real world yet, but the more I play with programmable devices, the more I think the whole enterprise flounders without it.  Here’s a concrete example.  The Wired article talks about  automating our daily tasks.  They mention that a computational system should be able to observe the rules you the user follow and then adopt them itself.  So, if you behave according to the rule “If  the sun hits your computer screen, then you lower a shade,” then the computer could do it for you and save you a step. That’s fine, but here’s the thing.  That’s not really the rule we follow.  It’s something more akin to “if I need to see the screen and I can’t, do something to make the screen visible.”  There is no sensor that can directly measure whether you can see the screen.  Such an event needs to be inferred from a set of observables like the brightness of the screen, the direction you are looking, how long it has been since the mouse was moved on the computer, the time of day, whether the shades have already been pulled, etc.  And the action of pulling the shades isn’t always the right response.  It might be more effective to brighten the screen, rotate it a little, or perhaps dim the lights in the room.  We need a computing infrastructure that (1) can integrate information across devices, (2) infer the desirability of possible interventions, and (3) orchestrate a coordinated response across devices.

From the end user’s perspective, this last step requires careful interaction with the devices.  They need to learn from you without distracting you.  The line between teaching, training, and programming begins to blur as we tell the computing systems what matters to us, how it can be measured, and what to do in response to undesirable situations.

Well, that’s the trajectory, anyway.  For now, the best bet is to focus on getting an infrastructure in place so we can start experimenting with different models of interaction.  We have lots of goals for this summer and I hope to be able to report on exciting and steady progress over the next few months!

Some things About SmartThings

It’s been a busy semester and I haven’t been able to find time to write.  And the longer I wait, the more I feel like the next post should be really good to justify the gap.  But, now it’s Spring Break and maybe I should just write something bad just to take the pressure off.  🙂  Here goes.

Yesterday, I was fortunate to have a visit from the former CTO of iRobot, the makers of the Roomba vacuum cleaning robot and perhaps history’s only profitable robotics company.  Tom Wagner stopped by mainly to talk about the Human-Robot Interaction initiative my colleagues at Brown and I are trying to sell.  But, since he was there, I took the opportunity to tell him a bit about the end-user programmable devices project.  Blase Ur participated from Pittsburgh via a robotic telepresence device called a Vgo.

vgo

We demoed several programmable devices—a fan, a blender, and two strings of colored lights—and asked Tom for feedback.  He was amazingly insightful and refreshingly forthright on a number of issues.  Unfortunately, many of the juiciest tidbits he shared were prefaced with “Don’t blog this, but…”.  Oh, well.  But, one excellent comment he made that got us thinking was “How does your project relate to SmartThings?”

Mmmm.  Well, I didn’t recognize the name at first, but SmartThings is a project that Vukosi Marivate brought to my attention as a great example of work relevant to end-user programming that was seeking funding through KickStarter.  I probably should have followed up right away when he first told me, but I always feel very much teased by KickStarter campaigns.  The pleas for funds are very compelling, but so many of the amazing ideas never materialize that it’s often a big letdown.

Briefly, SmartThings is a venture to create a networking infrastructure for programmable physical devices.  In the words of their homepage, “SmartThings adds intelligence to everyday things in your world, so that your life can be more awesome.”  Huh.  Awesome is good.  They’ve got over a million dollars in funding through KickStarter and three million in seed funding.  They’ve got over 30 people working on programmable devices.

Their vision is to have a platform so that developers can create apps that interface with physical devices.  They’ve got a cool hub that all the various devices in your home can talk to.

hub

The hub acts as a relay station for messages between the devices and the SmartThings cloud server.  Then, you can create apps that communicate with the cloud to monitor and control the devices.  It’s even possible to reconfigure and automate the behavior of the devices.  In addition to their own custom devices, they are hard at work making other wifi devices compatible with their system.  All the devices I’ve bought for my apartment in the last few months appear in their preliminary list of compatible devices, which is very encouraging!

I think SmartThings is an exciting development, but it’s hard not to feel a little competitive with them.  They did a demo of their system January 2013 that features the creation of an app that monitors a switch to turn a light on and off.  The audience loved it.  But, our upod project demoed precisely that same functionality (and more!) in November 2011.  We had a cloud server, a switch, lights, and a programming interface.  We didn’t have dozens of people and millions of dollars.  We had a handful of undergraduates and a few thousand dollars for parts.

For me, the frustrating part is that nothing like SmartThing existed back in 2010 when we started.  If it had, we wouldn’t have put the energy into creating our infrastructure and would have used theirs.  But, we couldn’t use theirs because it didn’t exist.

Based on their demo, it appears that they have two programming interfaces for their system.  One is very very close to the When-do interface we built for upod (which is, itself, inspired by the if-this-then-that design).  In our experience, this interface style is extremely easy to use, but also quite limited in its ability to express anything beyond the most basic programming functionality.  Their second programming interface is essentially raw code—lots of keywords and curly braces and colons.  It provides much more complete functionality but at the expense that non computer scientists would find it impenetrable.

In the upod project, we experiments with a kind of middle ground, which is a fully functioning programming language that is friendly and easy to use.  Specifically, we extended Scratch with blocks for listening to and controlling devices.

Looking ahead, we think there’s a lot more that needs to happen to make device programming friendly for the end user.  In particular, real-world devices get their information from real-world sensors.  Whereas if-this-then-that has recipes that can be specified with absolute precision like “When a new book is added to Kindle Top 100 Free eBooks, send me an email”, real-world sensors are more vague and messy.  Consider a recipe that appears in a screen shot in SmartThings’ advertising video: “when it’s going to rain, check if Barkley is in the yard”.

smartthings3

smartthings1 (1)

 

It’s a perfectly reasonable recipe and exactly what you’d want to be able to say as an end user.  But, questions like whether it’s going to rain or if a dog is outside hide a tremendous amount of complexity about how information is gathered from the physical world.  I think it’s terribly unlikely that SmartThings has this functionality working in a way an end-user can leverage yet.  But, it’s a lovely and important research question.  And, it’s what I’m anxious to work on assuming SmartThings really has solved the infrastructure problem.  I can’t wait until summer!

 

Love Letters to a Machine

Love letters are wonderful and romantic.  The very notion that the written word can help initiate or nurture a lifelong commitment is awe-inspiring.

From a practical standpoint, though, what is a love letter?  There are four important attributes I want to focus on.

  1. Literary: A love letter is a written form of communication.  You can’t have love letters without literacy.  In fact, some forms of love letters (sonnets, for example) put severe syntactic restrictions on the writer.  The writer has to work hard to put thoughts and feelings into the right structure.
  2. Dyadic: A love letter is a message between members of a pair—the writer and the reader.  As such, as much as I care about you, a blog post does not count as a love letter.
  3. Aspirational: A love letter makes the reader aware of the writer’s desires or wishes. It isn’t about facts and figures, but about revealing the writer’s hopes and dreams.
  4. Petitionary: A love letter beseeches the reader to share in the goals of the writer.  Ideally, the two adopt a common purpose and the writer achieves a state of delight as a result.

My thesis is that, in a manner of speaking, a computer program is a love letter to a machine.

Ok, maybe that’s a bit of a stretch.  Let’s compare.  Here’s a snippet of a computer program—a beautiful algorithm known as binary search.  It can be used for finding a target value in a sorted list of values, like looking up a name in a phonebook or a word in the dictionary (back when we had to do that sort of thing):

int binary_search(int A[], int key, int imin, int imax) {
  if (imax < imin): return KEY_NOT_FOUND;
  else {
      int imid = midpoint(imin, imax);
      if (A[imid] > key) return binary_search(A, key, imin, imid-1);
      else if (A[imid] < key) return binary_search(A, key, imid+1, imax);
      else return imid;
    }
}

Ah, lovely.  Now, here’s a snippet of a recent love song (from Jason Mraz):

Well open up your mind and see like me
Open up your plans and damn you're free
Look into your heart and you'll find love love love love

 

There are some important similarities.  They are both a little cryptic (“imid”?  see what like me?).  They both provide instructions on how to find something (key and love, respectively).  Both may or may not be successful (KEY_NOT_FOUND, love not won).  But, they are definitely not the same.  The song is a bit more—I don’t know—soul stirring.

Nevertheless, programs share the four features of love letters I enumerated earlier.  Programs are literary—they are written down, sometimes under severe syntactic constraints.  They are dyadic—the programmer is the writer and the computer is the reader.  They are aspirational—they express the programmer’s wish for what should happen.  Finally, they are petitionary—the programmer entreats that the computer adopt these goals as its own.

In most modern languages, the wish is expressed in a detailed, imperative style.  Programs read more like recipes (“add a stick of butter”) or scripts (“Luke, saddened by the loss of Obi-Wan Kenobi, stares off blankly as the robots look on”) or driving directions (“Turn left at the second light”) than wish lists.  But, some languages do support a declarative programming style.  Instead of giving step-by-step instructions, the programmer can describe the desired state of affairs and the computer has some latitude in how to bring it about.

Nevertheless, I really like this analogy.  One of the things that I find interesting and significant is that love letters require proficiency with written language.  This skill can take people a decade or more to develop.  Quite honestly, in spite of all that work, most people aren’t all that good at it.  I’ve been writing professionally for 24 years, and I still look with envy at the Pinkers and Gladwells of the world who make me appear to be doodling with a crayon.  Writing well is supremely difficult.  But, you don’t have to be J. K. Rowling to compose shopping lists or post youtube comments.  Or to pen love letters, for that matter.  The effort it takes to learn to write is worth it even if only a handful of people ever read your words.  It’s that valuable.  Your thoughts are your own and writing is the skill that lets you share it with other people you want to connect with.  Just because Shakespeare said it better than you ever could, doesn’t mean you shouldn’t try to say it.

Programming is like that.  There are gifted programmers who change the world and make a good living writing code.  You’ll never be that good at it.  I’ll never be that good at it.  Nevertheless, even if you never sell a line of code, it’s worth being able to program.  How else can you tell machines what you really want?

 

News Roundup

I love to get pointers to interesting relevant stuff going on in the world at large.  Here are a few links I was sent recently.

Ari Weinstein shared the news that Belkin announces WeMo Smart, coming to your coffee pot later this year.  I haven’t talked about WeMo yet on the blog, but it’s a gadget that I use in my apartment to control my lights.  It connects to my local wifi network and lets me use an iPad app or even IFTTT to write simple programs.  They sell an outlet and a motion sensor, so one program I have turns on my bedroom light for a minute in the evening whenever motion is detected.  I’m a fan of wifi-connected devices, so I appreciate Belkin’s existing product.  The new product isn’t described in any detail in the press release, but, reading between the lines, I think the idea is that they will produce a piece of electronics that will be embedded in the coffeemaker so it can be controlled and monitored via the WeMo cloud.   I don’t know if Belkin’s protocol is open, but they were at least willing to team up with IFTTT, so I’m encouraged by this development!  (Neither Belkin nor IFTTT has embraced general programming as an interface, so there’s still work to be done here.)

I also like the model that both Belkin and IFTTT have used—don’t solve all possible problems up front.  Instead, offer something small and simple that works and then grow from there.  Had Belkin attempted to tackle “home automation” in one go, they would have failed.  If IFTTT had attempted to tackle “end-user programming” in one go, they would have failed.  I very much like the way they are sneaking up on their targets.

The next item, sent by my colleague Shriram Krishnamurthi to the Brown CS faculty, is tangentially related: Ford, GM roll out red carpet for app developers.  Apparently, high end Ford and GM cars run a user-facing computerized system called Sync that interfaces with smartphones.  The news is that they are opening up the system to third-party developers to create car-based apps.  As an example, the article mentions one that reads newspaper articles out loud to the driver.  They are planning to prohibit software that will distract drivers with video, games, and the like.

According to Wired’s Exclusive: Ford Wants to Create the Android of Automotive Apps, they will also go further and offer their platform to other automakers; the idea being that the existence of a standardized platform will make it easier for software developers to invest the time needed to create outstanding applications.  One touchy point, apparently, is whether the system will be integrated with the car’s functioning (beyond the screen and audio).  The more integrated it is, the cooler the apps can be, but the less likely competing car manufacturers will be to trust it.  Plus, I suppose it could be quite risky to allow an app to mess with steering, braking, or even headlights and such.

For what it’s worth, I’ll note that the move to open up the platform to third party developers is far from inevitable.  I was just getting excited about the apps available for the Livescribe computerized ballpoint pen.  (The Zork port is pretty neat and the demo where you draw a piano and then you can play it with the pen is a kick!)  I was hoping students might be interested in developing software that can use the pen as an interface to household devices and then I discovered a page proclaiming Livescribe Discontinuing Developer Program.  That’s so sad and it seems like precisely the wrong direction.  We want to talk to our machines!

Partner-in-crime Blase Ur sent me Computer security and the modern home, an academic paper that analyzes the vulnerabilities of the collection of networked, computerized devices that are becoming available.  Just as programmable devices bring the flexibility of software into the physical world, they also amplify the opportunities for software to enable physical crimes.  The paper  provides a thorough, if dry, analysis of what makes computerized home devices risky from a security standpoint.  They did use one wry phrase I particularly liked, though.  The authors were describing the risks inherent in the fact that home devices are not generally administered by professionals.  Toys, in particular, may end up being particular problematic because they are effectively given to kids to administer

…despite the child’s likely lack of experience with computer security and different stance on privacy issues.

I assume the authors’ experiences with kids’ “privacy stances” are similar to mine—kids are willing to say just about anything to just about anyone.  Even in the unlikely case they understand how to keep their electronic toys from violating the privacy of people in the home, they wouldn’t understand why.

I respect the authors’ efforts in highlighting security concerns.  I recognize the importance of securing our homes and the risks that come with extending global communications networks inside them.  Nevertheless, reading about all the horrible things that can go wrong makes me feel all icky.

Lastly, the story Welcome to the home of the future sounds like it should be relevant; it’s pretty close to my first blog post title, in fact.  I didn’t find it interesting, though.  It talks about the excitement at companies like Apple and Microsoft about the market for advanced household devices, which is is a good thing.  But it focuses on things that are weirdly futuristic and doesn’t really tell a convincing story of how we might get there.  On the positive side, I lost the information about who sent me the link, so no one needs to feel bad about it.

Survey of Programmable Appliances

My motivation for creating this blog was to convince you of the importance of being able to talk to your machines by programming them.  With that goal in mind, I thought it might be worthwhile to take a look at what sort of devices are already considered “programmable” and what people think that means.

Here’s what I did.  I went to amazon.com and searched for the term “programmable” in their Appliances section.  If my calculations are correct, Amazon sells 231,647 products in its Appliances section, 452 of which (a mere 0.2%) use the word “programmable” in the product name.

I organized the products into 18 categories ranging from “air conditioner” to “wall control” and counted up how many appeared in each category.  One thing I found surprising is that coffeemakers make up more than half of all programmable appliances on amazon.com.  On further reflection, this discovery is consistent with my theory that our nemeses in the coming robot apocalypse really just want to make us breakfast.

mrcoffee

To get a sense of what these products do, I hunted down user manuals for at least one in each category.  Based on this documentation, I was able to further group products by what “programmable” means for each.  Aside from one that I couldn’t figure out why it was called programmable (a Meiko dishwasher with its “award-winning MIKE 2 controller” ) and one that I couldn’t get any sense of what the product was intended to do (“GE Part Number RAK148P1 T-Stat 2 Stage H-Pump (Programmable)”), their programming functionality fit into five classes:

  1. Timers
  2. ‘Stats
  3. Schedules
  4. Scripts
  5. Presets

Here’s what they do.

Timers

For the vast majority of the programmable devices—by which I mean the coffeemakers—programming simply means setting a time at which the device is switched on.

coffee

From a logical standpoint, a programmable coffeemaker is a clock radio.  Except it makes coffee.

Several other devices came up in the Amazon list with this same basic functionality.  There was an air conditioner that can be set to start at a given time and a fan that can be set to turn off after a particular delay.  But, the most common kind of product (after coffeemakers, of course) that supports this feature was “slow cooker”.  Similar to coffeemakers, which people like to be able to use in their sleep, slow cookers can be set to do their job while people are busy at work.

‘Stats

The slow cookers I looked at sported another class of programmability that I call “‘stats”.slow

In “probe mode”, the slow cooker can be made to turn off when a target temperature is reached.  Logically, this functionality is quite similar to that of a timer.  It’s about activating or deactivating a device, but it is based on temperature instead of time.  However, the reason that I consider ‘stats a different category from timers is that they are most often used in a feedback loop.  In particular, thermostats and humidistats (both of which show up in the Amazon list) and other kinds of ‘stats  measure some physical quantity (temperature or humidity, say) and hold it steady.  (The “stat” in “thermostat” shares its etymological history with the “stat” in “stationary”.)

What makes a ‘stat cool is that its sensor is sensitive to a condition that its effector corrects.  For example, a thermostat in a space heater triggers warming when the room gets too cold.  A humidistat in a dehumidifier triggers drying if the room becomes too wet.  Similarly, a barostat maintains pressure and a chemostat maintains chemical concentrations, although I didn’t run  across either of these on Amazon.  I remember many many years ago my father-in-law’s car had an automated windshield wiper that you might call a transparency-o-stat.  It activated the wiper whenever the windshield was obscured by rain.  It was pretty cool.

Whereas timers are most useful when a device needs to be turned on or off when you aren’t around, a ‘stat is most useful when there’s a quantity that can be controlled without interrupting you.  Perhaps one reason the transparency-o-stat didn’t become standard feature in new cars is that you are looking out the window anyway when you are driving—noticing that the window is wet doesn’t require any extra attention.

Schedules

Household thermostats are often bundled with another programming feature—weekly schedules.  In a sense, schedules combine timers and ‘stats with a “sensor” for the day of the week.  A schedule in a thermostat is useful because it’s common for people to try to save energy on heating or cooling during the day when they are at work or school.  But, during the weekend, things need to be handled differently as people are more likely to be home.  Schedule programming lets you specify target temperatures as a function of time of day and day of week.

The interfaces for programming schedules tend to be more complex because of the multiple dimensions (day of week, time, target) that have to be considered simultaneously.  The most extreme interface I ran across on Amazon was for a programmable towel warmer that lets you have a toasty warm towel waiting for you at 10pm on a Thursday, if that’s what you want.

towel

The complexity of this interface along with the seeming futility of its purpose (how many people know a week in advance that they need a warm towel at a specific time yet are unwilling to simply turn on a towel warmer as they get into the shower?) make me quite skeptical that this programmable device gets much use.  Of course, if we lived in a world where all our devices were programmable (hint, hint), it wouldn’t seem quite as odd.  Well, assuming you think towel warmers are normal.

The coolest example of schedule programming that I’ve read about is in the Nest thermostat, which combines schedules, timers, and ‘stats with proximity sensors (to find out whether you really are around on the weekend) and learning algorithms that adapt to your activity patterns and preferences.  I plan to write more about Nest in a future blog post.

Scripts

Amazon sells a dozen programmable breadmakers, and they embody a different kind of “programming”.  The process of making a loaf of bread consists of multiple steps like kneading, baking, waiting for the dough to rise, and, sometimes, adding in raisins.  Yum.  The timing of these steps depends on the kind and quantity of dough used as well as the preferred darkness of the bread at the end.  Programmable bread makers have scripts or sequences baked into their controllers (as it were) so you don’t have to spell out the steps yourself each time you use the device.

bread

The breadmaker is “programmable” in that you can select among a baker’s dozen or so of these pre-specified scripts.  The device is flexible enough to allow for a huge space of sequences, but the resulting complexity is mostly hidden from you since the useful sequences are encoded in advance.

Presets

Blenders, like breadmakers, can carry out sequences of actions with different timings.  For example, BarBoss Advance from Vitamix includes a set of scripts for producing different kinds of drinks.

blender

Each of these scripts is bound to its own digit on a dial on the face of the blender.  This means, apparently, that putting in the right ingredients, selecting Setting 4, then starting the blender will produce ice cream.  What’s not to love about programming?

blenderdial

The cool thing about this blender, though, is that you can connect it to a computer and create your own scripts.  You tell the blender how fast to go and for how long, how quickly to ramp up from one speed to the next, and when to add water or ice.

blenderprogram

The script you create is downloaded onto a chip that you can insert into the base of the blender, replacing the original pre-programmed scripts with your own “presets”.  Using a preset means that the complex sequence you’ve painstakingly created can be called back into being simply by selecting the right setting on the dial.

Presets are a great tool in the programming arsenal.  Other examples of devices with presets that I’ve seen include video gaming headsets/keyboards/controllers (allowing you to optimize your interface for “different gaming situations”, whatever that means), microwave ovens (allowing you to lay out different cooking stages that vary in time and power) and, of course, car radios (allowing you to jump to your favorite stations without having to hunt through the whole FM spectrum).

Composition

One of the delights of working with a true programming language is that it is possible to combine/compose programming features together in creative and powerful ways.  Although none of the devices I encountered on Amazon makes use of this idea, several do combine features in inflexible but interesting ways.  For example, the blender I mentioned uses scripts and presets together.  The same combination is used in message fans and fryers.  Thermostats and towel warmers use schedules and ‘stats.  The same combination is used in wall control / ventilation systems.  Rice cookers combine timers and presets.

All in all, I think it’s important to note which kinds of programmability have proven popular enough to be included in commercial products.  Whether and what kind of programming a product supports boils down to three factors:

  1. The behavior should be something that you can’t do yourself, either because it’s hard to remember (scripts, presets, schedules), boring or distracting (‘stats), or simply because you aren’t around to do it (timers).
  2. The behavior should be something that the machine can do on your behalf.  Noting the time or temperature is straightforward.  Remembering detailed sequences of mixing or baking steps is, you could say, a piece of cake.  Recognizing a face, understanding spoken dialogue, or moving with precision in natural environments is still quite harder.  (Although progress has been steady.)
  3. Finally, there must be a way of transferring your goals and intentions from your mind to the machine.  At present, the possibilities are still quite limited, but if devices supported general forms of programming and more people were familiar with this form of communication, the cost-benefit analysis would shift.

That pretty much covers it for the “programmable” appliances on Amazon.  As they exist today, it’s a little odd that they all are described using the same word.  But, if they supported a common programming language, it could make them easier to use and expose the deep connections between the different ways we can ask machines to work on our behalf.

Merry Chr-IFTTT-Mas

It’s Christmas morning and I awoke to find a single email in my inbox.

ifttt

It’s from a service called IFTTT (“if this, then that”) and it is a friendly reminder that, if I want to make a blog post once a week, it’s that time again.  It even helpfully gives me a link to the blogging site, since I’m inclined to have to look that sort of thing up.

I really like the form of this reminder mainly because I made it myself.  It’s a gift from me to me, made possible by IFTTT.  And it’s just what I wanted!

If I were giving out an award for “best attempt so far at an end-user-programming system” award, IFTTT definitely gets a nomination.  It has three of the four attributes that I think are critical:

  1. Relevance.  It lets people control something that actually matters to them in their lives.  In this case, it’s primarily social networking sites.
  2. Simplicity.  The designers have bent over backwards to make creating programs (“recipes”) easy, friendly, and even a little fun.
  3. Universality.  Instead of each social networking site developing its own end-user programming interface, IFTTT is a single system that works for all of them.  That’s key because it means the investment of time and energy a user makes in learning about one site pays off when applying her knowledge to other sites.  It also means that the user can start to mix and match functionality across sites, which isn’t possible if each site’s interface is independent of the others.
  4. Expandibility.  Unfortunately, IFTTT falls short in this category.  The simple form all programs must take is not completely general and it doesn’t take long before a user can think of a perfectly reasonable program that cannot be constructed within the constraints of IFTTT.

I’ll give you a little tutorial to give you a sense of what IFTTT can do.  (It’s a free service, so consider this description an invitation to sign up and experiment.)  The central concept in IFTTT is that of a channel.  To a first approximation, a channel is essentially a web service that you subscribe to.  IFTTT supports 59 channels right now, with maybe one or two added each month.  (I’ve set up 23 of the channels so far, which often means I need to give IFTTT my passwords to these services.  Hmmm, we’ll need to discuss privacy and security issues another time!)

channels

Most sites that are on my radar are included as channels, like youtube, Facebook, Google docs, and Gmail.  There are a bunch of channels that are on the periphery of my awareness that show up as well, things like bitly, Flickr, and Delicious.  Then there are a bunch that I’m utterly oblivious to and in many cases not even sure how to pronounce, like ffffound!, Svpply, Diigo, and Moped.  Aside from making me feel old and clueless, the presence of these channels doesn’t seem to cause any trouble.

An IFTTT recipe consists of filling in the “this” and the “that” in “if this, then that”.  A “this” is filled in with a “trigger” and a “that” is filled in with an “action”.  Each channel includes these two categories of functionality, capturing a piece of what you can do on the channel’s corresponding site.  I think an example will help make things more concrete.  My friend Tim was writing a personal blog for a healthcare technology site.  I didn’t want to miss any of his posts, so I made the following IFTTT recipe:

timalert

It checks the company’s blog feed for new posts that include the phrase “tim’s blog” (something that I had noticed was consistent to all of Tim’s articles).  When this trigger from the “Feed” channel fires, it results in an action from the “Email” channel that sends an email to me with a link to Tim’s writing.  This recipe successfully kept me informed of Tim’s posts and, I hope, will let me know if he starts writing again.

Other IFTTT recipes I use on a regular basis include letting me know (via something called Boxcar) whenever my daughter Molly posts a picture on Instagram, posting a status update for my Facebook friends whenever I put up a new video on youtube, and the latest addition that sends me an email reminder every Tuesday that it’s time to blog.  By and large, I’ve found it useful to be able to make these little programs to fine-tune my online life and then not have to think about them.

One recipe that didn’t quite work was my attempt to log temperatures during the day.   (I’m interested in the question of how reliable weather reports are and can never seem to find data that links predictions to the actual weather that happened!)  IFTTT has a clock channel with a trigger that can be made to fire every hour.  It has a weather channel that includes forecast information and current weather conditions.  And, it includes a Google Drive channel that has an action for creating a row at the bottom of a Google spreadsheet to log data.  These are exactly the pieces I need to collect my own weather data!  Unfortunately, IFTTT recipes are combinations of triggers and actions from two channels, not three.  There’s no way to put together the behavior I want.  I’ve contented myself with a recipe that logs the predicted high and low temperature each day at sunrise.

weather

Ok, not completely contented.  In fact, I’m not contented at all.  I want a language that fixes this kind of limitation.  I’ll let you know when I’ve got it.

But, in the spirit of the day, let me just say that I’m a fan of IFTTT.  Not only have I found it interesting as a case study in end-user programming, but I’ve incorporated into my life and I’m glad I did.  Merry Chr-IFTT-mas, everyone!

 

Welcome to the Future

My name is Michael Littman.  I’m a computer scientist at Brown University and I’ve had the privilege to work with a team of creative and energetic people at Rutgers University on a project we call “Scratchable Devices”.  Welcome to our blog!

The aspiration of the “Scratchable Devices” team is to help move us to a future in which end-user programming is commonplace.  The short version of the pitch goes like this.  We are all surrounded by computers—more and more of the devices we interact with on a daily basis are general purpose CPUs in disguise.  The marvelous thing about these machines is that they can carry out activities on our behalf: activities that we are too inaccurate or slow or fragile or inconsistent or frankly important to do for ourselves.  Unfortunately, most of us don’t know how to speak to these machines  And, even those of us who do are usually barred from doing so by device interfaces that are intended to be friendly but in fact tie our hands.

We seem to be on the verge of an explosion of new opportunities.  There are new software systems being created, more ways to teach people about programming, and many many more new devices that we wish we could talk to in a systematic way.  The purpose of this blog is to raise awareness of developments, both new and old, that bear on the question of end-user programming.  I welcome email from interested readers who have spotted something cool (mlittman@cs.brown.edu), although I don’t claim to be able to present anything more or less than my own personal take on what I read.

In the days to come, I will sift through my backlog of interesting tidbits and then maybe we’ll be in a good position to start figuring out what the future could look like.