Monthly Archives: January 2013

Love Letters to a Machine

Love letters are wonderful and romantic.  The very notion that the written word can help initiate or nurture a lifelong commitment is awe-inspiring.

From a practical standpoint, though, what is a love letter?  There are four important attributes I want to focus on.

  1. Literary: A love letter is a written form of communication.  You can’t have love letters without literacy.  In fact, some forms of love letters (sonnets, for example) put severe syntactic restrictions on the writer.  The writer has to work hard to put thoughts and feelings into the right structure.
  2. Dyadic: A love letter is a message between members of a pair—the writer and the reader.  As such, as much as I care about you, a blog post does not count as a love letter.
  3. Aspirational: A love letter makes the reader aware of the writer’s desires or wishes. It isn’t about facts and figures, but about revealing the writer’s hopes and dreams.
  4. Petitionary: A love letter beseeches the reader to share in the goals of the writer.  Ideally, the two adopt a common purpose and the writer achieves a state of delight as a result.

My thesis is that, in a manner of speaking, a computer program is a love letter to a machine.

Ok, maybe that’s a bit of a stretch.  Let’s compare.  Here’s a snippet of a computer program—a beautiful algorithm known as binary search.  It can be used for finding a target value in a sorted list of values, like looking up a name in a phonebook or a word in the dictionary (back when we had to do that sort of thing):

int binary_search(int A[], int key, int imin, int imax) {
  if (imax < imin): return KEY_NOT_FOUND;
  else {
      int imid = midpoint(imin, imax);
      if (A[imid] > key) return binary_search(A, key, imin, imid-1);
      else if (A[imid] < key) return binary_search(A, key, imid+1, imax);
      else return imid;
    }
}

Ah, lovely.  Now, here’s a snippet of a recent love song (from Jason Mraz):

Well open up your mind and see like me
Open up your plans and damn you're free
Look into your heart and you'll find love love love love

 

There are some important similarities.  They are both a little cryptic (“imid”?  see what like me?).  They both provide instructions on how to find something (key and love, respectively).  Both may or may not be successful (KEY_NOT_FOUND, love not won).  But, they are definitely not the same.  The song is a bit more—I don’t know—soul stirring.

Nevertheless, programs share the four features of love letters I enumerated earlier.  Programs are literary—they are written down, sometimes under severe syntactic constraints.  They are dyadic—the programmer is the writer and the computer is the reader.  They are aspirational—they express the programmer’s wish for what should happen.  Finally, they are petitionary—the programmer entreats that the computer adopt these goals as its own.

In most modern languages, the wish is expressed in a detailed, imperative style.  Programs read more like recipes (“add a stick of butter”) or scripts (“Luke, saddened by the loss of Obi-Wan Kenobi, stares off blankly as the robots look on”) or driving directions (“Turn left at the second light”) than wish lists.  But, some languages do support a declarative programming style.  Instead of giving step-by-step instructions, the programmer can describe the desired state of affairs and the computer has some latitude in how to bring it about.

Nevertheless, I really like this analogy.  One of the things that I find interesting and significant is that love letters require proficiency with written language.  This skill can take people a decade or more to develop.  Quite honestly, in spite of all that work, most people aren’t all that good at it.  I’ve been writing professionally for 24 years, and I still look with envy at the Pinkers and Gladwells of the world who make me appear to be doodling with a crayon.  Writing well is supremely difficult.  But, you don’t have to be J. K. Rowling to compose shopping lists or post youtube comments.  Or to pen love letters, for that matter.  The effort it takes to learn to write is worth it even if only a handful of people ever read your words.  It’s that valuable.  Your thoughts are your own and writing is the skill that lets you share it with other people you want to connect with.  Just because Shakespeare said it better than you ever could, doesn’t mean you shouldn’t try to say it.

Programming is like that.  There are gifted programmers who change the world and make a good living writing code.  You’ll never be that good at it.  I’ll never be that good at it.  Nevertheless, even if you never sell a line of code, it’s worth being able to program.  How else can you tell machines what you really want?

 

News Roundup

I love to get pointers to interesting relevant stuff going on in the world at large.  Here are a few links I was sent recently.

Ari Weinstein shared the news that Belkin announces WeMo Smart, coming to your coffee pot later this year.  I haven’t talked about WeMo yet on the blog, but it’s a gadget that I use in my apartment to control my lights.  It connects to my local wifi network and lets me use an iPad app or even IFTTT to write simple programs.  They sell an outlet and a motion sensor, so one program I have turns on my bedroom light for a minute in the evening whenever motion is detected.  I’m a fan of wifi-connected devices, so I appreciate Belkin’s existing product.  The new product isn’t described in any detail in the press release, but, reading between the lines, I think the idea is that they will produce a piece of electronics that will be embedded in the coffeemaker so it can be controlled and monitored via the WeMo cloud.   I don’t know if Belkin’s protocol is open, but they were at least willing to team up with IFTTT, so I’m encouraged by this development!  (Neither Belkin nor IFTTT has embraced general programming as an interface, so there’s still work to be done here.)

I also like the model that both Belkin and IFTTT have used—don’t solve all possible problems up front.  Instead, offer something small and simple that works and then grow from there.  Had Belkin attempted to tackle “home automation” in one go, they would have failed.  If IFTTT had attempted to tackle “end-user programming” in one go, they would have failed.  I very much like the way they are sneaking up on their targets.

The next item, sent by my colleague Shriram Krishnamurthi to the Brown CS faculty, is tangentially related: Ford, GM roll out red carpet for app developers.  Apparently, high end Ford and GM cars run a user-facing computerized system called Sync that interfaces with smartphones.  The news is that they are opening up the system to third-party developers to create car-based apps.  As an example, the article mentions one that reads newspaper articles out loud to the driver.  They are planning to prohibit software that will distract drivers with video, games, and the like.

According to Wired’s Exclusive: Ford Wants to Create the Android of Automotive Apps, they will also go further and offer their platform to other automakers; the idea being that the existence of a standardized platform will make it easier for software developers to invest the time needed to create outstanding applications.  One touchy point, apparently, is whether the system will be integrated with the car’s functioning (beyond the screen and audio).  The more integrated it is, the cooler the apps can be, but the less likely competing car manufacturers will be to trust it.  Plus, I suppose it could be quite risky to allow an app to mess with steering, braking, or even headlights and such.

For what it’s worth, I’ll note that the move to open up the platform to third party developers is far from inevitable.  I was just getting excited about the apps available for the Livescribe computerized ballpoint pen.  (The Zork port is pretty neat and the demo where you draw a piano and then you can play it with the pen is a kick!)  I was hoping students might be interested in developing software that can use the pen as an interface to household devices and then I discovered a page proclaiming Livescribe Discontinuing Developer Program.  That’s so sad and it seems like precisely the wrong direction.  We want to talk to our machines!

Partner-in-crime Blase Ur sent me Computer security and the modern home, an academic paper that analyzes the vulnerabilities of the collection of networked, computerized devices that are becoming available.  Just as programmable devices bring the flexibility of software into the physical world, they also amplify the opportunities for software to enable physical crimes.  The paper  provides a thorough, if dry, analysis of what makes computerized home devices risky from a security standpoint.  They did use one wry phrase I particularly liked, though.  The authors were describing the risks inherent in the fact that home devices are not generally administered by professionals.  Toys, in particular, may end up being particular problematic because they are effectively given to kids to administer

…despite the child’s likely lack of experience with computer security and different stance on privacy issues.

I assume the authors’ experiences with kids’ “privacy stances” are similar to mine—kids are willing to say just about anything to just about anyone.  Even in the unlikely case they understand how to keep their electronic toys from violating the privacy of people in the home, they wouldn’t understand why.

I respect the authors’ efforts in highlighting security concerns.  I recognize the importance of securing our homes and the risks that come with extending global communications networks inside them.  Nevertheless, reading about all the horrible things that can go wrong makes me feel all icky.

Lastly, the story Welcome to the home of the future sounds like it should be relevant; it’s pretty close to my first blog post title, in fact.  I didn’t find it interesting, though.  It talks about the excitement at companies like Apple and Microsoft about the market for advanced household devices, which is is a good thing.  But it focuses on things that are weirdly futuristic and doesn’t really tell a convincing story of how we might get there.  On the positive side, I lost the information about who sent me the link, so no one needs to feel bad about it.

Survey of Programmable Appliances

My motivation for creating this blog was to convince you of the importance of being able to talk to your machines by programming them.  With that goal in mind, I thought it might be worthwhile to take a look at what sort of devices are already considered “programmable” and what people think that means.

Here’s what I did.  I went to amazon.com and searched for the term “programmable” in their Appliances section.  If my calculations are correct, Amazon sells 231,647 products in its Appliances section, 452 of which (a mere 0.2%) use the word “programmable” in the product name.

I organized the products into 18 categories ranging from “air conditioner” to “wall control” and counted up how many appeared in each category.  One thing I found surprising is that coffeemakers make up more than half of all programmable appliances on amazon.com.  On further reflection, this discovery is consistent with my theory that our nemeses in the coming robot apocalypse really just want to make us breakfast.

mrcoffee

To get a sense of what these products do, I hunted down user manuals for at least one in each category.  Based on this documentation, I was able to further group products by what “programmable” means for each.  Aside from one that I couldn’t figure out why it was called programmable (a Meiko dishwasher with its “award-winning MIKE 2 controller” ) and one that I couldn’t get any sense of what the product was intended to do (“GE Part Number RAK148P1 T-Stat 2 Stage H-Pump (Programmable)”), their programming functionality fit into five classes:

  1. Timers
  2. ‘Stats
  3. Schedules
  4. Scripts
  5. Presets

Here’s what they do.

Timers

For the vast majority of the programmable devices—by which I mean the coffeemakers—programming simply means setting a time at which the device is switched on.

coffee

From a logical standpoint, a programmable coffeemaker is a clock radio.  Except it makes coffee.

Several other devices came up in the Amazon list with this same basic functionality.  There was an air conditioner that can be set to start at a given time and a fan that can be set to turn off after a particular delay.  But, the most common kind of product (after coffeemakers, of course) that supports this feature was “slow cooker”.  Similar to coffeemakers, which people like to be able to use in their sleep, slow cookers can be set to do their job while people are busy at work.

‘Stats

The slow cookers I looked at sported another class of programmability that I call “‘stats”.slow

In “probe mode”, the slow cooker can be made to turn off when a target temperature is reached.  Logically, this functionality is quite similar to that of a timer.  It’s about activating or deactivating a device, but it is based on temperature instead of time.  However, the reason that I consider ‘stats a different category from timers is that they are most often used in a feedback loop.  In particular, thermostats and humidistats (both of which show up in the Amazon list) and other kinds of ‘stats  measure some physical quantity (temperature or humidity, say) and hold it steady.  (The “stat” in “thermostat” shares its etymological history with the “stat” in “stationary”.)

What makes a ‘stat cool is that its sensor is sensitive to a condition that its effector corrects.  For example, a thermostat in a space heater triggers warming when the room gets too cold.  A humidistat in a dehumidifier triggers drying if the room becomes too wet.  Similarly, a barostat maintains pressure and a chemostat maintains chemical concentrations, although I didn’t run  across either of these on Amazon.  I remember many many years ago my father-in-law’s car had an automated windshield wiper that you might call a transparency-o-stat.  It activated the wiper whenever the windshield was obscured by rain.  It was pretty cool.

Whereas timers are most useful when a device needs to be turned on or off when you aren’t around, a ‘stat is most useful when there’s a quantity that can be controlled without interrupting you.  Perhaps one reason the transparency-o-stat didn’t become standard feature in new cars is that you are looking out the window anyway when you are driving—noticing that the window is wet doesn’t require any extra attention.

Schedules

Household thermostats are often bundled with another programming feature—weekly schedules.  In a sense, schedules combine timers and ‘stats with a “sensor” for the day of the week.  A schedule in a thermostat is useful because it’s common for people to try to save energy on heating or cooling during the day when they are at work or school.  But, during the weekend, things need to be handled differently as people are more likely to be home.  Schedule programming lets you specify target temperatures as a function of time of day and day of week.

The interfaces for programming schedules tend to be more complex because of the multiple dimensions (day of week, time, target) that have to be considered simultaneously.  The most extreme interface I ran across on Amazon was for a programmable towel warmer that lets you have a toasty warm towel waiting for you at 10pm on a Thursday, if that’s what you want.

towel

The complexity of this interface along with the seeming futility of its purpose (how many people know a week in advance that they need a warm towel at a specific time yet are unwilling to simply turn on a towel warmer as they get into the shower?) make me quite skeptical that this programmable device gets much use.  Of course, if we lived in a world where all our devices were programmable (hint, hint), it wouldn’t seem quite as odd.  Well, assuming you think towel warmers are normal.

The coolest example of schedule programming that I’ve read about is in the Nest thermostat, which combines schedules, timers, and ‘stats with proximity sensors (to find out whether you really are around on the weekend) and learning algorithms that adapt to your activity patterns and preferences.  I plan to write more about Nest in a future blog post.

Scripts

Amazon sells a dozen programmable breadmakers, and they embody a different kind of “programming”.  The process of making a loaf of bread consists of multiple steps like kneading, baking, waiting for the dough to rise, and, sometimes, adding in raisins.  Yum.  The timing of these steps depends on the kind and quantity of dough used as well as the preferred darkness of the bread at the end.  Programmable bread makers have scripts or sequences baked into their controllers (as it were) so you don’t have to spell out the steps yourself each time you use the device.

bread

The breadmaker is “programmable” in that you can select among a baker’s dozen or so of these pre-specified scripts.  The device is flexible enough to allow for a huge space of sequences, but the resulting complexity is mostly hidden from you since the useful sequences are encoded in advance.

Presets

Blenders, like breadmakers, can carry out sequences of actions with different timings.  For example, BarBoss Advance from Vitamix includes a set of scripts for producing different kinds of drinks.

blender

Each of these scripts is bound to its own digit on a dial on the face of the blender.  This means, apparently, that putting in the right ingredients, selecting Setting 4, then starting the blender will produce ice cream.  What’s not to love about programming?

blenderdial

The cool thing about this blender, though, is that you can connect it to a computer and create your own scripts.  You tell the blender how fast to go and for how long, how quickly to ramp up from one speed to the next, and when to add water or ice.

blenderprogram

The script you create is downloaded onto a chip that you can insert into the base of the blender, replacing the original pre-programmed scripts with your own “presets”.  Using a preset means that the complex sequence you’ve painstakingly created can be called back into being simply by selecting the right setting on the dial.

Presets are a great tool in the programming arsenal.  Other examples of devices with presets that I’ve seen include video gaming headsets/keyboards/controllers (allowing you to optimize your interface for “different gaming situations”, whatever that means), microwave ovens (allowing you to lay out different cooking stages that vary in time and power) and, of course, car radios (allowing you to jump to your favorite stations without having to hunt through the whole FM spectrum).

Composition

One of the delights of working with a true programming language is that it is possible to combine/compose programming features together in creative and powerful ways.  Although none of the devices I encountered on Amazon makes use of this idea, several do combine features in inflexible but interesting ways.  For example, the blender I mentioned uses scripts and presets together.  The same combination is used in message fans and fryers.  Thermostats and towel warmers use schedules and ‘stats.  The same combination is used in wall control / ventilation systems.  Rice cookers combine timers and presets.

All in all, I think it’s important to note which kinds of programmability have proven popular enough to be included in commercial products.  Whether and what kind of programming a product supports boils down to three factors:

  1. The behavior should be something that you can’t do yourself, either because it’s hard to remember (scripts, presets, schedules), boring or distracting (‘stats), or simply because you aren’t around to do it (timers).
  2. The behavior should be something that the machine can do on your behalf.  Noting the time or temperature is straightforward.  Remembering detailed sequences of mixing or baking steps is, you could say, a piece of cake.  Recognizing a face, understanding spoken dialogue, or moving with precision in natural environments is still quite harder.  (Although progress has been steady.)
  3. Finally, there must be a way of transferring your goals and intentions from your mind to the machine.  At present, the possibilities are still quite limited, but if devices supported general forms of programming and more people were familiar with this form of communication, the cost-benefit analysis would shift.

That pretty much covers it for the “programmable” appliances on Amazon.  As they exist today, it’s a little odd that they all are described using the same word.  But, if they supported a common programming language, it could make them easier to use and expose the deep connections between the different ways we can ask machines to work on our behalf.