Gesturing Government

A lot of people I’ve been talking to recently are excited by the emergence of NUI (Natural User Interfaces), but are asking “Isn’t this just for gamers?”

My immediate answer is “yes”. Gamers have been the first to embrace the technology (the Nintendo Wii remote controller  is one example of this). Future gaming consoles are likely to require no handheld controller at all, but will simply recognise your hand/foot gestures, facial expressions and vocal commands.

NUI is  also great for music, performance and artistic expression. See the inspirational experiments created by the Auckland Collective Purple Spheres. Additionally, museums around the world have been making use of this sort of technology for years.

But what about practical uses for NUI in the “real world”?  Recently I’ve been seeking examples beyond the realm of gaming.  Here are  a few examples:

Information Graphics


Pictured above:  It seems that analysing data is well suited to NUI and the NBC recently rolled out a Microsoft Surface to do real time analysis on the Obama and McCain campaigns. The presenter is in-control of the presentation and can trigger information-graphics via the touch-screen or by dropping objects (in this case a card for each month) on top of table.

Crisis Management


Pictured above: GIS for Crisis Management (The DAVE_G prototype). The research showed that even people without experience using gestural interfaces quickly learnt the system. It was also found to better for collaboration within teams. Case Study (PDF, 565kb).

And at a recent presentation to the Wellington Usability Professionals’ Association I suggested a number of ideas for NUI within Government, including the following:

Property Information


Pictured above: What if your District Council supplied your property records on a touch-screen? You could intuitively zoom in to view details on your property, scroll forward for town planning and overlay hazard maps such as flood and earthquake information.

Opinion Polls


Pictured above: Gestural voting. Voting (or, more likely opinion polls), would be more fun if you could simply give the thumbs up or thumbs down to any party or candidate presented on-screen (and perhaps the system could be smart enough to recognise  a few alternative gestures if  some people  felt  strongly about a particular party?!).

So beyond art and gaming it just takes a little bit of imagination to see how we might be able to utilise NUI for commercial and government applications.