I was intrigued by the Ted Talk we had as an essay assignment so I decided to browse. I found this video of James Patten: The best computer interface? Maybe … your hands. He is an interactive designer, inventor and visual artist. He earned his doctorate at the MIT Media Lab. This says enough right? His video describes his inventions of moving objects that are digitally and magnetically based and are controlled by our hands. His first invention was a technological display for Barney’s New York (retail) and it displayed a pair of women’s shoes moving away from a man’s pair of shoes. Pretty cool.


Right here in Chicago at the Museum of Science and Industry, he built an interactive chemistry exhibit. It allowed people to use physical objects to drag elements off of the periodic table and put them together and watch chemical reactions happen. Also pretty cool.  Research concluded that this exhibit was allowing people to learn more effectively and it showed that they were learning in a more social way. For subjects like chemistry, this is a great way to interactively get people to understand the complexity of the subject matter in a fun way. Patten says that our hands and our minds are optimized for us to think tangibly. One example he gave was to think about the way we use keyboards, what is easier to use, a physical keyboard or touchscreen? 




Patten realized that all of these projects had to be built from scratch; everything from technology to the software. So he wanted to design an interface where that does not have to be the case all the time.

He first tried this with the help of his professor Hiroshi Ishii by building a system of magnets that allowed objects to move on top of a surface. Problem was they were so heavy they caved in the table they were resting on. So he came up with the brilliant idea of creating an army of small robots that had special omni wheels. These wheels allow the object to move in any direction. He coupled the mini robots with video projectors. So now you have physical tools that change the way we interactively communicate. 


He also designed an interface where emergency response can be physically controlled and the mini robots can tell dispatchers where to go. The positions of the robots on the map are linked with the position of the units in the real world. How awesome is that?


Patten took it further by introducing the new possibilities of interactive chats and the new interconnectivity of revolutionizing the way we play board games. The mini robots were demonstrated to bring alive face chatting and physically playing games. The emotional connection that comes from a physical object is unique and Patten’s designs show us how much more connected we can become. I have never imagined in this great detail the realities of controlling things and communicating with my hands the way Patten is presenting to us. However, when you look at movies (I immediately thought of Tom Cruise’s “Minority Report), its not that hard to think that we are going to go a long way with technology. In that movie especially and in new series like NCIS Los Angles, how people communicate and solve problems are on screens that they physically control by dragging objects and pressing in mid air. Pretty interesting when you sit back and think about it all. But to see it in its form, real and useful, as Patten has shown here, takes it from fiction and some movie like fantasy to reality. And not just any reality, but the reality that this new type of communication has benefits beyond measure. Broadening the efforts of everything from medicine to verbal and physical communication.






I think the most interesting to me, was how this kind of technology can change the way we research medicine. He specifically wants to use this in a way and for problems that cannot be solved with people and computers alone. He used one of the robots to control a molecular protein (protein folding). By changing the way a protein is built or its properties, we can find a world of possibilities. In the demonstration, Patten says that if a certain connection does not work you can feel the robot pulling back. This is the next big thing. Can you imagine how useful looking at molecules and manipulating them so that we find cures, or causes and explanations for diseases can be? I was really excited about how he came up with this. 



The point? Patten says that this kind of new interaction, controlling things that are digital with our hands will change the way we interact with everything. Objects that seem to be just as they are, can be transformed into a catalyst for digital control. He says that the power we have on screens are confined to our cellphones and computers. He wants to bring this interaction to our physical word. The possibilities to this kind of integrated technology are numerous. I can’t wait to see what unfolds next. What do you think?




Patten, James “Speakers”, retrieved from http://www.ted.com

Ted Fellows Retreat, Patten James, August 2013, retrieved from  www.ted.com/talks

Pictures are screenshots from James Patten actual talk/powerpoint during the Ted Fellows Retreat


  1. This awesome little video only has 7 views (https://www.youtube.com/watch?v=Z6JcMIBsOt4), but makes me excited the same way reading your entry does Tika. The speed of technological development is truly remarkable, and really is like playing with fire in way. I know that economic matters are part of what holds up the quick release of these fast developing technologies… though I also wonder on an anthropological level what would happen to us if our tech developments continue at this acceleration.


  2. I saw a similar conversation on Ted about opportunities in controlling everything with our hands. I understand your excitement about this future possibilities. I always hope that scientific and technological progress realized by humans will be use for the good of the society, like you wrote talking about medicine apps.


Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s