GI 2017 Paper: Ivy

Also at GI this year will be another project that I was part of while at Autodesk Research. Barrett Ens, who was interning with Fraser (Anderson) and Tovi Grossman, had a keen interest in VR and 3D user interfaces and a really interesting idea to explore the next generation of programming environments that were based in VR and powered by Internet of Things devices and activities. Thus, Ivy was born! Ivy explored how intelligent, aware environments could be programmed in-situ, within VR representations of the target environment (or possibly in the future, AR representations of it). The project not only presented possible programming constructs and corresponding visualizations that would be useful for programmers of such spaces, but also explored how to integrate and represent real world data and breakpoints in a manner that was appropriate for spatially-situated environments. Aside from loving the name, I personally really appreciated seeing data flowing from sensors to other machines or equipment. The simple act of perceiving data as moving entities, really brings the notion of such programming paradigms life. The paper, Ivy: Exploring Spatially Situated Visual Programming for Authoring and Understanding Intelligent Environments, will be presented by Barrett and was also co-authored by Pourang Irani (University of Manitoba) and George Fitzmaurice (Autodesk Research).


The availability of embedded, digital systems has led to a multitude of interconnected sensors and actuators being distributed among smart objects and built environments. Programming and understanding the behaviors of such systems can be challenging given their inherent spatial nature. To explore how spatial and contextual information can facilitate the authoring of intelligent environments, we introduce Ivy, a spatially situated visual programming tool using immersive virtual reality. Ivy allows users to link smart objects, insert logic constructs, and visualize real-time data flows between real-world sensors and actuators. Initial feedback sessions show that participants of varying skill levels can successfully author and debug programs in example scenarios.

Leave a Reply

Your email address will not be published. Required fields are marked *