Professor Marco Gillies

Staff details

Position Senior Lecturer / Director of Distance Learning
Department Computing
Email m.gillies (@gold.ac.uk)
Phone +44 (0)20 7717 3378
Professor Marco Gillies

Marco is the Academic Director of Goldsmiths' Teaching and Learning Innovation Centre. He has a long standing interest in pedagogy and particularly online and technology enhanced learning. He co-created the first MOOC from an English University and was instrumental in developing Goldsmiths' large scale online degrees: BSc Computer Science and MSc Data Science.

Marco is also co-director (with Xueni 'Sylvia' Pan) of both the Masters in Virtual and Augmented Reality and the Social, Empathic and Embodied Virtual Reality Lab. He has been researching Virtual Reality and Artificial Intelligence since the late 1990s and has particularly worked on virtual humans and movement based interaction with VR.

In the past he has also been Academic Director: Distance Learning and co-founded the BSc Creative Computing degree at Goldsmiths, which was a key part of establishing Goldsmiths Computing as a key centre for applying computing to the arts and creative sectors.

Academic qualifications

  • PhD in Computer Science, University of Cambridge 2001
  • BA in Computer Science, University of Cambridge 1997

Teaching and Supervision

Marco Co-directs and teaches on the Masters in Virtual and Augmented Reality.

He supervises masters and PhD projects in many areas relating to Virtual Reality and Immersive media, particularly the interface of VR and AI/Machine Learning, Movement-based and Embodied forms of interaction and social interaction in VR and AR.

Research interests

Marco's research covers a wide range of areas of virtual reality and immersive media, particularly in three major areas:

  • The application of AI and Machine Learning to VR
  • The use of full body movement for interaction with immersive media
  • Social interaction in VR, AR and MR

These three strands are tightly intertwined. Machine learning is a key technology for recognising and designing movement based interaction; body movement and body language are a vital part of what makes VR a powerful medium for social interaction, and social interaction can be with AI characters as well as real people.

Marco is also interested in human-centred approaches to AI and machine learning that use HCI methods and epistemologies to enable machine learning as a creative tool.

His work is highly interdisciplinary and includes applications and collaboration with fields such as dance and performance, medicine, education, games development, critical disability studies, developmental psychology and social neuroscience.

Publications and research outputs

Article

Dewe, Hayley; Gottwald, Janna; Bird, Laura-Ashleigh; Brenton, Harry; Gillies, Marco and Cowie, Dorothy. 2022. My virtual self: the role of movement in children's sense of embodiment. IEEE Transactions on Visualization & Computer Graphics, 28(12), pp. 4061-4072. ISSN 1077-2626

Dobre, Georgiana Cristina; Gillies, Marco and Pan, Xueni. 2022. Immersive Machine Learning for Social Attitude Detection in Virtual Reality Narrative Games. Virtual Reality, 26(4), pp. 1519-1538. ISSN 1359-4338

Slyman, Souad; Gillies, Marco and Lytra, Vally. 2022. Developing an Evaluation Framework for Analysing Educational Simulation Games. European Conference on Games Based Learning (Proceedings of the 16th European Conference on Games Based Learning), 16(1), pp. 526-534. ISSN 2049-0992

Aburumman, Nadine; Gillies, Marco; Ward, Jamie A. and Hamilton, Antonia F.de C.. 2022. Nonverbal communication in virtual reality: Nodding as a social signal in virtual interactions. International Journal of Human-Computer Studies, 164, 102819. ISSN 1071-5819

Gibbs, Janet K.; Gillies, Marco and Pan, Xueni. 2022. A Comparison of the Effects of Haptic and Visual Feedback on Presence in Virtual Reality. International Journal of Human-Computer Studies, 157, 102717. ISSN 1071-5819

Collingwoode-Williams, Tara; O'Shea, Zoe; Gillies, Marco and Pan, Xueni. 2021. The Impact of Self-Representation and Consistency in Collaborative Virtual Environments. Frontiers in Virtual Reality, 2, 648601. ISSN 2673-4192

Cole, Tom and Gillies, Marco. 2021. Thinking and Doing: Challenge, Agency, and the Eudaimonic Experience in Video Games. Games and Culture, 16(2), pp. 187-207. ISSN 1555-4120

Drewett, Olivia; Hann, Gayle; Gillies, Marco; Sher, Carmel; Delacroix, Sylvie; Pan, Xueni; Collingwoode-Williams, Tara and Fertleman, Caroline. 2019. A Discussion of the Use of Virtual Reality for Training Healthcare Practitioners to Recognize Child Protection Issues. Frontiers in Public Health, 7(255),

Gillies, Marco. 2019. Understanding the role of Interactive Machine Learning in Movement Interaction Design. ACM Transactions on Computer-Human Interaction, 26(1), 5. ISSN 1073-0516

Fiebrink, Rebecca and Gillies, Marco. 2018. Introduction to the Special Issue on Human-Centered Machine Learning. ACM Transactions on Interactive Intelligent Systems, 8(2), ISSN 2160-6455

Albertini, Niccolò; Brogni, Andrea; Olivito, Riccardo; Taccola, Emanuele; Caramiaux, Baptiste and Gillies, Marco. 2017. Designing natural gesture interaction for archaeological data in immersive environments. Virtual Archaeology Review, 8(16), pp. 12-21.

Pan, Xueni; Gillies, Marco and Slater, Mel. 2015. Virtual character personality influences participant attitudes and behavior – an interview with a virtual human character about her social anxiety. Frontiers in Robotics and AI, 2, 1. ISSN 2296-9144

Zamborlin, Bruno; Bevilacqua, Frédéric; Gillies, Marco and d'Inverno, Mark. 2014. Fluid gesture interaction design: applications of continuous recognition for the design of modern gestural interfaces. ACM Transactions on Interactive Intelligent Systems, 3(4), 22. ISSN 2160-6455

Kleinsmith, Andrea and Gillies, Marco. 2013. Customizing by Doing for Responsive Video Game Characters. International Journal of Human-Computer Studies, 71(7-8), pp. 775-784. ISSN 1071-5819

Pan, Xueni; Gillies, Marco; Barker, Chris; Clark, David M. and Slater, Mel. 2012. Socially Anxious and Confident Men Interact with a Forward Virtual Woman: An Experimental Study. PLoS ONE, 7(4), e32931.

Gillies, Marco and Spanlang, Bernhard. 2010. Comparing and Evaluating Real Time Character Engines for Virtual Environments. Presence: Teleoperators and Virtual Environments, 19(2), pp. 95-117. ISSN 1054-7460

Gillies, Marco; Pan, Xueni and Slater, Mel. 2010. Piavca: a framework for heterogeneous interactions with virtual characters. Virtual Reality, 14(4), pp. 221-228. ISSN 1359-4338

Gillies, Marco. 2009. Learning Finite State Machine Controllers from Motion Capture Data. IEEE Transactions on Computational Intelligence and AI in Games, 1(1), pp. 63-72. ISSN 1943-068X

Zhang, Li; Gillies, Marco; Dhaliwal, Kulwant; Gower, Amanda; Robertson, Dale and Crabtree, Barry. 2009. E-Drama: Facilitating Online Role-play using an AI Actor and Emotionally Expressive Characters. International Journal of Artificial Intelligence in Education, 19(1), pp. 5-38. ISSN 1560-4292

Gillies, Marco; Pan, Xueni; Slater, Mel and Shawe-Taylor, John. 2008. Responsive Listening Behavior. Computer Animation and Virtual Worlds, 19(5), pp. 579-589. ISSN 1546-4261

Gillies, Marco; Crabtree, I. Barry and Ballin, Daniel. 2004. Customisation and Context for Expressive Behaviour in the Broadband World. BT Technology Journal, 22(2), pp. 7-17. ISSN 1358-3948

Gillies, Marco and Dodgson, N.. 2004. Behaviourally rich actions for user-controlled characters. Computers & Graphics, 28(6), pp. 945-954.

Gillies, Marco and Dodgson, N.. 2002. Eye Movements and Attention for Behavioural Animation. Journal of Visualization and Computer Animation, 13(5), pp. 287-300. ISSN 1049-8907

Conference or Workshop Item

Dobre, Georgiana Cristina; Gillies, Marco; Ranyard, David C; Harding, Russell and Pan, Xueni. 2022. 'More than buttons on controllers: engaging social interactions in narrative VR games through social attitudes detection'. In: ACM International Conference on Intelligent Virtual Agents (IVA ’22). Faro, Portugal 6 - 9 September 2022.

Cole, Tom and Gillies, Marco. 2022. 'Emotional Exploration and the Eudaimonic Gameplay Experience: A Grounded Theory'. In: CHI '22: ACM CHI Conference on Human Factors in Computing Systems. New Orleans, LA, United States 30 April - 5 May 2022.

Cole, Tom and Gillies, Marco. 2022. 'More than a bit of coding: (un-)Grounded (non-)Theory in HCI'. In: CHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI ’22 Extended Abstracts). New Orleans, LA, United States 30 April - 5 May 2022.

Dobre, Georgiana Cristina; Wilczkowiak, Marta; Gillies, Marco; Pan, Xueni and Rintel, Sean. 2022. 'Nice is Different than Good: Longitudinal Communicative Effects of Realistic and Cartoon Avatars in Real Mixed Reality Work Meetings'. In: CHI Conference on Human Factors in Computing Systems Extended Abstracts. New Orleans, LA, United States 29 April - 5 May 2022.

Hilton, Clarice; Plant, Nicola; Fiebrink, Rebecca; Perry, Phoenix; González Díaz, Carlos; Gibson, Ruth; Martelli, Bruno; Zbyszynski, Michael and Gillies, Marco. 2021. 'InteractML: Making machine learning accessible for creative practitioners working with movement interaction in immersive media'. In: VRST ’21: ACM Symposium on Virtual Reality Software and Technology. Osaka, Japan 8-10 December 2021.

Dobre, Georgiana Cristina; Gillies, Marco; Falk, Patrick; Ward, Jamie A; Hamilton, Antonia F de C and Pan, Xueni. 2021. 'Direct Gaze Triggers Higher Frequency of Gaze Change: An Automatic Analysis of Dyads in Unstructured Conversation'. In: International Conference on Multimodal Interaction (ICMI ’21). Montréal, QC, Canada 18 - 22 October 2021.

Gillies, Marco. 2021. 'Learnable Computing'. In: ACM CHI '21: CHI Conference on Human Factors in Computing Systems Extended Abstracts. Yokohama, Japan 8 – 13 May 2021.

Plant, Nicola; Hilton, Clarice; Gillies, Marco; Fiebrink, Rebecca; Perry, Phoenix; González Díaz, Carlos; Gibson, Ruth; Martelli, Bruno and Zbyszynski, Michael. 2021. 'Interactive Machine Learning for Embodied Interaction Design: A Tool and Methodology'. In: Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI ’21). Salzburg, Austria 14–17 February 2021.

Plant, Nicola; Gibson, Ruth; Diaz, Carlos Gonzalez; Martelli, Bruno; Zbyszynski, Michael; Fiebrink, Rebecca; Gillies, Marco; Hilton, Clarice and Perry, Phoenix. 2020. 'Movement Interaction Design for Immersive Media Using Interactive Machine Learning'. In: Proceedings of the 7th International Conference on Movement and Computing. Jersey City, NJ, United States 15 – 17 July 2020.

Gillies, Donald A and Gillies, Marco. 2020. 'Artificial Intelligence and Philosophy of Science from the 1990s to 2020'. In: Conference on the Philosophy of Science today: Seven perspectives (XXV Conference on Contemporary Philosophy and Methodology of Science). Ferrol, Spain 12-13th March 2020.

Gillies, Marco and Pan, Xueni. 2018. 'Virtual reality for social skills training'. In: VR/AR in Higher Education Conference 2018. Swansea, United Kingdom 12 September 2018.

Gillies, Marco. 2018. 'Creating Virtual Characters'. In: MOCO 2018 the 5th International Conference of Movement and Computing. Genova, Italy 28-30 June.

Gillies, Marco. 2016. 'What is Movement Interaction in Virtual Reality for?'. In: 3rd International Symposium On Movement & Computing. Thessaloniki, Greece.

Gillies, Marco. 2016. 'What is Movement Interaction in Virtual Reality for?'. In: MOCO 2016: 3rd International Symposium On Movement & Computing. Thessaloniki, Greece.

Gillies, Marco; Fiebrink, Rebecca; Tanaka, Atau; Garcia, Jérémie; Amershi, Saleema; Lee, Bongshin; Bevilacqua, Frédéric; Heloir, Alexis; Nunnari, Fabrizio; Mackay, Wendy; Kulesza, Todd; Caramiaux, Baptiste; d’Alessandro, Nicolas and Tilmanne, Joëlle. 2016. 'Human-Centered Machine Learning'. In: CHI '16 Extended Abstracts on Human Factors in Computing Systems. San Jose, United States 7 - 12 May 2016.

Donald, Nicky and Gillies, Marco. 2015. 'Novel Dramatic and Ludic Tensions Arising from Mixed Reality Performance as Exemplified in Better Than Life'. In: International Conference on Interactive Digital Storytelling. Copenhagen, Denmark.

Cole, Tom; Cairns, Paul and Gillies, Marco. 2015. 'Emotional and Functional Challenge in Core and Avant-garde Games'. In: CHI Play 2015. London, United Kingdom.

Gillies, Marco; Brenton, Harry; Yee-King, Matthew; Grimalt-Reynes, Andreu and d'Inverno, Mark. 2015. 'Sketches vs Skeletons: Video Annotation Can Capture What Motion Capture Cannot'. In: Proceedings of the 2Nd International Workshop on Movement and Computing. Vancouver, Canada.

Gillies, Marco; Brenton, Harry and Kleinsmith, Andrea. 2015. 'Embodied Design of Full Bodied Interaction with virtual humans'. In: 2nd International Conference on Movement and Computing. Vancouver, Canada.

Gillies, Marco; Kleinsmith, Andrea and Brenton, Harry. 2015. 'Applying the CASSM Framework to Improving End User Debugging of Interactive Machine Learning'. In: ACM Intelligent User Interfaces (IUI). Atlanta, United States.

Brenton, Harry; Kleinsmith, Andrea and Gillies, Marco. 2014. 'Embodied Design of Dance Visualisations'. In: International Workshop on Movement and Computing. paris, France.

Mohd Mokhtar, Ezwan and Gillies, Marco. 2013. 'How I Animate: A Hand Gesture-Centric Technique'. In: EVA London 2013: Electronic Visualisation and the Arts. London, United Kingdom.

Gillies, Marco; Zapata Gomez, Shakti and Zimmer, Sebastian. 2013. 'Suzie’s Dance: reliving dance through motion capture animation of photographs'. In: Electronic Visualisation and the Arts. London, United Kingdom.

Gillies, Marco; Worgan, Max; Peppe, Hestia; Robinson, Will and Kov, Nina. 2011. 'Exploring Choreographers’ Conceptions of Motion Capture for Full Body Interaction'. In: The BCS Conference on Human Computer Interaction. Newcastle, United Kingdom.

Gillies, Marco. 2010. 'Bodily Non-verbal Interaction with Virtual Characters'. In: Kansei Engineering and Emotion Research. Paris, France.

Gillies, Marco; Zhang, Li and Barnden, J A. 2008. 'EMMA: an Automated Intelligent Actor in E-drama'. In: Proceedings of International Conference on Intelligent User Interfaces. Canary Islands, Spain 13th - 16th Jan 2008.

Pan, Xueni; Gillies, Marco and Slater, Mel. 2008. 'Male Bodily Responses during an Interaction with a Virtual Woman'. In: Intelligent Virtual Agents. Tokyo, Japan 2008.

Zhang, Li; Gillies, Marco; Barnden, J A and Hendley, R J. 2007. 'An Improvisational AI Agent and Emotionally Expressive Characters'. In: Workshop on Narrative Learning Environments, Supplementary Proceedings of the 13th International Conference of Artificial Intelligence in Education. Los Angeles, United States July 9-13, 2007.

Zhang, Li; Gillies, Marco; Roberson, D.; Oldroyd, Amanda and Barnden, J A. 2007. 'Emotionally Expressive Characters in E-drama'. In: In Proceedings of International Conference of Internet Technology and Secured Transactions. London, United Kingdom.

Dhaliwal, Kulwant; Gillies, Marco; O’Connor, John; Oldroyd, Amanda; Robertson, Dale and Zhang, Li. 2007. 'eDrama: Facilitating online role-play using emotionally expressive avatars.'. In: The AISB Workshop on Language, Speech and Gesture for Expressive Characters. UNDEFINED 2007.

Pan, Xueni; Gillies, Marco; Sezgin, T M and Loscos, C. 2007. 'Expressing Complex Mental States Through Facial Expressions'. In: Second International Conference on Affective Computing and Intelligent Interaction. Lisbon, Portugal.

Gillies, Marco. 2006. 'Applying direct manipulation interfaces to customizing player character behaviour.'. In: International Conference on Entertainment Computing. UNDEFINED 2006.

Vinayagamoorthy, Vinoba; Gillies, Marco; Steed, A.; Tanguy, E.; Pan, Xueni; Loscos, C. and Slater, M.. 2006. 'Building Expression into Virtual Characters'. In: Eurographics Conference State of the Art Reports. Vienna, Austria 4-8 September, 2006.

Gillies, Marco; Crabtree, I. Barry and Ballin, Daniel. 2006. 'Individuality and Contextual Variation of Character Behaviour for Interactive Narrative.'. In: AISB Workshop on Narrative AI and Games. UNDEFINED 2006.

Friedman, Doron and Gillies, Marco. 2005. 'Teaching Characters How to Use Body Language'. In: Intelligent Virtual Agents. Kos, Greece 12-14 September, 2005.

Gillies, Marco and Ballin, Daniel. 2004. 'Affective Interactions Between Expressive Characters'. In: Proceedings of the IEEE conference on Systems, Man and Cybernetics. The Hague, Netherlands.

Gillies, Marco and Ballin, Daniel. 2004. 'Integrating autonomous behavior and user control for believable agents'. In: Third international joint conference on Autonomous Agents and Multi-Agent Systems. Columbia University, New York 2004.

Gillies, Marco; Crabtree, Barry and Ballin, Daniel. 2004. 'Expressive characters and a text chat interface'. In: AISB workshop on Language, Speech and Gesture for Expressive Characters. University of Leeds, UK March, 2004.

Ballin, Daniel; Gillies, Marco and Crabtree, Barry. 2004. 'A Framework For Interpersonal Attitude And Non-Verbal Communication In Improvisational Visual Media Production'. In: 1st European Conference on Visual Media Production (CVMP). IEE, London, UK 15-16 March 2004.

Gillies, Marco; Ballin, Daniel and Csáji, Balázs Csanád. 2004. 'Efficient Clothing Fitting from Data.'. In: 12th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG). Plzen, Czech Republic 2-6 February 2004.

Gillies, Marco and Ballin, Daniel. 2003. 'A Model of Interpersonal Attitude and Posture Generation'. In: Workshop on Intelligent Virtual Agents. UNDEFINED.

Gillies, Marco; Ballin, Daniel and Dodgson, Neil. 2002. 'Integrating Internal Behavioural Models with External Expression'. In: PRICAI '02 International workshop on Lifelike Animated Agents: Tools, Affective Functions and Applications. Tokyo, Japan 2002.

Gillies, Marco; Dodgson, Neil and Ballin, Daniel. 2002. 'Autonomous Secondary Gaze Behaviours'. In: AISB workshop on Animating Expressive Characters for Social Interactions. Imperial College, London 2002.

Gillies, Marco and Dodgson, Neil. 1999. 'Invariants and Affordances for Walking in a Cluttered Environment'. In: Second Workshop on Intelligent Virtual Agents. University of Salford 1999.

Gillies, Marco and Dodgson, Neil. 1999. 'Ball Catching: an example of psychologically-based behavioural animation'. In: Eurographics UK 17th annual conference. UK 1999.

Gillies, Marco and Dodgson, Neil. 1999. 'Psychologically-based Walking in a Cluttered Environment'. In: Eurographics `99 Short Papers and Demos. Milan, Italy 1999.

Book Section

Gillies, Marco and Kleinsmith, Andrea. 2014. Non-representational Interaction Design. In: Mark (J. M.) Bishop and Andrew Martin, eds. Contemporary Sensorimotor Theory. 15 Switzerland: Springer International Publishing, pp. 201-208. ISBN 978-3-319-05106-2

Giannachi, Gabriella; Gillies, Marco; Kaye, Nick and Swapp, David. 2009. Mediating Performance Through Virtual Agents. In: Zsófia Ruttkay; Michael Kipp; Anton Nijholt and Hannes Högni Vilhjálmsson, eds. Intelligent Virtual Agents 2009. Amsterdam: Springer, pp. 439-445. ISBN 978-3-642-04379-6

Gillies, Marco; Ballin, Daniel; Pan, Xueni and Dodgson, Neil. 2008. Semi-Autonomous Avatars: A New Direction for Expressive User Embodiment. In: Lola Cañamero and Ruth Aylett, eds. Animating Expressive Characters for Social Interaction. John Benjamins Publishing Company, pp. 235-255. ISBN 9789027252104

Zhang, Li; Gillies, Marco; Barnden, J A; Hendley, R J; Lee, M G and Wallington, A W. 2007. Affect Detection and an Automated Improvisational AI Actor in E-drama. In: , ed. Special Issue on Artificial Intelligence for Human Computing. Lecture Notes in Artificial Intelligence. (4451) Berlin/Heidelberg: Springer, pp. 339-358.

Gillies, Marco; Ballin, Daniel and Dodgson, Neil. 2003. Semi-Autonomous Avatars and Characters. In: Ruth Aylett and Lola Canamero, eds. Animating Expressive Characters for Social Interactions. John Benjamins.

Report

Mees, Annettte; Wright, Tim; Donald, Nicky; Gillies, Marco; Milne, Austin and Prime, Sian. 2015. Coney: Better Than Life. Project Report. NESTA, LONDON.

Gillies, Marco. 2001. Practical behavioural animation based on vision and attention. Technical Report. UCCL, Cambridge.

Software

Gillies, Marco. 2014. Gestyour.

Other

Devlin, Kate and Gillies, Marco. 2014. CO3355 Advanced graphics and animation subject guide. UCL, Goldsmtihs, University of London International Programme.

Further profile content

Goldsmiths Research Centres/Groups

Research projects

The plasticity of the bodily self: how function and age shape the acceptance of virtual bodies.
A ESRC funded collaboration with Dorothy Cowie at the University of Durham that investigates how adults and children can adapt to virtual bodies with different appearances and functions

Immersive, Innovative, and Interactive Experience (IIIE): A Technology Driven London-Shanghai Partnership in Mixed Realities and Performing Arts
Led by Xueni 'Sylvia' Pan, this AHRC funded project aims to foster collaboration between the creative sectors in the UK and China around immersive media and performance

2019-2021: 4i: Immersive Interaction Design of Indie Developers using Interactive Machine Learning
An EPSRC funded collaboration with University of Coventry CDARE, UAL CCI and Gibson/Martelli, looking at using machine learning to enable dancers and artists to design movement interaction for VR

2018-2019: AI-Driven Characters for Immersive Narrative Games
An InnovateUK funded collaboration with game studios Maze Theory and Dream Reality Interactive that applied machine learning to social interaction with immersive game characters for Peaky Blinders

2017-2020: Understanding and generating real-time face-to-face social interactions
A Leverhulme funded project led by Antonia Hamilton of the UCL Institute of Cognitive Neuroscience using motion capture and machine learning to understand non-verbal social interaction.

Articles related to Professor Marco Gillies