PhD students

The early history of video games has seen corporations producing titles with the primary objective to entertain their consumers. Nowadays, the stereotype of video games has been undermined, as video games are no longer perceived as exclusively a form of entertainmentand their use serves a much broader spectrum than in the past.

The positive instructional impact of video games, and the increasingly popularity of e-learning systems, has drawn the attention of researchers and academics who further consider the integration of games in formal education. Gamification, game-based learning, and simulations are all contemporary game-based techniques adopted in educational contexts to enhance students’ engagement, enjoyment, and motivation.

Andrea has been working as a programmer and educational experience designer for the BSc Computer Science online programmes offered by Goldsmiths, University of London, and Coursera for the past two years, developing instructional game-based learning web activities.

The research attempts to shed light on game-based learning in online education, a less discussed technique compared to gamification but more closely related to the motivational and, most importantly, instructional nature of video games.

The focus of my research is on the development of an abstract audiovisual composition practice within the emerging paradigms of machine learning and immersive technologies. The work examines theoretical concerns of audiovisual composition, methods for mapping audio and visual material using interactive machine learning techniques and the development of a software toolkit aimed at creating abstract audiovisual art within virtual environments. The research output includes theoretical writing, a software toolkit and a body of audiovisual artwork.

Through this research, a fully customisable and stand-alone wearable device is being developed, that employs machine learning techniques to recognise individual hand gestures and translate them into text, images and speech. The aim is to have the device recognise and translate custom hand gestures by training a personal classifier for each user, relying on a small training sample size, that works offline with a high classification accuracy rate. This is to be achieved by planning and executing a series of iterative case studies, with user testing carried out by real users in their every day environments and in public spaces.

I study interactions between people and computer systems that employ learning algorithms to adapt their behaviour to users. My main focus is the application of such systems to assistive technologies for people who are blind and visually impaired. The aim of my research is to create technology that give people greater control over and improve accessibility of the systems they rely on in their daily lives.

LivePrinter is an open source system for live, intuitive, immediate drawing and fabrication with 3D printers demonstrating the future possibility of people becoming creatively integrated into systems of semi-automated, digital manufacturing.

To allow users to create physical forms using different functions that take into account physical properties like speed and temperature, instead of the usual visual modelling techniques and explore how musical concepts and visual aesthetics can directly influence digital manufacturing toolpaths and vice versa.

Blackwell and Aaron’s (2015) notion of “practice-led craft research” was influential, where the role of the designer-academic is to bring their critical and analytical tools to bear on the design process itself, along with “cybernetic design process” where the design process involves the researcher themselves and results in the abductive emergence of theories from their experiments from Wolfgang Jonas (2015).

I created a working software system for controlling 3D printers, a series of physical artefacts illustrating different generative/intuitive techniques for making and a series of live hybrid physical/musical performances documented in videos.

On reflection, it is entirely possible to use code as a means for creating and specifying the design of objects and new expressive languages like the LivePrinter minigrammar will need to be developed to support further practice and research, alongside hardware design and demonstrated that coding objects directly can create new forms that are not limited by CAD tool metaphors and take full advantage of 3D printers, like printing sparse forms in the air.

My PhD is a practice based research project in games anthropology.

My investigation starts from an interest in games and play and in particular in how play can be seen as a form of language and communication. In this respect I have a deep interest in a specific landscape of small independent productions that go beyond the fictional and escapist content to instead depicting deep and intense bits of the human experience. 

Taking from anthropology and from the study of ritual I plan to study how creators use the peculiar aspects of games in order to support and reinforce the message they want to deliver; how they craft the game space as a space where to use a specific language to communicate feelings and how all this is received and decoded by the audience. By focusing my study on digital games, the way algorithms act as a platform for this language to emerge is crucial.

The practical based part of my research aims to collect and put in relation all my findings in an interactive documentary.

Friendred is an installation & computational artist /researcher. He is currently a PhD candidate in ACT Goldsmiths, researching the intertwined relationship between technology and performance arts in the field of HCI. Since 2015, Friendred has been focused on disciplines crossing arts, technology and sciences. His recent work aimed to bridge the conversation between movement and algorithmic machines, exploring the sensory apparatus and interactive systems. Through exploring technologized elements and influences in the society and their correlations and entangled relationships with other disciplines, the emergence of the embodiment, technologized performance and architectural body became his main practice.

Social interaction between humans and virtual characters (VC) within a virtual space has a particular importance for the development of immersive Virtual Reality (VR) experiences. VCs are a powerful tool for building communication-based interactions for immersive VR applications for science, education, training, travel, entertainment and news.

Humans have finely honed social observational skills and they can easily spot errors made by the VC. Frequently, these errors are not seen as technological flaws, but as the VC’s mannerisms. A great effort is put into developing VCs that are able to perform contextual behavioural cues. Examples of these include gaze, body posture, hand gestures, facial reactions, or backchannels. My current focus is on the gaze behaviour, the pattern of eyes movement, showing where someone looks at.

My research involves using machine learning models to generate and explore gaze and other appropriate nonverbal behaviour for virtual humans suitable for immersive virtual environments. Using multimodal datasets (including video eye data, upper-body motion-capture, audio and video recordings) I’m working on building nonverbal behaviour models that drive virtual humans gaze behaviour based on the user’s actions.

Designing communication and other social interactions in immersive VR can be a challenging task, and aspects on this are addressed in my research. The findings from these studies can help game designers and game developers determine the appropriate non-player character's non-verbal (and verbal) behaviour in games, especially in VR games.

Along with its applications in the games industry, the findings would be useful for other applications such as designing multi-modal human-machine interactions and other systems for medical purposes, for social anxiety disorders therapy, simulations, training or learning.

In this work we investigate dynamic models for the prices of financial assets when prices can jump. In particular, we consider a class of financial models within which the prices of assets are Lévy-Ito processes driven by an n-dimensional Brownian motion and an independent Poisson random measure that is associated with an n-dimensional pure-jump Lévy process.

Each such model consists of a pricing kernel, a money market account, and one or more risky assets. We show that the excess rate of return of a risky asset in a pure-jump model is given by an integral of the product of a term representing the riskiness of the asset and a term representing market risk aversion.

The integral is over the state space of the Poisson random measure and is taken with respect to the Lévy measure associated with the n-dimensional Lévy process. The resulting framework is applied to the theory of interest rates and foreign exchange, allowing one to construct new models as well as various generalizations of familiar models.

As an application of the theory of Lévy-Ito processes, we consider the problem of optimal hedging in an incomplete market with an established pricing kernel.

In such a market, prices are uniquely determined, but perfect hedges are usually not available. Given a position in need of hedging and the instruments available as hedges, we demonstrate the existence of an optimal hedge portfolio, where optimality is defined by use of a least expected squared error criterion over a specified timeframe, and where the numeraire with respect to which the hedge is optimized is taken to be the benchmark process associated with the designated pricing kernel.

As another application we consider the problem of determining the Lévy exponent in a Lévy model for asset prices given the price data of derivatives. The model, formulated under the real-world measure, consists of a pricing kernel together with one or more risky assets driven by the same Lévy process. We show that if the initial prices of power-payoff derivatives are given, then the Lévy exponent is determined up to a linear term. In such a setting, derivative prices embody complete information about price jumps: in particular, the spectrum of the price jumps can be worked out from current market prices of derivatives.

The aim of my PhD project is to research algorithms and human / computer interaction methods that can be used to create a tool that will let a game designer direct and evolve believable digital creatures, from nothing to full model using machine learning, with the purpose of being able to directly use and export those creatures’ animations and meshes into existing asset pipelines.

Through this research, a fully customisable and stand-alone wearable device is being developed, that employs machine learning techniques to recognise individual hand gestures and translate them into text, images and speech.

The aim is to have the device recognise and translate custom hand gestures by training a personal classifier for each user, relying on a small training sample size, that works offline with a high classification accuracy rate. This is to be achieved by planning and executing a series of iterative case studies, with user testing carried out by real users in their everyday environments and in public spaces.

Based on established neurobiological correlates of reinforcement learning (RL), I investigate animal learning and decision making using cognitive modelling techniques, such as probabilistic programming and machine learning.

Animals somehow manage to create useful representations of incoming sensory information, representations then used for learning and decision making.

How these representations of states of the world are integrated into task structure and models of the world is an open question. In search of answers, my research uses RL combined with several complementary methods to model human behavioural data.

The potential findings of these experiments have promise to not only elucidate the workings of the animal brain but also provide valuable contributions to artificial intelligence, where improved models of state representations could vastly improve data efficiency and generalisability over current-generation systems.

Janet is exploring how multi-modal perceptual feedback contributes to a player's sense of presence in the virtual world. Jaron Lanier described Virtual Reality (VR) as the substitution of the interface between a person and their physical environment with an interface to a simulated environment. This interface is of particular significance in understanding how presence depends on the nature, extent and veridicality of our sensorimotor interaction with the virtual environment, and how that relates to our normal engagement with the real world.

Building on sensorimotor theory of perception, Janet aims to establish how our perceptual system responds to multi-modal feedback that almost, but not quite, matches what we are used to, in making sense of the simulated environment of VR.

My research aims to reveal insight into how we can best mediate difficult conversation using technology — studying experiences that sit within the newly developing sub-genre of interactive theatre, which explicitly uses technology to facilitate participation. At the heart of these experiences are a series of complex moral dilemmas that the audience must navigate by conversing amongst each other and forming a collective decision.

This discourse presents an opportunity to study an individual’s participation throughout the experience, including ascertaining their understanding of key plot points. Using this information, this project looks to explore the elements of interactions with technology that can affect participation, positively or conversely, including the potential for external stimuli such as the spatial and social properties of the containing space. 

Using cameras to obtain accurate body dimensions rather than measuring these manually, is a developing research direction in apparel (fashion) industry with high practical potential. This technology promises to significantly reduce costs and improve efficiency.

The purpose of our research is to study and progress where needed how state-of-the-art body and motion capture technology can be made a reliable and affordable option for the apparel/fashion sector. We review in the paper many of the available body scanning technologies (including 3D body scanners, Mo-Cap systems).

To study how body movements, as well as body changes, impact the experience of wearing a piece of clothing is another, more challenging focus of our work. We also conduct a state-of-the-art survey of how recent machine learning methods are providing new ways to conduct such studies and analyses as a function of new and growing databases.

As the habit of shopping online for clothing becomes more popular, so is the tendency to return garments with poor fit often due to a mismatch on what is visually experienced online and the real-life experience of wearing the order. The high rate of returns (for some items up to and over 80%) represents a major challenge for the industry. Better capture and understanding of body measurements are known to be one of the major factors limiting progress on this issue.

Scientific data are intrinsically high dimensional - that is, 3D or even higher. However, most of these data are currently being projected onto a 2D screen. The goal for my research is not only to visualise scientific data in 3D immersive VR but also to make it interactive.

This is motivated by the fact that Virtual Reality has improved significantly over the past decade, VR devices are now more affordable and offer more features than ever before.

The traditional method of navigation and interaction in VR is through the use of device controllers however with improvement in tracking technology and machine learning we can now simply use our hands and interact with virtual world without the need of controllers or external tracking devices.

Another field which my research reflects on is social VR, the idea of allowing multiple users to join in VR experience.

Artists have been experimenting with software as an artistic medium since the late 1960s, however, in recent years software-based art has developed from being a niche medium seen only at media festivals to entering mainstream collections of contemporary art museums and private collectors.

The fast-changing nature of the technological context in which software-based artworks must survive means that many artists, particularly those with practices beginning in the 1990s and early 2000s, have had to resolve the same problems that conservators in collecting institutions are responding to.

This doctoral research will provide a systematic study of what artists are doing to preserve, document, maintain and conserve their work, and how this relates to their production technologies in comparison with practices within the gaming industry and conservation.

It will consider these practices of maintenance within the context of studies of communities that modify and adapt technologies, exploring the nature of the ecosystem in which software developers from these three domains operate and their relationship to broader social relationships to technology. This will inform our understanding of obsolescence in the context of the gaming industry, artistic practice and conservation, while also probing differences of meaning and understanding.

The redefinition and reusing of creative methods of modding, crossover and shipping investigate a possibly queer Maximalism aesthetic practices and judgments for the development of cognitive and affective capabilities in ACGN and fandom contexts, especially with an Asian cultural background, for example, mainland China, Hong Kong, Taiwan, Japan, and South Korea.

Modding means modification of various elements, objects, and properties in video game. Crossover aims to present two characters at the same time from different fictional worlds together in ACGN community. Shipping in fandom is making affectionate relationship between bodies.

Queer Maximalism HyperBody is supposed to make aesthetic relationships, entanglements, and intra-actions via modding, crossover and shipping in the cosmotechnics of multi-fandoms.

This thesis presents a set of experimental studies to understand the benefits of utilising architectural design to create virtual environments optimised for completing a series of cognitively demanding tasks. Each field of investigation is reviewed separately.

The first field of investigation relates to spatial design and analysis from an architectural standpoint. The second is concerned with memory, spatial abilities, and embodied cognition. Two VR-based user studies are designed to further explore the potential interactions between these fields of knowledge.

An initial experiment called “Archimemory” is based on a memory palace, a historical mnemonic technique, to explore how spatial knowledge representation can enhance memory retrieval.

It compares the benefits of using different architectural designs in VR to support participants’ recall accuracy of a sequence of playing cards. The main user study, called the ""Immersive Virtual Architecture Studio"" (IVAS), validates a new methodology to study the effect of spatial qualities on embodied cognition related tasks.

A spatial analysis using the isovist technique provides an objective approach to measure spatial qualities such as ""openness"" and ""complexity"". Participants have to perform a batch of cognitive tasks in the IVAS. Results from the spatial analysis are compared to participants subjective rating of the same spatial qualities as well as their performance.

Findings suggest that a spatial performance metric can be evaluated for each room, for instance, it was the highest in the case of the more closed (fewer windows) and more complex (with columns) condition.

The combination of spatial analysis and performance metrics obtained from these two novel VR applications, Archimemory and IVAS, leads this research to form a Virtual Architecture Framework.

Guidelines are proposed for VR architects, UX designers and scientists to adopt this framework to support further exploration and evaluation of spatial design to enhance human cognitive abilities when experiencing immersive learning environments.

My research includes the study of large clinical datasets, such as primary care data, using Deep Artificial Neural Networks (""Deep Learning"") and other technologies. This requires the rigorous use of statistical techniques and measures, including those used in clinical research.

I am also analysing datasets such as the Alzheimer’s Disease Neuroimaging Initiative (ADNI) data, which includes MRI imaging and bio-metric data, using Deep Learning models such as Convolutional Neural Networks (CNNs). Further research planned includes analysing text data and the use of Long Short-Term Memory (LSTM) models to study changes over time.

Autism is a spectrum condition with a wide range of severity levels, individuals with Autism are generally impacted by the condition in various ways, some individuals with autism are able to live relatively independent lives while others need a lifetime of support.

Despite the range of the spectrum, individuals affected with autism share a group of similarities that become more apparent when interacting with people and the environment. Autism is typically characterised by the following impairments, difficulty with 1) Social Communication, (2) Social interaction, and (3) Rigidity of thought. These characteristics that define autism are described as the triad of impairment.

Due to their highly visual environments, assistive technology devices or systems can be used in enhancing learning and the daily functioning of autistic individuals. The variation in the behavioural limitations and strengths of autistic individuals leads to variabilities in their support needs.

Current assistive technology designed for autistic learners do not provide users the ability to tailor various elements of the system to address their specific usage goals. They assume all autistic individuals are affected by the impairment in the same way. The motivation for this research is due to the primary issues with assistive technology designed for individuals with autism and the gaps in existing research in this area.

This research contributes a formal approach for designing personalisable assistive technology for autistic learners, this approach is called the Personalisable Impairment Design (PID). The PID principles have been devised to address each aspect of the autism triad of impairment.

This study suggests that software designed for autistic learners should adopt the PID (Personalisable Impairment Design) methodology, in order to effectively address the changing expressions of autism and the challenges currently encountered in educating autistic learners.

Souad is an experienced business computing educator & researcher. She has a multidisciplinary scientific, educational and business background, starting with 1st class degree in Mathematics and Computing (BSc), continued with Master of Arts in Mathematics Education (MA) and currently conducting a PhD in Computer Science in game studies, design and development at Goldsmiths, University of London. Souad's research focuses on evaluating the impact of games that is purposefully designed to improve learning, attitudes and skills, in a range of academic areas:

• Simulation Games
• Serious Games for Education and Medicine
• HCI
• Information System
• Software Engineering
• Leadership
• Machine Learning

Souad’s current research focuses on games studies, games design and development and finding out ways of improving statistical learning using game-based learning. Souad is an experienced professional in quantitative and qualitative research methods and she is keen to explore potential research partnerships in this area/ interesting projects.

"The use of Virtual Reality (VR) has become a common medium for social interaction. With this rise in demand for Head Mounted Displays (HMDs), so is the need to create Embodied Shared Virtual Environments (ESVE) where users may experience authentic social interactions.

This thesis presents an exploratory examination of Embodiment - meaning the subjective feeling of owning a virtual representation in VR, and specifically Consistency in Embodiment - relating to how we prioritize and synchronise objective attributes of embodiment (i.e avatar representation) in order to create ESVEs which supports more intuitive social interaction.

The goal is to understand how different technical setups could have a psychological impact on participants' experiences in ESVE. This research hopes to inform the development of successful social interaction in a variety of applications in VR, ranging from training to gaming.

Study 1 assessed the impact of consistent and inconsistent arm movement and lipsync whilst being embodied in a virtual avatar. The survey results showed an interaction effect which suggested that there were higher levels of Embodiment in consistent conditions where there was agency, and that the addition of Lip Sync in itself is not critical to facilitating higher levels of Embodiment.

Study 2 explored the impact of the visualisation of a full-body self-avatar versus just controllers in a collaborative ESVE, as well as the consistency of avatar representation between the two users, with one of them being a confederate (Exp1) or both participants (Exp2). The level of trust post VR was measured using a questionnaire and a trust game.

Exp1 suggested that participants were more likely to give higher scores to the same confederate (regardless of confederate’s representation) when they themselves have an avatar. Exp2 suggested participants trusted each other more in consistent conditions, and that consistency also had an impact on the pattern of their performance in the collaborative task.

Study 3 will look at consistency in embodiment in a medical training application. Participants will be given the instruction to give bad news to an angry parent regarding the operation of their child whilst embodied in an avatar.

We will be testing how different manipulation of embodiment would change their experience, and whether being able to review the virtual consultation from the virtual patient’s perspective will enhance learning.

In my research, I am investigating new methods for analysing and manipulating deep generative models. Generative models, such as Generative Adversarial Networks (GANs), are very powerful tools that can learn from data to generate all sorts of outcomes, such as images, audio and 3D models.

While they can learn to effectively reproduce the kinds of things that you have data for, they aren’t able to generate novel outcomes that are unlike the training data, and they are also notoriously difficult to interpret and control.

With my research I am looking at how to take already trained models, and develop methods to better understand their internal workings, then to apply interventions in the computational graph of these models, to alter the flow of information inside them so they begin generating results that diverge from the original training data.

So far I have done this through fine-tuning models to alter their weights, or by adding extra layers inside networks. I recently presented an approach that I have called Network Bending, where I have applied image filters inside GANs that can be deterministically controlled by a user.

I have also developed a novel algorithm for understanding the inner working of GANs in an unsupervised fashion, leading to the discovery of sets of features that control the generation of different semantic properties in the image that can then be manipulated in tandem.

This work has applications in the interpretability of deep generative models as well as in developing methods and tools for creative expression. In future work, I am looking to apply these methods to generative models for other domains, like audio and 3D. 

While the inclusion of original soundtracks is on the rise in the board games industry, in their current form, these soundtracks can only accommodate games with low consequence randomness – a type of randomness that causes unpredictable changes to the game state and gameplay experience.

As tabletop games are becoming increasingly narrative-driven – a feature that results in higher immersion levels and higher consequence randomness – a dynamic, adaptive soundtrack system is needed that is able to follow game state changes accordingly, resulting in gaming experiences on par with video games.

The goal of this research is to create such a system, exploring the concept of hidden interaction (interaction without the user’s conscious interference) with the tools of machine learning and adaptive music while attempting to gain an understanding of immersion in analogue games from an experiential perspective.

Yajuan's research focuses on exploring the relationship between female identities and virtual spaces in relation to issues of agency within the Chinese context.

Operating across the paradigms of game design, immersive technologies and art, the study aims to examine the potential influence on and eventual transformation of attitudes towards female positioning and identities within the gaming industry and beyond. By creating an interactive art experience that adopts the language and conventions of gaming, the practice-based research establishes conversations between art, female identities, technology and games.

The outcome aims to introduce potential breakthroughs for investigating alternative modes of female empowerment in China and reveal innovative possibilities in game design for female Chinese players.

Alex's research focuses on human-computer-interaction, expression with sound, and game design. His research explores the role of expression in player experience and immersion.

He is a multi-disciplinary creative-practitioner with a background in music, designing and developing devices for musical expression, building interactive installations and computational studio art.