MA/MSc Virtual & Augmented Reality

The course will introduce students to Virtual Reality and cover a selection of the topics below:

• VR history and its place in broader media art history
• Introduction to VR hardware
• Examples of VR applications across creative disciplines
• Human Perception and the Psychology behind VR
• VR graphics
• 3D interaction (navigation, object interaction)
• Narrative in VR and critical approaches to VR
• Embodiment
• Technical topics: character rigging in 3D, 3D audio etc
• Design of VR experience as part of a creative practice

Students will gain practical experiences in the above areas creating VR applications. The technical focus of the module will be on coding for VR using game engines (C# in Unity), with opportunities for exploration of web-based VR (e.g. AFrame).

This module introduces students to a range of contemporary research in virtual reality and related fields. Students will be asked to read a number of research papers of their choosing and then focus on one paper to develop in a small group. In the first half of the course, the groups will take turns to make a short presentation about their chosen paper, in a seminar context. In the second half they will perform a replication of the paper: which will either be an experimental study or an implementation of a VR system, depending on the nature of the chosen paper, the results of which will be presented to the class at the end of the session. This work will be complemented by a series of guest lectures by researchers in the field from both Goldsmiths and other institutions. The researchers will present their current work.

Students will learn advanced techniques for immersively combined real world with virtual contents including augmented reality, photogrammetry, motion capture, and computer vision. In the lab, they will be able to apply these techniques to create a mixed reality application that combines real and virtual content appropriately, and leverage the body in this experience.

This module covers the basics of 3d modelling, texturing, rigging and animation. Students will learn how to use modelling software to create a range of 3D rigid body assets varying from buildings to vehicles to household objects to vegetation to roads and terrain, etc. Lighting and texturing will be taught as well as simple rigging and animation workflows.

At the end of the module, the students will be able to use Industry standard export pipelines, to integrate the assets they have created into simple prototype projects in a game engine (such as Unity or Unreal).

You will learn:

• Navigation in 3d space, primitives and hierarchies
• Hard surface poly modelling
• Modelling from reference and concept art
• Ligh8ng and simple shader workflows
• Animation principles, forward and inverse kinematics
• So; body modelling
• Simple low poly character, skinning and rigging
• Simple character animation, walk cycles
• Export pipelines using Ox and obj formats

In this module, you'll learn foundational game technologies through practical in-engine work.  We'll explore the concepts underlying the functioning of a typical videogame and the operational pipeline to produce interactive content in a game engine with a minimal amount of coding. 

The creation of digital games as well as interactive experiences of various natures needs to be supported by an understanding of the technical aspects as well as by the knowledge of a typical development pipeline.

The development of an interactive experience doesn’t stop at coding. but it involves a series of visual tools and technology that work in concert towards the final outcome. Whatever the game engine is used in a studio or in a project, the underlying skills and concepts needed to create a satisfying product are common. 

This module focuses on the understanding of the visual tools used in game engines and on the use of them to create an interactive experience and aims to give students the tools for easily move to different creation environments, by making them experience concepts like cameras, composition, interactivity, animation and greyboxing. 

The first part of the module will introduce you to the general elements of a game engine. One side is exploring how we visualise and animate objects with a computer and the other is how we formalise these elements through the concept of classes, instances and components. You'll also understand what a game loop is, and the difference between frame-based and time-based approaches. At the same time, you'll work on simulated physics, sensors and collisions. 

The second part will give you an understanding of the visual tools of a typical game engine, ranging from the UI builder to camera management and animation systems, to simple shaders and VFXs with a particular focus on visual scripting. You'll learn how to integrate and control these aspects with minimal code, and which hybrid visual/coding approach is best suited for creating small games and prototypes. 

The lab part of the module offers in-lab challenges and tutorials finalized to the creation of a final small interactive experience based on a given brief. 

This module covers the basics of multi-platform game development, using one of the most popular game engines in the world: Unity. You'll learn the fundamentals of C#, which is used to implement everything from your game logic to character controllers. 

At the end of the module, you'll be able to create your very own game and publish it on different platforms.

Linear algebra for games (vectors, matrices) with applications to 2D and 3D graphics, procedural generation of assets (L-systems), complex algebra (complex numbers, quaternions) and applications (fractals, 3D orientation).

This module covers the basics of multi-platform game development, using one of the most popular game engines in the world: Unity. You'll learn the fundamentals of C#, which is used to implement everything from your game logic to character controllers. 

At the end of the module, you'll be able to create your very own game and publish it on different platforms.

This module builds on the previous introduction to Modelling and Animation and engages the student in a range of harder and more expert problems. Students will be expected to build a rich 3D fantasy world of their own design, with objects such as fantasy buildings and architecture and assets suitable for importing into a 3D games engine world. Working from their concept art in a selected game genre, students will learn how to model characters within poly budgets, UV texture then rig them to create animated walk and run cycles with blended set key moves, using Inverse Kinematics, Physics, cloth, fluids and AI packages where necessary. 

Lessons will cover the following:

• Nurbs surface modelling and texturing
• SoN body modelling of human forms.
• Using industry-standard texturing workflows - Complex UV mapping and substance painter
• Using Constraints, deformers, driven keys, Kinema3cs and Spline IKs and vehicle rigging
• Complex Animation Walk and Run cycles, baked anima3on, trax editor and animation export
• Paint Effects, Particles, Dynamics, Dynamic Effects, Fluids and nCloth clothing

This one term long module (following the introduction module) is targeted directly at those who wish to work in the games industry. Technical and art positions at major or independent studios are difficult to secure, with many requiring tough tests and interviews. The approach is highly practical focusing on the key skills valued by employers in senior staff.

As well as focusing on the required programming the module also teaches the organisational skills required to work at a high level. Artists talking this module will benefit from knowing how the asset pipeline works.

A single term module focused on games design and how to use player behaviour to maximise acquisition/retention.

This module is taught by industry veteran Richard Leinfellner, who has more than 40 published game credits ranging from Programmer to Executive Producer.

You will study how to deconstruct designs from a designer vs a player perspective allowing you to build your own design patterns, reward cycles and hence optimise player enjoyment. We will cover how to rapidly prototype your designs using a game engine. Mobile considerations, multiplayer and the use of analytics is covered allowing the student to make informed design choices.

The BBC 1960’s TV programme “It’s a Knockout” inspired the UK games developer Mediatonic to create Fall Guys. In the 1990’s, dolls houses provided the inspiration for Will Wright’s game The Sims. Kafka’s novel, ‘The Castle,’ was a major influence on the Japanese games designer, Suda 51. Thus, culture and history have had a major impact on games design innovation and provide invaluable source material and inspirational starting points for games designers and artists.

On this course, taking a games industry perspective, students will learn about the history of computer games development, art and animation, starting in Renaissance times with three-point perspective through to computer graphics in the 1960s and 70s and the emergence and growth of interactive entertainment from the games of the 80s and 90s to the VR industry today.

With a view to enhancing students’ “games design potential”, they will learn about Surrealism, Cubism, Pop Art, Dada, the History of Perspective, Computer Art, Rave Culture, Cyber Culture and Punk Rock. Creative research is then carried out by the student on these cultural themes to source ideas, images and designs that can be translated into innovative new games designs for mobile, console, VR and PC. This process uses standard games industry games design document templates.

The students then use the research methods and templates acquired to create their own original games designs based on cultural themes. Giving students a “cultural-based” games design strategy that they can use in their own games development practice to develop new and original gameplay mechanics and designs to set them apart from the competition.

A historical section of the course is games-industry-focussed, examining some of the world’s most famous games designers and also development trends and commercial drivers on an international scale and how research and creativity can be translated into commercial and indie games.

You will learn:

• Historical and cultural research methods to explore, find and select material suitable for conversion to computer games.
• Learn how to take the research outcomes and translate them into your original games designs.
• How to use and complete games industry games design templates to a professional standard.
• Create eight mini-games concept documents of your own design spanning all the cultural themes covered. This will include diagrams, sketches and drawings.
• Create a major games design document and video animatic within a fixed budget and development constraints. This will include diagrams, sketches and drawings.
• How to be an innovative games designer and researcher.

This module introduces students to professional-grade game engines (such as Unreal) that can be used across a wide array of disciplines including games, computational arts, film, design and science to build virtual worlds. Students will learn how to combine various components of a complex 3D rendering engine, such as geometry, lighting, particle systems, 3D audio and user input, to realize their ideas. Coursework is project-based and students are encouraged to work in mixed artist/engineer teams.

Modern games demand a lot in terms of complex computer-generated behaviour and content. This is a single term module that teaches you a set of core concepts that you can use to build your own state-of-the-art AI systems. It focuses on practical techniques and architectures that can be directly applied in game development: pathfinding, reactive movement, behaviour trees, HTN planning, procedural content generation. It will also give you an understanding of more advanced AI concepts that are increasingly finding their way into games, such as Monte Carlo Tree Search and techniques from Machine Learning, such as Convolutional Neural Networks (CNNs), Deep Reinforcement Learning (DRL) and Generative Adversarial Networks (GANs).

The course will introduce you to a range of techniques and practices for creating interactive audiovisual software using generative techniques. This will include computational and process-based thinking, perspectives on audio and visual perception, algorithms for creating graphics and sound, advanced topics in computational media and project development. Every week a theme from art is introduced and then replicated and examined using code.

Objectives

• Provide the student with a fundamental understanding of code and modern computer literacy.
• Introduce and apply programming concepts and techniques using openFrameworks/C++.
• Approach programming from an artistic perspective.
• Allow for the emergence of open dialogue regarding the content being instructed.
• Invoke the student's interests to apply what they have learned outside of class.

On successful completion you will be able to:

•  Program interactive installations
•  Develop algorithms for generating images and sound
•  Reason about the aesthetics of computer art pieces 

Sample Syllabus:

• Introduction to art, tech, free software
• Introduction to openFrameworks
• Animation and intro to generative art
• Repeat, repeat, repeat: loops and arrays
• Number generators
• Images & video
• Algorithmic thiking
• Sound with maximilian
• Revision and guest artist
•  ----- project work -----

This module builds on Workshops in Creative Coding 1 by assuming that students have mastered the basics of C++ in introduces them to topics in computational arts such as: computer vision, machine learning / AI, networking, genetic algorithms, 3D graphics, emergence of complexity and more.

 Sample Syllabus:

• Emergence and object oriented programming
• Computer vision A (part 1)
• Computer vision B (part 2)
• Networked art with OSC messages
• Machine learning / AI
• Sound with maximilian (part 2)
• Projection mapping
• Genetic algorithms and other advanced generative techniques
• 3D graphics
• ----- project work ----- 

Other topics include:

• Data visualization
• Physics engines
• Mobile
• Shaders / GLSL
• Augmented reality
• DMX
• Delaunay / voronoi
• Art with typography
• Swarm intelligence
• Using 3rd party APIs

This module will expose students to state-of-the-art techniques, tools, and open questions related to creative uses of data, signal processing, and machine learning. The emphasis will be on developing hands-on skills using these techniques in creative projects, and on exploring the creative potential of these techniques.

Specifically, students will learn about topics including:

• Representations and feature engineering for sensor data, audio data, image and video data, social media data, etc.
• Signal processing techniques for working effectively with this data (e.g., perceptual audio and video features, smoothing filters,
  onset detection)
• Communication protocols for working with real-time data (e.g., OpenSoundControl, Web Sockets, serial)
• Applications of classification to creative and interactive contexts: e.g., human pose recognition, activity recognition, semantic
  audio analysis
• Applications of regression to creative and interactive contexts: e.g., creating continuous gestural controllers and multimodal
  mappings (such as music visualisations, gesturally-controlled instruments)
• Applications of temporal modeling to creative and interactive contexts: e.g., gesture recognition, temporal analysis of music or
  video
• Current topics in signal processing and machine learning in music, art, and other creative industries (e.g., Google's "Deep
  Dream," chat bots, image style transfer)
• Tools for working with data, signal processing, and machine learning in creative projects, including tools for real-time data
  analysis
• Reasoning about fundamental questions in machine learning and data mining, including e.g., how can an algorithm learn from
  data? What feature representations should we use for a given problem? How do we know whether one algorithm is better than
  another?

From basic design tools to conceptualising, prototyping and play testing an array of games, this module will teach you how to create compelling game mechanics within playable experiences.

You'll explore how emotion, sensory experience, interaction design, framing and purpose unfold through game design, and grapple with the tools which make play compelling. Additionally, this module looks a the different kind of possible models for play such as cooperation, skill, experience, chance, whimsy, performance, expression and simulation.

In addition to learning game design, students will learn how to talk about and understand games. From designers working with a formalised ludic approach to artists exploring liminoid spaces within play, the range of approaches will be explored.

You will leave this class with a clear understanding of the state of games and play as well as with a lexicon on how to discuss work within this space.

What are the keys to creating successful playable spaces? What kinds of digital play experiences work in physical environments? Leveraging your ability to experiment, this course will give you the opportunity to rapidly prototype and explore the environmental aspects of play. 

Along the way you'll be brainstorming, pitching and storyboarding experience design and physical games, analysing critical interventions using technology and play in public space and leveraging technology in multiple contexts such as theatre and museum installations.

You'll also be looking at movement in physical play, using technology to enhance experiences, multi and cross sensory based experience design, current uses of emerging technologies in physical games and museum settings and the ways in which user testing and iteration cam improve play experiences. 

Physical Computing is of increasing interest to artists, musicians, choreographers and other creative practitioners for the creation of novel artworks and also for forms of computational interaction between these objects and people. There are many other applications of Physical Computing, for example in museums, ubiquitous and embedded computing, robotics, engineering control systems and Human Computer Interaction.

A physical environment may be sonic, tangible, tactile, visually dynamic, olfactory or any combination of these. In this module, you will learn how the environment, which is essentially continuous, can be monitored by analogue electrical and mechanical sensors. Computers, however, are digital machines programmed by software. One element which you will focus on, therefore, is the interface between the digital and the analogue.

This study will encompass basic physics, electronics, programming and software engineering. The practical objective of this module is the development of the skills you will need for designing and building interactive physical devices.

A large amount of data is available in electronic resources, both offline and online. This module will give a broad introduction to techniques for gathering data from electronic sources, such as databases and the internet. It will cover both fundamental ideas and the use of some of the most important currently available tools. The module will also present tools and ideas for more effectively using the internet to communicate, visualise and generate news stories. 

This module will address the fundamentals of working with motion capture and theories behind digital embodiment. The principles explored in the module can be applied to a range of contexts that involve the human body in movements, such as video games, animations, interactive experiences, performances, social VR, training and rehabilitation, and much more.

You'll explore pipelines for capturing and using recorded motion data, as well as using mocap for real-time applications. You'll discover the full workflow from asset rigging to using the mocap data for animations and creating live interactions. The module will include hands-on sessions working directly with motion capture systems, as well as guest lectures from researchers, artists and industry leaders. The module is accessible to students with reduced mobility and assessments can be performed even on students who cannot physically wear the mocap suit.