MA/MSc Virtual & Augmented Reality

The lectures will cover the following topics:

1. Introduction to VR hardware and VR history
2. VR applications (training, education, therapy, communication, entertainment)
3. VR graphics
4. The three illusions - the Psychology behind VR
5. 3D interaction (navigation, object interaction)
6. Virtual humans: body and face animation
7. Embodiment
8. Introduction to 3D audio

The students will also gain practical experiences in the above areas with a game engine (Unity3D).

Introduction to Unity, Physics 2D and Collisions, Game Logic, Character Controller, Physically Based Rendering, Animator and Animation, Linear Interpolation.

Students will learn advanced techniques for immersively combined real world with virtual contents including augmented reality, photogrammetry, motion capture, and computer vision. In the lab, they will be able to apply these techniques to create a mixed reality application that combines real and virtual content appropriately, and leverage the body in this experience.

This module introduces students to a range of contemporary research in virtual reality and related fields. Students will be asked to read a number of research papers of their choosing and then focus on one paper to develop in a small group. In the first half of the course, the groups will take turns to make a short presentation about their chosen paper, in a seminar context. In the second half they will perform a replication of the paper: which will either be an experimental study or an implementation of a VR system, depending on the nature of the chosen paper, the results of which will be presented to the class at the end of the session. This work will be complemented by a series of guest lectures by researchers in the field from both Goldsmiths and other institutions. The researchers will present their current work.

The scope of this module includes a range of skills to be acquired, which will empower the students to create their own assets for their own game designs, and to work in a team on group projects during the course, giving them the necessary basic art, modelling and animation skills to work in industry. 

The aim is to train students in the basic use of software packages (for example, 2D paint packages such as Photoshop and 3D Modelling Packages such as 3D Studio Max, Blender and Maya) for the production of game and special effects assets and animation data sets ready for importing into computer games. Skills developed include 2D texturing and associated mapping for 3D worlds. 

Initially, the students’ work will involve “re-skinning” in-game art assets from existing games such as PacMan, Tetris, Space Invaders and Lemmings using paint packages to produce animating 2D Sprite texture maps, to enable students to get visually pleasing results quickly. 

Students will then learn how to use modelling software to create a range of 3D assets varying from buildings to cars to household objects to vegetation to roads and terrain etc. keeping within set polygon budgets, thereby creating the building blocks of a small 3D world with moveable elements ready for importing into a game engine (such as Unity or Unreal). 

Students will learn how to hand draw in a “game concept art style” suitable for production design and how to pitch to clients. They will learn how to storyboard and draw in cartoon / popular art style. 

Working from their concept art of simple moving human characters, students will learn how to model and “rig” them, to create animated walk and run cycles with blended set key moves. Advanced export pathways including intermediate formats will be also be covered. 

At the end of the module the students will take the assets they have created and integrate them into an actual simple prototype game for PC or tablet. 

Throughout the module you’ll be taught project planning, resource management, risk analysis in the context of computer games art development and design.

Linear algebra for games (vectors, matrices) with applications to 2D and 3D graphics, procedural generation of assets (L-systems), complex algebra (complex numbers, quaternions) and applications (fractals, 3D orientation).

The scope of this module builds on the Introduction to Modelling and Animation Course and engages the student in a range of harder and more expert problems. 

Students will be expected to build a rich 3D fantasy world of their own design, with objects such as fantasy buildings and architecture and assets suitable for importing into a 3D games engine world. Working from their concept art in a selected game genre, students will learn how model characters then “rig them”, to create animated walk and run cycles with blended set key moves, using Physics, Inverse Kinematics and AI packages where necessary. These animated characters are then used to populate and game 3D world as NPCs (Non Player Characters). 

The students will learn how to use a range a set of specialist modelling tools including SpeedTree and ZBrush and fit them into games art production pipeline. Students will learn the basics of Procedural content generation in the context of game engines and the distinct artistic and technical approach necessary. 

A section of the course in the later stages will cover the development of assets and the artistic styling of games on emerging platforms such as Virtual Reality platforms, using Location Based Gaming, Serious and Gamification Games, Crowd Sourced Games and Robotics platforms. Work will included the development of novel analytics techniques designed initially on paper. 

A section of the course will include teaching Advanced Project Planning, Quality control and early testing of assets, Resource Management using Gantt Charts, Excel and Bug Tracking Management Tools. Outsourcing and other production management approaches will be explored also in the context of Risk Analysis.

This one term long module (following the introduction module) is targeted directly at those who wish to work in the games industry. Technical and art positions at major or independent studios are difficult to secure, with many requiring tough tests and interviews. The approach is highly practical focusing on the key skills valued by employers in senior staff.

As well as focusing on the required programming the module also teaches the organisational skills required to work at a high level. Artists talking this module will benefit from knowing how the asset pipeline works.

You'll study:

• Kinect, Freemium gameplay
• Level design
• Modding games
• Games Analytics techniques

Students will learn about the history of games design, art and animation on a world-wide basis since the 1960’s covering the major trends and directions, including computer graphics and special effects and the interactive entertainment industry. Students will also learn about the broader cultural movements in the History of Art and Culture including Surrealism, Cubism, Pop Art, Ready Made Art and Post Modernism and the emergence of Computer Art.

Basic cultural theory will be covered within the course. The course will explore the relationship between technological innovation and how this has been historically capitalised on by creative individuals. The course will include short sections on the Renaissance and Impressionism. The course will include key influential (such as Benoît Mandelbrot, Jon Lassetter) and events such as The SIGGRAPH Conference, The Games Developer Conference, CES Consumer Electronics Show and New York Armoury Show.

This course will help give students a broad cultural perspective but also is expected to “feed into” shaping their own personal creative games ideas and designs and give them creative inspiration to be used on the other modules. The timing of this in second term is to give students the necessary research to work on projects or undertake placements with broader cultural understanding.

This single term module focuses on building asset pipelines for triple-A game studios and the film industry. There is a focus on cloud computing, Python scripting in servers and in art tools, automation of content delivery and Unix command line basics. Students who take this course will acquire an understanding of how to build and maintain art asset pipelines in commercial environments.

Modern games demand a lot in terms of complex computer-generated behaviour and content. This is a single term module that teaches you a set of core concepts that you can use to build your own state-of-the-art AI systems. It focuses on practical techniques and architectures that can be directly applied in game development: pathfinding, reactive movement, behaviour trees, HTN planning, procedural content generation. It will also give you an understanding of more advanced AI concepts that are increasingly finding their way into games, such as Monte Carlo Tree Search and techniques from Machine Learning, such as Convolutional Neural Networks (CNNs), Deep Reinforcement Learning (DRL) and Generative Adversarial Networks (GANs).

The course will introduce you to a range of techniques and practices for creating interactive audiovisual software using generative techniques. This will include computational and process-based thinking, perspectives on audio and visual perception, algorithms for creating graphics and sound, advanced topics in computational media and project development. Every week a theme from art is introduced and then replicated and examined using code.

Objectives

• Provide the student with a fundamental understanding of code and modern computer literacy.
• Introduce and apply programming concepts and techniques using openFrameworks/C++.
• Approach programming from an artistic perspective.
• Allow for the emergence of open dialogue regarding the content being instructed.
• Invoke the student's interests to apply what they have learned outside of class.

On successful completion you will be able to:

•  Program interactive installations
•  Develop algorithms for generating images and sound
•  Reason about the aesthetics of computer art pieces 

Sample Syllabus:

• Introduction to art, tech, free software
• Introduction to openFrameworks
• Animation and intro to generative art
• Repeat, repeat, repeat: loops and arrays
• Number generators
• Images & video
• Algorithmic thiking
• Sound with maximilian
• Revision and guest artist
•  ----- project work -----

This module builds on Workshops in Creative Coding 1 by assuming that students have mastered the basics of C++ in introduces them to topics in computational arts such as: computer vision, machine learning / AI, networking, genetic algorithms, 3D graphics, emergence of complexity and more.

 Sample Syllabus:

• Emergence and object oriented programming
• Computer vision A (part 1)
• Computer vision B (part 2)
• Networked art with OSC messages
• Machine learning / AI
• Sound with maximilian (part 2)
• Projection mapping
• Genetic algorithms and other advanced generative techniques
• 3D graphics
• ----- project work ----- 

Other topics include:

• Data visualization
• Physics engines
• Mobile
• Shaders / GLSL
• Augmented reality
• DMX
• Delaunay / voronoi
• Art with typography
• Swarm intelligence
• Using 3rd party APIs

This module will expose students to state-of-the-art techniques, tools, and open questions related to creative uses of data, signal processing, and machine learning. The emphasis will be on developing hands-on skills using these techniques in creative projects, and on exploring the creative potential of these techniques.

Specifically, students will learn about topics including:

• Representations and feature engineering for sensor data, audio data, image and video data, social media data, etc.
• Signal processing techniques for working effectively with this data (e.g., perceptual audio and video features, smoothing filters,
  onset detection)
• Communication protocols for working with real-time data (e.g., OpenSoundControl, Web Sockets, serial)
• Applications of classification to creative and interactive contexts: e.g., human pose recognition, activity recognition, semantic
  audio analysis
• Applications of regression to creative and interactive contexts: e.g., creating continuous gestural controllers and multimodal
  mappings (such as music visualisations, gesturally-controlled instruments)
• Applications of temporal modeling to creative and interactive contexts: e.g., gesture recognition, temporal analysis of music or
  video
• Current topics in signal processing and machine learning in music, art, and other creative industries (e.g., Google's "Deep
  Dream," chat bots, image style transfer)
• Tools for working with data, signal processing, and machine learning in creative projects, including tools for real-time data
  analysis
• Reasoning about fundamental questions in machine learning and data mining, including e.g., how can an algorithm learn from
  data? What feature representations should we use for a given problem? How do we know whether one algorithm is better than
  another?

From basic design tools to conceptualising, prototyping and play testing an array of games, this module will teach you how to create compelling game mechanics within playable experiences.

You'll explore how emotion, sensory experience, interaction design, framing and purpose unfold through game design, and grapple with the tools which make play compelling. Additionally, this module looks a the different kind of possible models for play such as cooperation, skill, experience, chance, whimsy, performance, expression and simulation.

In addition to learning game design, students will learn how to talk about and understand games. From designers working with a formalised ludic approach to artists exploring liminoid spaces within play, the range of approaches will be explored.

You will leave this class with a clear understanding of the state of games and play as well as with a lexicon on how to discuss work within this space.

What are the keys to creating successful playable spaces? What kinds of digital play experiences work in physical environments? Leveraging your ability to experiment, this course will give you the opportunity to rapidly prototype and explore the environmental aspects of play. 

Along the way you'll be brainstorming, pitching and storyboarding experience design and physical games, analysing critical interventions using technology and play in public space and leveraging technology in multiple contexts such as theatre and museum installations.

You'll also be looking at movement in physical play, using technology to enhance experiences, multi and cross sensory based experience design, current uses of emerging technologies in physical games and museum settings and the ways in which user testing and iteration cam improve play experiences. 

Physical Computing is of increasing interest to artists, musicians, choreographers and other creative practitioners for the creation of novel artworks and also for forms of computational interaction between these objects and people. There are many other applications of Physical Computing, for example in museums, ubiquitous and embedded computing, robotics, engineering control systems and Human Computer Interaction.

A physical environment may be sonic, tangible, tactile, visually dynamic, olfactory or any combination of these. In this module, you will learn how the environment, which is essentially continuous, can be monitored by analogue electrical and mechanical sensors. Computers, however, are digital machines programmed by software. One element which you will focus on, therefore, is the interface between the digital and the analogue.

This study will encompass basic physics, electronics, programming and software engineering. The practical objective of this module is the development of the skills you will need for designing and building interactive physical devices.

A large amount of data is available in electronic resources, both offline and online. This module will give a broad introduction to techniques for gathering data from electronic sources, such as databases and the internet. It will cover both fundamental ideas and the use of some of the most important currently available tools. The module will also present tools and ideas for more effectively using the internet to communicate, visualise and generate news stories.