Augmenting Assembly Line Operations through Multimodal Interaction

Watch this on Vimeo!


Human dynamism is vital to the final assembly line of large industries. Assembly tasks are often complex and costly to automate, despite being repetitive and monotonous. This project aims to develop a system of digital augmentation, to improve performance and reduce monotony of assembly line work. The concept is a high level exploration into how multimodal interaction, including augmented reality, haptic and conversational interfaces, could seamlessly meet the challenges of training and accomplishing complex tasks. The resulting super operators move across various tasks. They can gain skills and experiences helping them fulfil their potential, ultimately increasing satisfaction.

Inspiration and Method

Assembly line work is hard, it is both physically and mentally taxing. Improvements over the past century have led to efficiency and increased production rates and also higher safety and better working conditions for humans. Although, some things have still remained unchanged - monotony of work and cognitive overload from the instructions and complex set of tasks.

To understand these problems first hand, visits were made to factories in Sweden, viz. Volvo Cars, Volvo Trucks, GE Healthcare and Komatsu Forest AB. Information was collected through observations and in-depth interviews with operators. After laying down some opportunity areas, I used quick Lo-Fi prototypes to ideate and develop my final concept.


The final concept is a Multi-modal interface, comprising of augmented reality glasses, audio and haptic interface that helps train and assist operators for work across the assembly line. As a result breaking monotony of work today, improving performance and developing highly skilled operators - Super Operators.

As part of the project, a framework was developed to evaluate which modality was suitable for the different kind of information. The assembly line is one context for such an application of multimodality. This could be applied to many different interactions and especially for emerging technologies like AR & VR.