Introduction

Computing is increasingly intertwined with our physical world. From smart watches to connected cars, to the Internet of Things and 3D-printing, the trend towards combining digital and analogue materials in design is no longer an exception, but a hallmark for where interaction design is going in general. Computational processing increasingly involves physical materials, computing is increasingly manifested and expressed in physical form, and interaction with these new forms of computing is increasingly mediated via physical materials. Interaction Design is therefore increasingly a material concern. – Welcome to a book on the materiality of interaction, welcome to a book on material-centered interaction design! In this introduction to this book, “The Materiality of Interaction – Notes on the Materials of Interaction Design”, I describe the contemporary trend in interaction design towards material interactions, I describe how interaction design is increasingly about materials, and I propose “Material-centered interaction design” as a method for working with materials in interaction design projects.

A New Materiality

The previous chapter provided us with a theory of the materiality of interaction. So, where do we go from here? Well, in order to move forward, I use this chapter to suggest that we might now need to look back in order to see the road ahead of us more clearly. In this chapter I therefore present how a focus on the materiality of interaction one the one hand leaves any distinctions between the physical and the digital behind, and how it on the other hand presents us with three distinct challenges as we move forward through the material turn.

Material-Centered Interaction Design

The growing interest in the materiality of interaction in the field of HCI (Human-Computer Interaction) indicates that there is a value in acknowledging the material aspects and dimensions of interaction design. However, and if only relying on a representation-driven approach to interaction design the notion of materiality only works, at best, as a “metaphorical maneuver, while still pushing for an interaction design paradigm oriented towards the immaterial aspects of interaction design (for instance the use of symbols and metaphors in interaction design). So what would an alternative perspective and approach be? In short, can we not only shift perspectives here, but also imagine different approaches and methods to interaction design that truly accepts the digital as a design material, that focuses on interaction as the form being designed, and an approach that do not introduces any categorical distinctions between different matters. In this book I have suggested that we should make no metaphysical or ontological distinction between physical and digital materials, between atoms, bits, and cells, between “visible” or “invisible” materials, and even avoid distinctions between what might be considered as “material” or “immaterial” in the first place (like radio waves). In the same way as wood or iron are typical examples of physical materials I consider code, algorithms, sensors and processors as digital materials. Still, from the viewpoint of interaction design it is the composition and activation of these different materials as to give the interaction a particular form that is essential – not each materials ontological or metaphysical status. So, instead of focusing on what a particular interactive system represents, the material-centered approach to interaction design as proposed in this book focuses on how interaction is presented and accordingly materially manifested in the world (in all imaginable forms – from completely embedded and “invisible” interactive systems to the gadgets, pads, and tabs we surround ourselves with in our everyday lives). In this chapter I take this as a point of departure for the development of an approach to interaction design that I label material-centered interaction design.

The Materiality of Interaction

This book about the materiality of interaction is not only concerned with how materials and how the notion of materiality is related to interaction design, but as we shift perspectives from representations to materials we also need to look closely at this central notion of interaction and we need to discuss what it is, and if a shift of perspectives here also has implications for how this term should and can be used in the context of interaction design – and when designing for interaction. And even more fundamentally, the title of this book is ‘the materiality of interaction’ and not ‘the materiality of a computer’ so at this point it is probably a good idea to reflect upon what we mean when we say that we do interaction design, how we define this term interaction, and how we see interaction as being at least partly a material matter. This chapter is devoted to these essential questions.

Representations

Computing, and human interaction with these computational machines, has commonly been thought of as a kind of abstract activity where we manipulate digital objects on glossy screens. It is abstract in the sense that it is hands-on in terms of working with the computer, but still abstract in terms of how that work is to a great extent about arranging and re-arrange ‘painted bits’, click on virtual ‘buttons’, and about processing symbols, work with representations, and even store and access data in ‘the cloud’. How can all of this be even remotely related to materials? And does it really make sense to talk about the materiality of interaction in the context of human-computer interaction (HCI) and interaction design? In short, is materiality really a concern for interaction design? And accordingly, is it meaningful to talk about the materiality of interaction? In this chapter I dwell into these essential questions for interaction design. I do so by revisiting some examples from the early days of computing, and through one such historical backdrop I illustrate how computing was in its early days enabled through material configurations, and how we since then have tried to uphold various distinctions between the material and the immaterial, between the physical and the digital, between the virtual and the real (most recently in the current debate about skeuomorphic versus non-skeuomorphic design), and how computing has always been, and will continue to be a material concern.

The Material Turn

No matter if we think about interaction design as a design tradition aimed at giving form to the interaction with computational objects, or if we think about interaction design as being simply about user interface design it is hard to escape the fact that the user interface to a large extent defines the scene and the form of the interaction. Without adopting a fully deterministic perspective here it is still a fact that if the user interface is screen-based and graphical and the input modality is mouse-based, then it is likely that the form of that interaction, that is what the turn-taking looks like and what is demanded by the user, is very similar to other screen-based interfaces with similar input devices. However, the design space for the form of interaction is growing fast. While command-based interfaces and text-based interfaces sort of defined the whole design space in the 1970s, the development since then, including novel ways of bringing sensors, actuators, and smart materials to the user interface has certainly opened up for a broader design space for interaction design. But it is not only the range of materials that has been extended over the last few decades, but we have also moved through a number of form paradigms for interaction design. With this as a point of departure I will in this chapter reflect on how we have moved from early days of command-based user interfaces, via the use of metaphors in the design of graphical user interfaces (GUIs), towards ways of interacting with the computer via tangible user interfaces (TUIs). Further on, I will describe how this movement towards TUIs was a first step away from building user interfaces based on representations and metaphors and a first step towards material interactions.

Moving Forward

At last - everything must come to an end. However, and instead of concluding this book with a final chapter that summarizes some of the main arguments presented here, I see this final chapter as an opportunity to look back, and as an opportunity for thinking about how to move forward. The focus for this book is on the ‘material turn’ in our field – how it has played out historically across the history of computing, how it has fueled a discussion on materiality in HCI, and how now sets the scene for a material-centered approach to interaction design. With one such focus I think that its important to understand that a turn is not a state, but rather a movement, and accordingly a process we should not just follow, understand and describe, but something that we should follow, probably try to trace the origins of, and probably also a process that we should try to make some predictions about in terms of where we are going. In short, how did we end up here? And where are we going next? So, in line with these lines of thinking, I suggest that it is now time for us to look back, in order to move forward.

Interactive Compositions

Interaction design is a relational practice where interaction designers bring together many different materials into compositions that in return enable particular forms of interaction. However, and even though interaction design is about the design of interaction, interaction as this act that unfolds between a user and a digital artifact cannot be completely “pre-designed” in terms of fully controlled from a design point of view. Users might use a piece of technology as intended, but sometimes technologies are also used in unintended ways. Therefore, what the interaction designer can do is not to design the interaction, but to design good preconditions for a particular form of interaction, a particular materiality. When this materiality is used during acts of interaction we can refer to this intertwined relation of interaction enabled by, and performed with and through material configurations as the materiality of interaction. Clearly there are many aspects and dimensions to take into account when designing interaction (an in addition to thinking about how to materialize its intended form), so in this chapter I present a set of ideas for how an interaction designer can think about, and do, compositional and material-centered interaction design.

Designing across Substrates

Is interaction design a single material design tradition? In the same way as other areas of craftsmanship have had this single material focus (including knitting, woodcraft, or smiths working with iron) we can ask if interaction design as an area of craftsmanship can be said to be about digital materials as the “single material” of interaction design? At a first glance it is temping to say yes here, not at least if we review the total outcome interaction design projects we can probably say that 99.9% of all interaction design projects are screen-based, and maybe even web-based. Further on, all these projects are made out of code and even the tools, including the programming languages, the scripts and the code compilators are made of digital materials (ultimately bits). In this chapter I take a point of departure in a material-centered approach to interaction design. In particular and in setting out to take a material stance in the construction of an approach to interaction design I ask if contemporary interaction design is restrained to only one material, hence this introduction to this chapter.