Research
My research interests centre on investigating human-algorithm interaction design: how can design shape the ways that interact with computational systems at scales from small and personal to societal and global. I’m most interested in where we need to develop new ways of thinking to account for human experience within digital architectures, in order to build systems that are humane and supportive. Currently, this involves looking at the interactions between humans and AI/complex information processing systems..
Current projects:
- I am director of the AI Futures Lab: Rights and Justice, with Ben Wagner. The AI Futures Lab on Rights and Justice uses relational prototyping to radically re-imagine the regulation of digital technologies. We carry out impact-oriented research to support rights and justice around AI and related technologies through imagining possible futures. Our research aims to center and support those with less power in their interactions and relations with technology, working towards more just futures for all.
- I co-lead the Design and AI Symposium with Phil van Allen, Mathias Funk and Stephan Wensveen. The Symposium brings together academics, industry and publics to explore the connections between design and AI, balancing criticality and optimism.
Relational Prototyping
In order to better understand the relationships formed through and around technology, relational prototyping looks at how to understand current relations, imagine future relations, and manifest those future possibilities.
Speculative relations
- Towards Just Futures: A Feminist Approach To Speculative Design For Policy Making Dideriksen, Sofie Amalie Torp and Verma, Himanshu and Cila, Nazli and Murray-Rust, Dave (2024) DRS2024
Towards Just Futures: A Feminist Approach To Speculative Design For Policy Making
There is a call for and growing use of future-oriented design methods like speculative design in the development of policies. While these methods offer potential benefits in helping future-proof policies, they also run the risk of solidifying existing structures of power if not applied critically. In this paper, we describe a case study where we created a speculative design exhibition grounded in feminist theory in order to challenge the existing power structures in the public domain. We then discuss the insights from our design process and the reaction the exhibition received from policymakers in light of how feminist theory can help ensure a critical application of future-oriented design methods in policy design. - Spatial Robotic Experiences as a Ground for Future HRI Speculations Murray-Rust, Dave and Lupetti, Maria Luce and Ianniello, Alessandro and Gorbet, Matt and Van Der Helm, Aadjan and Filthaut, Liliane and Chiu, Adrian and Beesley, Philip (2024) Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction
Spatial Robotic Experiences as a Ground for Future HRI Speculations
This work illustrates how artistic robotic systems can provide a reservoir of unfamiliarity and a basis for speculation, to open the field toward new ways of thinking about HRI. We reflect on a collaborative project between design students, a media art studio, and design researchers working with the baggage handling department of a strategic European airport. Engaging with the industrial context, we developed ’meta-behaviours’ - abstracted ideas of processes carried out on the worksite and passed these over to the students who translated them into robotic enactions based on hardware and a form language developed by the media art studio. The resulting visit experience challenges the audience to decode the installation in terms of meta-behaviours and their possible relations to industrial HRI. We used this to reflect on the value of conducting artistic and speculative work in HRI and to distil actionable recommendations for future research.
Understanding AI Relations
- Prototyping with Uncertainties: Data, Algorithms, and Research through Design Giaccardi, Elisa and Murray-Rust, Dave and Redström, Johan and Caramiaux, Baptiste (2024) ACM Trans. Comput.-Hum. Interact.
Prototyping with Uncertainties: Data, Algorithms, and Research through Design
Seen both as a resource and an obstacle to clarity, uncertainty is a concept that permeates many areas of design. As the concept gains prominence in Human-Computer Interaction (HCI), this special issue specifically explores the interplay between uncertainty and prototyping in Research through Design (RtD). We first outline three histories of uncertainty in design, in relation to its philosophical significance, its role in statistical and algorithmic processes, and its importance in prototyping. The convergence of these aspects is crucial as design evolves toward more agentive and entangled systems, introducing challenges such as Design as a Probabilistic Outcome. We then investigate the design spaces for engaging with “being uncertain” that emerge from the papers: from nuancing the relationship between designers and quantitative data to blurring the line between humans, fungi, and algorithms. Finally, we illuminate some preliminary threads for how RtD can navigate and engage with these shifting technological and design landscapes thoughtfully. - (Un)Making AI Magic: A Design Taxonomy Lupetti, Maria Luce and Murray-Rust, Dave (2024) Proceedings of the CHI Conference on Human Factors in Computing Systems
(Un)Making AI Magic: A Design Taxonomy
This paper examines the role that enchantment plays in the design of AI things by constructing a taxonomy of design approaches that increase or decrease the perception of magic and enchantment. We start from the design discourse surrounding recent developments in AI technologies, highlighting specific interaction qualities such as algorithmic uncertainties and errors and articulating relations to the rhetoric of magic and supernatural thinking. Through analyzing and reflecting upon 52 students’ design projects from two editions of a Masters course in design and AI, we identify seven design principles and unpack the effects of each in terms of enchantment and disenchantment. We conclude by articulating ways in which this taxonomy can be approached and appropriated by design/HCI practitioners, especially to support exploration and reflexivity. - Metaphors for Designers Working with AI Murray-Rust, Dave and Nicenboim, Iohanna and Lockton, Dan (2022) DRS Biennial Conference Series
Metaphors for Designers Working with AI
In this paper, we explore the use of metaphors for people working with artificial intelligence, in particular those that support designers in thinking about the creation of AI systems. Metaphors both illuminate and hide, simplifying and connecting to existing knowledge, centring particular ideas, marginalising others, and shaping fields of practice. The practices of machine learning and artificial intelligence draw heavily on metaphors, whether black boxes, or the idea of learning and training, but at the edges of the field, as design engages with computational practices, it is not always apparent which terms are used metaphorically, and which associations can be safely drawn on. In this paper, we look at some of the ways metaphors are deployed around machine learning and ask about where they might lead us astray. We then develop some qualities of useful metaphors, and finally explore a small collection of helpful metaphors and practices that illuminate different aspects of machine learning in a way that can support design thinking. - Metaphor Gardening: Experiential Engagements for Designing AI Interactions Murray-Rust, Dave and Lupetti, Maria Luce and Nicenboim, Iohanna (2024) DRS2024
Metaphor Gardening: Experiential Engagements for Designing AI Interactions
Designers deploy metaphors in various constructive ways but there is a challenge in noticing and selecting helpful metaphors to describe AI systems. Metaphors serve to highlight certain aspects of AI but their influence can be so potent that envisioning or discussing AI in alternative ways becomes challenging, with unwanted expectations, lazy tropes and hidden biases. Alternative metaphors help designers grasp distinctive qualities of AI and move past hidden assumptions. Hence, it is key to support designers in precise, plural and intentional metaphor use to grasp unique qualities of AI and explore its relationalities. We illustrate this through a selection of prototyping journeys in which metaphors directly shaped students’ design trajectories and allowed them to explore the relational, entangled complexities of AI systems. Finally, ‘metaphor gardening,’ provides a series of recommendations for designers when designing AI with metaphors, which we hope can ultimately support a generative and responsible approach to AI technologies. - Respect as a Lens for the Design of AI Systems Seymour, William and Van Kleek, Max and Binns, Reuben and Murray-Rust, Dave (2022) Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society
Respect as a Lens for the Design of AI Systems
Critical examinations of AI systems often apply principles such as fairness, justice, accountability, and safety, which is reflected in AI regulations such as the EU AI Act. Are such principles sufficient to promote the design of systems that support human flourishing? Even if a system is in some sense fair, just, or ’safe’, it can nonetheless be exploitative, coercive, inconvenient, or otherwise conflict with cultural, individual, or social values. This paper proposes a dimension of interactional ethics thus far overlooked: the ways AI systems should treat human beings. For this purpose, we explore the philosophical concept of respect: if respect is something everyone needs and deserves, shouldn’t technology aim to be respectful? Despite its intuitive simplicity, respect in philosophy is a complex concept with many disparate senses. Like fairness or justice, respect can characterise how people deserve to be treated; but rather than relating primarily to the distribution of benefits or punishments, respect relates to how people regard one another, and how this translates to perception, treatment, and behaviour. We explore respect broadly across several literatures, synthesising perspectives on respect from Kantian, post-Kantian, dramaturgical, and agential realist design perspectives with a goal of drawing together a view of what respect could mean for AI. In so doing, we identify ways that respect may guide us towards more sociable artefacts that ethically and inclusively honour and recognise humans using the rich social language that we have evolved to interact with one another every day.
Robot Relationality
- Cosmic Troubleshooting: Exploring Third-Person View for Error Handling in Telerobotic Planetary Infrastructure Maintenance Filthaut, Liliane and Murray-Rust, Dave and Lupetti, Maria Luce and Lii, Neal Y. and Schmaus, Peter and Leidner, Daniel (2024) 2024 International Conference on Space Robotics (iSpaRo)
Cosmic Troubleshooting: Exploring Third-Person View for Error Handling in Telerobotic Planetary Infrastructure Maintenance
This study investigates error handling intricacies in supervised autonomy orbit-to-ground teleoperation for space exploration robots, emphasizing scenarios with communication delays that render Earth-based ground control assistance unfeasible. In this setting, one major challenge lies in empowering the crew to independently mitigate robot errors that may occur as the robot plans its actions. To address this limitation of current supervised autonomy interfaces, we propose a third-person perspective and game design principles to improve environmental awareness in error situations. 16 experts with similar technical background as the target crew members tested the interface in a physical user study, while 42 people assessed it in an online study. We conclude that a third-person view brings significant improvements to mental workload, overall experience and the ability to identify and rectify planning errors. - Actor-Flower-Mesh-Work: Making Environments Together von Jungenfeld, Rocio and Murray-Rust, Dave (2023) Creating Digitally: Shifting Boundaries: Arts and Technologies—Contemporary Applications and Concepts
Actor-Flower-Mesh-Work: Making Environments Together
In this paper we use an Ingoldian ‘meshwork’ and a Latournian ‘actor network’ approach to unpack the complexities of devising and constructing complex situations that combine creativity and technology. To do this, we draw on our experience of creating and exhibiting an interactive installation. We draw on our observations of the interactions, relations and correspondences between the installation, its elements and the people that visited the space. We develop a discussion of these theories somewhat auto-ethnographically around the artwork Lichtsuchende, a group of robots with social behaviours enacted through light. Their trajectories of becoming are not that straightforward, and by looking at the moments of becoming we can delve into understanding the artwork better using two distinctive approaches: Ingold’s meshwork and Latour’s actor-network. The two approaches enable us to investigate from within and from outside. We seek to understand the ways in which these viewpoints can be applied as methodologies to unpack the factors involved in the creation of an artificial society and the emergence of a shared environment made of human and non-human things. - First International Workshop on Workers-Robot Relationships Zaga, Cristina and Lupetti, Maria Luce and Forster, Deborah and Murray-Rust, Dave and Prendergast, Micah and Abbink, David (2024) HRI2024
First International Workshop on Workers-Robot Relationships
In Industry 5.0, cognitive robots and workers will engage in evolving and reciprocal relations, which we call workers-robot relationships (WRRs). To enable evidence-based work futures with workers, we must co-develop WRRs and understand their impact on work, workers, management, and society. To this end, we posit that the HRI field should work beyond disciplines and include value-driven and plural perspectives through transdisciplinary research done with and for workers. However, WRRs and transdisciplinarity pose unique technical, philosophical, and methodological challenges yet to be explored. We propose a workshop to engage the HRI community working on Industry 5.0, aiming at 1) taking stock of current WRR-related challenges in relevant disciplines, 2) collectively kick-off the exploration of a joint research agenda, 3) preliminary examining if and how transdisciplinarity could help the HRI community, and 4) start discussing how to deal with such complex knowledge integration in practice.
Designing AI Systems
- Grasping AI: Experiential Exercises for Designers Murray-Rust, Dave and Lupetti, Maria Luce and Nicenboim, Iohanna and van der Hoog, Wouter (2023) AI & Society
Grasping AI: Experiential Exercises for Designers
Artificial intelligence (AI) and machine learning (ML) are increasingly integrated into the functioning of physical and digital products, creating unprecedented opportunities for interaction and functionality. However, there is a challenge for designers to ideate within this creative landscape, balancing the possibilities of technology with human interactional concerns. We investigate techniques for exploring and reflecting on the interactional affordances, the unique relational possibilities, and the wider social implications of AI systems. We introduced into an interaction design course (n=113) nine ‘AI exercises’ that draw on more than human design, responsible AI, and speculative enactment to create experiential engagements around AI interaction design. We find that exercises around metaphors and enactments make questions of training and learning, privacy and consent, autonomy and agency more tangible, and thereby help students be more reflective and responsible on how to design with AI and its complex properties in both their design process and outcomes. - Unpacking Human-AI Interactions: From Interaction Primitives to a Design Space Tsiakas, Konstantinos and Murray-Rust, Dave (2024) ACM Transactions on Interactive Intelligent Systems
Unpacking Human-AI Interactions: From Interaction Primitives to a Design Space
This paper aims to develop a semi-formal representation for Human-AI (HAI) interactions, by building a set of interaction primitives which can specify the information exchanges between users and AI systems during their interaction. We show how these primitives can be combined into a set of interaction patterns which can capture common interactions between humans and AI/ML models. The motivation behind this is twofold: firstly, to provide a compact generalisation of existing practices for the design and implementation of Human-AI interactions; and secondly, to support the creation of new interactions by extending the design space of HAI interactions. Taking into consideration frameworks, guidelines and taxonomies related to human-centered design and implementation of AI systems, we define a vocabulary for describing information exchanges based on the model’s characteristics and interactional capabilities. Based on this vocabulary, a message passing model for interactions between humans and models is presented, which we demonstrate can account for existing HAI interaction systems and approaches. Finally, we build this into design patterns which can describe common interactions between users and models, and we discuss how this approach can be used towards a design space for HAI interactions that creates new possibilities for designs as well as keeping track of implementation issues and concerns. - Towards a Multi-Stakeholder Value-Based Assessment Framework for Algorithmic Systems Yurrita, Mireia and Murray-Rust, Dave and Balayn, Agathe and Bozzon, Alessandro (2022) 2022 ACM Conference on Fairness, Accountability, and Transparency
Towards a Multi-Stakeholder Value-Based Assessment Framework for Algorithmic Systems
In an effort to regulate Machine Learning-driven (ML) systems, current auditing processes mostly focus on detecting harmful algorithmic biases. While these strategies have proven to be impactful, some values outlined in documents dealing with ethics in ML-driven systems are still underrepresented in auditing processes. Such unaddressed values mainly deal with contextual factors that cannot be easily quantified. In this paper, we develop a value-based assessment framework that is not limited to bias auditing and that covers prominent ethical principles for algorithmic systems. Our framework presents a circular arrangement of values with two bipolar dimensions that make common motivations and potential tensions explicit. In order to operationalize these high-level principles, values are then broken down into specific criteria and their manifestations. However, some of these value-specific criteria are mutually exclusive and require negotiation. As opposed to some other auditing frameworks that merely rely on ML researchers’ and practitioners’ input, we argue that it is necessary to include stakeholders that present diverse standpoints to systematically negotiate and consolidate value and criteria tensions. To that end, we map stakeholders with different insight needs, and assign tailored means for communicating value manifestations to them. We, therefore, contribute to current ML auditing practices with an assessment framework that visualizes closeness and tensions between values and we give guidelines on how to operationalize them, while opening up the evaluation and deliberation process to a wide range of stakeholders.
Data Relations
(also data KEM)
- Exploring The Future of Data-Driven Product Design Gorkovenko, Katerina and Burnett, Daniel J. and Thorp, James K. and Richards, Daniel and Murray-Rust, Dave (2020) Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
Exploring The Future of Data-Driven Product Design
Connected devices present new opportunities to advance design through data collection in the wild, similar to the way digital services evolve through analytics. However, it is still unclear how live data transmitted by connected devices informs the design of these products, going beyond performance optimisation to support creative practices. Design can be enriched by data captured by connected devices, from usage logs to environmental sensors, and data about the devices and people around them. Through a series of workshops, this paper contributes industry and academia perspectives on the future of data-driven product design. We highlight HCI challenges, issues and implications, including sensemaking and the generation of design insight. We further challenge current notions of data-driven design and envision ways in which future HCI research can develop ways to work with data in the design process in a connected, rich, human manner. - Data-Enhanced Design: Engaging Designers in Exploratory Sensemaking with Multimodal Data Gorkovenko, Katerina and Jenkins, Adam and Vaniea, Kami and Murray-Rust, Dave (2023) International Journal of Design
Data-Enhanced Design: Engaging Designers in Exploratory Sensemaking with Multimodal Data
Research in the wild can reveal human behaviours, contexts, and needs around products that are difficult to observe in the lab. Telemetry data from the use of physical products can help facilitate in the wild research, in particular by suggesting hypotheses that can be explored through machine learning models. This paper explores ways for designers without strong data skills to engage with multimodal data to develop a contextual understanding of product use. This study is framed around a lightweight version of a data enhanced design research process where multimodal telemetry data was captured by a GoPro camera attached to a bicycle. This was combined with the video data and conversation with the rider to carry out an exploratory sensemaking process and generate design research questions that could potentially be addressed through data capture, annotation and machine learning. We identify a range of ways that designers could make use of the data for ideation and developing context through annotating and exploring the data. Participants used data and annotation practices to connect the micro and macro, spot interesting moments, and frame questions around an unfamiliar problem. The work follows the designers’ questions, methods, and explorations, both immediate concerns and speculations about working at larger scales with machine learning models. This points to the possibility of tools that help designers to engage with machine learning, not just for optimisation and refinement, but for creative ideation in the early stage of design processes. - Entangled Ethnography: Towards a Collective Future Understanding Murray-Rust, Dave and Gorkovenko, Katerina and Burnett, Dan and Richards, Daniel (2019) Proceedings of the Halfway to the Future Symposium 2019
Entangled Ethnography: Towards a Collective Future Understanding
In this work, we develop a vision for entangled ethnography, where constellations of people, artefacts, algorithms and data come together to collectively make sense of the relations between people and objects. This is grounded in New Materialism’s picture of a world understood through entanglement, through resonant constellations, through a multiplicity of unique individual viewpoints and their relationships. These perspectives are especially relevant for design ethnography, in particular for research around smart connected products, which collect data about their environment, the networks they are a part of, and the ways they are used. However, we are concerned about the current trend of many connected systems towards surveillance capitalism, as data is colonised, machinations are hidden, and a narrow definition of value is extracted. There is a key tension that while design, particularly of networked objects, attempts to go beyond human centeredness, the infrastructures that support it are moving towards a less than human perspective in their race to accumulate and dispossess. Our work tries to imagine the situations where participants in networked systems are richly engaged, rather than exploited. We hope for a future where human agency is central to a respectful and acceptable collaborative development of understanding.
Infrastructural Relations
- Token Gesture: Non-Transferable NFTs, Digital Possessions and Ownership Design Elsden, Chris and Morgan, Evan and Black, Suzanne R. and Disley, Martin and Schafer, Burkhard and Murray-Rust, Dave and Speed, Chris (2024) CSCW
Token Gesture: Non-Transferable NFTs, Digital Possessions and Ownership Design
This paper presents the design, deployment and qualitative study of a large-scale, public, generative art exhibition, through which passers-by could create artworks, and mint a non-fungible-token (NFT). Following the month-long exhibition, during which 229 anonymous participants produced artworks, 69 non-transferable NFTs were minted, we surveyed (33) and interviewed (14) expert and novice participants about their experiences. We explored contemporary challenges of owning digital things, and the extent to which NFTs, and ‘Web3’ technologies offer meaningful forms of ownership. Our findings describe how the inability to trade this NFT, and its unique circumstances of acquisition, made it meaningful in ways that extended beyond its immediate (limited) utility and offered participants something through which to construct identity. Reflecting on the aspirations, contradictions, and misconceptions of forms of ownership enabled by NFTs, we conclude with proposals for renewed attention in HCI to the nature of digital possessions, and the potential for ‘ownership design’. - Blockchain and Beyond: Understanding Blockchains through Prototypes and Public Engagement Murray-Rust, Dave and Elsden, Chris and Nissen, Bettina and Tallyn, Ella and Pschetz, Larissa and Speed, Chris (2022) Transactions on Computer-Human Interaction
Blockchain and Beyond: Understanding Blockchains through Prototypes and Public Engagement
This paper presents an annotated portfolio of projects that seek to understand and communicate the social and societal implications of blockchains, distributed ledgers and smart contracts. These complex technologies rely on human and technical factors to deliver cryptocurrencies, shared computation and trustless protocols but have a secondary benefit in providing a moment to re-think many aspects of society, and imagine alternative possibilities. The projects use design and HCI methods to relate blockchains to a range of topics, including global supply chains, delivery infrastructure, smart grids, volunteering and charitable giving, through engaging publics, exploring ideas and speculating on possible futures. Based on an extensive annotated portfolio we draw out learning for the design of blockchain systems, broadening participation and surfacing questions around imaginaries, social implications and engagement with new technology. This paints a comprehensive picture of how HCI and design can shape understandings of the future of complex technologies. - Enacting the Last Mile: Experiences of Smart Contracts in Courier Deliveries Tallyn, Ella and Revans, Joe and Morgan, Evan and Fisken, Keith and Murray-Rust, Dave (2021) Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Enacting the Last Mile: Experiences of Smart Contracts in Courier Deliveries
Smart contract systems could change the nature of last-mile delivery for the better through enhanced precision, coordination and accountability. However, technological complexity poses a challenge for end-users participating in the design process, making it hard to explore their experiences and incorporate their perspectives. We describe a case study where technological prototypes create smart contract experiences for professional couriers and receptionists, allowing them to speculate about emerging possibilities, whilst remaining grounded in their current practices. Participants enacted a series of deliveries, choreographed by smart contracts, and their responses were explored in post-experience, one-to-one interviews. Working with professionals to explore the potential impact of smart contract technologies, revealed the systemic webs of value underlying their existing work practices. This has implications for design of such technologies, in which increased automation, efciency and accountability must be delicately balanced with the benefts of sustaining personal values, relationships and agency.
AI Art Relations
- Agency and Legibility for Artists through Experiential AI Hemment, Drew and Vidmar, Matjaz and Panas, Daga and Murray-Rust, Dave and Belle, Vaishak and Ruth, Aylett (2023) XAIxArts at Creativity and Cognition
Agency and Legibility for Artists through Experiential AI
Experiential AI is an emerging research field that addresses the challenge of making AI tangible and explicit, both to fuel cultural experiences for audiences, and to make AI systems more accessible to human understanding. The central theme is how artists, scientists and other interdisciplinary actors can come together to understand and communicate the functionality of AI, ML and intelligent robots, their limitations, and consequences, through informative and compelling experiences. It provides an approach and methodology for the arts and tangible experiences to mediate between impenetrable computer code and human understanding, making not just AI systems but also their values and implications more transparent, and therefore accountable. In this paper, we report on an empirical case study of an experiential AI system designed for creative data exploration of a user-defined dimension, to enable creators to gain more creative control over the AI process. We discuss how experiential AI can increase legibility and agency for artists, and how the arts can provide creative strategies and methods which can add to the toolbox for human-centred XAI. - Experiential AI: Between Arts and Explainable AI Hemment, Drew and Murray-Rust, Dave and Belle, Vaishak and Aylett, Ruth and Vidmar, Matjaz and Broz, Frank (2024) Leonardo
Experiential AI: Between Arts and Explainable AI
Experiential artificial intelligence (AI) is an approach to the design, use, and evaluation of AI in cultural or other real-world settings that foregrounds human experience and context. It combines arts and engineering to support rich and intuitive modes of model interpretation and interaction, making AI tangible and explicit. The ambition is to enable significant cultural works and make AI systems more understandable to nonexperts, thereby strengthening the basis for responsible deployment. This paper discusses limitations and promising directions in explainable AI, contributions the arts offer to enhance and go beyond explainability and methodology to support, deepen, and extend those contributions.
Other
Creative Industries
- Designing New Socio-Economic Imaginaries Speed, Christopher and Nissen, Bettina and Pschetz, Larissa and Murray-Rust, Dave and Mehrpouya, Hadi and Oosthuizen, Shaune (2019) The Design Journal
Designing New Socio-Economic Imaginaries
This short paper recovers the term ‘imaginaries’ which is often used in the social sciences to describe a meaning system that frames individuals lived experience of an inordinately complex world. The paper goes on to reflect on the extent to which design has the capability to disrupt imaginaries through the development of products in order for people to construct new ones, or whether the discipline is perpetuating old models of the world. The paper uses a workshop method to explore socio-economic models in order to better balance the multiple imaginaries that participants hold with the opportunity to design disruptive and critical propositions. Reflections upon the workshop and the concept of imaginaries allows the authors to identify a challenge for design in which it must accept its role as mediator and exacerbator. - Enacting the Last Mile: Experiences of Smart Contracts in Courier Deliveries Tallyn, Ella and Revans, Joe and Morgan, Evan and Fisken, Keith and Murray-Rust, Dave (2021) Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Enacting the Last Mile: Experiences of Smart Contracts in Courier Deliveries
Smart contract systems could change the nature of last-mile delivery for the better through enhanced precision, coordination and accountability. However, technological complexity poses a challenge for end-users participating in the design process, making it hard to explore their experiences and incorporate their perspectives. We describe a case study where technological prototypes create smart contract experiences for professional couriers and receptionists, allowing them to speculate about emerging possibilities, whilst remaining grounded in their current practices. Participants enacted a series of deliveries, choreographed by smart contracts, and their responses were explored in post-experience, one-to-one interviews. Working with professionals to explore the potential impact of smart contract technologies, revealed the systemic webs of value underlying their existing work practices. This has implications for design of such technologies, in which increased automation, efciency and accountability must be delicately balanced with the benefts of sustaining personal values, relationships and agency.
Active Grants
- Centre for the Decentralised Digital Economy (DECaDE) (UKRI/EPSRC, £4M, 2020 - 2025) , Centre for the Decentralised Digital Economy - creativity around distributed ledgers (with Surrey, Digital Catapult)
- DCODE (Horizon 2020 ITN, Jan 2020 - Jan 2025) , Designing human-machine relations, trusted interactions, socio-economic models for democratic futures (with Delft, Edinburgh, Umea, Copenhagen, Aarhus, TTI, Philips, AAS )
- AI Futures Lab: Rights and Justice in Remote Work (Delft AI Initiative , 2022 - 2027) , AI Futures Lab: Rights and Justice in Remote Work - part of the Delft AI labs initiative, exploring the impact of AI on remote work, and how we can use design to support new systems that preserve rights and justice
Current PhD Students
- Yuxi Liu (through DCODE, with Elisa Giaccardi and Johann Redstrom) is investigating the challenges of decentralized interaction with data-driven systems, and the development of novel design principles for multi-intentional interaction.
- Mahan Mehrvarz (through AI Futures Lab, with Elisa Giaccardi) is investigating designing justice oriented AI artefacts.
- Sofie-Amalie Torp Dideriksen (through AI Futures Lab, with Elisa Giaccardi) is investigating critical feminist approaches to designing with AI (all PhDs)
Research Themes
AI Futures
I am director of the AI Futures Lab: Rights and Justice in Remote Work, with Ben Wagner and Filippo Santoni de Sio. The AI Futures Lab will address the current knowledge gap by combining IDE post-industrial design research and methodologies. These will be applied to machine ethnography, to experiential AI and to in-the-wild AI prototyping using TPM methodologies of comprehensive engineering and design for values. We will explore configurations of people and AI around remote collaboration and distributed work, aiming to expand both scientific knowledge and public understanding of AI capabilities. Our goal is a tangible and vibrant set of prototypes, experiences and theories that map out ways in which design can be engaged to deploy AI and machine learning in support of new ways of working.
This is closely related to the work of the DCODE project, where I am co-supervising:
- Yuxi Liu (with Elisa Giaccardi and Johann Redstrom) who is investigating the challenges of decentralized interaction with data-driven systems, and the development of novel design principles for multi-intentional interaction.
Continuous digital ethnography and use driven design
Entangled ethnography develops ways to use continuous, realtime data to develop understanding of use and behaviour. In order to make sure inferences are valid, it includes contextual enquiry techniques to ground truth data against lived experience. We are also creating tools for exploring and annotating the data, in collaboration with machine learning, and supporting designers in asking questions of existing and yet to be collected data.
This started through the Chatty Factories, but continues within TU Delft’s design department.
- Murray-Rust, D., Gorkovenko, K., Burnett, D., & Richards, D. (2019). Entangled Ethnography: Towards a Collective Future Understanding. Proceedings of the Halfway to the Future Symposium 2019, 1–10. New York, NY, USA: Association for Computing Machinery.
- Gorkovenko, K., Burnett, D. J., Thorp, J. K., Richards, D., & Murray-Rust, D. (2020). Exploring The Future of Data-Driven Product Design. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–14. New York, NY, USA: Association for Computing Machinery.
- Gorkovenko, K., Burnett, D., Thorp, J., Richards, D., & Murray-Rust, D. (2019). Supporting Real-Time Contextual Inquiry Through Sensor Data. Ethnographic Praxis in Industry (EPIC2019). https://doi.org/10.1111/1559-8918.2019.01307
Prototyping with Emerging infrastructures
New computational systems are springing up around us, shaping society as they go. This includes artificial intelligence ecosystems, smart contracts and decentralised ledgers, IoT and more. I’m interested in how we can bring design to bear in this space, offering open experiences that foster critical engagement based on a developed understanding of these technologies. This started with work on GeoCoin, prototyping interactions around geolocated currencies and continued with GeoPact - a system of intelligent objects that interact with location based smart contracts (Funded through B-IoT, IoT-Tram and BLING - BLockchain IN Government). A large part of this work spanning 5 years is captured in an annotated portfolio, that covers work from the Institute for Design Informatics.
- Annotated portfolio: Murray-Rust, D., Elsden, C., Nissen, B., Tallyn, E., Pschetz, L., & Speed, C. (2022). Blockchain and Beyond: Understanding Blockchains through Prototypes and Public Engagement. Transactions on Computer-Human Interaction. https://doi.org/https://dl.acm.org/doi/10.1145/3503462
- Tallyn, E., Revans, J., Morgan, E., Fisken, K., & Murray-Rust, D. (2021). Enacting the Last Mile: Experiences of Smart Contracts in Courier Deliveries. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1–14. Yokohama Japan: ACM.
Tallyn, E., Revans, J., Morgan, E., & Murray-Rust, D. (2020). GeoPact: Engaging Publics in Location-Aware Smart Contracts through Technological Assemblies. Designing Interactive Systems 2020 Conference, 799–811. ACM.
- Nissen, B., Pschetz, L., Murray-Rust, D., Mehrpouya, H., Oosthuizen, S., & Speed, C. (2018). GeoCoin: Supporting Ideation and Collaborative Design with Location-Based Smart Contracts. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM.
Experiential AI, art-science and embodied algorithms
A cruicial societal ability is the ability to debate and engage with emerging technology. Part of my work is to support this, by bringing art-science thinking to bear on AI and robotics. This is different from approaches such as Explainable AI - here I’m interested in creating experiences that help to understand and engage viscerally and critically with technologies. In the emerging Experiential AI research them, we are exploring how creative practice can shape the way publics and computer scientists understand artificial intelligence.
- Hemment, D., Aylett, R., Belle, V., Murray-Rust, D., Luger, E., Hillston, J., … Broz, F. (2019). Experiential AI. AI Matters, 5, 25–31.
- Hemment, D., Belle, V., Aylett, R., Murray-Rust, D., Pschetz, L., & Broz, F. (2019). Toward Fairness, Morality and Transparency in Artificial Intelligence through Experiential AI. Leonardo, 52. https://doi.org/10.1162/leon_a_01795
- Murray-Rust, D., & von Jungenfeld, R. (2017). Thinking through Robotic Imaginaries. RTD2017. https://doi.org/10.6084/m9.figshare.4746973
- Perelló, J., Murray-Rust, D., Nowak, A., & Bishop, S. R. (2012). Linking Science and Arts: Intimate Science, Shared Spaces and Living Experiments. European Physical Journal - Special Topics, 214, 597–634.
Humane Data Interaction
Driven in part by understanding the social within social machines, I have been concerned with aspects of the ways in which people interact with data that have to do with privacy, identity and autonomy. This means opening up areas around humane data interaction – how we can live as humans within an increasingly algorithmically mediated society. Key themes are pro social deception, identity, wayfaring, personal data and developing acceptability as a lens for technical systems.
- Rooksby, J., Morrison, A., & Murray-Rust, D. (2019). Student Perspectives on Digital Phenotyping: The Acceptability of Using Smartphone Data to Assess Mental Health. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1–14. New York, NY, USA: Association for Computing Machinery.
- Gorkovenko, K., & Murray-Rust, D. (2021). User Perspectives on the Acceptability of Realtime Data Capture for Design Research by Connected Products. In G. Bruyns & H. Wei (Eds.), [ ] With Design: Reinventing Design Modes (pp. 2201–2221). Singapore: Springer Nature Singapore.
- van Kleek, M., Murray-Rust, D., Guy, A., O’Hara, K., & Shadbolt, N. (2016). Computationally Mediated Pro-Social Deception. CHI 2016, 552–563. ACM.
- Murray-Rust, D., Tarte, S., Hartswood, M., & Green, O. (2015). On Wayfaring in Social Machines. WWW ’15 Companion Proceedings of the 24th International Conference on World Wide Web, 1143–1148. ACM.
- Kleek, M. V., Smith, D. A., Shadbolt, N. R., Murray-Rust, D., & Guy, A. (2015). Self Curation, Social Partitioning, Escaping from Prejudice and Harassment: The Many Dimensions of Lying Online. WWW 2015 Companion, 371–372.
Data Manifestation
‘How does data give rise to experiences? How does data science become the basis for communicative work?’. I’m investigating these questions through a combination of reasearch and teaching. I’m collaborating with Benjamin Bach to understand how to make and use Data Comics to communicate about data. I’m collaborating with Bettina Nissen on WallVis to explore the possibilities of a dynamic, modular data physicalisation system.
- Wang, Z., Romat, H., Chevalier, F., Riche, N., Murray-Rust, D., & Bach, B. (2022). Interactive Data Comics. IEEE Transactions on Visualization and Computer Graphics. https://doi.org/10.1109/TVCG.2021.3114849
- Bach, B., Wang, Z., Farinella, M., Murray-Rust, D., & Henry Riche, N. (2018). Design Patterns for Data Comics. ACM Conference on Human Factors in Computing Systems (CHI). ACM.
Algorithms and Music
My PhD was in artificial intelligence and music - creating a system of musical agents that could be improvisational partners.
- Murray-Rust, D., & Smaill, A. (2011). Towards a Model of Musical Interaction and Communication. Artificial Intelligence, 175, 1697–1721.
- Murray-Rust, D., Smaill, A., & Edwards, M. (2006). MAMA: An Architecture for Interactive Musical Agents. In G. Brewka, S. Coraeschi, A. Perini, & P. Traverso (Eds.), ECAI 2006, PROCEEDINGS (pp. 36–40). I O S PRESS.
- Murray-Rust, D., Smaill, A., & Maya, M. C. (2005). VirtuaLatin - Towards a Musical Multi-Agent System. In H. Selvaraj, B. Verma, & A. DeCarvalho (Eds.), ICCIMA 2005: Sixth International Conference on Computational Intelligence and Multimedia Applications, Proceedings (pp. 17–22). United States: Institute of Electrical and Electronics Engineers (IEEE).
Engineering Social Machines
From Wikipedia to Facebook, social machines have become integral to our daily lives. Digital networked technology now routinely enables coordination of collective action, releasing the power of decentralised hybrid human-machine problem-solving at scale. Supported through the SociaM Project, summed up in The Theory and Practice of Social Machines. Within this, I’ve been working on using Process Calculus to create open interactions on the web.
- Murray-Rust, D., Papapanagiotou, P., & Robertson, D. (2016). Softening Electronic Institutions to Support Natural Interaction. Human Computation, 2. https://doi.org/10.15346/hc.v2i2.3
- Papapanagiotou, P., Davoust, A., Murray-Rust, D., Manataki, A., van Kleek, M., Shadbolt, N., & Robertson, D. (2018). Social Machines for All. 17th International Conference on Autonomous Agents and Multiagent Systems, 1208–1212.
- Murray-Rust, D., & Robertson, D. (2014). LSCitter: Building Social Machines by Augmenting Existing Social Networks with Interaction Models. Proceedings of the Companion Publication of the 23rd International Conference on World Wide Web Companion, 875–880. ACM.
Understanding Mathematical practice
I have a longstanding interest in the ways that digital tools can support mathematical practice, starting with developing a declarative, semantic model of mathematics that can be used both for display and calculation. Through the SociaM project, I became interested in the Social Machines of Mathematics, developing formal structures for modelling online collaboration.
- Corneli, J., Martin, U., Murray-Rust, D., Nesin, G. R., & Pease, A. (2019). Argumentation Theory for Mathematical Argument. Argumentation, 1–42.
- Lane, L., Martin, U., Murray-Rust, D., Pease, A., & Tanswell, F. (2018). Journeys in Mathematical Landscapes: Genius or Craft? In Mathematics Education in the Digital Era. Proof Technology in Mathematics Research and Teaching. Springer.
Completed Grants
- Blockchains in Government (BLING) (EU North Sea Region INTERREG, £1.9M, Jan 2019 - Dec 2022) (with (see project page for partners))
- IoT Transport and Mobility (IoT-TRAM) (PETRAS Demonstrator + Lloyds Registry, £143K, Oct 2018 - Feb 2019) (with Warwick (lead), Edinburgh, Imperial, Surrey)
- Blockchains for Intelligent Travel and the Internet of Things (B-IoT) (PETRAS Strategic Fund, £350K, Oct 2017 - Aug 2019) (with Imperial (lead), Edinburgh, Surrey and Warwick)
- Chatty Factories (EPSRC [EP/R021031/1] , £1.8M, Jan 2018 - Dec 2020) (with Cardiff (lead), Edinburgh, Lancaster, Nottingham, Essex)