HomeProjects and Grants

I have been part of several EPSRC-funded projects:

Please see the project pages linked above and associated publications for further details of this research. See also my additional work page for details of public engagement, impact, and smaller projects and grants.

UnMute

A public space speech assistant prototype in Dharavi, Mumbai, India

UnMute: opening spoken language interaction to the currently unheard is a project in collaboration with colleagues at the University of Edinburgh, Translators Without Borders and Auris Tech, aiming to address the limitations of today's speech and voice-based interactions and open up intelligent interfaces to the currently digitally ‘unheard’. While state-of-the-art natural language systems are beginning to address the needs of “conventional” users (i.e., those who speak a widely spoken and written language; and who have relatively high degrees of literacy, exposure to digital interactions and other resources), there are many hundreds millions of people who are being excluded globally. Paradoxically, these users who have resource constraints (such as low digital and textual literacy) could be the ones to most benefit from advances in speech-based interactive systems, opening up economic, social and educational possibilities that are currently unmet. In advancing this area of research, we hope to produce a blueprint and toolkit that can be used by many other low or zero-resource language communities, worldwide. For more information, publications and contacts, see the project website.

Rethinking public technology

A contactless lift button deployed as an augmentation to an existing appliance

Rethinking public technology in a COVID-19 era focuses on the ubiquitous yet essential public infrastructures that rely on buttons or touchscreens: common interactions that, until recently, were considered perfectly safe to perform. The Covid-19 pandemic has created unprecedented opportunities — and an urgent need — for technologies that minimise interaction with shared devices. While recent advances in sensor technology and artificial intelligence have enabled consumer products that can respond to other modalities such as speech, gestures and faces, the underlying interaction paradigms have not been widely studied in public-facing settings. This project will develop and integrate a suite of touchless technologies into exemplar public-facing services, working closely with Swansea City Council as a test-bed location. These flexible longitudinal deployments will allow us to study usage in-situ to determine the most appropriate, robust and cost-effective ways of facilitating safe, touchless interfaces for the COVID-19 era (and any subsequent outbreaks). The result of this work will be a public-facing touchless interaction design methodology and open-source toolkit of blueprints and techniques that can be integrated into existing infrastructures and used to inform the design of future installations.

PV interfaces

A decorative self-powered display that responds to light levels and gesture inputs

PV interfaces: self-powered interfaces and interactions via photovoltaic surfaces is a project in collaboration with colleagues in DST Innovations, Hewlett Packard, Google, HAB Housing and IIT Bombay to develop approaches to capture the benefits of the IoT future while making it sustainable, delightful and universally accessible. The work involves a team of material scientists and human computer interaction researchers, working together with partners to develop a new form of physical material that can generate the power it needs to drive digital interfaces and interactions. That is, we will drive towards attractive, flat and flexible solar energy harvesting tiles, which may incorporate input and output features to enable people to interact with them and other connected devices. These tiles will be able to be integrated into buildings (in walls and floors, for instance) and objects (like tables, clothes and book covers). The surfaces capture the energy from indoor and ambient light and at the same surface can present digital displays and interfaces to the user. For more information, publications and contacts, see the project website.

Breaking the glass

Breaking the glass: multimodal, malleable interactive mobile surfaces for hands-in interactions is a project in collaboration with colleagues in the University of Sussex, Université Grenoble Alpes, University of Bristol and BBC R&D that looks to pioneer hands-in and hands-on interaction on deformable mobile display surfaces, leading to dramatic new ways for users to interact with content and services. We will ambitiously drive developments in both physical display materials and interaction techniques, enabling a richer, expressive toolkit of gestures and manipulation on touch surfaces. For more information, publications and contacts, see the project website.

Reshaping the expected future

A miniature robot that draws lines or letters using everyday foodstuffs

Re-shaping the expected future: novel interactions for emergent users is a project in collaboration with colleagues in iHub Research, IBM Research India, Microsoft Research UK, Microsoft Research India, University of Cape Town, MercyCorps, IIT Bombay and Social Impact Lab. Emergent users are the hundreds of millions of people from the poorest regions of the world. Challenges in these areas range from low technological and textual literacy, a paucity of relevant, appropriate content, to a lack of affordable, high-bandwidth data connections. Our aim is to radically innovate for key information interaction needs, drawing on a network of organisations and individuals deeply connected to emergent users, along with emergent end-users themselves. Our aspiration is to uncover fundamental, generally applicable interaction and transaction techniques for these sorts of users. In the developed world there are key base interfaces and interactions, such as textually-rich query-result search, drag and drop, document cutting and pasting, and persistent network-dependent content. A driving question in our work is, “what are the equivalents of these sorts of enabling interfaces for emergent users?” For more information, publications, contacts and source code, see the project website and resulting Digital Inclusion and Better Together toolkits.

Scaling the rural enterprise

A trial of Spoken Web audio icons near Dharwad, India

Scaling the rural enterprise is a project in collaboration with colleagues in the University of Nottingham, Imperial College London, Centre for Technology & Development, IBM Research Delhi, Microsoft Research Bangalore and University of Cape Town to develop approaches to “scaling up” rural enterprise by combining engineering, economics, interaction design and computing. The problem we are addressing is a complex, socio-technical one requiring a strongly inter-disciplinary user led approach. We work iteratively with real-world communities in India and the UK. As well as hosting interventions, these "living labs" help us understand how diverse networks of actors shape rural enterprises in both the UK and India, and stretch our thinking to ensure we guard against too specific a set of solutions. We focus on two particular distributed communities of practice that produce goods through shared labour which allows us to transfer lessons learned between India and the UK and vice versa. For more information, publications, contacts and source code, see the project website and resulting Enterise toolkit.

The next billion

Community-generated media for the next billion is a project in collaboration with colleagues in the University of Glasgow, University of Surrey, University of Cape Town and Transcape NPO to provide innovative, appropriate media sharing technologies and methods to give a voice to the hundreds of millions of users on the wrong side of the so-called “digital divide”. The work will develop solutions for the hard problems of designing effective interfaces and information access algorithms for users with low levels of textual and computer literacy; limited access to digital services; and patterns of living and working that differ to those of people in “developed” regions.

Our role in Swansea is to design and build mobile media sharing tools and interfaces as part of the project’s community media toolkit. For more information, publications, contacts and source code, see the project website and resulting Digital Economy toolkit.

Individual project pages

Negotiated interaction

Multimodal, negotiated interaction in mobile scenarios is a project in collaboration with colleagues in the University of Glasgow to think about the future of mobile digital-physical world interactions. Our proposal is to investigate alternative ways to allow users to interact with content and services in their environment, such that the actions they make and feedback they receive are continuous, with the user and system negotiating their interactions in a fluid, dynamic way.

Our role in Swansea is to examine mobile scenarios for negotiated interaction, developing effective physical-digital interaction methods and toolkits. For more information, publications and contacts, see the project website.