BEST - Beyond Screen readers and Alt Text - Designing Multisensory Alternative to Text for Different Reading Abilities

This project explores how technology can support different forms of reading and sense making of text and multimedia content. Reading is a complex process that includes seeing the character (accessibility via visibility), being able to distinctly identify each character from the other (recognition via legibility) and finally making sense of the characters as a group (interpretation via readability). The aim of our project is to provide useful and usable assistive tools for people with different reading abilities for a rewarding reading experience. We take inspiration from works on screen readers as these are widely available to help users make sense of written text and images, and we move on considering how this paradigm could be further expanded to provide a multisensory reading experience. We will start from a seminal work by Morris et at., (2018) revisiting the concept of alternative text (alt text), descriptive text associated to images in HTML and other types of documents, to provide a rich reading experience to blind and visually impaired users. The main outcome of their study is a taxonomy of 5 categories (interactivity, stability, representation, structure and personalisation) to define the design space for producing prototypes of innovative representations of visual content in a multisensory way. BEST will adopt and further expand that design space for producing alternative representations of text (Alt to Text) to fit different reading abilities and provide an effective and rewarding reading experience. We believe that properly designed multisensory representations have the potential to serve, not only readers visually impaired, but also those experiencing either temporarily or permanently a variety of reading difficulties.

Our main contribution will be the design, implementation and evaluation of tools to support reading while catering for different abilities. By considering different combinations of context (formal and non), age (from pre-school children to adults), and abilities, we will produce and test a series of prototypes to support reading. We will focus on two specific sub-projects: EPPics (Enhanced Personalised Picture Stories) and Muses (Multisensory story-telling).

EPPics explores a formal context, pre-school, and engages as co-designers pre-school children and teachers in a collaborative design approach focusing on the concept of personalised stories, as a way to foster pre-reading skills. It aims at producing an authoring tool to cater for individual reading needs and follows pupils building and developing reading skills from scratch. Personalisation and gamification will be at its core.

Muses will involve adults with mild cognitive disabilities as visitors in a museum, a non formal context, together with their care givers and educators. They will take an active role in collaborative critical design. The objective of this sub-project is to develop tools for supporting reading before, during and after the visit, to suit individual needs and preferences.

Both sub-projects are based on feasibility studies conducted by the applicant, and by running them in parallel, we will be able to exchange and take advantage of each other findings.