WP6 Year 3: implementation & evaluation

During year 3, WP6 has focussed on refining the evaluation methodology. This has included five Patient and Public Involvement (PPI) sessions (in Nottingham and at INTRAS) to review the design prototypes as well as consultation with staff who will be involved in trying out the prototype designs (Intras, Zorggroep Sint Marten, Alexianer St Hedwig Hospital, Alzheimer Society).

Semi-annual update: March – August 2018

The MinD evaluation workplan was refined in the hosted secondments in the UK (V22, May) and Luxembourg (V23, June). Leading on from the earlier Soft Evaluation workshop, the discussions and development of the plan for evaluation were linked to the user populations of the member sites and to the Patient and Public Involvement activity.

Formative evaluation of the Social Engagement Map

In the UK secondment an opportunity for people with lived experience to try out and feedback option on a table-top mockup of the Social Engagement Map (SEM) on an LCD screen, including function and layout. This was facilitated during the main PPI session. Participants were asked to provide a photo of themselves in advance of the PPI session to be used in the mock up. The session enabled participants to comment on ease of use, aesthetics and suggestions for improvement. Users were delighted when they were able to identify their own faces and interests floating across the screen and imagine how this would enable future potential connections and shared activities. The ‘tangibles’ (hand‐held objects) were of particular interest with variation in preference for weight, size and shape. Many useful comments and suggestions were recorded through a brief questionnaire, the results of which were presented in the second week of the UK secondment and developed into a specification and guidelines for the next design iteration.

Mock-up of social engagement map Mock-up of social engagement map

Evaluation Framework

A framework for the ongoing and summative evaluation of the designs was developed in Luxembourg. A model for supporting evaluation in three areas was previously devised. The AIR model considers the impact of design on 3 areas: Activities, Internal World and Relationships of the person living with dementia. We will focus on using the AIR model for use in ongoing formative and summative evaluation of the design prototypes to assess their potential for change from the perspectives of (depending on location and setting) a) people with dementia, b) carers or c) clinical staff in different roles, through small focus groups. A template for the evaluation question will be developed at the next secondments. The intention is to present prototypes to users in the four countries and user populations from which needs were originally determined: Germany, Netherlands, UK and Spain.

In addition to the evaluation of designs, an evaluation of the design process is also being conducted. At the Luxembourg secondment a paper was presented reflecting on the use of design probes. We also intend to evaluate the PPI process of MinD since the involvement experience has been captured in detail in the UK workshops.


Semi-annual update: September 2018 – February 2019

Evaluation tools were co-created over several iterations during secondments in Germany, Netherlands, the UK and Spain with participation from across the MinD partnership, including ongoing contributions from partners with lived experience who were instrumental in finalising the tools to ensure their accessibility and acceptability for end-users.

The set of evaluation tools developed includes information about the co-creation project and process, embracing the purpose of the product in the first person and as a mutual exercise of ‘we’ rather than ‘you’ to emphasise partnership

e.g. ‘Also definitions of key principles are given in plain language i.e. mindfulness as the relationship we have with our own experience; self-empowerment enhancing personal independence and control; purpose fulfilment that makes a real difference to our lives and those of others; and better connection to nurture the growth of body and spirit’.

Additional material produced for the evaluation work includes co-creation of:

— Participation information for the evaluation, and consent forms.

— Evaluation questions and general guide to use, for participants.

— Guide for facilitators/interviewers, including prompts.

— Feedback questionnaires about the tasks and the session for all.

— Mindful reflective questions for facilitators, interviewers and observers

Collaborative (PPI*) events were held in Spain and the UK to present, discuss and respond to the prototypes of both the Social Engagement Map and the Good Life Kit and findings were conveyed to the development teams who committed to make requested adaptations where feasible.

Two researchers with lived experience from Nottingham participated in two secondments outside of the UK, hosted by INTRAS (Valladolid), during November 2018 and February/March 2019. During their secondments, the researchers supported Spanish people with lived experience of dementia and their carers, to organise for involvement as active partners within INTRAS and to further assist in evaluating MinD prototypes. Additionally, the researchers met with academics and policy makers to raise awareness about MinD and to generally promote co-production and public engagement strategy across Spain.

A poster was published on 15 November 2018 “Researching together with members of the Centre for Dementia PPI group” at a conference ’International Perspectives on Evaluation of PPI in Research’ where the MinD PPI work was featured as one of 5 case studies where Nottingham has been involved in PPI work.

During the November 2018 secondment in Valladolid, colleagues from Italy, Germany and UK worked with INTRAS Foundation partners to present the MinD project at the 3rd MinD symposium “Application of mindfulness in the work with people affected by mental and neurological diseases” co-organised by Intras and hosted at Universidad Europea Miguel de Cervantes (UEMC) on 20th November 2018. Two lived experience researchers spoke from the platform; for one it was their first public presentation and a big leap towards their enhanced confidence, self-esteem and feelings of well-being.

In February 2019 a paper was written and submitted to the MinD conference (Dresden) in the form of an extended abstract by authors from the MinD partnership, entitled: “Evaluation methods as part of the co-design process for assistive and well-being products for and with people living with dementia focussed on social engagement and mindfulness”. Lived experience input to conference planning during February resulted in the inclusion of an accessible /practical workshop format to balance the academic nature of paper presentations.

* PPI  – Public and patient involvement