WP6 Year 2: evaluation framework

Semi-annual update 1 – February 2018

WP6 dovetails with the co-design process of WPs 4 and 5 and seeks first to determine the appropriate evaluation methodology for assessing the value of designs developed in MinD and then apply an evaluation scheme to the prototypes that are realised. The aim of MinD is to provide new design solutions to facilitate improved social engagement and to empower people living with dementia and support their emotional well-being through mindful activity. This fits with a psychosocial outlook on wellbeing. To evaluate solutions in the most participatory manner, it is useful to review measures that have been designed for either formative evaluation (during the period of prototype design) or summative evaluation (at the end of design or at different points where solution prototypes will be available). It has been noted in the literature that, until recently, there were few tools to specifically measure psychosocial wellbeing of persons with dementia, although a large number of more general quality of life (QoL) measures exist. On the other hand, the attempt to measure psychosocial impact of assistive technologies has resulted in a number of useful tools for their summative evaluation, although these are not dementia specific.

In October 2017 during a UK hosted secondment, a workshop on ‘Soft Evaluations’ was held at the University of Wolverhampton to discuss evaluation in a multiprofessional expert panel: designers, professionals in dementia research, Public and Patient Involvement (PPI) and scientific methodologists. The general notion of soft evaluation is an early stage of evaluation during the development of design solutions. Participants in the workshop included members of the MinD consortium and invited speakers. The following topics were discussed during the day:

  • Levels of evaluation, their methods and their validity Transferabilty of results and their influence on public decision-making
  • Non-functional requirements e.g. impact of resourcing, time on design and evaluation choices
  • Ethical framework and human equity considerations
  • Challenges of multidisciplinary working and communication between disciplines (particularly clinical/public health and arts/design)
  • Emotional impact on researchers of conducting research with people with dementia

At the short secondment in the Netherlands in February 2018, a document on evaluation was produced including an overview of technology evaluation measures, positive psychology measures and technology adoption models that have been suggested for use in the dementia domain.