ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study

For both lay representatives and researchers, a positive experience created a virtuous cycle whereby PPI became increasingly embedded.

“For both lay representatives and researchers a positive experience created a virtuous cycle whereby PPI became increasingly embedded.”

A dual role of participation/involvement was sometimes used to capture the target population perspective.

“A dual role of participation/involvement was sometimes used to capture the target population perspective.”

• AssumptionRests on an assumption: Normalisation Process Theory provides four main areas of work to embed a complex intervention such as PPI as normal practice.

An assigned, resourced PPI co-ordinator role was important in sustaining PPI.

“An assigned, resourced PPI co-ordinator role was important in sustaining PPI.”

• AssumptionRests on an assumption: Normalisation Process Theory provides four main areas of work to embed a complex intervention such as PPI as normal practice.

Flexible approaches to enable use of lay representatives' individual skill sets and their personal circumstances were required.

“Flexible approaches to enable use of lay representatives' individual skill sets and their personal circumstances were required.”

• AssumptionRests on an assumption: Normalisation Process Theory provides four main areas of work to embed a complex intervention such as PPI as normal practice.

There was some evidence that establishing and maintaining good relationships was hard to achieve when PPI was conducted purely through virtual media.

“There was some evidence that this was hard to achieve when PPI was conducted purely through virtual media.”

• AssumptionRests on an assumption: Normalisation Process Theory provides four main areas of work to embed a complex intervention such as PPI as normal practice.

There was little systematic appraisal within the case studies.

“There was little systematic appraisal within the case studies.”

• AssumptionRests on an assumption: Normalisation Process Theory provides four main areas of work to embed a complex intervention such as PPI as normal practice.

Establishing and maintaining good relationships between researchers and lay representatives was crucial.

“Establishing and maintaining good relationships between researchers and lay representatives was crucial.”

• AssumptionRests on an assumption: Normalisation Process Theory provides four main areas of work to embed a complex intervention such as PPI as normal practice.

In newly formed groups of researchers and lay representatives, achieving agreement on the purpose of PPI may take time.

“In newly formed groups of researchers and lay representatives, this may take time to achieve.”

• AssumptionRests on an assumption: Normalisation Process Theory provides four main areas of work to embed a complex intervention such as PPI as normal practice.

The lay representatives in the outreach model had strong links and networks with populations the research was aiming to recruit.

“The lay representatives in this model had strong links and networks with populations the research was aiming to recruit.”

Junior members of the research team tended to have more difficulties in differentiating between participation and involvement.

“Junior members of the research team tended to have more difficulties in differentiating between participation and involvement.”

• AssumptionRests on an assumption: Normalisation Process Theory provides four main areas of work to embed a complex intervention such as PPI as normal practice.

Of the 478 eligible studies identified in the national scoping, information was obtained on 38% (n=182).

“Of the 478 eligible studies identified, we obtained information on 38% (n = 182).”

All the case studies had evidence of PPI outcomes, including research priority/question setting, study marketing, changes to the design including interventions, ensuring participant safety and recruitment.

“All the case studies had evidence of PPI outcomes, including research priority/question setting, study marketing, changes to the design including interventions, ensuring participant safety and recruitment.”

The outreach model of PPI had regular points of contact between researchers and lay representatives throughout the research.

“This model had regular points of contact between researchers and lay representatives throughout the research.”

In the one-off model of PPI, potential outcomes were limited by the researcher.

“In the one-off model of PPI, potential outcomes were limited by the researcher.”

Higher levels of agreement on the purpose of PPI usually led to more embedded PPI.

“Higher levels of agreement on the purpose of PPI usually led to more embedded PPI.”

• AssumptionRests on an assumption: Normalisation Process Theory provides four main areas of work to embed a complex intervention such as PPI as normal practice.

In total, 1464 studies in the six broad topic/exemplar areas were downloaded from the UKCRN database.

“In total, 1464 studies in the six broad topic/exemplar areas were downloaded from the UKCRN database.”

• AssumptionRests on an assumption: The UKCRN portfolio is a sufficient and representative source for identifying relevant PPI studies.

Thematic content analysis established how different contexts affected processes, mechanisms and outcomes of PPI.

“This established how different contexts (e.g.topic area, type of research and setting) affected processes (e.g.training and support), mechanisms (e.g.relationships and communication) and outcomes of PPI (e.g.evidence of changes, responsiveness to PPI input and impact on dissemination).”

The scoping exercise obtained information about PPI from 182 studies out of a possible total of 478 (38%).

“The scoping exercise obtained information about PPI from 182 studies out of a possible total of 478 (38%) (Table 6).”

• AssumptionRests on an assumption: The UKCRN portfolio is a sufficient and representative source for identifying relevant PPI studies.

Even when the protocol was available on the funder's or study's website, there was little information about PPI.

“Significantly, even when the protocol was available on the funder's or study's website, there was little information about PPI.”

One of our findings from this initial scoping exercise is that there is minimal direct open access to such study documentation.

“Indeed, one of our findings from this initial scoping exercise is that there is minimal direct open access to such study documentation.”

Replicating the findings from the RAPPORT study, the importance of relationships emerged as central in this study.

“Replicating the findings from the RAPPORT study, the importance of relationships emerged as central in this study.”

Having a local person to co-ordinate PPI travel, expenses and other practical matters was found to be very important.

“Having a local person to co-ordinate PPI travel, expenses and other practical matters was found to be very important, as was accommodating preferences for written/hard copy materials rather than electronic communications:”

Within the scoping we were unable to find evidence of PPI in 49% of studies.

“Within the scoping we were unable to find evidence of PPI in 49% of studies.”

The original project's assumptions about the different roles, expectations and capacities of the various PPI representatives within the RAPPORT study needed revisiting over time.

“The original project's assumptions about the different roles, expectations and capacities of the various PPI representatives within the RAPPORT study needed revisiting over time: 'Being on an advisory group I found quite limiting'.”

There was evidence of a dual role: study participants also acting as advisors either concurrently with being a participant, or sequentially.

“There was also evidence of a dual role: study participants also acting as advisors either concurrently with being a participant, or sequentially.”

Co-researchers gained personally from carrying out the research, finding it stimulating.

“As found within the RAPPORT study, the co-researchers gained personally from carrying out the research: stimulating to get me back to using my brain a bit more and think about research.”

Paradoxically, open-text responses in the survey emphasised the challenge of representativeness.

“Paradoxically, open-text responses in the survey emphasised the challenge of representativeness.”

Study design alone was not a predictor of engagement with PPI.

“Study design alone was not a predictor of engagement with PPI.”

In a short survey focused on gaining descriptive detail on the current extent of PPI, it was not possible to identify any specific rationale for not having PPI within a study.

“In a short survey focused on gaining descriptive detail on the current extent of PPI, it was not possible to identify any specific rationale for not having PPI within a study (see Chapter 3).”

Within the scoping we were unable to find evidence of any PPI in nearly half of the studies.

“Within the scoping we were unable to find evidence of any PPI in nearly half of the studies.”

Respect for and accommodation of PPI representatives' different skill mixes and requirements was an issue.

“Respect for and accommodation of PPI representatives' different skill mixes and requirements was an issue.”

There were some differences in how PPI was approached by research topic and study design.

“However, there were some differences in how PPI was approached by research topic and study design.”

Over a 2.5-year study, illness meant that at times it was difficult for some PPI representatives to travel long distances or into London and be as involved in the RAPPORT study as they would have liked.

“Over a 2.5-year study, illness has meant that at times it was difficult for some PPI representatives to travel long distances or into London and be as involved in the RAPPORT study as they would have liked.”

From the survey we found that nearly half of PPI representatives were already known to the study team when it invited them to be lay advisers.

“From the survey we found that nearly half of PPI representatives were already known to the study team when it invited them to be lay advisers.”

In both the scoping and survey it was clear that some respondents did not distinguish between involvement and study participation.

“In both the scoping and survey it was clear that some respondents did not distinguish between involvement and study participation.”

Accommodating preferences for written/hard copy materials rather than electronic communications was found to be very important.

“Having a local person to co-ordinate PPI travel, expenses and other practical matters was found to be very important, as was accommodating preferences for written/hard copy materials rather than electronic communications:”

With some noteworthy exceptions, there was sparse evidence of systematic evaluation of PPI within studies.

“With some noteworthy exceptions, there was sparse evidence of systematic evaluation of PPI within studies.”

While still challenging for a number of studies and funders, recruiting a diverse range of lay people was enhanced when the study setting interfaced with a diverse population.

“While still challenging for a number of studies and funders, recruiting a diverse range of lay people was enhanced when the study setting interfaced with a diverse population.”

There was relatively sparse evidence of systematic evaluation happening overall.

“However, there was relatively sparse evidence of systematic evaluation happening overall.”

Survey results indicated that 29% of studies had 21 or more individual patient or public representatives involved.

“Survey results indicated that 29% of studies had 21 or more individual patient or public representatives involved.”

The research approach did appear to influence what particular aspects of PPI were highlighted by respondents as challenging.

“The research approach did appear to influence what particular aspects of PPI were highlighted by respondents as challenging.”

Basic science or tissue bank design studies appear to have a limited role for PPI.

“Basic science or tissue bank design studies appear to have a limited role for PPI.”

There were similar proportions of studies having PPI in the older studies (as measured by being 'closed' on the UKCRN portfolio) and in the open studies.

“There were similar proportions of studies having PPI in the older studies (as measured by being 'closed' on the UKCRN portfolio) and in the open studies.”

It was therefore not possible to make an accurate assessment about PPI models.

“It was therefore not possible to make an accurate assessment about PPI models.”

A surprising finding was the existence of a dual role.

“A surprising finding was the existence of a dual role.”

Low levels of involvement were reported in data analysis, data collection and report writing in either the scoping study or the survey.

“Low levels of involvement were reported in data analysis, data collection and report writing in either the scoping study or the survey (Table 10).”

There is little difference in the overall range of activities between each topic area.

“However, Figure 7 shows that there is little difference in the overall range of activities between each topic area.”

Researchers' descriptions of 'usual suspects' as lay representatives reflected a perception of concerns about the professionalisation of PPI.

“In contrast, researchers' descriptions of 'usual suspects' as lay representatives reflected a perception of concerns about the professionalisation of PPI.”

Seventy-nine per cent of researcher respondents said they knew of a local or regional and public group that had supported research.

“Seventy-nine per cent of researcher respondents said they knew of a local or regional and public group that had supported research.”

PPI was most likely to be embedded when hosted in well-resourced research centres with a dedicated budget and PPI co-ordinator.

“It was clear from the case studies that PPI was most likely to be embedded when hosted in well-resourced research centres with a dedicated budget and PPI co-ordinator.”

The most frequent type of answer given was 'the design stage/at the beginning/as early as possible'.

“The most frequent type of answer given was 'the design stage/at the beginning/as early as possible'.”

We also found evidence in the survey and case studies that a dual role of being both participant and lay representative was being undertaken.

“We also found evidence in the survey and case studies that a dual role of being both participant and lay representative was being undertaken.”

In both the scoping and the survey, IDD studies, compared with the other topic areas, had the highest percentage of studies with evidence of the presence of PPI (scoping 91%; survey 100%).

“In both the scoping and the survey, IDD studies, compared with the other topic areas, had the highest percentage of studies with evidence of the presence of PPI (scoping 91%; survey 100%) (Table 9).”

It also exemplified a 'one-off' model whereby PPI was limited to a single task.

“It also exemplified a 'one-off' model whereby PPI was limited to a single task.”

In the survey there was an overall trend for much higher levels of PPI across all topic areas than in the scoping.

“However, in the survey there was an overall trend for much higher levels of PPI across all topic areas than in the scoping.”

This was further influenced by the topic area's history of PPI and the tacit models that had evolved.

“This was further influenced by the topic area's history of PPI and the tacit models that had evolved.”

An unexpected finding was the invisibility of publicly accessible information on PPI in research studies.

“An unexpected finding was the invisibility of publicly accessible information on PPI in research studies.”

The resulting impact was culturally sensitive, participant-focused research designs.

“The resulting impact was culturally sensitive, participant-focused research designs.”

The main way in which the researchers said they had altered their practice had been by consulting PPI at proposal writing stage to 'define research questions' and making objectives 'more realistic' and 'patient-orientated'.

“The main way in which the researchers said they had altered their practice had been by consulting PPI at proposal writing stage to 'define research questions' and making objectives 'more realistic' and 'patient-orientated'.”

The four studies reporting this in the survey suggested the benefit of this was the provision of direct feedback which could be used to improve research processes and outcomes.

“The four studies reporting this in the survey suggested the benefit of this was the provision of direct feedback which could be used to improve research processes and outcomes:”

The findings from the scoping and survey suggest that funder requirements and study design appear to be the biggest influences on the extent of PPI within a study.

“The findings from the scoping and survey suggest that funder requirements and study design appear to be the biggest influences on the extent of PPI within a study.”

Breakdown in this relationship had a profound effect on levels of PPI embeddedness over time.

“Breakdown in this relationship had a profound effect on levels of PPI embeddedness over time.”

An important finding in both the survey and the case studies was the dual role, whereby the lay advisor was also a study participant.

“An important finding in both the survey and the case studies was the dual role, whereby the lay advisor was also a study participant.”

The most common theme that emerged was recruitment and how to gain access to 'representative' service users, mentioned by about one-third of respondents.

“The most common theme that emerged was recruitment and how to gain access to 'representative' service users, mentioned by about one-third of respondents:”

Our analysis showed that various interpretations of PPI were apparent among researchers.

“Our analysis showed that various interpretations of PPI were apparent among researchers.”

Fewer than half of the survey studies had actually costed for PPI in their proposals.

“We also expected that resources would also be an enabler of PPI and yet fewer than half of the survey studies had actually costed for PPI in their proposals.”

In testing whether or not topic area had an influence on the embedding of PPI, we found similarity between the nature of the clinician-patient relationship and the way PPI was enacted.

“In testing whether or not topic area had an influence on the embedding of PPI, we found similarity between the nature of the clinician-patient relationship and the way PPI was enacted.”

Particular PPI structures helped, including an outreach model with either a specific lay person or a co-ordinator acting as a bridge to a broader community.

“Particular PPI structures helped, including an outreach model with either a specific lay person or a co-ordinator acting as a bridge to a broader community.”

Both mixed-methods and clinical trials had higher levels of PPI than basic science (scoping 21%, survey 67%), tissue bank (scoping 33%, survey 43%) and other (observational studies and 'cohort') (scoping 0%, survey 71%).

“Both of them had higher levels of PPI than basic science (scoping 21%, survey 67%), tissue bank (scoping 33%, survey 43%) and other (observational studies and 'cohort') (scoping 0%, survey 71%).”

Studies which were clinical trials had the highest proportion of respondents naming PPI as having the most impact at the design stage.

“Studies which were clinical trials had the highest proportion of respondents naming PPI as having the most impact at the design stage.”

Topic area was therefore not predictive of the range of PPI roles within a study.

“Topic area was therefore not predictive of the range of PPI roles within a study.”

The survey results suggested that a minority of respondents were equating PPI with participation in research.

“However, the survey results suggested that a minority of respondents were equating PPI with participation in research:”

The quality of such connections was important to create the context for involvement.

“The quality of such connections was important to create the context for involvement.”

How PPI was operationalised in individual studies could be traced back to the funder's ethos of PPI.

“In addition, how PPI was operationalised in individual studies could be traced back to the funder's ethos of PPI.”

For case studies not hosted by organisations with existing PPI structures, an alternative was to recruit an ad hoc, project-specific group of people or individuals to support the study as PPI.

“For case studies not hosted by organisations with existing PPI structures, an alternative was to recruit an ad hoc, project-specific group of people or individuals to support the study as PPI.”

In CS08, the PPI respondent was also a study participant.

“This dual role was overt in another arthritis-related study (CS08) (Box 13), where the PPI respondent was also a study participant.”

Clinicians who were also researchers had the added advantage of having ready access to patients as potential PPI representatives.

“Clinicians who were also researchers had the added advantage of having ready access to patients as potential PPI representatives.”

Other funding bodies taking part in the RAPPORT study had only recently started to implement PPI strategies in their organisations.

“The other funding bodies taking part in the RAPPORT study had only recently started to implement PPI strategies in their organisations.”

Funding organisations that received government public funding as part of a national research agenda clearly articulated the reasons for PPI in methodological terms.

“The funding organisations that received government public funding as part of a national research agenda clearly articulated the reasons for PPI in methodological terms:”

The systematic review design required more effort from the research team to ensure that parents were involved and the review was rigorous, compared with the ongoing qualitative research being undertaken in the programme of work.

“The study design (systematic review) (C3), compared with the ongoing qualitative research being undertaken in the programme of work, required more effort from the research team to ensure that parents were involved and the review was rigorous.”

Providing PPI infrastructure often involved having a designated lead for PPI.

“This often involved having a designated lead for PPI.”

Structures and processes for PPI within funder organisations varied and strongly reflected the influence of their core funding source, mission, scale and length of service.

“Structures and processes for PPI within funder organisations themselves inevitably varied and strongly reflected the influence of their core funding source, mission, scale and length of service.”

If the whole team was not engaged with PPI, the loss of the PPI champion had a detrimental impact.

“If the whole team was not engaged with PPI, the loss of the PPI champion had a detrimental impact.”

There was a strong focus on positive relationships that were developed over time and enhanced by social activities.

“There was a strong focus on positive relationships (C5) that were developed over time and enhanced by social activities.”

Continuity of personnel with responsibility for PPI was an issue.

“Continuity of personnel with responsibility for PPI was an issue.”

In CS14, the distinction between being involved and being a research participant was often blurred, with PPI being understood by many respondents as a dual role.

“However, the distinction between being involved and being a research participant was often blurred in this case study, with PPI being understood by many of the respondents as a dual role.”

Case studies funded by NIHR funding streams were required to have PPI as a different way of working within the actual research process because of the emphasis on the methodological importance of PPI.

“Case studies funded by NIHR funding streams were required to have PPI as a different way of working within the actual research process because of the emphasis on the methodological importance of PPI.”

CS21 exemplified fully embedded PPI which was enabled by all eight identified contexts.

“Case study 21 exemplified fully embedded PPI which was enabled by all eight identified contexts.”

Six CMO configurations explained salient case study actions, potential contexts, plausible mechanisms and potential outcomes, both intended and unintended.

“We identified six CMO configurations that explained salient case study actions, potential contexts, plausible mechanisms and potential outcomes, both intended and unintended.”

The difficulties for public health studies in accessing representative PPI were also linked to the setting (e.g. broad communities) in which they took place.

“The difficulties for public health studies was also linked to the setting (e.g.broad communities) in which they took place.”

CS21 demonstrated an 'entwined' model of PPI in which research and PPI were wrapped together much like a double helix.

“It also demonstrated an 'entwined' model of PPI in which research and PPI were wrapped together much like a double helix.”

Two tissue bank studies also had lay representatives who advised but also had donated samples.

“Two tissue bank studies also had lay representatives who advised but also had donated samples.”

Difficulties in accessing 'representative' PPI were particularly marked for public health studies when the focus was on healthy or 'at-risk' populations with particular lifestyles.

“Difficulties in accessing 'representative' PPI were particularly marked for public health studies when the focus was on healthy or 'at-risk' populations with particular lifestyles.”

Having someone to lead and co-ordinate was important in embedding PPI.

“As described in Purpose, structure and role of patient and public involvement, having someone to lead and co-ordinate was important in embedding PPI.”

Even with funder expectations, only a limited number of case studies were reflecting on and evaluating PPI in an ongoing way to improve the quality of PPI processes and outcomes.

“However, even with funder expectations, only a limited number of case studies were reflecting on and evaluating PPI in an ongoing way to improve the quality of PPI processes and outcomes.”

The findings suggested there were eight enabling contexts for the mechanisms.

“The findings suggested there were eight enabling contexts for the mechanisms (Box 5).”

Host organisations with an established PPI infrastructure, such as an established group or panel and access to training, facilitated the purpose, structure and role of PPI.

“It was also facilitated by host organisations with an established PPI infrastructure, such as an established group or panel, and access to training.”

Reimbursement of out-of-pocket expenses and time contributed to maintaining trust.

“Reimbursement of out-of-pocket expenses and time contributed to maintaining trust.”

Having adequate resources to maintain the PPI group in terms of administration and reimbursement was vital for sustained engagement with PPI in research.

“Having adequate resources to maintain this group in terms of administration and reimbursement was vital for sustained engagement with PPI in research.”

At the time of data collection, some topic areas (dementia and diabetes mellitus) provided this enabling context with network support for PPI.

“At the time of data collection, this enabling context was provided by some topic areas (dementia and diabetes mellitus) with network support for PPI.”

Three additional enabling contexts (training, positive relationships, positive experience) became visible during data collection and analysis.

“The other three (training, positive relationships and positive experience) became visible during data collection and analysis.”

The entwined model generated some research programmes that would not have been run without lay involvement.

“This generated some research programmes that would not have been run without lay involvement.”

Novice researchers were more likely to be unable to fully distinguish PPI as separate from research participation.

“Novice researchers were more likely to be unable to fully distinguish PPI as separate from research participation.”

For some case studies, the purpose, structure and role for PPI were an inherent part of the research design.

“For some case studies, the purpose, structure and role for PPI were an inherent part of the research design.”

Funder requirements played a key role in influencing levels of reimbursement.

“We found that funder requirements played a key role in influencing levels of reimbursement.”

A major finding from interviews with funder representatives was that their stances or values (and thus PPI structures and processes) were actively shaped by the origin of their funding and their histories.

“A major finding from the interviews with funder representatives was that their stances or values (and thus PPI structures and processes) were actively shaped by the origin of their funding and their histories.”

The need for time to develop a PPI infrastructure that could support embedded PPI was also clear in CS22.

“The need for time to develop a PPI infrastructure that could support embedded PPI was also clear in CS22 (Box 7).”

In CS13, the dual role was carefully constructed and managed in a longitudinal cohort study.

“In contrast, this dual role was carefully constructed and managed in a longitudinal cohort study (CS13, see Box 16).”

Host organisations, and in particular CLAHRCs, were instrumental in providing a PPI infrastructure.

“Host organisations, and in particular CLAHRCs, were instrumental in providing a PPI infrastructure.”

Medical charities reliant on fundraising spoke of PPI in terms of an ethical rationale.

“In contrast, the medical charities reliant on fundraising spoke of PPI in terms of an ethical rationale:”

A characteristic of studies with fully embedded PPI was that opportunities for PPI in all parts of the research process were fully exploited.

“A characteristic of studies with fully embedded PPI was that opportunities for PPI in all parts of the research process were fully exploited.”

The benefits of PPI representatives having the opportunity to get to know each other also provided support later in the project.

“The benefits of PPI representatives having the opportunity to get to know each other also provided support later in the project.”

There was a varying sense of equality within the committee.

“As shown in CS04 (see Box 22), there was a varying sense of equality within the committee:”

Sustained feedback to PPI representatives was a clear enabler of trust and continued engagement.

“Sustained feedback to PPI representatives was a clear enabler of trust and continued engagement:”

Those case studies which had high levels of PPI activity could achieve this through flexibility and allowing PPI representatives to 'dip in and out' as their other commitments also varied.

“Those case studies which had high levels of PPI activity could achieve this through flexibility and allowing PPI representatives to 'dip in and out' as their other commitments also varied:”

There was also some evidence that researchers felt concerned that some PPI representatives who lacked scientific understanding could jeopardise a study at the design stage.

“There was also some evidence that researchers felt concerned that some PPI representatives who lacked scientific understanding could jeopardise a study at the design stage.”

Researchers often reported that the pressure to recruit research participants (during data collection) resulted in less capacity to enable involvement of PPI representatives.

“Researchers often reported that the pressure to recruit research participants (during data collection) resulted in less capacity to enable involvement of PPI representatives.”

We found that the relationship between the researchers and PPI representatives was a key factor for successful PPI.

“We found that the relationship between the researchers and PPI representatives was a key factor for successful PPI.”

The distinctive characteristic of case studies that enabled involvement at all stages of the research process was the presence of a dedicated individual facilitating PPI (e.g. CS21, see Box 6).

“The distinctive characteristic of case studies that enabled involvement at all stages of the research process was the presence of a dedicated individual facilitating PPI (e.g.CS21, see Box 6).”

This was often enabled by the host organisation, such as a CLAHRC or research centre.

“This was often enabled by the host organisation, such as a CLAHRC or research centre.”

One finding from the case studies was that the majority of PPI representatives had a previous working relationship with the research team.

“One finding from the case studies was that the majority of PPI representatives had a previous working relationship with the research team.”

Only two of the case studies provided any evidence of systematically appraising the PPI contribution to their study.

“Only two of the case studies provided any evidence of systematically appraising the PPI contribution to their study.”

Performing the necessary tasks for PPI appeared limited to certain stages of the research cycle.

“Performing the necessary tasks for PPI appeared limited to certain stages of the research cycle.”

This was further enabled by having a pool of lay representatives (C6) with sufficient interest and expertise to contribute effectively at all stages of the research.

“This was further enabled by having a pool of lay representatives (C6) with sufficient interest and expertise to contribute effectively at all stages of the research.”

There were reservations from some researchers on the potential for PPI in the early work of basic science designs (C3).

“There were reservations from some researchers on the potential for PPI in the early work of basic science designs (C3).”

The relationship between the researchers and the PPI representatives was key to fully embedding PPI within the research study.

“The relationship between the researchers and the PPI representatives was key to fully embedding PPI within the research study.”

Despite it often being seen as good practice, there was little evidence that this relational integration was being formalised through job descriptions or contracts for the PPI representatives.

“Despite it often being seen as good practice, there was little evidence that this relational integration was being formalised through job descriptions or contracts for the PPI representatives.”

Among the five case studies reimbursing for time, the amounts ranged from £25 per meeting to £75.

“Among the five case studies reimbursing for time, the amounts ranged from £25 per meeting to £75.”

In more formal advisory or steering committee meetings, PPI representatives spoke of being specifically welcomed into the meeting (sometimes being met beforehand).

“In more formal advisory or steering committee meetings, PPI representatives spoke of being specifically welcomed into the meeting (sometimes being met beforehand).”

Case study 15 exemplified how, even when resources were minimal for PPI, the very existence of mutual trust resulted in positive PPI impact.

“Case study 15 (Box 19) exemplified how, even when resources were minimal for PPI, the very existence of mutual trust resulted in positive PPI impact.”

Trust in CS16 was lost because of a lack of formal agreements about access and ownership of a database.

“However, trust in CS16 (Box 20) was lost because of a lack of formal agreements about access and ownership of a database.”

Hence, the host organisations that managed reimbursement and arrangements seamlessly made lay representatives feel a collegial partner with the organisation.

“Hence, the host organisations that managed reimbursement and arrangements seamlessly made lay representatives feel a collegial partner with the organisation.”

Many researchers felt comfortable involving PPI representatives in the general review of study documents but not so certain about involvement in other parts of the research process.

“Many researchers felt comfortable involving PPI representatives in the general review of study documents but not so certain about involvement in other parts of the research process.”

The forum in which researchers and lay representatives came together was also an important enabling context for trust.

“The forum in which researchers and lay representatives came together was also an important enabling context for trust.”

Being unable to meet in person also caused problems in CS18, where the CI and lay representative were in different regions.

“As presented in Strategies for recruiting lay representatives, being unable to meet in person also caused problems in CS18, where the CI and lay representative were in different regions.”

There was evidence of an ongoing debate about whether or not the experience of a condition (direct or indirect) was seen as less legitimate because of that person's professional background, experience or training in research or health care.

“There was evidence of an ongoing debate about whether or not the experience of a condition (direct or indirect) was seen as less legitimate because of that person's professional background, experience or training in research or health care:”

In contrast, host organisations (C7) that had embedded PPI infrastructures were more likely to evaluate and fine-tune PPI processes in a collaborative way.

“In contrast, host organisations (C7) that had embedded PPI infrastructures were more likely to evaluate and fine-tune PPI processes in a collaborative way.”

There was evidence that this focus on evaluation, although not fully assessing and comparing impact, was resulting in some modification to the funder's PPI work, for example developing advice to support lay representatives on trial steering committees.

“There was evidence that this focus on evaluation, although not fully assessing and comparing impact, was resulting in some modification to the funder's PPI work, for example developing advice to support lay representatives on trial steering committees.”

Approaches to evaluating the outcomes and impact of PPI were shaped by the funding organisation's stance towards PPI.

“approaches to evaluating the outcomes and impact of PPI were shaped by the funding organisation's stance towards PPI.”

In addition to the outcomes which were identified in terms of the research project, the researchers and PPI representatives also identified personal outcomes associated with their PPI role.

“In addition to the outcomes which were identified in terms of the research project, the researchers and PPI representatives also identified personal outcomes associated with their PPI role.”

These five case studies were continuing to work with their lay representatives to generate further research questions.

“These five case studies were continuing to work with their lay representatives to generate further research questions.”

In a longitudinal cohort study (CS14) there was evidence that being a case study within the RAPPORT study provided the opportunity for participants to reflect on PPI and plan things differently for the future.

“in a longitudinal cohort study (CS14) there was also evidence that being a case study within the RAPPORT study provided the opportunity for participants to reflect on PPI and plan things differently for the future”

Data collection tools such as questionnaires were 'softened' by PPI representatives in terms of language and sensitivity (CS09, CS21).

“Data collection tools such as questionnaires were 'softened' by PPI representatives in terms of language and sensitivity (CS09, CS21).”

Lay representatives were also used as a link to recruit study participants.

“Lay representatives were also used as a link to recruit study participants.”

PPI representatives made comments on the protocol and design of the study in 13 of the case studies, particularly regarding lay summaries, data collection instruments and study information.

“Patient and public involvement representatives made comments on the protocol and design of the study in 13 of the case studies, particularly regarding lay summaries, data collection instruments and study information (Box 31).”

In those studies which contained an intervention as part of their study design, PPI representatives helped to design or make suggestions to existing interventions to make them more acceptable/feasible.

“In those studies which contained an intervention as part of their study design, PPI representatives helped to design or make suggestions to existing interventions to make them more acceptable/feasible.”

A positive experience of PPI was often the start of the process to establish PPI as normal practice.

“A positive experience of PPI was often the start of the process to establish PPI as normal practice”

Three main models of PPI were identified: a bounded, one-off model, an outreach model and a fully intertwined model.

“Three main models of PPI were identified: a bounded, one-off model, an outreach model and a fully intertwined model.”

In CS07 PPI representatives were involved in the validation of 260 pre-and post-questionnaires (which were used as part of the intervention).

“In another case study (CS07) PPI representatives were involved in the validation of 260 pre-and post-questionnaires (which were used as part of the intervention).”

Researchers commonly attributed recruitment success to the PPI input to materials (suggestions on layout, size, language, colour, length, readability, clarity and design; evidenced by tracked changes/before and after versions of documents) used for recruitment (PIS, invitation letters, consent forms).

“Researchers commonly attributed recruitment success to the PPI input to materials (suggestions on layout, size, language, colour, length, readability, clarity and design; evidenced by tracked changes/before and after versions of documents) used for recruitment (PIS, invitation letters, consent forms).”

Our tracking interviews suggested that, in a number of cases where it had been planned that PPI representatives would be involved in dissemination, this had not happened.

“our tracking interviews suggested that, in a number of cases where it had been planned that PPI representatives would be involved in dissemination, this had not happened.”

One of the most frequently stated outcomes for PPI in our case studies was a perceived increase in recruitment rates to the research study (18 of the 22 case studies).

“One of the most frequently stated outcomes for PPI in our case studies was a perceived increase in recruitment rates to the research study (18 of the 22 case studies).”

Funders' requirements (C1) were a strong enabling context for evaluation.

“Funders' requirements (C1) were a strong enabling context for evaluation.”

Among the eight completed case studies, we found no evidence of any immediate outcomes of PPI for the research findings.

“Among the eight that were completed, we found no evidence of any immediate outcomes of PPI for the research findings.”

• AssumptionRests on an assumption: The case studies included in RAPPORT are sufficient to identify the impacts and outcomes of PPI in NIHR-funded research.

There was no systematic appraisal of the observable impact of PPI by condition-specific medical charities.

“However, there was no systematic appraisal of this.”

Where managed carefully, such experience for both research teams and lay representatives was also evidenced in the RAPPORT study.

“Where managed carefully, such experience for both research teams and lay representatives was also evidenced in the RAPPORT study.”

Our findings suggested that, in a minority of case studies, simply providing feedback on, for example, the study results, despite being one-way information giving, was seen by some participants as active involvement.

“Our findings suggested that, in a minority of case studies, simply providing feedback on, for example, the study results, despite being one-way information giving, was seen by some participants as active involvement.”

In CS02, PPI views (through a national charity panel) changed the application to include a safety questionnaire.

“PPI views (through a national charity panel) changed the application to include a safety questionnaire”

Some case studies agreed to be part of the RAPPORT study as an aid to meet the requirement to report on PPI.

“some case studies agreed to be part of the RAPPORT study as an aid to meet this requirement.”

Echoing the earlier survey of Barber et al., the RAPPORT survey findings suggested that some researchers still have difficulty in distinguishing between research participation and PPI activities.

“Echoing the earlier survey of Barber et al., 163 the RAPPORT survey findings suggested that some researchers still have difficulty in distinguishing between research participation and PPI activities.”

• cites Barber et al., 163Not checked yet

  Not checked in the free run.

There was evidence that researchers perceived an unintended outcome of PPI.

“However, there was also evidence that researchers perceived an unintended outcome of PPI.”

A number of researchers also felt PPI was worthwhile to them personally beyond impact just on the research process and outcomes.

“A number of researchers also felt PPI was worthwhile to them personally beyond impact just on the research process and outcomes.”

The requirement to report on PPI when NIHR-funded projects were completed was a major incentive for researchers to review processes and benefits of PPI in the project.

“the requirement to report on PPI when NIHR-funded projects were completed was a major incentive for researchers to review processes and benefits of PPI in the project.”

Involving patients' groups in the governance of a study had evidence of potentially improving patient safety.

“Involving patients' groups in the governance of a study had evidence of potentially improving patient safety.”

As presented in each CMO configuration, case study actions led to a number of methodological and moral impacts and outcomes.

“As presented in each CMO configuration, case study actions led to a number of methodological and moral impacts and outcomes.”

• AssumptionRests on an assumption: The case studies included in RAPPORT are sufficient to identify the impacts and outcomes of PPI in NIHR-funded research.

Having developed over time, embedded PPI infrastructures also enabled positive relationships (C5) in which researchers and lay members could reflect honestly on what and what had not worked well.

“Having developed over time, embedded PPI infrastructures also enabled positive relationships (C5) in which researchers and lay members could reflect honestly on what and what had not worked well.”

The respondents most likely to confuse these terms were those less involved in submitting grant applications, such as research assistants or research nurses.

“The respondents most likely to confuse these terms were those less involved in submitting grant applications, such as research assistants or research nurses.”

PPI as a dynamic process was significantly compromised when relationships broke down.

“PPI as a dynamic process was significantly compromised when relationships broke down.”

PPI representatives also made a contribution to data analysis.

“Patient and public involvement representatives also made a contribution to data analysis (Box 33).”

Case studies involving marginalised groups, such as those with learning disabilities and IDDs, did not have particular problems in recruiting PPI representatives thanks to the existing infrastructure for PPI developed in these areas.

“Interestingly, those of our case studies which involved marginalised groups, such as those with learning disabilities and IDDs, did not have particular problems in recruiting PPI representatives thanks to the existing infrastructure for PPI developed in these areas.”

There was qualifying evidence that outcome measures had been changed or added to in the research design.

“There was qualifying evidence here that the outcome measures had been changed or added to in the research design.”

Patient and public involvement within a study was also partially shaped by the lay recruitment method.

“Patient and public involvement within a study was also partially shaped by the lay recruitment method.”

A third stance (policy) was the initial trigger for many of the researchers and the centrally funded funders to engage with PPI.

“A third stance (policy) described in Chapter 2, was the initial trigger for many of the researchers and the centrally funded funders to engage with PPI.”

The quest to find a person (or a group) with characteristics as close as possible to a target study population was particularly challenging for research teams in public health studies, where the target population would not necessarily see themselves as having a health condition or need.

“The quest to find a person (or a group) with characteristics as close as possible to a target study population 173 was particularly challenging for research teams in public health studies, where the target population would not necessarily see themselves as having a health condition or need.”

• cites [173] Beresford, 2005, Developing the theoretical basis for service user/survivor-led research and equal involvement in research, Epidemiologia e Psychiatria Sociale sourceNot checked yet

  Not checked in the free run.

Some case studies from the IDD topic area clearly presented a strongly coherent moral stance on PPI.

“In contrast, some case studies from the IDD topic area clearly presented a strongly coherent moral stance on PPI.”

Analysis of the data from these respondents suggested three pathways.

“Analysis of the data from these respondents suggested three pathways (see Figures 9 and 10).”

Many of these groups were able to reimburse lay members for work in developing grant applications, a particularly challenging issue raised by many that could then be addressed.

“Many of these groups were also able to reimburse lay members for work in developing grant applications, a particularly challenging issue raised by many 17 that could then be addressed.”

• cites [17] Staniszewska, 2007, User involvement in the development of a research bid: barriers, enablers and impacts, Health Expect sourceNot checked yet

  Not checked in the free run.

This often shifted to a subsequent 'conversion' to PPI when researchers appraised outcomes of PPI as having had a benefit on the research outcomes.

“However, this often shifted to a subsequent 'conversion' to PPI when they appraised outcomes of PPI as having had a benefit on the research outcomes.”

How often PPI representatives were involved in successive research stages in our case studies could be presented as a decreasing double bell shape, with activity focused on project set-up and to a lesser extent the dissemination phase.

“How often PPI representatives were involved in successive research stages in our case studies could be presented as a decreasing double bell shape, with activity focused on project set-up and to a lesser extent the dissemination phase.”

Overall quality of research improved through PPI.

“Overall quality of research improved”

The most common forum in our case studies was some form of study management committee.

“The most common forum in our case studies was some form of study management committee.”

Four case studies utilised PPI to provide links to the wider community.

“Four case studies utilised PPI to provide links to the wider community.”

A frequently found narrative was of researchers 'having to do' PPI because of revised governance requirements.

“A frequently found narrative, illustrated in Figure 9, was of researchers 'having to do' PPI because of revised governance requirements.5”

• cites [5] Department of Health, 2005Not checked yet

  Not checked in the free run.

Five of the case studies reported that PPI representatives had been involved in research priority setting.

“Five of the case studies reported that PPI representatives had been involved in research priority setting.”

The PPI costs required in a grant application proved a surprise to a minority of researchers.

“However, the PPI costs required in a grant application proved a surprise to a minority of researchers, suggesting that further work needs to be done in helping researchers understand the resource implications of supporting PPI.”

In CS02, the single lay representative on the steering committee consulted with a wider range of people from the study target population.

“Case study 02 exemplified this approach, with the single lay representative on the steering committee consulting with wider range of people from the study target population.”

Varying the methods of PPI recruitment was one means by which researchers tried to address the issue of PPI representativeness.

“Varying the methods of PPI recruitment was one means by which researchers tried to address the issue of PPI representativeness, an often-questioned issue.”

The RAPPORT study was provided evidence of PPI impact in identifying research questions.

“We were provided evidence of PPI impact in identifying research questions.”

Research teams advocated the use of PPI for outcome measures that were relevant to patients.

“Research teams advocated the use of PPI for outcome measures that were relevant to patients; this was particularly relevant to one arthritis study:”

Two main stances, methodological and moral, appeared to shape the ways in which respondents viewed and operationalised PPI.

“Two main stances, methodological and moral, appeared to shape the ways in which our respondents viewed and operationalised PPI.”

Two studies had PPI representatives involved directly in data collection.

“Two studies had PPI representatives involved directly in data collection.”

In many of these committees lay representatives felt well supported and that their voice was heard, often primarily enabled by relational work.

“In many of these committees lay representatives felt well supported and that their voice was heard, often primarily enabled by relational work (discussed in Contextual enablers for patient and public involvement: relationships).”

We had no evidence of any of our lay respondents 'shopping around' for research projects to be involved in.

“Although we had no evidence of any of our lay respondents 'shopping around' for research projects to be involved in, it could also be seen that one of the condition-specific charities had framed the end-user of the research as being the consumer, and consequently including both patient and clinician as their 'user representatives'.”

The RAPPORT data suggested a far more dynamic framework than that of Oliver et al., 132, recognising subtle changes over time.

“However, the RAPPORT data suggested a far more dynamic framework (Figure 11) than that of Oliver et al., 132 recognising subtle changes over time.”

• cites Oliver et al., 132Not checked yet

  Not checked in the free run.

Despite the growing body of literature, there is relatively little evidence of PPI's impact and outcomes.

“Despite the growing body of literature outlined in the previous section, there is relatively little evidence of PPI's impact and outcomes.99”

• cites [99] Jagosh, 2014, Critical reflections on realist review: insights from customizing the methodology to the needs of participatory research assessment, Res Synthesis Methods sourceNot checked yet

  Not checked in the free run.

Oliver et al. found detailed reports of 87 examples of PPI influencing the identification or prioritisation of research topics.

“They found detailed reports of 87 examples of PPI influencing the identification or prioritisation of research topics.”

• cites [102] Oliver, 2004, Involving consumers in research and development agenda setting for the NHS: developing an evidence-based approach, Health Technol Assess sourceNot checked yet

  Not checked in the free run.

The RAPPORT study design, data collection, analysis, management and dissemination were conducted in partnership with lay co-applicants/co-researchers, service user reference groups and lay members of the study advisory group.

“The RAPPORT study design, data collection, analysis, management and dissemination have been conducted in partnership with two lay coapplicants/ co-researchers, one co-researcher, two service user reference groups and four lay members of the study advisory group.”

One reason for the weak evidence on PPI impact is the difficulty that confused terminology or poor reporting poses for finding evidence of PPI impact in the literature.

“One reason for this is the difficulty that confused terminology or poor reporting poses for finding evidence of PPI impact in the literature.”

In addition, the majority of included reports were descriptive accounts written by researchers rather than service user representatives.

“In addition, the majority of included reports were descriptive accounts written by researchers rather than service user representatives.”

• cites [102] Oliver, 2004, Involving consumers in research and development agenda setting for the NHS: developing an evidence-based approach, Health Technol Assess sourceNot checked yet

  Not checked in the free run.

The case study approach provided the methodological flexibility to generate theoretical insights from the results.

“It also provided the methodological flexibility to generate some theoretical insights from our results.147”

• cites [147] Marinetto, 1998Not checked yet

  Not checked in the free run.

A co-researcher's independent inductive coding added to the rigour and validity of data analysis as similar themes emerged.

“This co-researcher devised her own set of inductive codes and analysed the transcripts using her own coding system.The use of this separate coding process added to the rigour and validity of data analysis as similar themes emerged:”

The system of checking and critiquing the analysis was a way of more deeply interrogating the data and exploring alternative explanations, not merely promoting consensus.

“This system of checking and critiquing the analysis was not merely to promote consensus but was a way of more deeply interrogating the data and exploring alternative explanations.157”

• cites [157] Barbour, 2001, Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?, BMJ sourceNot checked yet

  Not checked in the free run.

The survey findings suggested that certain topic areas and study designs have different influences on researchers when working with PPI representatives.

“In terms of NPT, this highlights the construct of collective action where the necessary tasks for PPI are performed; as per our hypothesis, the survey findings suggested that certain topic areas and study designs have different influences on researchers when working with PPI representatives.”

In applying the NPT framework to the findings there were indications of barriers to embedding PPI.

“However, in applying the NPT framework to the findings there were indications of barriers to embedding PPI.”

It was also indicative of a lack of training or experience in PPI, particularly in novice researchers.

“It was also indicative of a lack of training or experience in PPI, particularly in novice researchers.”

Funders influenced PPI in subtly different ways depending on their organisational values.

“Hence, funders influenced PPI in subtly different ways depending on their organisational values.”

This level of resources was necessary for an entwined model of PPI.

“This level of resources was necessary for an entwined model of PPI (see Chapter 7).”

This move towards 'virtual' contact as opposed to 'face to face' has implications for the establishment of relationships.

“However, this move towards 'virtual' contact as opposed to 'face to face' has implications for the establishment of relationships.”

A relationship framed by trust was the minimum requirement for PPI to have a positive impact.

“A relationship framed by trust was the minimum requirement for PPI to have a positive impact.”

As an enabling context for embeddedness, it was not necessarily how the PPI was organised but how much PPI representatives were encouraged to be involved and were exposed to different areas of the study throughout its stages where they could have an impact.

“As an enabling context for embeddedness, it was not necessarily how the PPI was organised but how much PPI representatives were encouraged to be involved and were exposed to different areas of the study throughout its stages where they could have an impact.”

If PPI is limited to engagement, this precludes the full potential of involvement.

“If PPI is limited to engagement, this precludes the full potential of involvement.”

Longer-term outcomes influenced by PPI need an extensive time to detect.

“It is clear that, as in all evaluation of research impact, longer-term outcomes influenced by PPI need an extensive time to detect.”