Patient involvement (PI) has become a key priority to the Spanish Network of Agencies for Assessing National Health System Technologies and Performance (RedETS). As part of the national strategy to promote PI, an interest group was created in 2017 to share knowledge, develop methodologies and standardize PI processes. The aim of this work is to analyze the main activities of the Patient Interest Group 5 years after its launch and to reflect on possible needs and challenges.

Narrative description and an in-depth analysis of the main activities of the Interest Group from 2017 to the present.

The group is composed of HTA researchers from the 8 regional agencies in Spain and is supported by the Ministry of Health and the RedETS council. It currently has the participation of 26 researchers, organized into different working subgroups. The initial lines of work were the analysis of the situation, the development of procedures, and the initiation of training materials for patients on HTA. At present, the main projects are the development of metrics for evaluating the impact of patient participation, the development of procedural materials to promote methodological process standardizaton (e.g., a flowchart with the main process steps, checklists, templates), and the design and piloting of virtual training for patients in HTA. New lines include the analysis of the ethical challenges of PI and the feasibility of setting up an HTA patient registration system and a patient forum to facilitate participation. In addition, the interest group has promoted the exchange of relevant information for PI and the organization of capacity building activities.

The RedETS Patient Interest Group is encouraging the development of activities, reflection on collective experiences, and tools that facilitate PI in Spain. Among the main challenges are the need to ensure the quality and applicability of PI and to analyze the views of patients who have actively participated in HTAs.

The aim of this scoping review is to map the available evidence about the use of health technology assessment (HTA) in the assessment of whole genome sequencing (WGS).

A scoping review methodology was adopted. The population, concept, and context framework was used to build up the research question and to establish the eligibility criteria. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews was adopted to implement a comprehensive search strategy. Evidence was retrieved from scientific databases and HTA organizations Web sites. Reports were classified as full HTA, mini-HTA, rapid reviews or other.

The search strategy identified seven reports. Five HTA organizations from five countries elaborated the reports: one full HTA, four rapid reviews, and two classified as others. The reports were mainly focused on the evaluation of the clinical utility and cost-effectiveness of genome-wide sequencing as well as informing policy questions by providing analyses of organizational and ethical considerations.

Few HTA organizations are drafting reports for WGS. It is essential to stimulate a critical reflection during the elaboration of HTA reports for WGS to steer choices of decision makers in the establishment of priorities for research and policy and reimbursement rates.

Patient preferences (PPs) are an important source of evidence in health technology assessment (HTA). However, a methodological framework to achieve their integration in decision-making is lacking. We aim to investigate the potential role of evaluative frameworks to integrate PP evidence into HTA and decision-making.

We undertook a scoping review to identify potential methodological frameworks to consider PP evidence in HTA and evidence of the acceptability of these frameworks for decision-makers. We searched PubMed, Cochrane, and the grey literature to identify relevant studies, reports, or guidance documents. We restricted our search to the use of PP rather than patient experience data and excluded articles solely relating to deliberative approaches.

Frameworks identified as having the potential to integrate PP evidence included cost-utility analysis, cost-consequence analysis (CCA), the efficiency-frontier approach, and multi-criteria decision analysis. All have been used in various HTA contexts, but not necessarily for inclusion of PP evidence. Distinct benefits and challenges of integrating PP data were identified for each framework. These included the theoretical basis of the frameworks, their ability to consider non-health as well as health outcomes, and their ability to separate outcomes based on PPs from outcomes based on population preferences. There is limited evidence and no consensus on the application of these frameworks to consider PPs in HTA or on their acceptability for decision-makers. However, CCA has the advantage that it is both based on economic decision theory and it leaves patient preferences disaggregated from population preferences in an HTA.

The frameworks identified in this review offer potential approaches to systematically and transparently integrate PPs into HTA and decision-making. Based on the review findings, we propose a research agenda to explore the potential of CCA in particular. We anticipate that our findings will augment the recommendations of the Innovative Medicines Initiative PREFER project, which are expected to report in 2022.

Symptom-checkers are digital health applications (DHA) with diagnostic algorithms. These symptom-checkers claim to improve the diagnostic process and patient guidance. After asking the user to describe the symptoms using a chatbot interface, the symptom-checkers offer a list of potential diagnoses, and/or give recommendations for appropriate action (self-care, doctor’s visit, or emergency care). Because of the growing number and increasing use of these diagnostic DHA, there is a need to evaluate the evidence.

We updated a British evidence synthesis on symptom-checkers from the National Institute for Health Research (NIHR, 2019). For the systematic update search, we selected four databases. The following endpoints were selected: effectiveness, safety, diagnostic accuracy, triage accuracy, organizational and patient-relevant endpoints. For accuracy studies included from the update search, we assessed the risk of bias (RoB) using the quality assessment tool of diagnostic accuracy studies (QUADAS-2).

The NIHR-report included 27 studies. We added 14 additional studies via update search. One randomized-controlled-trial (RCT) reported a prolonged illness duration when using symptom-checkers (statistically non-significant). No harms when using symptom-checkers were identified (six observational studies). The diagnostic accuracy ranged from 14-84.3 percent (ten observational studies), the triage accuracy ranged from 33-100 percent (eleven observational studies). For organizational endpoints, the results were inconsistent (one RCT, six observational studies). The patient perspective indicates a high usability for symptom-checkers, but the limited description of symptoms and the missing verbal interaction with health personnel were mentioned as hindering factors (nine survey-studies). The QUADAS-2 assessment for RoB was low in one, and high in seven studies.

The studies were often conducted using fictitious case-vignettes, limiting the validity of the evidence. Therefore, the results for the diagnostic and triage accuracy are insufficient to demonstrate a benefit in real-world settings. Additionally, there is a concern for misdiagnosis and overdiagnosis. We recommend a continuous monitoring of these diagnostic DHA, using high-quality studies.

To evaluate the cost-effectiveness of the nonsurgical periodontal treatment (NSPT) compared with supragingival therapy in type II diabetics with periodontitis.

A decision tree analysis was used to estimate the costs and health outcomes of two periodontal therapies in a hypothetical cohort of type II diabetics with periodontitis. The analysis was developed from the perspective of a third-party payer at 1 year and 5 years. Probabilities were derived from two systematic reviews. The costs and resource use were validated by a Delphi expert panel. All costs were expressed in USD, using the 25 May 2021 Colombian pesos market exchange rate (USD 1 = COP 3,350).

NSPT was a dominant alternative compared with subsidized supragingival therapy in type II diabetics with periodontitis, generating savings of USD 87 and 400, during the first year or up to 5 years, respectively, and improving dental survival from 32 to 69 percent.

NSPT can generate savings by reducing the complications derived from uncontrolled periodontitis and tooth loss.

Artificial intelligence (AI) is seen as a major disrupting force in the future healthcare system. However, the assessment of the value of AI technologies is still unclear. Therefore, a multidisciplinary group of experts and patients developed a Model for ASsessing the value of AI (MAS-AI) in medical imaging. Medical imaging is chosen due to the maturity of AI in this area, ensuring a robust evidence-based model.

MAS-AI was developed in three phases. First, a literature review of existing guides, evaluations, and assessments of the value of AI in the field of medical imaging. Next, we interviewed leading researchers in AI in Denmark. The third phase consisted of two workshops where decision makers, patient organizations, and researchers discussed crucial topics for evaluating AI. The multidisciplinary team revised the model between workshops according to comments.

The MAS-AI guideline consists of two steps covering nine domains and five process factors supporting the assessment. Step 1 contains a description of patients, how the AI model was developed, and initial ethical and legal considerations. In step 2, a multidisciplinary assessment of outcomes of the AI application is done for the five remaining domains: safety, clinical aspects, economics, organizational aspects, and patient aspects.

We have developed an health technology assessment-based framework to support the introduction of AI technologies into healthcare in medical imaging. It is essential to ensure informed and valid decisions regarding the adoption of AI with a structured process and tool. MAS-AI can help support decision making and provide greater transparency for all parties.

Innovative digital health technologies (DHTs) may present new aspects of value that are not appropriately accounted for in current health technology assessments. In discovering what value means in the context of DHTs, multi-stakeholder collaboration is essential.

A scoping literature review was conducted to identify current value assessment criteria and proposed methodologies across three health systems: United States of America, United Kingdom, and Germany. A Delphi exercise was conducted with stakeholders from the following groups: users, healthcare practitioners, decision-makers, supply-side actors, and influencers. Based on a review of assessment frameworks in the study countries and consultations with experts from each stakeholder group, researchers proposed value constructs in five domains: health inequalities, data rights and governance, technical and security, economic characteristics, clinical characteristics, and user preferences. In Delphi round one, participants commented on the proposed constructs and submitted their own. A thematic analysis identifying key concepts and themes of the participant proposed constructs and comments was used to incorporate this information for round two. Then, participants rated each value construct on an ‘importance’ Likert scale in two decision contexts: user-facing DHTs and system-facing DHTs. In round three, participants were presented with the consensus judgement for each construct, with the opportunity to change their answer. Value constructs with equal to or greater than 70 percent consensus were included in the final framework. Rounds four and five were, respectively, value judgements on a Likert scale and a presentation of consensus for a therapeutic area to test the final framework.

Initially 32 value constructs were proposed by researchers, 20 of which were changed or removed based on round one feedback. Additional constructs were added based on participant suggestions resulting in forty-five value constructs in round two. The final framework will be available after round three closes on 20 December 2022.

The multi-stakeholder Delphi approach ensures that all suggestions and value judgements are weighted equally across stakeholder groups. The resultant value framework can be used to inform policymaking around health technology assessment of DHTs.

We explore how the scope of analysis is defined in health technology assessment (HTA) and economic evaluation and consider the potential implications of these decisions.

The scope of analysis, including decisions about which methods and domains of HTA to include in the assessment, which costs, and health outcomes are most meaningful, and which comparators are the most relevant are typically informed by the needs of the decision-maker. We undertook two systematic scoping reviews to assess: (i) to what extent systems thinking is considered in literature-based technology assessments; and (ii) how the scope of the analysis is defined in economic evaluation using Clostridioides difficile infection as an exemplar. We synthesized the findings from these reviews and offer three key observations for future research and exploration in the field of HTA.

Our scoping reviews found that the scope of analysis in economic evaluations typically focus on single interventions, often ignoring upstream and downstream interventions. Similarly, published technology assessments have narrowly defined and inconsistent scopes of analysis, with limited consideration of indirect health and non-health impacts. Three key observations for the field of HTA include: (i) economic evaluations focus on the value of single heath interventions. A focus on a single health intervention may simplify the analysis; however, will this siloed decision-making lead to optimal health resource allocation? (ii) published assessments have inconsistently defined scopes of analysis. A decision problem that focuses on the needs of the decision-maker is practical; however, will inconsistencies in perspectives across assessments create unfair conceptualizations of value? (iii) HTA is technology-focused, not patient-focused. A technology-focused HTA system aligns with the technology diffusion process; however, does this move us away from the patient-centered mandate of HTA?

The dynamic nature of HTA leads to many conceptualizations of value. Considering the potential implications of narrowly defined, inconsistent, and technology-focused scopes of analyses may have consequences on achieving a patient-centered high-quality health system.

Conducting a cost-effectiveness analysis (CEA) is resource consuming, and therefore the Dutch National Health Care Institute (ZIN) only performs those for interventions with a high budget impact. Sometimes, cost-effectiveness (CE) estimates are clearly below or far above reference values, which makes full cost-effectiveness assessments less vital. The objective of this study was to develop an efficient and simplified method to identify interventions that are clearly cost-(in)effective.

The method makes use of headroom analysis. Several HTA experts and other relevant stakeholders have been asked to provide feedback on a preliminary version of the CE signal.

The method consists of five steps. In the first step (i) the relevant willingness-to-pay threshold is determined. Reference values are used by ZIN for the maximum willingness-to-pay per incremental quality-adjusted life year (QALY), depending on burden of disease.

In next step (ii) the health gain that can realistically be obtained with the new treatment is estimated. Hereby the effect of the intervention on the clinical outcomes, quality of life and gained life years is determined to estimate the number of QALYs gained, including uncertainty. Then (iii) the societal cost maximum (SCM) of the new treatment is calculated by multiplying step 2 with step 1. In step four (iv) the incremental treatment costs are estimated looking at both the costs and savings for both treatments options for the average patient. In the final step (v) the incremental treatment costs are compared to the SCM to determine if the intervention is probably cost effective, probably not cost-effective or if a conclusion cannot be drawn.

This method has proven to be feasible and could be a valuable addition to the current cost-effectiveness assessment toolbox. The CE-signal is being validated against performing a full cost-effectiveness analysis.

Wearable digital health technologies (DHTs) have the potential to improve chronic kidney disease (CKD) management through patient engagement. This study aimed to investigate and elicit preferences of individuals with CKD toward wearable DHTs designed to support self-management of their condition.

Using the results of our review of the published literature and after conducting qualitative patient interviews, five-choice attributes were identified and included in a discrete-choice experiment. The design consisted of 10-choice tasks, each comprising two hypothetical technologies and one opt-out scenario. We collected data from 113 adult patients with CKD stages 3–5 not on dialysis and analyzed their responses via a latent class model to explore preference heterogeneity.

Two patient segments were identified. In all preference segments, the most important attributes were the device appearance, format, and type of information provided. Patients within the largest preference class (70 percent) favored information provided in any format except the audio, while individuals in the other class preferred information in text format. In terms of the style of engagement with the device, both classes wanted a device that provides options rather than telling them what to do.

Our analysis indicates that user preferences differ between patient subgroups, supporting the case for offering a different design of the device for different patients’ strata, thus moving away from a one-size-fits-all service provision. Furthermore, we showed how to leverage the information from user preferences early in the R&D process to inform and support the provision of nuanced person-centered wearable DHTs.

The very high costs of orphan drugs, together with the uncertainties regarding their (cost-)effectiveness raise questions regarding the efficiency and legitimacy of their health technology assessment (HTA) and appraisal process. The aim of the present, qualitative study was to investigate how experts on the reimbursement of these treatments perceive the HTA and appraisal process in their country. Moreover, it aimed to provide specific conditions and practical recommendations for their improvement.

Twenty-two European experts from 19 different countries were included in a qualitative survey and semi-structured interviews. Transcripts were analyzed using the qualitative data analysis software Nvivo. A grounded theory approach was adopted to develop a set of well-defined concepts from the cyclic analysis of the empirical data.

First, analysis of the expert interviews yielded five good practices for an efficient HTA and appraisal of orphan drugs: a high level of transparency, patient involvement, a clear decision-making structure with room for flexibility, mechanisms to minimize bias and an explicit consideration of the opportunity cost. Meanwhile, participants highlighted several barriers to the overall process, such as a lack of trust between the different stakeholders and imbalances in negotiation power. In addition, the results allowed to identify a number of ‘contextual’ determinants that may undermine the legitimacy of the final decision, such as bias and the perverse effects of the orphan drug legislation. Drawing from the experts’ experiences, a toolkit was developed that includes an extensive number of specific recommendations (and conditions) for decision-makers to improve the legitimacy and efficiency of their HTA and appraisal of orphan drugs.

Overall, the results showed that decision-makers should focus on limiting the impact of the contextual determinants rather than improving the methods included in the HTA. This will contribute to further legitimize reimbursement choices for orphan drugs towards the wider public.

On a regular basis, new and innovative medicines come to the market; with uncertainties surrounding their exact effect in patients. To address these uncertainties, disease-specific registries are commonly being used. The Dutch National Health Care Institute (Zorginstituut Netherland [ZIN]) started, in collaboration with stakeholders, a national program on the management and coordination of disease-specific registries. The main goal is to improve the quality and consistency of these registries and thereby increase the value of the data in monitoring innovative, expensive medicines. As a starting point for this program, we performed a study on the current state of these registries in the Netherlands.

Using an initial list of 114 registries, we sent out questionnaires to 58 disease-specific registries that collected information on medicine use. Thirty registries responded and their registries were used for the analysis.

Of 30 registries that responded, 15 registries collect information on orphan medicinal products. Most registries are in the field of internal medicine (n = 8), oncology (n = 6), and rheumatology (n = 5). The size of the registries ranged between 250 patients (orphan diseases) and more than 10,000 patients (oncology). Only 13 registries collect information on patient reported outcomes. Data collection is mostly performed manually and standard coding systems such as Systemized Nomenclature of Medicine (SNOMED), Logical Observation Identifiers, Names, and Codes (LOINC), and The International Classification of Diseases 10th Revision (ICD-10) are not routinely used. Finally, our results show that most registries are (partly) dependent on the funding of pharmaceutical companies.

Our assessment shows variation in the type, goal of, governance, and funding of the disease-specific registries investigated. We believe that these results show the importance of further national coordination of the disease-specific registries to increase the usability of their data to address the uncertainties surrounding these innovative, expensive medicines.

The National Institute for Health and Care Excellence (NICE) interventional procedures programme (IPP) issues guidance on the safety and efficacy of new interventional procedures (IPS). The IPP considers a range of evidence from randomized controlled trials (RCTs) to case series to make final recommendations. Real-world evidence (RWE) can provide additional information on long-term outcomes and patient population characteristics that are not easily captured by RCTs. This study explores the impact of RWE in complementing RCTs on long-term safety and efficacy for national guidance development.

We review the impact of RWE in IPS guidance (IPG) updates and change in guidance recommendations in the last 5 years. This is done by analyzing NICE guidance updates and supporting evidence. A range of RWE was considered in the supporting evidence, for example, registries and clinical audits.

The addition of RWE evidence to IPP guidance update has led to significant changes in the recommendations. For example, in one recent IPG, standard arrangements recommendation based on short-term RCT evidence was changed to a research recommendation when RWE showed an increase in long-term adverse effects. In another recent IPG update, special arrangements recommendation based on short-term RCT evidence was changed to a research recommendation when RWE reported long-term inferior efficacy and safety for the new intervention compared to current standard of care. A complete overview of results of the last 5 years will also be presented at the meeting.

These findings indicate that increased availability of RWE in HTA has the potential to impact national guidance recommendations. In addition, it shows how RWE can fill the evidence base gaps created by RCTs. Such data can confirm or contradict the findings of RCTs, or generate questions needing further research, or support disinvestment in non-effective technologies.

Several diagnostic tests are often adopted into diagnostic pathways for specific indications without strong evidence to support their use. In this context, real-world prospective cohort studies in combination with decision modelling can generate evidence to support decision-making. The Early Detection of neovascular Age-Related Macular Degeneration (EDNA) study was a prospective cohort designed to assess the diagnostic accuracy and cost-effectiveness of several diagnostic monitoring tests used in routine practice for the detection of neovascular age-related macular degeneration (nAMD) in the second eye of patients being treated for unilateral disease.

Five-hundred and fifty-two participants with newly diagnosed unilateral nAMD were monitored for up to 3 years in 24 UK eye clinics. The diagnostic monitoring performance of five index tests was compared: self-reported change in visual function, Amsler test, clinic measured change in visual acuity, fundus assessment by clinical examination or colour photography, and spectral-domain optical coherence tomography (SD-OCT). The reference standard was fundus fluorescein angiography (FFA). A patient-level state transition model was used to simulate the onset of nAMD in the second eye, and assess the impact of different tests on the timing of detection and treatment, and associated costs and quality adjusted life years (QALYs) over a 25-year time-horizon.

One hundred and forty-five (26.3%) patients developed active nAMD in the study eye, of whom 120 had an FFA at detection. SD-OCT had the highest sensitivity (91.7 percent (95% CI: 85.2-95.6) and provided high specificity (87.8% (95% CI: 83.8-90.9)). It generated more QALYs and lower health and personal social care costs compared to all other monitoring tests. The combination of SD-OCT with fundus-examination provided a marginal increase in sensitivity over OCT alone, but the associated incremental cost-effectiveness ratios was >GBP 100,000 per QALY.

The efficiency of diagnostic pathways for nAMD may be improved by using SD-OCT alone to monitor the second eye of people being treated for unilateral disease. Prospective cohort studies embedded into routine practice offer value for informing decisions surrounding the use of technologies already in routine use.

The French health technology assessment (HTA) agency initiated a research between 2018 and 2019 with the aim of determining whether other HTA organizations (agencies, bodies, institutes, and expert networks) and researchers had developed an evaluation framework of organizational impacts (OIs).

Three types of investigation were carried out: (i) an analysis of documents published by selected HTA organizations, (ii) a rapid review on the OI issues, (iii) a questionnaire survey to experts of the International Network of Agencies for Health Technology Assessment.

The analyses highlight six key points: (i) there is no explicit conceptual definition of OIs; (ii) OIs are often not included in a specific dimension of the evaluation or in the same dimensions; (iii) three recurring categories emerge from the assessment of OIs: processes, structure, and culture; (iv) despite its limitations, the European Network for Health Technology Assessment framework (Core Model) is the most mature assessment model to date; (v) the question of the scope of OIs to be considered is unresolved (micro-meso-macro); and (vi) the delineation between OI assessment and economic assessment must be addressed.

Although the issue of considering OI in HTA has been raised for many years, it remains largely unresolved. Defining the concept of OI is a prerequisite for taking the next step toward an evaluation framework. As the question of the impact of innovation goes beyond the health sector, extensive research on how to define and take into account these OIs may be relevant.