Deakin Health Economics, Centre for Population Health Research, Faculty of Health, Deakin University, Victoria, Australia.
Inclusion of economic evaluations alongside cancer clinical trials necessitates the collection and analysis of resource utilisation and cost data alongside outcomes. The purpose of this paper is to describe and discuss the measurement of cost in clinical studies, particularly resource utilisation. Cost data collection can be conducted retrospectively through linkage of treatment data with claims data, such as Medicare, or by patient recall (questionnaires). Prospective approaches include the patient diary. Measures and data collection tools are usually modified by researchers to fit the purpose and target population of their specific study. There is strong agreement on the inclusion of direct medical and non-medical costs in economic evaluations. The balance of opinion is that inclusion of indirect costs is appropriate; but agreement on exactly ‘which indirect costs’ and in ‘what context’ differs. However, narrow study perspectives mean that inter-sectoral resources are often overlooked. In addition to the two cancer-specific instruments included in the Database of Instruments for Resource-Use Measurement, there are numerous resource utilisation measurement tools used in a broad range of clinical research with heterogeneous intervention characteristics and outcome measures. Despite this, very few studies report validated cost/resource use instruments. Further, many cost analyses ignore long-term care costs, non-medical costs borne by patients and important costs incurred in other sectors, such as social services. There is no ‘gold standard’ for resource utilisation instruments and the agenda for future research is lengthy. For example, many issues such as recall length, accuracy in recall of medical terms and medicines, specificity versus comprehensiveness of the instrument and missing data, remain to be addressed. Innovation in mobile technology will likely revolutionise data collection and may overcome many of the existing barriers to robust measurement of resource utilisation for cancer clinical trials and improve societal decision making.
Health economic assessment and economic evaluation is growing, owing to the need to demonstrate cost-effectiveness of a new health technology, pharmaceutical product, intervention, program or service.1,2 Hence, economic evaluation alongside a randomised controlled trial (RCT) is increasingly used in research to inform decision making for clinical practice and service planning.3 The purpose of this paper is to describe and discuss the measurement of cost in clinical studies, particularly resource utilisation in the healthcare sector for cancer.
Cost assessment in economic evaluation involves the identification, measurement and valuation of costs relevant to both the intervention/s under consideration and the comparator.4 Identification of relevant costs depends on the context of decision making and the study questions to determine the scope of cost inclusion. Measurement of costs assesses the quantity of the services and goods related to the delivery of the intervention/s and comparator. Valuation of costs assigns a unit cost or price to resource items related to the interventions of interest in a consistent year of analysis. Symmetry in methods across intervention and comparator is important to facilitate comparability and confidence in cost results.
Guidelines for economic evaluation and health technology assessment have been developed and increasingly utilised by health economists and evaluators. For example, the National Institute for Health and Care Excellence in the UK has a health economics guidelines manual underpinning clinical and public health guidance.2 The International Society for Pharmacoeconomics and Outcomes Research has published and updated the Good Research Practices for Cost-Effectiveness Analysis guidelines for conducting and reporting the economic evaluation alongside clinical studies.5 In Australia, the Pharmaceutical Benefits Advisory Committee has developed and regularly revises guidelines for the preparation of submissions for consideration of pharmaceutical product reimbursement.1 In the US, updated guidelines of the Second Panel on Cost-Effectiveness in Health and Medicine were published in 2016.6 The updates build on the original work of the 1996 recommendations for the conduct and reporting cost-effectiveness analyses.7 These guidelines, although acknowledging different contexts, settings and study aims, recommend the inclusion of health resource use data in conducting economic evaluations.
Examples of medical resource items include medicines, medical services and procedures, hospital services, diagnostic and investigational services, community-based services and any other direct medical costs. The guidelines also recommend the inclusion of indirect costs, although guidance on where and how these costs should be included varies. Indirect costs include time and travel costs, productivity impacts in the general economy and domestic production, together with informal care to patients provided by carers/families. An example of impact and cost considerations across various sectors from different perspectives is illustrated in figure 1.
Figure 1: Impact Inventory Template, Source: Sanders G et al 2016.6
Measurement of resource use varies in complexity from a macro top-down approach that focuses on frequency of use of pre-costed activity components, exemplified by the Australian Refined Diagnostic Related Groups,8 to a micro approach that identifies expenditure categories, including salaries and wages, capital, consumables and overheads, and individual patient utilisation data.9 Measurement can be conducted retrospectively, prospectively, or by using mixed methodologies, depending on the study context. Methods include reviewing relevant patient treatment records, using administrative data collections, linkage with claims data, questionnaire or survey administration, and the use of patient diaries.1,10,11 Within the Australian healthcare system, services and care pathways are segregated and financed by state and/or commonwealth governments. For example, palliative and supportive care for cancer patients requires a broad range of services provided by diverse disciplines across all healthcare sectors with primary, secondary and tertiary care providers.12 Therefore, many data sources are required to estimate healthcare costs along the care pathways. Data sources include Medicare claims, health records held by general practitioners, health professionals and hospitals, or patient recall.13-15
Questionnaires, logs and diaries are commonly used by health economists to record patient-level health services utilisation (including patient out-of-pocket costs) or indirect costs, e.g. travel costs, time costs and impact on their productivity.16-19 These patient-level costs are usually based on patient recall or prospective recording by clinical study participants. Retrospective questionnaires may be subject to recall bias, whereas prospective diaries can be burdensome and subject to partial completion.20,21 A study comparing these two collection approaches for rectal cancer patients concluded that the cost questionnaire with structured, closed questions could replace a cost diary for recall periods of up to six months.20 Our experience supports the view that diaries can be problematic and resource intensive. For example, in a clinical trial where young women were recruited into a life-style modification intervention, a paper-based diary was not efficient in data collection as the participants easily misplaced and overlooked the diary.22
There is no gold standard for the development of resource utilisation instruments. Numerous instruments, mostly not validated, have been designed to collect self-report cost and resource usage for economic studies in a broad range of clinical research.23 Information is reported by patients, their parents/carers, healthcare professionals and even researchers. Various administration methods are employed to complete the data collection, either in person by the researchers, by mail-out to patients, through telephone interviews, or via computer and internet interface.14,24,25 Costs can be borne by different sectors and are generally reported as ‘costs to government as the third-party payer of healthcare services’, ‘costs to individuals as out-of-pocket expense’, the ‘cost of informal care’ to families and ‘productivity gain/loss’ to the economy.26-28
A web-enabled Database of Instruments for Resource-Use Measurement (DIRUM) was developed by the Medical Research Council Network of Hubs for Trial Methodology Research in the UK.29 DIRUM offers a repository of methodological papers related to resource use and cost measurement. At its inception, there were 54 resource utilisation instruments used in the UK for inclusion in the database.23 The database has expanded to 81 instruments up to March 2017, incorporating those from other countries including Australia,30 and the scope has extended to include inter-sectoral cost measurement outside the healthcare sector. This web-based database is a very useful resource for researchers conducting cost assessments, but these instruments may not be generalisable to other countries due to the diversity in healthcare systems from country to country.
In the DIRUM database, there are two instruments specifically designed to collect costs incurred by cancer patients, the UK Cancer Costs Questionnaire and the Assessment of Nausea in Chemotherapy Research (ANCHoR) Questionnaire. The UK Cancer Cost Questionnaire was used to describe the economic burden of UK cancer survivorship one year post-diagnosis for breast, colorectal and prostate cancer patients treated with curative intent.31 Included resource usage items were community-based health and social care, medications, travel costs and informal care. The questionnaire is part of an electronic data collection system for obtaining relevant patient-level clinical and financial information to estimate social costs by using a standard cost-of-illness framework. Similarly, direct medical and non-medical costs and indirect costs to patient and families, including social care and workdays lost, were collected by the ANCHoR Health Economic Questionnaire.32 The ANCHoR questionnaire was designed to collect patient-level costs in a RCT for the management of chemotherapy-related nausea.
Ridyard and Hughes examined the DIRUM database’s instruments for reliability, validity, pilot testing and questionnaire completion rates.23 Little evidence of reliability testing existed in the instruments included in the DIRUM database. Some degrees of validity, including content validation, face validation, criterion validation or convergent validation, were observed in approximately half of these instruments. Less than half of instruments were piloted, using a variety of methods, and less than 10% tested the cognitive or patient comprehension of the instructions and questions. The review of these 54 instruments raised many unanswered questions which need to be addressed, including: i) the effect of question sequencing; ii) the optimal recall length; iii) accuracy in recall of medical terms and medicines; iv) specificity versus comprehensiveness of the instrument; v) treatment of missing data; vi) the appropriateness and transferability of generic instruments; and vii) the challenge of instrument development in multi-national trials.
There are also numerous resource measurement tools which have been used in many types of clinical studies with heterogeneous interventions characteristics and outcome measures for cancer patients. In Australia, the Cancer Research Economics Support Team at the University of Technology Sydney have published a series of factsheets to aid clinicians and researchers in understanding and developing economic evaluations alongside their clinical studies.15 These factsheets provide guidance on how to conduct health economic studies in cancer effectiveness trials, with the intention of creating opportunities to conduct cross-trial investigations in broader research areas. Practical guides and specific topics in trial design, data collection and analysis in health economics are provided by the online resources. However, details of cost measurement and instrument development are absent and the factsheets do not address issues of instrument acceptability and appropriateness.
Generally, data collection tools are modified by research teams to fit the purpose and target population for their specific studies. While understandable, this process may lack the rigour required to ensure modified questionnaires are reliable and valid. Pilot consultation with trial participants provides a way to validate the acceptance and appropriateness of the questions in the collection tool. Past and current economic studies show that response rates are often not ideal and there is little evidence of analysis in questionnaire completion rates that exists in the literature.23 Our experience confirms that study participants may not feel comfortable providing financial information, but a more detailed exploration of the factors influencing the completion of health resource utilisation data collection is needed.
The current literature on health resource use measures predominantly focuses on: i) comparison of data sources; ii) methods for data collection; and iii) the validation of self-report questionnaires with administrative data.33-35 A systematic review of validated self-reported questionnaires for measuring resource utilisation found very few studies reporting validated instruments, particularly compared to the sheer numbers of economic evaluations conducted.33 Among the 15 studies included in the systematic review, great variation existed in target populations, conditions studied, the age of patients, the length of questionnaires and the relevant resource sections included. On the other hand, validation of self-report questionnaires with clinical records or administrative databases does have limitations. For example, clinical records are often fragmented across the health system and therefore sufficient information for accurate costing may not be readily available from these sources.36
A recent study examined the full scope of out-of-pocket costs, lost income and the management of finances during cancer treatment.37 Many non-medical costs were identified in the qualitative exploration study, including modification to housing arrangements, special clothing, fitness costs and the impact of an altered diet. Although only 14 cancer patients completed the interview, these patients with a diagnosis of breast, colorectal, lung or prostate cancer, revealed a foundation issue of unexpected financial shock after the diagnosis. These costs impose a significant financial burden to cancer patients and are usually omitted by economic assessment in many studies.
Another review paper identified that many cost analyses ignore long-term care costs and costs occurring in other sectors, such as social service.36 Unsurprisingly, most clinical studies and trials focus on the measurement of cost and consequences within the healthcare sector due to its narrow perspectives of study analysis. The impact of the intervention on other sectors such as housing, education and social welfare is usually overlooked. Some guidelines, including those prepared by the Pharmaceutical Benefits Advisory Committee, recommend that only healthcare resources be included in assessments due to their decision making context, such as a Department of Health perspective. A key issue is that cost identification/measurement should be carefully considered based on the study questions and decision context from a broader societal perspective if possible. Other issues include the need for symmetry across costs and outcomes, particularly in regard to tracking costs through time and data tractability.
Mobile technology has advanced enormously in recent decades and mobile phone apps and tablets are gradually being adopted in healthcare service and research. The applications of mobile technology range from appointment reminders to enhancing treatment compliance, delivering interventions and data collection.38-41 This innovation, denoted as mobile health (mHealth), has become a powerful platform in the healthcare sector.42 An example of the technology assisted data collection in research is the mobile device application development, TherApp (Therapy App), currently in randomised control trials of upper limb orthoses for children with cerebral palsy.43 The application can be installed in mobile phones and tablets for recording frequency, duration and complications of wearing the orthoses each day during enrolment in randomised control trials. Alerts and prompts are sent to the parent of the child participant if no responses are received. Therapies received and attendance of medical appointments are also recorded by a weekly prompt question. Safety and privacy is ensured by password protected access to TherApp for each study participant and secure data storage and transfer.
Such mobile applications for collecting data in cancer patients are practical and are expected to improve the accuracy of data in cost measurement, as the app acts like a real-time cloud diary. Using a mobile device may assist in overcoming the problems of paper-based diary described earlier. The data collected can be compared with or complement other forms of data collection, such as questionnaires administered at time points during the trial or study. These innovative technologies can be further adopted through knowledge translation from research to routine clinical practice.44
Measurement of costs and resource utilisation in cancer clinical studies is a pivotal component of economic evaluation. Measurement instruments for resource utilisation are generally designed to fit the decision context, target population, disease and intervention for each study. Variation in data inclusion and administration methods exist due to the inherent nature of fit-for-purpose instruments. This then takes us back to clarity on costing principles and this is where the health economist can help to set research on the right footing.
In cancer clinical studies, there is a strong agreement on the inclusion of healthcare related resources that are direct medical and non-medical costs. The balance of opinion is that inclusion of indirect costs is appropriate, but agreement on exactly ‘which indirect costs’ and in ‘what context’ differs – there is more agreement on inclusion of costs in time, travel and informal care than for productivity impacts. Further, inter-sectoral resources outside the healthcare setting are often overlooked. The vast majority of tools used in clinical trials/studies are not validated and many issues, such as recall length, accuracy in recall of medical terms and medicines, specificity versus comprehensiveness of the instrument, missing data, etc. are yet to be addressed. Innovation in mobile technology will revolutionise future clinical studies in data collection, intervention delivery and adoption for routine practice.