Measurement of resource utilisation in cancer clinical studies – tools, issues and challenges



Deakin Health Economics, Centre for Population Health Research, Faculty of Health, Deakin University, Victoria, Australia.


Inclusion of economic evaluations alongside cancer clinical trials necessitates the collection and analysis of resource utilisation and cost data alongside outcomes. The purpose of this paper is to describe and discuss the measurement of cost in clinical studies, particularly resource utilisation. Cost data collection can be conducted retrospectively through linkage of treatment data with claims data, such as Medicare, or by patient recall (questionnaires). Prospective approaches include the patient diary. Measures and data collection tools are usually modified by researchers to fit the purpose and target population of their specific study. There is strong agreement on the inclusion of direct medical and non-medical costs in economic evaluations. The balance of opinion is that inclusion of indirect costs is appropriate; but agreement on exactly ‘which indirect costs’ and in ‘what context’ differs. However, narrow study perspectives mean that inter-sectoral resources are often overlooked. In addition to the two cancer-specific instruments included in the Database of Instruments for Resource-Use Measurement, there are numerous resource utilisation measurement tools used in a broad range of clinical research with heterogeneous intervention characteristics and outcome measures. Despite this, very few studies report validated cost/resource use instruments. Further, many cost analyses ignore long-term care costs, non-medical costs borne by patients and important costs incurred in other sectors, such as social services. There is no ‘gold standard’ for resource utilisation instruments and the agenda for future research is lengthy. For example, many issues such as recall length, accuracy in recall of medical terms and medicines, specificity versus comprehensiveness of the instrument and missing data, remain to be addressed. Innovation in mobile technology will likely revolutionise data collection and may overcome many of the existing barriers to robust measurement of resource utilisation for cancer clinical trials and improve societal decision making.

Health economic assessment and economic evaluation is growing, owing to the need to demonstrate cost-effectiveness of a new health technology, pharmaceutical product, intervention, program or service.1,2 Hence, economic evaluation alongside a randomised controlled trial (RCT) is increasingly used in research to inform decision making for clinical practice and service planning.3 The purpose of this paper is to describe and discuss the measurement of cost in clinical studies, particularly resource utilisation in the healthcare sector for cancer.

Cost assessment in economic evaluation involves the identification, measurement and valuation of costs relevant to both the intervention/s under consideration and the comparator.4 Identification of relevant costs depends on the context of decision making and the study questions to determine the scope of cost inclusion. Measurement of costs assesses the quantity of the services and goods related to the delivery of the intervention/s and comparator. Valuation of costs assigns a unit cost or price to resource items related to the interventions of interest in a consistent year of analysis. Symmetry in methods across intervention and comparator is important to facilitate comparability and confidence in cost results.

Cost assessment for economic evaluation alongside clinical studies

Guidelines for economic evaluation and health technology assessment have been developed and increasingly utilised by health economists and evaluators. For example, the National Institute for Health and Care Excellence in the UK has a health economics guidelines manual underpinning clinical and public health guidance.2 The International Society for Pharmacoeconomics and Outcomes Research has published and updated the Good Research Practices for Cost-Effectiveness Analysis guidelines for conducting and reporting the economic evaluation alongside clinical studies.5 In Australia, the Pharmaceutical Benefits Advisory Committee has developed and regularly revises guidelines for the preparation of submissions for consideration of pharmaceutical product reimbursement.1 In the US, updated guidelines of the Second Panel on Cost-Effectiveness in Health and Medicine were published in 2016.6 The updates build on the original work of the 1996 recommendations for the conduct and reporting cost-effectiveness analyses.7 These guidelines, although acknowledging different contexts, settings and study aims, recommend the inclusion of health resource use data in conducting economic evaluations.

Examples of medical resource items include medicines, medical services and procedures, hospital services, diagnostic and investigational services, community-based services and any other direct medical costs. The guidelines also recommend the inclusion of indirect costs, although guidance on where and how these costs should be included varies. Indirect costs include time and travel costs, productivity impacts in the general economy and domestic production, together with informal care to patients provided by carers/families. An example of impact and cost considerations across various sectors from different perspectives is illustrated in figure 1.

Figure 1: Impact Inventory Template, Source: Sanders G et al 2016.6

Figure 1- Impact Inventory Template






Measurement of resource use varies in complexity from a macro top-down approach that focuses on frequency of use of pre-costed activity components, exemplified by the Australian Refined Diagnostic Related Groups,8 to a micro approach that identifies expenditure categories, including salaries and wages, capital, consumables and overheads, and individual patient utilisation data.9 Measurement can be conducted retrospectively, prospectively, or by using mixed methodologies, depending on the study context. Methods include reviewing relevant patient treatment records, using administrative data collections, linkage with claims data, questionnaire or survey administration, and the use of patient diaries.1,10,11 Within the Australian healthcare system, services and care pathways are segregated and financed by state and/or commonwealth governments. For example, palliative and supportive care for cancer patients requires a broad range of services provided by diverse disciplines across all healthcare sectors with primary, secondary and tertiary care providers.12 Therefore, many data sources are required to estimate healthcare costs along the care pathways. Data sources include Medicare claims, health records held by general practitioners, health professionals and hospitals, or patient recall.13-15

Questionnaires, logs and diaries are commonly used by health economists to record patient-level health services utilisation (including patient out-of-pocket costs) or indirect costs, e.g. travel costs, time costs and impact on their productivity.16-19 These patient-level costs are usually based on patient recall or prospective recording by clinical study participants. Retrospective questionnaires may be subject to recall bias, whereas prospective diaries can be burdensome and subject to partial completion.20,21 A study comparing these two collection approaches for rectal cancer patients concluded that the cost questionnaire with structured, closed questions could replace a cost diary for recall periods of up to six months.20 Our experience supports the view that diaries can be problematic and resource intensive. For example, in a clinical trial where young women were recruited into a life-style modification intervention, a paper-based diary was not efficient in data collection as the participants easily misplaced and overlooked the diary.22

There is no gold standard for the development of resource utilisation instruments. Numerous instruments, mostly not validated, have been designed to collect self-report cost and resource usage for economic studies in a broad range of clinical research.23 Information is reported by patients, their parents/carers, healthcare professionals and even researchers. Various administration methods are employed to complete the data collection, either in person by the researchers, by mail-out to patients, through telephone interviews, or via computer and internet interface.14,24,25 Costs can be borne by different sectors and are generally reported as ‘costs to government as the third-party payer of healthcare services’, ‘costs to individuals as out-of-pocket expense’, the ‘cost of informal care’ to families and ‘productivity gain/loss’ to the economy.26-28

Tools used in collecting resource utilisation data in cancer patients

A web-enabled Database of Instruments for Resource-Use Measurement (DIRUM) was developed by the Medical Research Council Network of Hubs for Trial Methodology Research in the UK.29 DIRUM offers a repository of methodological papers related to resource use and cost measurement. At its inception, there were 54 resource utilisation instruments used in the UK for inclusion in the database.23 The database has expanded to 81 instruments up to March 2017, incorporating those from other countries including Australia,30 and the scope has extended to include inter-sectoral cost measurement outside the healthcare sector. This web-based database is a very useful resource for researchers conducting cost assessments, but these instruments may not be generalisable to other countries due to the diversity in healthcare systems from country to country.

In the DIRUM database, there are two instruments specifically designed to collect costs incurred by cancer patients, the UK Cancer Costs Questionnaire and the Assessment of Nausea in Chemotherapy Research (ANCHoR) Questionnaire. The UK Cancer Cost Questionnaire was used to describe the economic burden of UK cancer survivorship one year post-diagnosis for breast, colorectal and prostate cancer patients treated with curative intent.31 Included resource usage items were community-based health and social care, medications, travel costs and informal care. The questionnaire is part of an electronic data collection system for obtaining relevant patient-level clinical and financial information to estimate social costs by using a standard cost-of-illness framework. Similarly, direct medical and non-medical costs and indirect costs to patient and families, including social care and workdays lost, were collected by the ANCHoR Health Economic Questionnaire.32 The ANCHoR questionnaire was designed to collect patient-level costs in a RCT for the management of chemotherapy-related nausea.

Ridyard and Hughes examined the DIRUM database’s instruments for reliability, validity, pilot testing and questionnaire completion rates.23 Little evidence of reliability testing existed in the instruments included in the DIRUM database. Some degrees of validity, including content validation, face validation, criterion validation or convergent validation, were observed in approximately half of these instruments. Less than half of instruments were piloted, using a variety of methods, and less than 10% tested the cognitive or patient comprehension of the instructions and questions. The review of these 54 instruments raised many unanswered questions which need to be addressed, including: i) the effect of question sequencing; ii) the optimal recall length; iii) accuracy in recall of medical terms and medicines; iv) specificity versus comprehensiveness of the instrument; v) treatment of missing data; vi) the appropriateness and transferability of generic instruments; and vii) the challenge of instrument development in multi-national trials.

There are also numerous resource measurement tools which have been used in many types of clinical studies with heterogeneous interventions characteristics and outcome measures for cancer patients. In Australia, the Cancer Research Economics Support Team at the University of Technology Sydney have published a series of factsheets to aid clinicians and researchers in understanding and developing economic evaluations alongside their clinical studies.15 These factsheets provide guidance on how to conduct health economic studies in cancer effectiveness trials, with the intention of creating opportunities to conduct cross-trial investigations in broader research areas. Practical guides and specific topics in trial design, data collection and analysis in health economics are provided by the online resources. However, details of cost measurement and instrument development are absent and the factsheets do not address issues of instrument acceptability and appropriateness.

Issues and challenges in resource utilisation measurement

Generally, data collection tools are modified by research teams to fit the purpose and target population for their specific studies. While understandable, this process may lack the rigour required to ensure modified questionnaires are reliable and valid. Pilot consultation with trial participants provides a way to validate the acceptance and appropriateness of the questions in the collection tool. Past and current economic studies show that response rates are often not ideal and there is little evidence of analysis in questionnaire completion rates that exists in the literature.23 Our experience confirms that study participants may not feel comfortable providing financial information, but a more detailed exploration of the factors influencing the completion of health resource utilisation data collection is needed.

The current literature on health resource use measures predominantly focuses on: i) comparison of data sources; ii) methods for data collection; and iii) the validation of self-report questionnaires with administrative data.33-35 A systematic review of validated self-reported questionnaires for measuring resource utilisation found very few studies reporting validated instruments, particularly compared to the sheer numbers of economic evaluations conducted.33 Among the 15 studies included in the systematic review, great variation existed in target populations, conditions studied, the age of patients, the length of questionnaires and the relevant resource sections included. On the other hand, validation of self-report questionnaires with clinical records or administrative databases does have limitations. For example, clinical records are often fragmented across the health system and therefore sufficient information for accurate costing may not be readily available from these sources.36

A recent study examined the full scope of out-of-pocket costs, lost income and the management of finances during cancer treatment.37 Many non-medical costs were identified in the qualitative exploration study, including modification to housing arrangements, special clothing, fitness costs and the impact of an altered diet. Although only 14 cancer patients completed the interview, these patients with a diagnosis of breast, colorectal, lung or prostate cancer, revealed a foundation issue of unexpected financial shock after the diagnosis. These costs impose a significant financial burden to cancer patients and are usually omitted by economic assessment in many studies.

Another review paper identified that many cost analyses ignore long-term care costs and costs occurring in other sectors, such as social service.36 Unsurprisingly, most clinical studies and trials focus on the measurement of cost and consequences within the healthcare sector due to its narrow perspectives of study analysis. The impact of the intervention on other sectors such as housing, education and social welfare is usually overlooked. Some guidelines, including those prepared by the Pharmaceutical Benefits Advisory Committee, recommend that only healthcare resources be included in assessments due to their decision making context, such as a Department of Health perspective. A key issue is that cost identification/measurement should be carefully considered based on the study questions and decision context from a broader societal perspective if possible. Other issues include the need for symmetry across costs and outcomes, particularly in regard to tracking costs through time and data tractability.

Future trends in resource utilisation data collection for clinical studies

Mobile technology has advanced enormously in recent decades and mobile phone apps and tablets are gradually being adopted in healthcare service and research. The applications of mobile technology range from appointment reminders to enhancing treatment compliance, delivering interventions and data collection.38-41 This innovation, denoted as mobile health (mHealth), has become a powerful platform in the healthcare sector.42 An example of the technology assisted data collection in research is the mobile device application development, TherApp (Therapy App), currently in randomised control trials of upper limb orthoses for children with cerebral palsy.43 The application can be installed in mobile phones and tablets for recording frequency, duration and complications of wearing the orthoses each day during enrolment in randomised control trials. Alerts and prompts are sent to the parent of the child participant if no responses are received. Therapies received and attendance of medical appointments are also recorded by a weekly prompt question. Safety and privacy is ensured by password protected access to TherApp for each study participant and secure data storage and transfer.

Such mobile applications for collecting data in cancer patients are practical and are expected to improve the accuracy of data in cost measurement, as the app acts like a real-time cloud diary. Using a mobile device may assist in overcoming the problems of paper-based diary described earlier. The data collected can be compared with or complement other forms of data collection, such as questionnaires administered at time points during the trial or study. These innovative technologies can be further adopted through knowledge translation from research to routine clinical practice.44


Measurement of costs and resource utilisation in cancer clinical studies is a pivotal component of economic evaluation. Measurement instruments for resource utilisation are generally designed to fit the decision context, target population, disease and intervention for each study. Variation in data inclusion and administration methods exist due to the inherent nature of fit-for-purpose instruments. This then takes us back to clarity on costing principles and this is where the health economist can help to set research on the right footing.

In cancer clinical studies, there is a strong agreement on the inclusion of healthcare related resources that are direct medical and non-medical costs. The balance of opinion is that inclusion of indirect costs is appropriate, but agreement on exactly ‘which indirect costs’ and in ‘what context’ differs – there is more agreement on inclusion of costs in time, travel and informal care than for productivity impacts. Further, inter-sectoral resources outside the healthcare setting are often overlooked. The vast majority of tools used in clinical trials/studies are not validated and many issues, such as recall length, accuracy in recall of medical terms and medicines, specificity versus comprehensiveness of the instrument, missing data, etc. are yet to be addressed. Innovation in mobile technology will revolutionise future clinical studies in data collection, intervention delivery and adoption for routine practice.


  1. Australian Government Department of Health. Guidelines for preparing a submission to the Pharmaceutical Benefits Advisory Committee (Version 5.0). Communication Branch DoH. Canberra: Commonwealth of Australia; 2016.
  2. National Institute for Health and Care Excellence (NICE), Methods for the development of NICE public health guidance (third edition). UK2012.
  3. Glick HA, Doshi JA, Sonnad SS, et al. Economic Evaluation in Clinical Trials. Second Edition ed: Oxford University Press; 2007.
  4. Drummond M, Sculpher M, Claxton K, et al. Methods for the Economic Evaluation of Health Care Programmes. 4th Edition ed. UL: Oxford University Press; 2015.
  5. Ramsey SD, Willke RJ, Glick H, et al. Cost-Effectiveness Analysis Alongside Clinical Trials II-An ISPOR Good Research Practices Task Force Report. Value in Health. 2015;18(2):161-72.
  6. Sanders G, Neumann P, et al. Recommendations for Conduct, Methodological Practices, and Reporting of Cost-effectiveness Analyses: Second Panel of Cost-Effectiveness in Health and Medicine. JAMA 2016;316(10):1093-103.
  7. Gold M, Siegel J, Russell L, et al. Cost-Effectiveness in Health and Medicine. Oxford University Press. 1996.
  8. Jackson T, Dimitropoulos V, Madden R, et al. Australian diagnosis related groups: Drivers of complexity adjustment. Health Policy. 2015;119(11):1433-41.
  9. Mathers C, Stevenson C, Carter R, et al. Disease costing methdology used in the disease costs and impact study 1993-1994. Australian Institute of Health and Welfare. Canberra, Australia 1998.
  10. Janson M, Carlsson P, Haglind E, et al. Data validation in an economic evaluation of surgery for colon cancer. Int J Technol Assess Health Care. 2005;21(2):246-52.
  11. Al-Badriyeh D, Alameri M, Al-Okka R. Cost-effectiveness research in cancer therapy: a systematic review of literature trends, methods and the influence of funding. BMJ Open. 2017;7(1):e012648.
  12. Gardiner C, Ingleton C, Ryan T, et al. What cost components are relevant for economic evaluations of palliative care, and what approaches are used to measure these costs? A systematic review. Palliat Med. 2017;31(4):323-37.
  13. Ward RL, Laaksonen MA, van Gool K, et al. Cost of cancer care for patients undergoing chemotherapy: The Elements of Cancer Care study. Asia Pac J Clin Oncol. 2015;11(2):178-86.
  14. Chatterton ML, Chambers S, Occhipinti S, et al. Economic evaluation of a psychological intervention for high distress cancer patients and carers: costs and quality-adjusted life years. Psychooncology. 2016;25(7):857-64.
  15. Cancer Research Economics Support Team (CREST). CREST Factsheets – Step by step guide to economic evaluation in cancer trials 2015 Available from:
  16. Kessler RC, Barber C, Beck A, et al. The World Health Organization Health and Work Performance Questionnaire (HPQ). J Occup Environ Med. 2003;45(2):156-74.
  17. Chambers SK, Newton RU, Girgis A, et al. Living with prostate cancer: randomised controlled trial of a multimodal supportive care intervention for men with prostate cancer. BMC Cancer. 2011;11:317.
  18. Lambert SD, McElduff P, Girgis A, et al. A pilot, multisite, randomized controlled trial of a self-directed coping skills training intervention for couples facing prostate cancer: accrual, retention, and data collection issues. Support Care Cancer. 2016;24(2):711-22.
  19. Butow PN, Bell ML, Smith AB, et al. Conquer fear: protocol of a randomised controlled trial of a psychological intervention to reduce fear of cancer recurrence. BMC Cancer. 2013;13:201.
  20. van den Brink M, van den Hout WB, Stiggelbout AM, et al. Self-reports of health-care utilization: diary or questionnaire? Int J Technol Assess Health Care. 2005;21(3):298-304.
  21. Goossens ME, Rutten-van Molken MP, Vlaeyen JW, et al. The cost diary: a method to measure direct and indirect costs in cost-effectiveness research. J Clin Epidemiol. 2000;53(7):688-95.
  22. O’Reilly SL, Dunbar JA, Versace V, et al. Mothers after Gestational Diabetes in Australia (MAGDA): A Randomised Controlled Trial of a Postnatal Diabetes Prevention Program. PLoS Med. 2016;13(7):e1002092.
  23. Ridyard CH, Hughes DA, Team D. Development of a database of instruments for resource-use measurement: purpose, feasibility, and design. Value Health. 2012;15(5):650-5.
  24. Chon JK, Jacobs SC, Naslund MJ. The cost value of medical versus surgical hormonal therapy for metastatic prostate cancer. J Urol. 2000;164(3 Pt 1):735-7.
  25. Belisario JSM, Jamsek J, Huckvale K, et al. Comparison of self-administered survey questionnaire responses collected using mobile apps versus other methods. Cochrane Database of Systematic Reviews. 2015(7).
  26. Jayadevappa R, Schwartz JS, Chhatre S, et al. The burden of out-of-pocket and indirect costs of prostate cancer. Prostate. 2010;70(11):1255-64.
  27. Li C, Zeliadt SB, Hall IJ, et al. Burden among partner caregivers of patients diagnosed with localized prostate cancer within 1 year after diagnosis: an economic perspective. Support Care Cancer. 2013;21(12):3461-9.
  28. Nyman JA. Productivity costs revisited: Toward a new US policy. Health Econ. 2011:DOI: 10.1002/hec.795.
  29. DIRUM. Database of Instruments for Resource Use Measurement (DIRUM) 2011. Available from:
  30. Rowell D, Connelly L, Webber J, et al. What are the true costs of major trauma? J Trauma. 2011;70(5):1086-95.
  31. Marti J, Hall PS, Hamilton P, et al. The economic burden of cancer in the UK: a study of survivors treated with curative intent. Psycho-Oncology. 2016;25(1):77-83.
  32. Molassiotis A, Russell W, Hughes J, et al. The effectiveness and cost-effectiveness of acupressure for the control and management of chemotherapy-related acute and delayed nausea: Assessment of Nausea in Chemotherapy Research (ANCHoR), a randomised controlled trial. Health Technol Asses. 2013;17(26).
  33. Leggett LE, Khadaroo RG, Holroyd-Leduc J, et al. Measuring Resource Utilization: A Systematic Review of Validated Self-Reported Questionnaires. Medicine (Baltimore). 2016;95(10):e2759.
  34. Pollicino C, Viney R, Haas M. Measuring health system resource use for economic evaluation: a comparison of data sources. Aust Health Rev. 2002;25(3):171-8.
  35. Ridyard C, Linck P, Hughes DA. Review of the Use of Resource Use Instruments Based on Patient Recall in Relation to Other Methods of Cost Estimation. Value in Health. 2013;16(7):A466-A.
  36. Martin A, Jones A, Mugford M, et al. Methods Used to Identify and Measure Resource Use in Economic Evaluations: A Systematic Review of Questionnaires for Older People. Health Econ. 2012;21(8):1017-22.
  37. Longo CJ, Fitch M, Grignon M, et al. Understanding the full breadth of cancer-related patient costs in Ontario: a qualitative exploration. Support Care Cancer. 2016;24(11):4541-8.
  38. Clough BA, Casey LM. Technological adjuncts to increase adherence to therapy: A review. Clinical Psychology Review. 2011;31(5):697-710.
  39. Vodopivec-Jamsek V, de Jongh T, Gurol-Urganci I, et al. Mobile phone messaging for preventive health care. Cochrane Database of Systematic Reviews. 2012(12).
  40. de Jongh T, Gurol-Urganci I, Vodopivec-Jamsek V, et al. Mobile phone messaging for facilitating self-management of long-term illnesses. Cochrane Database of Systematic Reviews. 2012(12).
  41. Mobasheri MH, Johnston M, Syed UM, et al. The uses of smartphones and tablet devices in surgery: A systematic review of the literature. Surgery. 2015;158(5):1352-71.
  42. White A, Thomas DSK, Ezeanochie N, et al. Health Worker mHealth Utilization A Systematic Review. Cin-Computers Informatics Nursing. 2016;34(5):206-14.
  43. Imms C, Wallen M, Elliott C, et al. Minimising impairment: Protocol for a multicentre randomised controlled trial of upper limb orthoses for children with cerebral palsy. Bmc Pediatr. 2016;16.
  44. Pham Q, Wiljer D, Cafazzo JA. Beyond the Randomized Controlled Trial: A Review of Alternatives in mHealth Clinical Trial Methods. Jmir Mhealth and Uhealth. 2016;4(3).

Be the first to know when a new issue is online. Subscribe today.