COMBAT study – Computer based assessment and treatment – A clinical trial evaluating impact of a computerized clinical decision support tool on pain in cancer patients Sunil X. Raja,b,?, Cinzia Brunellid, Pål Klepstadc,e, Stein Kaasaa,b a European Palliative Care Research Centre (PRC), Department of Cancer Research and Molecular Medicine, Faculty of Medicine, Norwegian University of Science and Technology, Trondheim, Norway b Cancer Clinic, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway c Department of Anaesthesiology and Intensive Care Medicine, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway d Palliative Care, Pain Therapy and Rehabilitation Unit, Fondazione IRCCS Istituto Nazionale dei Tumori, Milano, Italy e Department of Circulation and Medical Imaging, Faculty of Medicine, Norwegian University of Science and Technology, Trondheim, Norway h i g h l i g h t s • Pain in cancer patients is not managed adequately and must be improved. • Modern information technology is widely used at many health care institutions. • This study examines utilization of information technology in pain management. • Modern information technology did not improve pain management in this study. • Lack of efficacy is probably related to insufficient implementation strategies. a r t i c l e i n f o a b s t r a c t Article history: Background and aims: The prevalence of pain in cancer patients are relatively high and indicate inad- Received 13 May 2017 equate pain management strategies. Therefore, it is necessary to develop new methods and to improve Received in revised form 12 June 2017 implementation of guidelines to assess and treat pain. The vast improvement in information technology Accepted 7 July 2017 facilitated development of a computerized symptom assessment and decision support system (CCDS) – the Combat system – which was implemented in an outpatient cancer clinic to evaluate improvement in Keywords: pain management. CCDS Methods: We conducted a controlled before-and-after study between patient cohorts in two consecutive Computer study periods: before (n=80) and after (n=134) implementation of the Combat system. Patients in the Pain Decision support first cohort completed questionnaires with the paper-and-pencil method and this data was not shown Cancer outpatients to physicians. Patients in the latter cohort completed an electronic questionnaire by using an iPad and the data were automatically transferred and presented to physicians at point of care. Additionally, the system provided computerized decision support at point of care for the physician based on the electronic questionnaires completed by the patients, an electronic CRF completed by physicians and clinical guidelines. Results: The Combat system did not improve pain intensity and there were no significant alterations in the prescribed dose of opiates compared to the cohort of patients managed without the Combat system. Conclusion: The Combat system did not improve pain management. This may be explained by several factors, however, we consider lack of proper implementation of the CCDS in the clinic to be the most important factor. As a result, we did not manage to change the behaviour of the physicians in the clinic. Implications: There is a need to conduct larger prospective studies to evaluate the efficacy of modern information technology to improve pain management in cancer patients. Before introducing new information technology in the clinics, it is important to have a well thought out implementation strategy. The trial is registered at Clinialtrials.gov, number NCT01795157. © 2017 Scandinavian Association for the Study of Pain. Published by Elsevier B.V. All rights reserved. ? Corresponding author at: Department of Cancer Research and Molecular Medicine, Faculty of Medicine, NTNU, St. Olavs University Hospital, NO-7006 Trondheim, Norway. E-mail address: sunil.raj@ntnu.no (S.X. Raj). http://dx.doi.org/10.1016/j.sjpain.2017.07.016 1877-8860/© 2017 Scandinavian Association for the Study of Pain. Published by Elsevier B.V. All rights reserved. 1. Introduction Cancer pain affects the health related quality of life (HRQOL) of patients and their families [1] and approximately 50% of cancer outpatients reports inadequate pain control [2,3]. Several trials indicate the presence of insufficient pain assessments. In a study by Cohen et al. pain assessment was documented in the medical records of 57% of cancer outpatients and reassessment after treatment was documented in 34% [4]. Several studies have also reported a lack of agreement between physicians’ observer ratings and cancer patients’ self-reports of symptoms [5,6] and this disagreement contribute to poor pain outcomes [7]. Therefore, it is recommended to employ patient reported outcome measures (PROMs), which are various type of patient data reported by the patients and collected without modification from clinicians or other health care personal, for assessment, classification and follow up of pain [8]. It has been shown that pain can be alleviated in 88% of the patients by adhering to WHO guidelines for treatment of cancer pain [9] and implementation of pain guidelines in routine clinical practice improves pain control according to a randomized clinical trial [10], but adherence to cancer pain guidelines is relatively poor [11]. The main purpose of information technology systems at hospitals is to record, store and retrieve patient data. A more complex health related computer system is a computerized clinical decision support systems (CCDS). CCDS provide decision support for patient management by integrating patient data from different sources [12]. Various subtypes of CCDS have been investigated like alerts and reminders system [13,14], computerized provider order entry systems [15] and expert systems [16,17] and several systematic reviews from other areas than cancer pain management have supported the effectiveness of CCDS in patient management [18–20]. For cancer patients, a CCDS could include both PROMs and computerized algorithms using PROMs to provide advice for clinicians. Patients can use a computer system, for instance a tablet, in order to collect PROMs and assure that the results are presented to the physician at point of care, which may improve patient–physician communication [21], cancer pain treatment and potentially patients’ HRQOL [22]. We developed the COMBAT (Computer Based Assessment and Treatment) system which includes the following procedures: (i) A computer based collection of PROMs, (ii) an immediate wireless transfer of PROMs to the oncologist computer at point-of-care and (iii) a CCDS system designed for cancer pain management. We have hypothesized that pain management in an outpatient oncology clinic could be improved by applying the COMBAT system and conducted a controlled before-and-after study by comparing two cohorts of patients, before and after implementation of the COMBAT system, to answer the following research questions: - Is there an improvement in pain control, measured as average pain last 24 h, within the first three weeks of treatment after implementing the COMBAT system? - Is there an improvement in pain control, measured as worst pain last 24 h, within the first three weeks of treatment after implementing the COMBAT system? - Are prescribed opioid doses modified after implementing the COMBAT system? 2. Method 2.1. Study design The Combat study is designed as a controlled before-and-after study comparing data between patient cohorts in two consecutive study periods: Before implementation of the COMBAT system (the pre-intervention period) and after implementation of the COMBAT system (intervention period). 2.2. Patients Patients were recruited from the outpatient department at the Cancer Clinic, St. Olavs University Hospital, between March 2010 and February 2013. All potentially eligible patients with an appointment with the physician at the outpatient department during the study period were approached by phone by a research assistant the day before an appointment with a physician and invited to participate in the study if they fulfilled the inclusion criteria (Fig. 1). The inclusion criteria were: Histologically verified malignancy, age 18 years or above, pain intensity (any of current pain, average pain last 24 h or worst pain last 24 h) of at least 4 on a 0–10 point numerical rating scale (NRS), physical and cognitive function appropriate to follow study instructions. Eligibility screening data was collected in 33 of 39 intervention weeks. Hundred-and-seventy-six patients were eligible for inclusion of which 141 (80%) were included in the study during these 33 weeks. The main reason for exclusion was low pain intensity. 2.3. Pre-intervention period In the pre-intervention period (Fig. 1) the patients completed a questionnaire by paper-and-pen method about 30 min prior to consultation with the clinician. This data was not presented for the physician. 2.4. Intervention period Fig. 1. Differences in procedures and assessment in pre-intervention and intervention period. The intervention consisted of applying the COMBAT system which involved three main facets (Fig. 1). (1) Collection of data on PROMs. Patients completed an electronic questionnaire and an electronic body map by using an iPad2 [23] about 30 min prior to consultation with their physician. Selected items among the electronic questionnaire were screening questions on pain, breakthrough pain (BTP), neuropathic pain, depression and pain medication. An affirmative response or a response above a predefined threshold to these screening questions resulted in additional follow up questions. (2) Transfer of PROMs to the physician as a part of the consultation: Data on PROMs were immediately and wirelessly transferred to the desktop computer employed by the physician. The data was available for physician before the consultation, both as an overview of the most important symptoms including the electronic body map, and as a structured digital output of the entire set of questionnaires. At the end of the consultation the physician completed an electronic CRF (Case Report Form) on diagnosis, tumour directed treatment, sites of metastasis and pain medication. (3) Computerized decision support: The electronic questionnaire and electronic CRF completed by the patients and the physician, respectively, were subsequently processed by the CCDS engine providing decision support for the clinician at point of care. The decision support presented for the clinician employed mathematical algorithms according to international guidelines [24,25] and involved four topics: Pain, BTP, neuropathic pain and depression. The decision support algorithm was based on responses to a limited amount of questions. For instance, if the patient responded with a value of at least 4 on a numerical rating scale on current pain, average pain last 24 h or worst pain last 24 h and answered “no” if he/she used strong opioids, then the following text was conveyed to the physician as decision support: “The patient has pain with pain intensity of at least 4 on a numerical rating scale (NRS), which may be current pain, average pain last 24 h or worst pain the last 24 h. The patient does not use opioids. If the pain is caused by cancer, please consider to prescribe Dolcontin tablets 10 mg twice a day (a long acting opiate) and Morphine tablets 10 mg as rescue medication. Physicians participated in an introductory lecture on how to use the software and written instructions were also provided. Physicians could ask for service from a study nurse stationed at the outpatient department if they needed assistance in terms of logging into the software, basic navigation within the software, completing the electronic CRF or other issues regarding the technical aspects of the CCDS. 2.5. Assessment Assessments were performed at baseline, 1 week and 3 weeks after inclusion. At baseline patients completed questionnaires on current pain medication, education and ethnicity. Data on cancer diagnosis, metastatic sites, tumour directed treatment, aim of treatment (curative, life-prolonging, symptomatic) pain aetiology and changes in pain medication were completed by the physicians by using an electronic CFR during the consultation. Is cases where the physician did not complete such information, this information was completed by the first author (SXR) from the electronic medical record. We also recorded the aetiology of pain and classified this as either pain due to malignant invasion, pain due to cancer treatment or pain due to other conditions and diseases than cancer or cancer treatment. Patients completed data on pain medication through questionnaires which was subsequently verified by checking the medical records and electronic prescriptions. Pain intensity was measured using the Brief Pain Inventory (BPI) [26], which is a pain self-assessment tool thoroughly validated and translated into different languages, including Norwegian [27]. For this analysis data from two questions were used (“average pain intensity last 24 h” and “worst pain intensity last 24 h”, both assessed on a 0–10 NRS). Additionally, the presence of BTP were assessed by a single screening question on breakthrough pain (“Have you experienced breakthrough pain past 24 h?”). The pre-intervention questionnaire was the same as the intervention questionnaire. Patients in both study periods were followed up by phone by a study nurse 1 and 3 weeks after consultation and completed the same set of questionnaires as baseline. If the patient were unable to reply phone calls at the day of follow up, the study nurse approached the patient by phone once a day for three consecutive days excluding weekends. If still unable to reach the patient, the follow up were defined as missing. A total of 30 physicians were involved in the Combat study. Three physicians included 14 patients each, two physicians included nine patients each, one physicians included 8 patients, two physicians included seven patients and three physicians included 6 patients. The remaining 50 patients were included by 19 physicians. 2.6. Study endpoints The three primary endpoints were average pain intensity last 24 h (as the mean of 1 and 3 weeks follow up assessments), worst pain intensity last 24 h (as the mean of 1 and 3 weeks follow up assessments) and prescribed opioid doses (oral Morphine equivalents in mg/24 h, as the mean of 1 and 3 weeks follow up dosages). 2.7. Statistical analyses Fig. 2. Consort diagram. Patients included and excluded in the study. We considered a pain reduction of 1.5 on a 0–10 point NRS as clinical significant [28] and hypothesized a standard deviation of 2.5 based on an earlier study at our institution [29] which resulted in an effect size of 1.5/2.5 = 0.6. Due to multiple outcomes, we decided upon an alpha value of 0.01. In the hypothesis of a detectable effect size of 0.6 in the three primary outcome measures using a t-test for the difference between two independent means, an alpha level of 0.01 and a power of 0.9 we estimated the requirement of at least 85 patients in each study period, 170 in total. To account for 20% lost to follow up we decided to include at least 102 patients in each study period. We scheduled a fixed time frame of 12 months for inclusion in both study periods, but due to delayed accrual we extended the inclusion period to 18 months in the preintervention period. We included more patients than planned in the intervention period to increase the power when analyzing multiple outcomes and to compensate for lost to follow up, which was an issue in the pre-intervention period. Patients demographics and pain characteristics were summarized by applying means, proportions and standard deviations (SD). Repeated follow-up assessments of the three main outcomes were analyzed using the summary measure approach as suggested by Frison and Pockok [30]. This method involves averaging the follow-up assessments on each patient and then applying this as dependent variable in an ANCOVA model with baseline data as covariate to measure the effect of the study period adjusting for baseline imbalance. In a second stage of analyses the effect of potentially confounding variables (age, gender, presence of cancer related pain, presence of rheumatic disease) were also tested in ANCOVA models. In the evaluation of pain and opioid prescription we included patients with at least one follow up examination, thus, patients who had completed only the baseline examination were excluded from this analysis. Opioid dose was converted to oral morphine equivalents (mg/24 h) [31]. Further on, we defined cancer related pain as pain due to malignant invasion and/or pain due to cancer treatment. This sub-classification of pain was done in order to examine the influence of the CCDS on patients with cancer related pain. Thus, we had to define cancer related pain which we defined as pain due to malignant invasion and/or pain due to cancer treatment. The trial is registered at Clinialtrials.gov, number NCT01795157. 3. Results 3.1. Study sample A total of 247 patients were included from March 2010 to February 2013, a total of 103 and 151 patients in the preintervention period and intervention period, respectively. Patients with at least one follow up interview after inclusion were included in the final analyses. This led to the exclusion of 23 patients in the pre-intervention period and 17 patients in the intervention period (Fig. 2). 3.2. Demographics Patients main demographic and clinical characteristics (Table 1) were well balanced between the pre-intervention and intervention periods except a higher frequency of rheumatic disease in the intervention period and higher frequency of cancer related pain and patients taking opioid medication in the pre-intervention period. 3.3. Pain intensity Fig. 3 shows that pain intensity scores are very similar both at baseline and at follow up between patients in the pre-intervention and intervention period without any significant differences. The mean of pain intensity scores of 1 and 3 weeks follow up were 3.6 and 3.3 for average pain intensity last 24 h in the preintervention period and intervention period, respectively, with a between group difference of 0.12 (95% CI – 0.33–0.58) after adjusting for baseline pain intensity (Table 2). For worst pain last 24 h the mean of pain intensity scores at 1 and 3 weeks follow up were 4.6 and 4.8 in the pre-intervention period and intervention period, respectively, with a between group difference of 0.32 (95% CI – 0.27–0.91) after adjusting for baseline pain intensity (Table 2). This indicates no differences in pain intensity scores after introducing the Combat system when adjusting for baseline pain intensity. Adjusting for potential confounding predictor variables such as age, gender, opioid consumption, the presence of rheumatic disease, presence of BTP, pain medication and cancer related pain, intervention with the Combat system did not improve the results significantly (Table 2). It is worth noting that 75% of the patients in the pre-intervention period and 42% of the patients in the intervention period had cancer-related pain (p ? 0.001). However, this imbalance did not alter the lack of efficacy of the intervention as shown in the regression analyses when adjusted for cancer related pain (Table 2, the far-right column). 3.4. Opioid prescription The mean of opioid doses at 1 and 3 weeks follow up were 48 mg and 59 mg in the pre-intervention and intervention periods, respectively, with a between group difference of 8.4 mg (95% CI – 11.3–28.0) after adjusting for baseline opioid dose (Table 2). This indicates no differences in opioid prescription after introducing the Combat system when adjusting for baseline opioid dose. Adjusting for potential confounding variables such as age, gender, presence of rheumatic disease, presence of BTP, use of pain medication and presence of cancer related pain did not alter the results (Table 2). Table 1 Baseline characteristics. Preintervention Intervention p-value N = 80 N = 134 n (%) n (%) Age (years ± SDa) Male gender 58.6 ± 13.25 41 (51.3) 61 ± 12.2 70 (52.2) 0.91 0.89 Diagnosis 0.78 Breast 20 (25) 32 (23.9) Prostate cancer 13 (16.3) 21 (15.7) Colorectal cancer 8 (10) 18 (13.4) Lymphoma 7 (8.8) 19 (14.2) Lung cancer 8 (10) 11 (8.2) Testicular cancer 4 (5) 5 (3.7) Anal 3 (3.8) 4 (3) Upper GI cancer 0 6 (4.5) Other 17 (21) 18 (13.4) Disease extent 0.06 Localized 15 (19.2) 16 (11.9) Metastatic 37 (46.1) 44 (32.8) Cancer absent 24 (29.5) 63 (47.0) Not relevantb 4 (5.1) 11 (8.2) Treatment 0.33 Chemotherapy 9 (11.2) 25 (18.7) Radiation therapy 2 (2.5) 10 (7.5) Radiochemotherapy 3 (3.8) 2 (1.5) Hormone therapy 20 (25.0) 28 (20.9) Targeted therapy 4 (5.0) 4 (3.0) Chemotherapy and targeted therapy 3 (3.8) 9 (6.7) No treatment 37 (46.2) 52 (38.8) Missing 2 (2.5) 4 (2.9) Treatment intention 0.07 Curative 38 (47.5) 84 (62.7) Life-prolonging 39 (48.8) 44 (32.8) Symptomatic 3 (3.8) 6 (4.5) Comorbidity Rheumatological disease 5 (6.3) 35 (26.1) <0.001 COPD 5 (6.3) 2 (1.5) 0.06 Diabetesc 6 (7.5) 10 (7.5) 0.99 Vascular diseased 22 (27.5) 49 (36.8) 0.17 Myocardial infarction 7 (8.8) 5 (3.7) 0.12 Presence of BTPe 49 (63) 74 (55) 0.23 Presence of cancer related painf 60 (75) 56 (42) <0.001 Pain medication 0.007 No pain medication 14 (18) 32 (24) Non-opioid pain medication 13 (16) 46 (34) Opioid pain medication 43 (54) 48 (36) Missing 10 (12.5) 8 (6) a SD = standard deviance. b Patients with active cancer where TNM staging is not applicable (lymphoma, CNS tumour, myeloma). c Diabetes without organ complication. d Peripheral vascular disease excluding coronary artery disease and cerebrova- suclar disease. e BTP = breakthrough pain. f Cancer related pain = pain due to tumour invasion and/or cancer treatment. The proportion of patients starting new opioid medication were 8.8% and 10.5% (Pearsons Chi square = 0.16, p = 0.69) in the pre-intervention and intervention period, respectively. In the preintervention period physicians changed the dose of opioids in 18.8% of the patients compared to 21.6% of the patients in the intervention period (Pearsons Chi square = 0.26, p = 0.61). 4. Discussion The primary aim of this study was to evaluate the efficacy of the Combat system on pain management. The study did not demonstrate improved pain control or an alternation in the prescribed opioid dose. Previous systematic reviews and meta-analyses on CCDS have provided information of the effects CCDS systems on clinical outcomes. Kawamoto et al. [32] conducted a meta analyses of 71 trials where the following factors were associated with improved practice: (i) Automatic treatment suggestions presented to the physician at point of care as opposed to treatment suggestions which was not available during consultation. (ii) CCDS which generated a treatment recommendation as opposed to only an assessment. (iii) A computer generated decision support as opposed to systems depending on manual procedures. Evidence that treatment suggestions at point of care is essential for successful CCDS has also been shown by others [33]. A recent meta-analysis [20] of 162 trials demonstrated that standalone CCDS were more likely to improve care than CCDS within an electronic charting or order entry systems. Additionally, CCDS systems providing advice to both clinician and patients and systems where clinician had to provide a reason for not accommodating to advice were shown to improve patient care. The CCDS used in our study provided decision support at point of care which is important for successful CCDS as shown in metaanalyses [32,33]. The CCDS provided a recommendation (rather than a simple assessment) and generated a computerized decision support to start opioid medication, prompting the presence of, and advising on treatment, of neuropathic pain as well as depression, issues which are crucial for efficient CCDS [32]. Further on, we employed a CCDS as a stand-alone product, which is a prognostic factor for improving care according to Roshanov [20]. However, our CCDS did not provide decision support for the patients. It was not compulsory for the clinicians to provide a reason for not following an advice. Both of these issues may have reduced the efficacy of the CCDS. Further limitations in our study may be related to lack of a proper implementation strategy of the CCDS. A well-planned implementation strategy is essential when introducing new technology in health care [34]. In the current study, we attempted to implement the CCDS through an introductory lecture for the physicians providing instructions on how to use the software and the decision support system as well as written information as a leaflet. When evaluating our efforts in retrospect we did not put enough emphasis into identifying barriers and solutions for a major change in routine clinical practice, hence, our method of implementing the CCDS was insufficient and not powerful enough to change the behaviour of the physicians. For future trials, we would recommend profound implementation strategies. Implementation strategies at the organization level in the context of this study could involve curtained areas in the clinic where patients could complete questionnaires with a tablet PC and a more thorough engagement of the nurse staff. In the present study patients completed questionnaires applying a tablet in the ordinary waiting room and the nurse staff was not involved in the study. Implementation strategies involving health care professionals in the context of this study could imply regularly audit and feedback sessions [35], a tighter collaboration between study nurse and physicians and encouraging communication between physicians and patients on the basis of the completed questionnaires. Follow up lectures, certification procedures or individual follow up are methods which may have facilitated behaviour changes and improved implementation of the CCDS [36]. A basic barrier at an oncology outpatient department, where the primary focus is to treat the tumour and not symptoms, is to change the attention and clinical behaviour towards symptom diagnosis and treatment including pain diagnosis and treatment. Therefore, the interventions need to be well planned, repetitive and with a meticulous attention to observe if the new technology is used as intended. It is our opinion that a well-founded implementation strategy may have changed the behaviour of physicians [37] and improved correct utilization of the Combat system in this study. Another limitation was that the software was not integrated with the electronic medical record. Even though Roshanov et al. argue that a stand-alone CCDS system may be beneficial, Miriovsky Fig. 3. Pain intensities and opioid dose at baseline, follow up 1 week and follow up 3 weeks in the pre-intervention and intervention period. Table 2 Pain intensity and opioid consumption in pre-intervention and intervention period. Mean (95% CI) Mean (95% CI) Adjusted for baseline Adjusted for other variables (*) Pain intensity N = 80 N = 134 Baseline average pain intensitya last 24h 4.4 (4.0–4.8) 4.0 (3.7–4.2) Mean pain intensitya of 1 and up on average pain last 24 h 3 weeks follow 3.6 (3.1–4.1) 3.3 (3.0–3.7) 0.12 (?0.33 to 0.58) 0.86 (?0.41 to 0.58) Baseline worst pain intensitya last 24 h 5.8 (5.4–6.2) 5.6 (5.3–5.9) Mean pain intensitya of 1 and up on worst pain last 24 h 3 weeks follow 4.6 (4.0–5.1) 4.8 (4.4–5.2) 0.32 (?0.27 to 0.91) 0.18 (?0.46 to 0.81) Opioid dosages N = 72 N = 125 Baseline opioid doseb 53 (32–74) 52 (20–85) Mean opioid dose of 1 and 3 weeks follow up 48 (24–72) 59 (22–96) 8.4 (?11.3 to 28) 8.0 (?14.1 to 30.2) a Pain intensity measured on 10-point numerical rating scale from 0 to 10. b Oral morphine equivalent in mg. * Adjusted for the following variables at baseline: Age, gender, presence of breakthrough pain, presence of cancer related pain, presence of rheumatological disease and use of pain medication. et al. [38] endorse that various IT systems used in hospital medicine, such as electronic radiological and laboratory data, may be presented alongside of other clinical data collected from the patient and presented to the physician in an integrated manner. However, this approach has not been, to our knowledge, tested in comparative clinical trials. An individual follow up may have been used to collect data on how the physicians applied the software at point-of-care. Such a data collection could have been applied as a part of the intervention itself, but also as background information to better understand the lack of clinical measurable effects in this study. Only 10.4% and 12% of the patients started with new pain medication and changed existing medication, respectively, among patients in the intervention period which is not significantly different compared to the pre-intervention period. This indicates that physicians did not take advantage of the information conveyed through the Combat system. We did not collect information on how often the physicians accessed or acknowledged recommendations from the CCDS at point of care, which is a limitation in the study. This would have provided valuable insight on the impact of this CCDS system. In future trials involving implementation of new technology such as CCDS, a thorough assessment of how the physicians utilize recommendations from CCDS systems is crucial to develop successful systems [39]. The content and the structure of the software applied in this study may have been another limitation. The software was developed with an emphasis on the electronic PROMs completed by the patients. The presentation of PROMs to the physician was less emphasized and may not have been as mature as the electronic PROMs. Additionally, the decision support was founded on very simple algorithms, hence, the advices may have been perceived as too superficial by the oncologists. We did not examine how the patients experienced the usefulness and the implementation of the CCDS system. Hence, we do not have information about limitations of the system from the patients point of view. One way of obtaining such information is to organize focus groups of patients and providers, which we aim to do in future trials. Twenty-five and 58% of the patients had non-malignant pain in the pre-intervention and intervention periods, respectively. The higher prevalence in the intervention period may be explained by a high number of patients with rheumatic disease. Others have reported a prevalence of non-malignant pain in 25% of cancer outpatients with pain [40,41]. These findings illustrate the importance of non-malignant pain also in a cancer population. The influence from non-malignant pain may be one of the factors that is different between relatively healthy patients at an out-patient cancer clinic and in cohorts with patients with advanced cancer disease. In this study, the decision support algorithm was not programmed to account for non-malignant pain, which may have influenced how the physician interacted with the software. The treatment of non-malignant pain is complex, often difficult to treat and not commonly managed by oncologists. Therefore, the finding of a high number of patients in an out-patient clinic with non-malignant pain may explain some of the lack of efficacy of a decision support defined with the purpose of treating cancer pain. The disadvantage of the controlled before-and-after study approach was organizational changes at our department during the pre-intervention period. This may have resulted in slightly imbalance of patient demographic as shown in Table 1. We omitted a randomized study with a parallel group design due to the risk that physicians in the control group could improve their pain management skills and thus biasing the results. A parallel group design may have been conducted by recruiting patients at other cancer outpatient departments in a cluster randomized trial in order to reduce potential bias. Further on, the study design did not distinguish between the effects of the PROMs and the decision support system. The amount of missing data was higher in the pre-intervention period. This may have biased the treatment effect estimates (differences between two study groups), especially if we hypothesize that patients excluded from the analysis due to missing follow-up assessment had a different pain outcome. 5. Conclusion Even though the current trial did not provide significant results of employing a CCDS to improve pain management, we believe that CCDS have a potential to improve management of cancer patients if correctly designed and carefully implemented as shown in clinical trials [16,18,19]. 6. Implications There is a need to conduct larger prospective studies to improve pain management by using modern information technology. However, such trials must be carefully planned in order to secure a proper implementation of new technology as well as paying close attention to how health care providers utilize modern information technology. It is also necessary to improve the CCDS software by utilizing existing information technology, for instance by facilitating electronic feedback from the patients to the physician about the efficacy of various pain management strategies and supporting automatic referral to a pain specialist when the aim of pain management is not achieved. Ethical issues The study was approved by the Regional Committees for Medical and Health Research Ethics in Norway and all patients provided written informed consent to participate in the study before inclusion. Conflict of interest The authors declare no conflict of interest. Cinzia Brunelli has provided consultancy for Mundipharma Pharmaceurticals. The main author has full control of all primary data and will allow the journal to review the data if requested. Acknowledgment The Norwegian Cancer Society supports the European Palliative Care Research Centre. References [1] Kwon JH. Overcoming barriers in cancer pain management. J Clin Oncol 2014;32:1727–33. [2] Fisch MJ, Lee JW, Weiss M, Wagner LI, Chang VT, Cella D, Manola JB, Minasian LM, McCaskill-Stevens W, Mendoza TR, Cleeland CS. Prospective, observational study of pain and analgesic prescribing in medical oncology outpatients with breast, colorectal, lung, or prostate cancer. J Clin Oncol 2012;30:1980–8. [3] Porta-Sales J, Nabal-Vicuna M, Vallano A, Espinosa J, Planas-Domingo J, VergerFransoy E, Julià-Torras J, Serna J, Pascual-López A, Rodríguez D, Grimau I, Morlans G, Sala-Rovira C, Calsina-Berna A, Borras-Andrés JM, Gomez-Batiste X. Have we improved pain control in cancer patients? A multicenter study of ambulatory and hospitalized cancer patients. J Palliat Med 2015. [4] Cohen MZ, Easley MK, Ellis C, Hughes B, Ownby K, Rashad BG, Rude M, Taft E, Westbrooks JB. Cancer pain management and the JCAHO’s pain standards. J Pain Symptom Manag 2003;25:519–27. [5] Gravis G, Marino P, Joly F, Oudard S, Priou F, Esterni B, Latorzeff I, Delva R, Krakowski I, Laguerre B, Rolland F, Theodore C, Deplanque G, Ferrero JM, Pouessel D, Mourey L, Beuzeboc P, Zanetta S, Habibian M, Berdah JF, Dauba J, Baciuchka M, Platini C, Linassier C, Labourey JL, Machiels JP, El Kouri C, Ravaud A, Suc E, Eymard JC, Hasbini A, Bousquet G, Soulie M, Fizazi K. Patients’ selfassessment versus investigators’ evaluation in a phase III trial in non-castrate metastatic prostate cancer (GETUG-AFU 15). Eur J Cancer 2014. [6] Di Maio M, Gallo C, Leighl NB, Piccirillo MC, Daniele G, Nuzzo F, Gridelli C, Gebbia V, Ciardiello F, De Placido S, Ceribelli A, Favaretto AG, de Matteis A, Feld R, Butts C, Bryce J, Signoriello S, Morabito A, Rocco G, Perrone F. Symptomatic toxicities experienced during anticancer treatment: agreement between patient and physician reporting in three randomized trials. J Clin Oncol 2015;33:910–5. [7] Cleeland CS, Gonin R, Hatfield AK, Edmonson JH, Blum RH, Stewart JA, Pandya KJ. Pain and its treatment in outpatients with metastatic cancer. N Engl J Med 1994;330:592–6. [8] Kaasa S, Apolone G, Klepstad P, Loge JH, Hjermstad MJ, Corli O, Strasser F, Heiskanen T, Costantini M, Zagonel V, Groenvold M, Fainsinger R, Jensen MP, Farrar JT, McQuay H, Rothrock NE, Cleary J, Deguines C, Caraceni A. Expert conference on cancer pain assessment and classification – the need for international consensus: working proposals on international standards. BMJ Support Palliat Care 2011. [9] Zech DF, Grond S, Lynch J, Hertel D, Lehmann KA. Validation of World Health Organization Guidelines for cancer pain relief: a 10-year prospective study. Pain 1995;63:65–76. [10] Du Pen SL, Du Pen AR, Polissar N, Hansberry J, Kraybill BM, Stillman M, Panke J, Everly R, Syrjala K. Implementing guidelines for cancer pain management: results of a randomized controlled clinical trial. J Clin Oncol 1999;17:361–70. [11] Cleeland CS, Gonin R, Baez L, Loehrer P, Pandya KJ. Pain and treatment of pain in minority patients with cancer. The Eastern Cooperative Oncology Group Minority Outpatient Pain Study. Ann Intern Med 1997;127:813–6. [12] Sucher JF, Moore FA, Todd SR, Sailors RM, McKinley BA. Computerized clinical decision support: a technology to implement and validate evidence based guidelines. J Trauma 2008;64:520–37. [13] Cleeland CS, Wang XS, Shi Q, Mendoza TR, Wright SL, Berry MD, Malveaux D, Shah PK, Gning I, Hofstetter WL, Putnam JBJ, Vaporciyan AA. Automated symptom alerts reduce postoperative symptom severity after cancer surgery: a randomized controlled clinical trial. J Clin Oncol 2011;29:994–1000. [14] van Wyk JT, van Wijk MA, Sturkenboom MC, Mosseveld M, Moorman PW, van der Lei J. Electronic alerts versus on-demand decision support to improve dyslipidemia treatment: a cluster randomized controlled trial. Circulation 2008;117:371–8. [15] Terrell KM, Perkins AJ, Hui SL, Callahan CM, Dexter PR, Miller DK. Computerized decision support for medication dosing in renal insufficiency: a randomized, controlled trial. Ann Emerg Med 2010;56:623–9. [16] Bertsche T, Askoxylakis V, Habl G, Laidig F, Kaltschmidt J, Schmitt SP, Ghaderi H, Bois AZ, Milker-Zabel S, Debus J, Bardenheuer HJ, Haefeli WE. Multidisciplinary pain management based on a computerized clinical decision support system in cancer pain patients. Pain 2009;147:20–8. [17] Holbrook A, Thabane L, Keshavjee K, Dolovich L, Bernstein B, Chan D, Troyan S, Foster G, Gerstein H. Individualized electronic decision support and reminders to improve diabetes care in the community: COMPETE II randomized trial. CMAJ 2009;181:37–44. [18] McCowan C, Neville RG, Ricketts IW, Warner FC, Hoskins G, Thomas GE. Lessons from a randomized controlled trial designed to evaluate computer decision support software to improve the management of asthma. Med Inform Internet Med 2001;26:191–201. [19] Schmidt-Kraepelin C, Janssen B, Gaebel W. Prevention of rehospitalization in schizophrenia: results of an integrated care project in Germany. Eur Arch Psychiatry Clin Neurosci 2009;259(Suppl 2):S205–12. [20] Roshanov PS, Fernandes N, Wilczynski JM, Hemens BJ, You JJ, Handler SM, Nieuwlaat R, Souza NM, Beyene J, Van Spall HG, Garg AX, Haynes RB. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ 2013;346:f657. [21] Takeuchi EE, Keding A, Awad N, Hofmann U, Campbell LJ, Selby PJ, Brown JM, Velikova G. Impact of patient-reported outcomes in oncology: a longitudinal analysis of patient–physician communication. J Clin Oncol 2011;29:2910–7. [22] Velikova G, Booth L, Smith AB, Brown PM, Lynch P, Brown JM, Selby PJ. Measuring quality of life in routine oncology practice improves communication and patient well-being: a randomized controlled trial. J Clin Oncol 2004;22:714–24. [23] A. Apple iPad2; 2015. Retrieved from https://www.apple.com/ipad/ [last accessed 08.06.15]. [24] Rayner L, Price A, Hotopf M, Higginson IJ. The development of evidence-based European guidelines on the management of depression in palliative cancer care. Eur J Cancer 2011;47:702–12. [25] Caraceni A, Hanks G, Kaasa S, Bennett MI, Brunelli C, Cherny N, Dale O, De Conno F, Fallon M, Hanna M, Haugen DF, Juhl G, King S, Klepstad P, Laugsand EA, Maltoni M, Mercadante S, Nabal M, Pigni A, Radbruch L, Reid C, Sjogren P, Stone PC, Tassinari D, Zeppetella G. Use of opioid analgesics in the treatment of cancer pain: evidence-based recommendations from the EAPC. Lancet Oncol 2012;13:e58–68. [26] Daut RL, Cleeland CS, Flanery RC. Development of the Wisconsin Brief Pain Questionnaire to assess pain in cancer and other diseases. Pain 1983;17:197–210. [27] Klepstad P, Loge JH, Borchgrevink PC, Mendoza TR, Cleeland CS, Kaasa S. The Norwegian brief pain inventory questionnaire: translation and validation in cancer pain patients. J Pain Symptom Manag 2002;24:517–25. [28] Dworkin RH, Turk DC, Wyrwich KW, Beaton D, Cleeland CS, Farrar JT, Haythornthwaite JA, Jensen MP, Kerns RD, Ader DN, Brandenburg N, Burke LB, Cella D, Chandler J, Cowan P, Dimitrova R, Dionne R, Hertz S, Jadad AR, Katz NP, Kehlet H, Kramer LD, Manning DC, McCormick C, McDermott MP, McQuay HJ, Patel S, Porter L, Quessy S, Rappaport BA, Rauschkolb C, Revicki DA, Rothman M, Schmader KE, Stacey BR, Stauffer JW, von Stein T, White RE, Witter J, Zavisic S. Interpreting the clinical importance of treatment outcomes in chronic pain clinical trials: IMMPACT recommendations. J Pain 2008;9:105–21. [29] Raj SX, Thronaes M, Brunelli C, Hjermstad MJ, Klepstad P, Kaasa S. A crosssectional study on prevalence of pain and breakthrough pain among an unselected group of outpatients in a tertiary cancer clinic. Support Care Cancer 2014;22:1965–71. [30] Frison L, Pocock SJ. Repeated measures in clinical trials: analysis using mean summary statistics and its implications for design. Stat Med 1992;11:1685–704. [31] Hanks G, Cherny N, Christakis NA, Fallon M, Kaasa S, Portenoy RK. Oxford textbook of palliative medicine. Oxford University Press; 2009. [32] Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005;330:765. [33] Blum D, Raj SX, Oberholzer R, Riphagen II, Strasser F, Kaasa S, EURO IMPACT, E. I. M. P. C. R. T. Computer-based clinical decision support systems and patientreported outcomes: a systematic review. Patient 2015;8:397–409. [34] Cresswell KM, Bates DW, Sheikh A. Ten key considerations for the successful implementation and adoption of large-scale health information technology. J Am Med Inform Assoc 2013;20:e9–13. [35] Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O’Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012;6:CD000259. [36] Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet 2003;362:1225–30. [37] Colquhoun HL, Squires JE, Kolehmainen N, Fraser C, Grimshaw JM. Methods for designing interventions to change healthcare professionals’ behaviour: a systematic review. Implement Sci 2017;12:30. [38] Miriovsky BJ, Shulman LN, Abernethy AP. Importance of health information technology, electronic health records, and continuously aggregating data to comparative effectiveness research and learning health care. J Clin Oncol 2012;30:4243–8. [39] Moja L, Passardi A, Capobussi M, Banzi R, Ruggiero F, Kwag K, Liberati EG, Mangia M, Kunnamo I, Cinquini M, Vespignani R, Colamartini A, Di Iorio V, Massa I, González-Lorenzo M, Bertizzolo L, Nyberg P, Grimshaw J, Bonovas S, Nanni O. Implementing an evidence-based computerized decision support system linked to electronic health records to improve care for cancer patients: the ONCO-CODES study protocol for a randomized controlled trial. Implement Sci 2016;11:153. [40] Williams JE, Yen JT, Parker G, Chapman S, Kandikattu S, Barbachano Y. Prevalence of pain in head and neck cancer out-patients. J Laryngol Otol 2010;124:767–73. [41] Valeberg BT, Miaskowski C, Hanestad BR, Bjordal K, Paul S, Rustøen T. Demographic, clinical, and pain characteristics are associated with average pain severity groups in a sample of oncology outpatients. J Pain 2008;9:873–82. 106 S.X. Raj et al. / Scandinavian Journal of Pain 17 (2017) 99–106 S.X. Raj et al. / Scandinavian Journal of Pain 17 (2017) 99–106 107