Measuring the quality and impact of 3D medical printing in surgical planning, procedures and communications using product usefulness surveys

Abstract

3D printing is an essential technology for clinical decision-making, affording significant improvements over conventional imaging alone. Product usefulness surveys offer potential in measuring the quality and impact of 3DP to assist clinical decision-making and patient communication. A narrative literature review (Scoping Review articles only) frames the benefits and limitations of 3D print surgical workflow and user experiences. A usability survey was co-designed with a company to measure the impact of their 3D prints on surgical practice, and secondary data analysis was conducted of to understand the effectiveness and usefulness of the technology in a real-world healthcare setting. Three categories of usefulness and relevance were identified. High-value relevance (positive Likert scores over 70%) related to pre-operative planning, clinical communication, and the value of the technology. Moderate scores (50% to 69%) related to time saving with pre-operative surgery, and effectiveness of patient care and diagnosis. Minor relevance (less than 50%) included direct cost savings, physical resource efficiencies, time in intra-operative surgery, or intra-operative risks. This research considers 3D printings relationship to medical error prevention, limitations and future recommendations for usability surveys of this type and it identified issues around gender inclusion in Design for Health research.

Keywords:

• 3D medical printing
• usefulness
• user experience
• surgical planning
• patient communication

Introduction

3D printing (3DP) or additive manufacturing is a transformative technology in health care. Academic publishing trends indicate a significant increase in publications related to 3DP applied in spinal surgery (Lin et al. 2023) and orthopaedics (Vaishya et al. 2018). 3DP technology was pioneered by Dr Hideo Kodama (1981) and refined by Charles Hull who patented the first stereolithography method in 1984, which laid the foundations for commercial additive manufacturing (Hideo 1981; Pirjan and Petroșanu 2013; Gokhare et al. 2017). The global 3DP market is consistently growing, and it is expected to grow from its 2022 value of $18.33 billion to $105.99 billion by 2030 (FBI 2023). While Design and Engineering account for 72% of the market, healthcare (medical and dental) account for 11% and is one of the fastest growing sectors. The range of medical 3DP applications is broad, including training, diagnostics, surgical simulation, design and the fabrication of implants, prosthetics, medical tools (Javaid and Haleem 2018a) as well as open-source Personal Protective Equipment (PPE) supplied during the COVID-19 pandemic (Ravi et al. 2021). The current literature documents a high level of enthusiasm for 3DP in surgical applications in virtually every area of specialism (Lal and Patralekh 2018) with 78% of maxillofacial, cardiothoracic and orthopaedic procedures utilizing 3DP (Eley 2017). It is changing workflows in cardiac surgery (Vukicevic et al. 2017) with high success rates reported for the particularly complex left atrial appendage closure (LLA) procedure reducing radiation load and optimizing device selection (Oliveira-Santos et al. 2019). In otolaryngology, it has revolutionized the treatment of complex airway malacia (Kaye et al. 2016), and in neurology, 3DP identified pathology that was invisible during imaging of intra-operative contrast agent (Vakharia et al. 2016). Pre-operative planning is the primary use for 3DP in surgery (Eley 2017) and some suggest it is becoming an ‘indispensable tool’ (Parikh and Sharma 2018). Surgical training, preoperative planning, intra-operative navigation, patient communication, education and counselling are among the main benefits 3DP applications, where the categories of use do not require medical device consideration (Yan et al. 2018).

User experience benefits of 3DP in design for health

Medical professionals, who may be from a broad range of professions, experience a challenge in cognitive processing of 2D images, generated from 2D or 3D image modalities, into a 3D representation within their minds (Farooqi and Mahmood 2018). As images are difficult to fully represent and imagine as real-world objects, they may be subject to variation in interpretation, especially for trainees and young surgeons (Bogdanova et al. 2015). 3DP has been termed the ‘4th dimension in surgical procedure’ (Garg and Mehta 2018) or ‘4D surgical navigation’ (Cacciamani et al. 2019) providing additional information where medical imaging visualization alone is inadequate, or where image projection and identification errors occur (Lin et al. 2018). Nevertheless, it remains reliant on 3D medical imaging technology.

A 3D printed anatomical model provides more accurate representation, visual spatial navigation (Marro et al. 2016; Garg and Mehta 2018; Haleem et al. 2018; Lal and Patralekh 2018; Tong et al. 2019), and realistic haptic feedback in surgical planning (Zheng et al. 2016; Oliveira-Santos et al. 2019), including structural density and stiffness simulation in spines (Tong et al. 2019) and physical flow analysis in cardiology (Vukicevic et al. 2017). 3DP has led to innovative processes for surgical planning using reverse/inverse models to mould tissue reconstruction where the patient’s own body tissue fills the gap, as it heals (Shafiee and Alta 2016; Nagarajan et al. 2018), which reduces tissue harvesting (Nagarajan et al. 2018).

With pre-operative and intra-operative planning, time-saving efficiencies up to 22% are widely reported across orthopaedics (Garg and Mehta 2018; Lin et al. 2018), liver surgery (Wang et al. 2018), cardiology (Haleem et al. 2018), spine surgery (Tong et al. 2019) and jaw reconstruction (Serrano et al. 2019). Other benefits through pre-operative planning optimization include reduced risks, blood loss and morbidity and intra-operative benefits of improved operative team communication, predictability and quality of treatment outcomes saving lives (Eley 2017; Martelli et al. 2016). The impact of 3DP on diagnostic surgical planning while positive, is variable and not directly comparable. For example, in Urology between 30% and 50% of surgeons changed their approach after using a 3D model replica of anatomy (Cacciamani et al. 2019), 65% within maxillofacial, cardiothoracic and orthopaedic surgery and there was a 52%–74% influence on the choice of implant material and site in complex spinal surgery (Tong et al. 2019).

Patient communication has been reported to have significant benefits across maxillofacial, cardiothoracic, orthopaedic, urology and neurosurgery (Vakharia et al. 2016; Vukicevic et al. 2017; Garg and Mehta 2018; Cacciamani et al. 2019; Eley 2019). Some described the technology as having ‘revolutionary potentials for patient counselling, pre- and intraoperative surgical planning, and education in urology’ reporting improved patient knowledge and understanding of kidney disease (Cacciamani et al. 2019). In medical education, 65% of cases showed improvements using 3DP compared to visualization methods (Tong et al. 2019) and clinical knowledge improvements were widely reported (Vakharia et al. 2016; Garg and Mehta 2018; Eley 2019), with higher scores in surgical planning and educational evaluation when using 3DP (Zheng et al. 2016; Tong et al. 2019). There has been >25% increase in informed consent for spinal surgery and increases in patient treatment compliance alongside reduced patient anxiety (D’Urso et al. 1999; Tong et al. 2019).

Current limitations of 3DP for healthcare

3DP must be used with other technologies, as it is not able to provide information on ‘blood loss, blood clot; chest wound infection, and metabolic abnormalities’ and is co-reliant on medical imaging data (Haleem et al. 2018). However, it is medical imaging which is the limiting factor for print resolution (Eley 2017). Furthermore, invasive imaging modalities are subject to strict guidelines (IRMER 2017), optimizing radiation exposures to the patient and operator, which may affect data resolution and data acquisitions may have artefacts affecting quality (Tong et al. 2019). Some anatomy may be too small to print in precision scale due to their fragility during the build process, like small perforating vessels which are normally <1mm (Vakharia et al. 2016). Although shrinkage may occur in the print process, this is accommodated by pre-emptive scaling in production (Eley 2017). Improvements have been suggested including validation of 3DP model accuracy across the dimensional variability of imaging modalities, consistency of 3DP production settings and variable surgical scenarios (Vukicevic et al. 2017). According to Vakharia et al. (2016) only one study verifying the accuracy of 3DP against imaging data exists (Anderson et al. 2015), which was highly favourable. Overall, there is a call for standardization (Oliveira-Santos et al. 2019; Farooqi and Mahmood 2018) alongside improvements for technology accessibility, clinical-utility and usefulness of 3DP. Other limitations include the materials available to replicate the diversity of tissue in the human body, production lead times, costs of printing and shipping and potentially patient privacy issues due to data transfer (Marro et al. 2016; Yan et al. 2018; Oliveira-Santos 2019; Tong et al. 2019; Cacciamani et al. 2019; Javaid and Haleem 2018b; Parikh and Sharma 2018). The segmentation process (converting a medical image into a 3D digital model) has been highlighted as a potential barrier, due to the time and cost, due to the manual process in some workflows. Even processes that are efficiently conducted within a 24 h delivery time remain too long, especially in emergency procedures (Oliveira-Santos et al. 2019) or in hospitals with high turnover (Garg and Mehta 2018). In some cases, 3DP models arrived after the operation was completed (Eley, 2017). A review by Oliverira-Santos et al. (2019) outlined that ‘proprietary automated segmentation software will swiftly generate 3D files ready for printing, with standard materials’. In this context, Axial3D, a Belfast based 3D medical printing company are recognized as among the top companies internationally (e.g. Global Digital Health 100 list, 2020) and innovators (KTP Awards 2022, Business Impact and Transformation finalist). Their software (Harpur et al. 2018) Axial Insight received approval clearance from the FDA in July 2023 as a Class II medical device (FDA: 510(k) NO: K222745) and provides a tool to better manage issues associated with 3DPoutlined above. Its workflow pipeline can produce 3D prints from medical imaging managed through the hospitals PACS department, securely for supply within 48 hours (Figure 1).

Figure 1. Axial3D medical printing workflow pipeline (Axial3D Ltd.).

There are four stages that involve (1) The remote upload of medical imaging data by the clinician (2); an artificial intelligence (AI) process of image segmentation to produce a 3D digital twin of the anatomy; (3) an interactive 3D visual representation used for clinical verification and automated technical verification process; (4) issue of a verified 3D digital twin used for print export, CAD export or to assist patient communication or pre-operative planning alongside a physical 3D print.

The current research study aims to understand the impact of the Axial Insight product, through the extent of its usefulness and the nature of its relevance, following operator (surgeon) feedback after they used the software and associated 3D models for surgical decision making. The iterative survey design in this study investigates the usefulness of 3DP across different stages of the surgical process and the critical evaluation thereafter aims to inform best practice protocols for data gathering in professional and commercial situations.

Methodology

Within the medical printing industry, it is important to understand the value of 3D printed products for clinical service and benefits to their patients. However, within a commercial setting they often seek positive affirmation as a means of market traction or security, essential to business. This can lead to a tension between market driven approaches, which may be positively affirmed, verses authentic healthcare quality requiring balance. A similar challenge is identified in the case of patient satisfaction surveys (Dunsch et al. 2018). Axial3D have conducted satisfaction surveys in an ongoing manner to improve their clinical service and commercial competitiveness. In the present study, the research team made recommendations for the company’s research and user feedback strategy. A critical analysis of their survey practices was conducted, including survey design structure, granularity of data fields, usability analysis, research integrity and reducing bias. We found several question types that were positively loaded with little or no option for disagreement, or critical response. This can lead to experimenter demand effects (EDE) with the participants aligning most of their views to match the survey objectives (Mummolo and Peterson, 2019). Following iterative review, the new survey (Figure 2) provided more granular information suitable for comparative analysis, yet in a succinct format to improve completion rates in a real-world setting.

Figure 2. Modified Axial3D survey following academic review.

Likert scales were introduced within questions affording balanced responses of agreement or disagreement. New categories of data collection included the age of the operator, product number identification (mapping to surgical category and purpose within company order record), the type of surgical procedure and its stage of process (such as pre-, intra- and post-surgical), purpose of planning (pre-operative and in theatre decisions), medical error/risk questions, qualitative feedback fields and GDPR Regulation (EU) 2016/679 (Art. 17) considerations, for example enabling anonymous yet traceable data to permit the participants ‘right to be forgotten’. The Axial3D survey data was all held on the company’s Google cloud instances within the European Union and meets GDPR compliance. As a result, there were 13 questions in the revised survey, three for participant data, seven that focused on specific areas of impact and effectiveness, one open ended feedback question and two relating to privacy and future communications. Of the seven specific questions, five used 5-point Likert scales that were equally balanced between positive and negative opinions (removing the positive bias) and two questions provided a drop-down menu with 13-time duration choices, in increments of 20 mins including six positive (20 to 120+ minutes shorter) and six negative impacts (20 to 120+ minutes longer) and one neutral option (N/A). These were:

1. Effect on surgical planning and diagnosis (Likert)

2. Economic impact (Likert)

3. Impact on pre-operative time (drop down menu)

4. Impact on intra-operative time (drop down menu)

5. Effectiveness in clinical communications (Likert)

6. Effectiveness of the model on patient operative processes (Likert)

7. Valued purpose of 3D model (Likert)

8. Open ended comment

In 2020, Axial3D supplied 75 bespoke 3DP models to clinicians based at hospitals in Northern Ireland to assist with the backlogs of clinical services, disrupted by COVID-19, funded by Invest Northern Ireland. Participating hospitals had the option to respond to the Axial3D survey. This resulted in a positive response rate of 71% (n = 53). The survey was managed by Axial3D and circulated between July 2020 and May 2021. Ulster University researchers were later approached to undertake secondary data and were granted ethical approval by the Art & Design Research Ethics filter committee, Ulster University (REF: FCART-23-002) in March 2023. Thereafter, Axial3D shared their anonymized data with the research team.

Data analysis methods

All data were supplied on a Microsoft Excel workbook, with two sets of data. One page provided information on the 3D model used, location and purpose. The other dataset provided data that were extracted from the survey with 21 columns of question response data. The participants’ demographics (age range), profession (department, specialty, anatomical region) and views on the impacts of personalized 3D anatomical models (planning, diagnosis, economic and time impacts, model effects, overall views) data were analysed using IBM SPSS Statistics v28.0.0. Descriptive statistics was run to compute means, frequencies and percentage distribution and the relationships between the different data points was explored by crosstabulation. The data tables were then exported to MS Excel to generate relevant charts for visualization. The qualitative data collected from the comments section was thematically analysed in MS Excel for emerging themes and sub themes.

Results

There were 59 responses to the survey, which included six duplications (all answers were identical). From the 53 participants, 13% were aged 55+ years, 26% aged 45–55 years, 55% aged 35–45 years, 2% aged 25–35 years and 4% were undisclosed. Responses were received from seven identifiable hospitals or trusts from across Northern Ireland, including Belfast HSC Trust, Royal Belfast Hospital for Sick Children, Royal Victoria Hospital, Southeast HSC Trust, Southern HSC Trust, Ulster Hospital and Western HSC Trust. There were six clinical professions represented in the response, which, in order of frequency were Cardiology (49%), Orthopaedics (28.3%), Maxillofacial (15%), Trauma (3.8%), Cardiothoracic (2%) and Oncology (1.9%). There were 17 different types of anatomy supplied as 3DP models (Figure 3)

Figure 3. Distribution of 3DP models by region of anatomy and clinical department (n = 53).

Effect on surgical planning and diagnosis

Here, there were three categories analysing the effect on (1) planning the procedure (2) alteration of diagnosis and (3) choice of equipment used (Figure 4). The participants strongly agreed that the anatomical models affected their surgery plans (98%). These respondents indicated that it altered their plans completely (34%), changed their plans a modest amount (19%), or the model was valuable to re-enforce their decision (45%). Only 2% said that the models were ‘not helpful’ and there were no 'negative impacts’ reported. 3DP would appear to have a key role in surgical planning and affirmation. On whether the 3D models influenced diagnosis 70% agreed that it altered or affected the patients’ diagnosis. 11.3% indicated that their diagnosis changed because of the 3D model, 47.2% indicated that it was valuable to re-enforce their decisions while 11.3% indicated that it altered the diagnosis by a modest amount. However, the remainder of the respondents (30.2%), said the models were not helpful in diagnosis, but none of these reported a ‘negative impact’. On a closer inspection of the largest pool of respondents who were from cardiology (49%) 12 (63% of this specialism) felt that the 3D models were valuable to re-enforce the decisions while seven (36% of the specialism) indicated that the models were not helpful. This has some correlation with the entire cohort analysis of 70% and 30%, however further research into each surgical specialism is necessary to identify specific variable characteristics. The surgeons agreed that the 3D models affected their choice of equipment with 24 (45.3%) of them saying it was valuable to re-enforce their decisions. The remaining 29 (44.7%) believed the models had a modest effect (20.8%), changed the choice of equipment (18.9%) and had no effect or were not helpful (15%).

Figure 4. Comparison of the effectiveness of 3D printing for planning and diagnosis across all surgical specialisms (n = 53).

Economic impact – cost savings

Here, there were two categories analysing the impact on (1) equipment used and (2) duration of patient hospital stay. On the aspect of the economic impacts on the equipment used in theatre, 20 (37.7%) of the surgeons indicated that the 3D models had no impact. Another 40% agreed that the models contributed from moderate to significant savings. The 20 who indicated that the models had no economic impact on theatre equipment were from three specialties cardiac (55%), orthopaedic (40%) and trauma (5%). More than half of the surgeons agreed that the models did not have an impact on patient hospital stay. 30 out of the 53 total respondents (56%) indicated they had no impact, and the rest selected not applicable (23%). Only 21% said that the models contributed from moderate to significant savings.

Impact on operative time – time savings

Here, there were two questions analysing (1) pre-operative planning and (2) intra-operative decision making (Figure 5). In each case, a dropdown menu with thirteen choices were presented. The feedback from the survey indicated that the 3D models allowed the surgery teams to save on time in pre-operative surgery planning. Collectively, 58% of the surgeons reported shorter planning time with only 6% reporting longer times. The final 36% reported that it was not applicable for various reasons. However, most of the surgeons (28%) noted a pre-planning time shorter than 20 min. On the other hand, intraoperative time was indicated as shortened by employing the models (57%), not applicable (41%) and one instance that selected longer intra-operative time (2%). A further assessment of the distribution of time, the feedback shows that while the models may have caused an increase in the planning time, they possibly led to a shorter intra-operative time. A possible cause and effect of the anatomical models.

Figure 5. Time effects of 3D printed models on intra-operative and post-operative planning times.

Effectiveness in clinical communications

Here, there were three categories analysing (1) clinical communications with the patient, (2) surgeon to surgeon communication, and (3) trainee or education communication. The surgeons selected that the 3D models were helpful (22.6%) and extremely helpful (49%) in patient communication and education. However, there were those who thought the models were not helpful (5.7%), did not have an impact (7.6%) or selected not applicable (15.1%). Those of the opinion that the 3D models did not have an impact were from the cardiac and orthopaedic specialties. The surgeons seem to agree that the 3D models had a positive effect on surgeon-to-surgeon communication and education. 90% (48 out of the 53 respondents) indicated that the models were extremely helpful (66.0%) or helpful (24.5%). However, the remainder of the surgeons indicated that this was not applicable (9.4%). As with the previous positive reporting of the effect the model had on surgeon-to-surgeon communication, the surgeons indicated similar views on the effects of the 3D models on trainee communication and education. 92% of the respondents indicated that the models were helpful or extremely helpful.

Effectiveness of the model patient operative processes

Here, there were four categories analysing (1) patient safety, (2) patient anxiety, (3) reducing patient risks, and (4) gaining the patient consent. The surgeons’ opinion on the models’ effect on patient safety was positively inclined with 72% (n = 38) agreeing that it had a very positive (39.6%) and positive (32.1%) effect. The rest of the respondents felt that it had no effect (28.3%). The question on the effect of the 3D models on patient anxiety indicated that 45.3% of the surgeons thought it had no effect. However, the remainder of respondents felt that it had a positive effect (n = 29 or 26.4%) and a very positive effect (28.3%) on patient anxiety. Regarding the question on the effect of the 3D models on reducing patient risk, 42 of the respondents (38%) believed that it had a positive or very positive effect (41%). However, the remaining 21% thought it had no effect whatsoever on reducing patient risk. The surgeons were of the opinion that the 3D models had a very positive (34%), positive (36%) while the rest thought it had no effect (30%).

Value purpose of 3D printed models

Here, there were three categories analyzing clinicians’ views on 3DP in the context of (1) comparison with 2D images for diagnosis, (2) improvements on surgical outcomes, and (3) as a cost saving tool (Figure 6). The surgeons seemed to be of the same opinion from their responses to 3D models being a better method than 2D images for diagnosis and planning with 88.7% indicating that they strongly agree and agree while the rest were neutral (11.3%). The surgeons’ views on the effect of 3D models on improving surgical outcomes was unanimously positive with 92.4% indicating they agreed and strongly agreed with the statement. The remaining four respondents were neutral (7.6%). On whether 3D models were a cost saving tool, 45.3% were neutral while 5.7% disagreed that they were cost saving. On the other hand, almost half of respondents (49%), agreed and strongly agreed that they were cost saving.

Figure 6. Perceived value purpose of 3D printed anatomical models on surgical outcomes, cost saving and as compared to 2D images.

Open ended question

From the 53 participants, there were only five who did not comment. From the 48 responses, 81% were positive with 6% neutral, and 4% negative. Upon further analysis of the comments, the recurring themes identified are fidelity and quality of the models; planning, time, and cost effects; visualization, communication, and training; patient consent and safety.

Discussion

The scope of user experience in this case as a subjective concept and how this shaped the approach taken to develop the questionnaire

The objective of user experience evaluation is to determine the user’s perceived feelings resulting from the anticipated or actual use of a product. A fitting definition for this context is given by Nielsen-Norman Group ‘user experience encompasses all aspects of the end-user’s interaction with the company, its services and its products’ (NN group). The scope of user experience varies when interacting with different products, a key characteristic is that it is situated in context which is the mainstay in the evaluation of medical devices. Consequently, various protocols and methods are employed in the evaluation of different devices (Liljegren 2006; Bhutkar et al. 2013). Standardized questionnaires are popular in the evaluation of user experience which can be administered before, during or retrospectively after interaction with a product (Sauro and Lewis 2012; Díaz-Oreiro et al. 2019; Rotaru et al. 2020; Schrepp 2021). However, there are instances that require custom questionnaires to measure the experiences of specific use cases or technologies (Schrepp, Hinderks, and Thomaschewski, 2017; Hinderks et al. 2020). In the Axial 3D case, a custom survey was designed to collect the participants’ views on the user experience after they employed the use of 3DP anatomical models in surgical planning. The emerging themes on diagnosis, surgical planning, communication, cost, and time savings are all perceived customer benefits. While positive and negative scales were used, alongside an open-ended question, EDE biases may remain, as surgeons would be aware of the business objectives, the value perceptions and needs of their profession. Schlegel et al. (2022), in a literature review to investigate 3DP evaluation survey questions identified three overarching constructs (1) anatomy, (2) utility, and (3) experience. The most common questions were on anatomical accuracy, diagnosis, communication, surgical planning, and general experience (Schlegel et al., 2022). This conveys the homogeneity of stakeholder interests in the use of patient-specific 3D printed models in surgery planning. The area of evaluation of 3DP anatomical models is still developing, and at the time of this study, there was no standardized (or proposed) evaluation questionnaire. The 3DP evaluation landscape is heterogenous with custom surveys designed to measure different aspects of interest. The Radiological Society of North America has since released a 3DP Registry Data Dictionary with guidance questions on user assessment (RSNA, 2023). Schlegel et al. also propose an evaluation survey of 3DP anatomical models, which incorporates questions from the 3DP registry Data Dictionary, which is a paradigm shift and may be the next standardized 3DP survey (Schlegel et al., 2022).

Wider implications for the use of 3D medical printing on medical error prevention

A major area of current concern is medical errors (or iatrogenic errors). Two major reviews have been conducted 10 years apart but report similar issues. In the UK, errors occured in 5%–15% of admissions (45% of these in surgery) with human error being a major contributor. The ‘Swiss Cheese Effect’ is where it is not a single error but a sequential accumulation (O’Connor et al. 2010). In the most recent review, medical errors result in 400,000 preventable deaths per year costing $20B and is ‘a leading causes of death’ in the USA (Rodziewicz and Hipskind 2020). Therefore, monitoring or measuring the multiple points of error reduction is a major data analysis focus, with patient benefits and commercial value. From the results of the Axial3D survey study, the 3DP model has a potentially significant role addressing specific issues. Using Rodziewicz and Hipskind’s (2020) observations, we specify the potential of 3DP error mitigation.

• A forum to question senior surgical staff about the procedure, ensuring the surgical team is listened to during planning:

• Intra-operative use of a model to avoid reliance on memory:

• Incomplete assessments avoided with enhanced visual and tactile information.

• Facilitating communication between multiple surgeons on one operation.

• Evidence-based operational efficiencies; time pressures are alleviated.

• Wrong site surgical issues avoided. While, in spine surgery a review found that the complication rate and screw misplacement rate remain unaffected when using 3DP (Garg and Mehta 2018). The Axial Insight survey study found a case in cardiology, where a specific error was avoided, with one surgeon reporting

• Potential to mitigate wastage of surgical instruments, i.e. specimens, e.g. multiple cardiac valves.

‘This was very helpful, when we were viewing the CT, the model had been reconstructed from the ventricular side so everything was a mirror of what it should be. The model instantly highlighted our mistake and helped us plan the operation better highlighting areas that we need to be aware of and how close the structures are when the valve is being inserted’.

Limitations

The questions in the survey were closely aligned to Axial3D’s perceived value of the anatomical models for surgery planning, due to positive customer responses through iterative development of a technology serving their needs. Thus, there was a level of inherent subjectivity in nature which influenced the possible evaluation of significant correlations. As the target participants of this survey were surgeons who were beneficiaries of 3DP anatomical models from Axial3D, they may be considered early adopters, and aware of the value proposition of personalized 3DP models in surgery which may have influenced their views positively, a typical early adopter characteristic (Rogers 2003; Dedhayir et al. 2017).

The data show a somewhat dispersed distribution of surgical specialism, with the majority in the field of cardiology (49%) and orthopaedics (28%); however, as the survey considers the value of the 3D printed model in context of its use the data remains useful.

One of the emerging findings, relevant to future research of this topic is a significant limitation in the demographic data, which did not include gender. Furthermore, it indirectly excludes females due to the focus on surgeons only. Within the UK and Australasia, there is a significant underrepresentation of women in surgery (11%), identifying six distinct contributing factors that negatively affect women’s career prospects in surgery (Liang, Dornan and Nestel 2019). Most responses in the Axial3D survey were from cardiology (49%) and orthopaedic (28%) consultants which are among the most male dominant practices with only 10.8% and 7.3% respectively for female representation in the UK (Newman et al. 2022). Their report outlined that across all surgical specialities 16.1% of consultants and 34.2% of registrars are female. One step to increase the female surgical perspective would be to ensure registrars are also included in future studies as it is estimated to be almost 60 years before parity might be achieved in some of these surgical specialist practices.

Another characteristic missing from the survey relates to ethnicity. Northern Ireland is a very homogenous population, nevertheless capturing and representing the view of individuals across ethnicity and gender would ensure their inclusion is monitored, reported and a diversity of voices shared within the research community.

Conclusion

The survey had an excellent response rate of 71% with 53 valid responses across seven identifiable hospitals in Northern Ireland from six different surgical professions, with cardiology being the largest representation with almost 50%. The Likert scale responses were largely in agreement with positive quality measures and benefits, meeting the expectations of Axial3d, for the intended use of their technology. The findings have been summarized into three categories, contextualizing the impact of the Axial3D Insight software and 3D printed models in terms of its usefulness and relevance, ranking each finding using the highest score in numeric order.

1. High usefulness or relevance (over 70%): This category tended to be associated with pre-operative planning, clinical communications and technological value.

   • 98% strongly agreed that 3D printed models affect surgical planning.

   • 92% reported helpfulness for trainee education.

   • 92% reporting that 3D printed models have improved surgical outcomes compared to imaging alone.

   • 90% reported helpfulness in surgeon-to-surgeon communication and education.

   • 87% felt that 3D models are a better method for diagnosis and planning than 2D imaging modalities.

   • 71% of surgeons reported that models were helpful in communication with the patient.

   • 71% agreed that the models were helpful for patient care.

   • 70% agreed that the models were useful in gaining patient consent.

   • 70% agreed that 3D printed models altered patient diagnosis.

2. Moderate usefulness or relevance (50% to 69%) in time saving with preoperative surgery, intraoperative surgery, and the effectiveness in respect of patient care and diagnosis.

   • 58% of surgeons reported a saving on time with preoperative planning.

   • 57% reported time saving in intraoperative surgery.

   • 54% felt that the models were useful to reduce patient anxiety.

3. Minor usefulness or relevance (less than 50%) was found in cost savings, use of equipment or hospital resources, time in intraoperative surgery, or risks during intraoperative procedures.

   • 49% believed that there was some cost savings overall (aligning with Tong et al. (2019) where total cost savings were still reported in 44% in spinal surgery after factoring in technological infrastructure and expenditure.

   • 41% believed that it had a positive effect on reducing operative risks.

   • 40% identified moderate or significant impacts on equipment used.

   • Only 21% identified that models impacted on the duration of the patients stay, which correlates with Garg and Mehta (2018) who found hospital stays unaffected.

From the open-ended questions, 81% of qualitative comments were positive, 6% were neutral and 4% were negative (Table 1). However, in respect of inclusive data, certain clinical professions have a gender bias therefore including other related professionals, in particular registrars as well as surgeons, would provide a broader and more inclusive voice within Design for Health research. In the current research topic, this would more than double the available pool of female participants.

Table 1. Analysis of emerging themes and quotes from respondent comments.

Comment themes	%Total	Quotes
Planning	23	Extremely useful model to visualize a complex fracture and plan reconstruction.
Design and quality	19	The quality of the model is far superior than that of leading brands in the industry.
Communication	12	Especially useful for discussion/explanation of injury and surgery with patient.
Patient Safety	10	Invaluable model which confirmed that surgery/intervention was too risky and for medical management.
Visualization	7	Really great view of the coronary arteries which allowed me to plan my valve that I would implant and give warning if it would occlude the Ostia
Confidence	6	Very helpful in confirming planned operation
No comment	6	n/a
Time	5	Saved significant amount of time in planning and surgery.
Patient consent	5	Great way of getting patients understanding and in consent
Fidelity	4	The 3D model is definitely better for visualizing the course on anomalous coronary arteries than standard CT scanning
Cost	1	I don’t really feel they help with cost reduction or time planning: however, the worth to me is in making sure we don’t dilate stents and crush coronary vessels-so major patient safety; and also, in consent etc
Training	1	This is one of my favourite models and I am going to use it for teaching.

Disclosure statement

Axial3D are a spin out company from biomedical engineering at Ulster University in 2016. The Axial 3D Insight software was included as a research output for Ulster University’s Art & Design Unit of Assessment and featured in an Impact Case Study for REF2021. The company survey evaluation and redesign were funded by Invest Northern Ireland, Innovation Voucher scheme (No. 130222014, 2020), as part of wider research making recommendations for the company’s future strategy (Justin Magee and Raymond Bond). The data analysis was completed by an independent PhD researcher (Ozelle Kimalel), studying usability of medical devices and then checked by the academic team for accuracy.

Additional information

Funding

Axial3D’s segmentation software Axial Insight was funded through a design-led Innovate UK Knowledge Transfer Partnership (2017-2019: KTP010763) with Ulster University (Magee and Wilson). We thank Axial 3D for permission of the images for Figures 1 and 2.

Notes on contributors

Justin Magee

Justin Magee is a professor of Design and Research Director for Belfast School of Art. He attained his PhD at Ulster University in 2010 titled ‘3D digital modelling of spinal posture’. His research spans Design for Health, Experience Design and Immersive Technology applications. His applied research has received national recognition for Impact and Knowledge Exchange in the Innovate UK KTP Awards, Business Transformation and Impact finalist (2022) and the Times Higher Education Leadership Management Awards, KE initiative of the year finalist (2018). He has secured £16.1m across 29 external design research awards and is Principal Investigator on 17 of these (£6.25m) securing funding from AHRC, UKRI, Innovate UK, Invest NI, Enterprise Ireland, British Council, DCAL and HSCNI. As a practicing product designer, he has worked on >85 commercial projects (e.g., Randox, Smart MCC, Mercedes and LEGO Systems). He has served on the HSCNI Office for Research Ethics Committee (2017-2019).

Ozelle Kimalel

Ozelle Kimalel is a third year PhD researcher in the Belfast School of Art at Ulster University. Her current project explores human factors evaluation in medical device design with a specific interest in measuring usability and user experience. Her research interests include health technology evaluation, medical devices, emerging digital health technologies, and healthcare delivery technologies. Ozelle completed her undergraduate degree in Information Systems at USIU-Africa in Kenya and her master’s degree in Human Computer Interaction at the University of Nottingham. She is also an alumnus (2019 cohort) of the Healthcare Entrepreneurship program by MIT Bootcamps and Havard Medical School.

Kyle Boyd

Kyle Boyd graduated with a BSc (Hons) Interactive Multimedia Design in 2007 and again in 2009 with a MA Multidisciplinary Design. In 2014 Kyle graduated with a PhD entitled ‘An investigation into improving the Usability of Social Media for older users and their carers’. All three awards were completed at Ulster University. Dr. Kyle Boyd has research interests which range from Interaction Design, Human-Computer Interaction, User Experience Design, User Interface Design, User Experience Research, Usability Engineering, Social Media and Digital Health which is the application of digital technology in healthcare. He has published in these areas with papers spanning usability, Interface design and measuring user experience. He has been involved on research projects funded by InvestNI, InterTrade Ireland, HSC, AHRC, ESPRC and Interreg.

Raymond Bond

Raymond Bond is a professor of Human Computer Systems at Ulster University. His research interests include the application of human-computer interaction and machine learning techniques within digital health and clinical decision making. He also has research interests in usability engineering/user experience data analysis methods to evaluate medical devices and is involved in designing and evaluating digital health and wellbeing interventions. Raymond has over 450 research outputs and has chaired/co-chaired national/international research events, including: 1) 32nd International BCS Human Computer Interaction conference, 2) the 45th/46th Annual Conference of the International Society for Computerized Electrocardiology, 3) the 31st Annual European Conference of Cognitive Ergonomics, 4) 2022 Irish Human Computer Interaction (iHCI) Symposium and 5) International Digital Mental Health and Wellbeing Conference.