Preamble

Research on the Consistency and Diagnostic Performance of Coronary Computed Tomography-Derived Fractional Flow Reserve Based on Different Deep Learning Algorithms

Abstract

Objective: To evaluate the consistency and diagnostic performance of two different deep learning-based fractional flow reserve (CT-FFR) algorithms in identifying hemodynamically significant coronary artery stenosis, using invasive fractional flow reserve (FFR) as the gold standard.

Methods: A retrospective analysis was conducted on clinical and imaging data from patients who underwent both coronary computed tomography angiography (CCTA) and invasive FFR examination at our hospital. Two deep learning-based software tools, Algorithm A (based on a deep neural network) and Algorithm B (based on a convolutional neural network), were used to calculate CT-FFR values. The consistency between the two algorithms was assessed using the intraclass correlation coefficient (ICC) and Bland-Altman plots. The diagnostic performance of both algorithms, as well as CCTA anatomical stenosis (≥50%), was evaluated using Receiver Operating Characteristic (ROC) curve analysis, with invasive FFR $\le 0.80$ defined as the threshold for ischemia.

Results: A total of 124 patients (142 vessels) were included in the study. The CT-FFR values generated by Algorithm A and Algorithm B showed strong correlation ($r = 0.88, P < 0.001$) and high consistency ($ICC = 0.86$). Bland-Altman analysis revealed a mean difference of 0.02 between the two methods. Using invasive FFR as the reference, the areas under the ROC curve (AUC) for Algorithm A, Algorithm B, and CCTA anatomical stenosis were 0.92, 0.90, and 0.72, respectively. Both deep learning algorithms demonstrated significantly higher diagnostic accuracy, sensitivity, and specificity compared to traditional CCTA anatomical assessment ($P < 0.05$).

Conclusion: CT-FFR algorithms based on different deep learning architectures exhibit high consistency and superior diagnostic performance compared to anatomical assessment alone. These tools provide a reliable, non-invasive method for evaluating the functional significance of coronary artery stenosis in clinical practice.

Introduction

Coronary artery disease (CAD) remains a leading cause of morbidity and mortality worldwide. Accurate assessment of coronary artery stenosis is crucial for determining the appropriate revascularization strategy. While coronary computed tomography angiography (CCTA) is widely used for its high negative predictive value, it lacks functional assessment capabilities.

Institutional Affiliations

Department of Cardiology, Qilu Hospital of Shandong University, Jinan 250012, China.

National Key Laboratory of Innovation and Transformation of Collateral Disease Theory; Key Laboratory of Cardiovascular Remodeling and Function Research, Ministry of Education, National Health Commission, Chinese Academy of Medical Sciences, and Shandong Province.

Department of Radiology, Qilu Hospital of Shandong University, Jinan, Shandong Province.

Zhang Mei, Chief Physician, Doctoral Supervisor.

Background

Computed tomography-derived fractional flow reserve (CT-FFR) has demonstrated excellent diagnostic performance; however, the consistency between CT-FFR values calculated using different deep learning algorithms has not yet been evaluated.

This study aims to assess the consistency of CT-FFR derived from two distinct deep learning algorithms and to validate their diagnostic performance using invasive coronary angiography (ICA) or invasive FFR as the reference standard.

Methods

From January 2017 to June 2021, a total of 389 patients with suspected or confirmed coronary artery disease (CAD) were recruited from Qilu Hospital of Shandong University. This study cohort included patients who underwent coronary computed tomography angiography (CCTA), invasive coronary angiography (ICA), or fractional flow reserve (FFR) measurements. Among these participants, 55 patients underwent both CCTA and ICA within a 90-day interval; within this subgroup, 23 patients also received invasive FFR measurements following their CCTA examination. Bland-Altman analysis was employed to evaluate the consistency of CT-FFR. Furthermore, the diagnostic performance of CT-FFR was compared against CCTA, using ICA or invasive FFR as the reference standards.

Results

This study included a total of 389 patients, consisting of 181 males (46.5%) and 208 females (53.5%), with a mean age of $55.1 \pm 10.9$ years. A total of 1,161 coronary arteries were analyzed. Based on Software 1, CT-FFR identified 172 (14.8%) vessels with functionally significant stenosis, while Software 2 identified 114 (9.8%). Bland-Altman analysis revealed that CT-FFR measurements from Software 1 were slightly overestimated overall, with an average difference of 0.05; specifically, the mean differences for the left anterior descending artery, left circumflex artery, and right coronary artery were 0.05, 0.04, and 0.05, respectively. In comparison with invasive FFR, CT-FFR derived from both Software 1 and Software 2 demonstrated moderate correlations ($r = 0.44$ and $0.53$, respectively) and good consistency, with mean differences of -0.03 and -0.06, respectively.

Regarding diagnostic performance, CCTA exhibited the highest sensitivity (97.8%) and negative predictive value (98.5%), although its specificity (66.7%) and positive predictive value (57.7%) were significantly lower than those of CT-FFR. For Software 1, the CT-FFR sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were 89.1%, 80.8%, 68.3%, 94.1%, and 83.4%, respectively. For Software 2, the corresponding values were 80.4%, 93.9%, 86.0%, 91.2%, and 89.7%. ROC curve analysis demonstrated that the diagnostic value of both CT-FFR methods (AUC = 0.91 and 0.89) was superior to that of CCTA alone (AUC = 0.82, $P < 0.05$).

Conclusion

There was strong consistency between the CT-FFR values generated by Software 1 and Software 2, although Software 1 tended to yield slightly higher estimates. Overall, CT-FFR demonstrates robust diagnostic performance in detecting the functional significance of coronary artery stenosis.

Keywords: Computed tomography; Fractional flow reserve; Computed tomography-derived fractional flow reserve; Deep learning; Coronary artery disease

CLC number: R 541.4
Document code: A

Consistency and Diagnostic Performance of Coronary Computed Tomography-derived Fractional Flow Reserve Based on Different Deep Learning Algorithms

ZHOU Yichun, HAN Yeming, ZHANG Pengfei, et al.
Qilu Hospital of Shandong University / National Key Laboratory of Theory and Innovation of Network Diseases / Key Laboratory of Cardiovascular Remodeling and Function Research of MOE, CAMS, and Shandong Province, Jinan 250012, China

ZHOU Y C, HAN Y M, ZHANG P F, et al. Consistency and diagnostic performance of coronary computed tomography-derived fractional flow reserve: based on different deep learning algorithms [J]. Chinese General Practice, 2025. [Epub ahead of print].

Editorial Office of Chinese General Practice. This is an open access article under the CC BY-NC-ND 4.0 license.

Chinese General Practice

Qilu Hospital of Shandong University, Jinan 250012, China

LYU Lijuan, Attending physician; ZHANG Mei, Chief physician/Doctoral supervisor

Background

Computed tomography-derived fractional flow reserve (CT-FFR) has been shown to have good diagnostic performance, but the consistency of CT-FFR calculated by different deep learning algorithms has not been evaluated.

Objective: This study aims to assess the consistency of CT-FFR based on two deep learning algorithms and validate its diagnostic performance using invasive coronary angiography (ICA) or invasive fractional flow reserve (FFR) as references.

Methods

From January 2017 to June 2021, a total of 389 patients with suspected or confirmed coronary artery disease (CAD) were enrolled at Qilu Hospital of Shandong University. The cohort included patients who underwent coronary computed tomography angiography (CCTA), ICA, or FFR measurement. Among them, 55 patients underwent ICA within 90 days after CCTA, and 23 patients underwent FFR measurement after CCTA. Bland-Altman analysis was used to evaluate the consistency of CT-FFR, and the diagnostic performance of CT-FFR was compared with that of CCTA, using ICA or invasive FFR as the reference.

Results

A total of 389 patients were included, comprising 181 men (46.5%) and 208 women (53.5%), with a mean age of $55.1 \pm 10.9$ years; in total, 1,161 coronary arteries were analyzed. Based on Software 1, 172 vessels (14.8%) were identified as having functionally significant stenosis, while Software 2 identified 114 vessels (9.8%). Bland-Altman plots showed that CT-FFR derived from Software 1 slightly overestimated values, with a mean difference of 0.05 overall (0.05 in LAD, 0.04 in LCX, and 0.05 in RCA). Correlation analysis demonstrated moderate associations between CT-FFR and invasive FFR ($r = 0.44$ for Software 1; $r = 0.53$ for Software 2). Bland-Altman analysis showed good agreement with invasive FFR, with mean differences of -0.03 (Software 1) and -0.06 (Software 2). In diagnostic performance, CCTA had the highest sensitivity (97.8%) and negative predictive value (98.5%), but lower specificity (66.7%) and positive predictive value (57.7%). In contrast, Software 1-based CT-FFR achieved 89.1% sensitivity, 80.8% specificity, 68.3% PPV, 94.1% NPV, and 83.4% accuracy, while Software 2-based CT-FFR showed 80.4% sensitivity, 93.9% specificity, 86.0% PPV, 91.2% NPV, and 89.7% accuracy.

ROC curve analysis confirmed that both CT-FFR outperformed CCTA, with AUC values of 0.91 (Software 1) and 0.89 (Software 2), compared to 0.82 for CCTA ($P < 0.05$).

Conclusion

Good consistency was observed between the CT-FFR values based on Software 1 and Software 2, although a slight overestimation was found for CT-FFR based on software 1. Overall, CT-FFR demonstrated good diagnostic performance in detecting the functional significance of coronary stenosis.

Keywords: Computed tomography; Fractional flow reserve; Computed tomography-derived fractional flow reserve; Deep learning; Coronary artery disease

Invasive coronary angiography (ICA) is a commonly utilized invasive technique for diagnosing coronary artery disease (CAD). Due to the limitations of ICA, its inability to extend life expectancy, and its low diagnostic yield, invasive fractional flow reserve (FFR) has garnered significant attention \cite{2-3}. Since it was first used 30 years ago to evaluate the functional significance of coronary artery stenosis, FFR has been widely applied in clinical decision-making, particularly regarding revascularization \cite{4-5}. However, the application of FFR is restricted by its invasive nature, the potential for serious complications, and high costs. Coronary computed tomography angiography (CCTA) is a non-invasive anatomical imaging tool used to detect potential CAD due to its high sensitivity and negative predictive value (NPV) \cite{7-8}. Consequently, CCTA was recommended as a Class I indication in the 2019 European Society of Cardiology (ESC) Guidelines and was suggested for symptomatic patients with chronic coronary disease (CCD) in the 2022 American College of Cardiology (ACC)/American Heart Association (AHA) guidelines. The primary disadvantage of CCTA is its inability to provide a functional assessment of coronary stenosis. A critical step in evaluating the severity of myocardial ischemia is determining the functional significance of coronary artery stenosis. Therefore, non-invasive FFR techniques have emerged. As the earliest iteration of this technology, computed tomography-derived fractional flow reserve (CT-FFR) based on full-order computational fluid dynamics (CFD) models represents a compelling approach. CT-FFR results are interpreted by independent core laboratories in a blinded manner. Multicenter clinical studies have confirmed that CT-FFR possesses excellent diagnostic performance in detecting the functional significance of coronary artery stenosis \cite{13-16}. Full-order CFD-based CT-FFR requires offline computation on supercomputers, which involves lengthy processing times. Recently, CT-FFR calculation software based on CFD models has enabled on-site computation, significantly reducing the time required to determine CT-FFR values. Furthermore, CT-FFR has demonstrated favorable diagnostic accuracy in detecting the functional significance of coronary stenosis. To address the shortcomings of full-order models, studies have utilized reduced-order models in CT-FFR, which have shown good accuracy in determining the functional significance of coronary stenosis \cite{19-20}. Another approach involves the digital remodeling of coronary arteries via deep learning algorithms to predict FFR values, similar to the application of CFD models.

The computation time required for on-site CT-FFR calculations based on deep learning is relatively short. Numerous clinical studies have indicated that deep learning-based CT-FFR can facilitate risk stratification, prognostic assessment, and guidance for treatment decisions. Furthermore, CT-FFR exhibits superior diagnostic accuracy compared to CCTA alone in identifying the functional significance of coronary artery stenosis \cite{22, 26-27}. Compared with other imaging modalities, the diagnostic performance of CT-FFR at both the patient and vessel levels is also superior to that of single-photon emission computed tomography (SPECT). CT-FFR is a functional assessment tool with broad potential applications. However, the consistency of CT-FFR results derived from different deep learning algorithms has not yet been evaluated. In clinical practice, the accurate interpretation of CT-FFR values from different models is crucial for clinicians. Therefore, this study aims to evaluate the consistency of CT-FFR based on two different deep learning algorithms and to validate their diagnostic performance using ICA or invasive FFR as the reference standard.

1.1 Study Cohort

From January 2017 to June 2021, 389 patients with suspected or confirmed coronary artery disease (CAD) were selected as research subjects at Qilu Hospital of Shandong University. This cohort included patients who underwent coronary computed tomography angiography (CCTA), invasive coronary angiography (ICA), or fractional flow reserve (FFR) measurements. Specifically, 55 patients underwent ICA within 90 days following CCTA, and 23 of these patients received FFR measurements after CCTA. The inclusion criteria were: (1) patients with suspected or known CAD; (2) patients with a moderate pre-test probability of CAD as defined by the Diamond and Forrester criteria; (3) patients clinically recommended for ICA; and (4) age $\ge 18$ years. The exclusion criteria were: (1) acute coronary syndrome; (2) severe left ventricular dysfunction (left ventricular ejection fraction $< 40\%$); (3) non-ischemic cardiomyopathy; (4) severe cardiac arrhythmias, such as atrial fibrillation or second- or third-degree atrioventricular block; (5) contraindications to adenosine or iodinated contrast media; (6) renal insufficiency (estimated glomerular filtration rate $< 60 \text{ mL} \cdot \text{min}^{-1} \cdot (1.73 \text{ m}^2)^{-1}$); (7) a history of severe chronic obstructive pulmonary disease or chronic asthma; (8) pregnancy; and (9) refusal to sign informed consent. This study followed the Declaration of Helsinki (as revised in 2013).

The study was approved by the Hospital Ethics Committee (KYLL-202212-015-1), and individual informed consent was waived for this retrospective analysis.

1.2.1 Clinical Data Collection

Body Mass Index (BMI) was calculated as weight (kg) / height ($m^2$). Obesity was defined as BMI $\ge 28$ kg/m$^2$. Smoking history was defined as smoking at least one cigarette per day for a continuous or cumulative period exceeding six months, or being a long-term smoker who has quit for less than six months. Hypertension was defined according to the criteria in reference \cite{30}: (1) Patients not receiving antihypertensive medication who presented with a systolic blood pressure (SBP) $>140$ mmHg ($1 \text{ mmHg} = 0.133 \text{ kPa}$) and/or a diastolic blood pressure (DBP) $>90$ mmHg based on three measurements taken on different days; or (2) patients with a confirmed history of hypertension who were currently receiving antihypertensive drug therapy. Diabetes mellitus was diagnosed if: (1) the patient exhibited typical clinical symptoms (polyphagia, polydipsia, polyuria, and unexplained weight loss) and met at least one of the following criteria: random blood glucose $>11.1$ mmol/L, fasting blood glucose $>7.0$ mmol/L, glycated hemoglobin (HbA1c) $>6.5\%$, or venous blood glucose $>11.1$ mmol/L two hours after an oral glucose tolerance test; or (2) the patient had a documented history of diabetes and was currently receiving hypoglycemic treatment. Dyslipidemia was diagnosed if any of the following were met: (1) serum total cholesterol $\ge 6.2$ mmol/L; (2) serum high-density lipoprotein cholesterol $<1.0$ mmol/L; (3) serum low-density lipoprotein cholesterol $\ge 4.1$ mmol/L; (4) serum triglycerides $\ge 2.3$ mmol/L; or (5) the patient had a history of dyslipidemia and was currently receiving lipid-lowering therapy.

Imaging data collection: All included patients were scanned using a third-generation dual-source computed tomography (DSCT) scanner (SOMATOM Force; Siemens Healthineers). The specific scanning parameters were: slice collimation $= 192 \times 0.6$ mm, gantry rotation time $= 250$ ms, temporal resolution $= 66$ ms, longitudinal z-axis coverage $= 105$ mm, and tube voltage $= 100$ kV with automatic tube current modulation. Radiation dose reduction was achieved using CARE kV and CARE Dose 4D. Coronary CT angiography (CCTA) was performed using a bolus-tracking technique, with the region of interest (ROI) placed in the ascending aorta. Contrast medium was injected via an antecubital vein at a rate of $4\text{--}5$ mL/s, followed by a 40 mL saline flush using a dual-head power injector. Patients undergoing CCTA received retrospective ECG-triggered sequential acquisition, with the trigger window centered in either diastole or systole depending on the heart rate. Automatic tube voltage and current regulation (CARE kV and CARE Dose 4D, Siemens Healthineers) were applied. The entire procedure took approximately $15\text{--}20$ minutes.

1.2.2 CCTA Image Post-processing and Analysis

Image reconstruction and post-processing were performed using a dedicated workstation (Syngo.via VB10, Siemens Healthcare, Forchheim, Germany). Advanced modeled iterative reconstruction (ADMIRE) based on raw data was employed for image processing. The slice thickness was set at 0.75 mm with an increment of 0.50 mm. To minimize phase-motion bias, the optimal diastolic phase was selected for analysis. Image quality was evaluated using a 4-point Likert scale: 1 point, vessel poorly visualized with significant artifacts; 2 points, vessel boundaries partially blurred with some artifacts; 3 points, clear vessel boundaries with mild artifacts; 4 points, clear vessel boundaries with no artifacts. Vessels with a score < 2 were excluded. A semi-automated workflow consisting of "algorithmic initial segmentation + manual interactive correction" was used to mask calcification artifacts.

The software first automatically outlined the lumen, after which physicians manually removed calcification artifacts and corrected boundaries slice-by-slice to reduce the risk of "pseudo-stenosis" overestimating the pressure gradient. An initial threshold of 45% of the mean CT value of the aorta at the same level was applied, with manual area corrections of $< 5\%$ permitted. These procedures were performed by two radiologists with 5 and 8 years of experience, respectively. Coronary arteries with a diameter $\ge 1.5$ mm were evaluated, including the coronary artery diameter stenosis rate. Functionally significant coronary artery disease was defined as diameter stenosis $\ge 50\%$ in at least one epicardial coronary artery.

1.2.3 CT-FFR Assessment Based on Software 1

CT-FFR was calculated using commercially available software (Shukun-FFR, Shukun Technology). The computation process for CT-FFR values involves the automated reconstruction of the coronary artery tree and the subsequent derivation of CT-FFR values. First, the coronary arteries are extracted using a U-net model to present the complete anatomical structure. Second, an enhanced convolutional neural network model, integrating both 3D and 2D modes, is employed for plaque detection. To ensure the accuracy of branch connections, a standard 3D U-net and a shortest-path search algorithm are applied. The automated reconstruction of the coronary artery tree takes approximately two minutes. Subsequently, a modified reduced-order model is used to calculate the pressure along the vessel centerline, which is partitioned into stenotic and non-stenotic regions. The pressure for each specific region is then calculated.

The final intravascular pressure is an aggregation of the pressure derived from the reduced-order model and the pressure estimated via the neural network. CT-FFR values were calculated by a core laboratory in a blinded manner, defined as the ratio of the pressure in the stenotic coronary artery to the theoretical pressure in the absence of such stenosis. CT-FFR was evaluated 2.0 cm distal to the final stenosis; if the vessel had no stenosis or if the vessel diameter at the site of stenosis was $<1.5$ mm, the measurement was taken at the point where the vessel diameter reached 1.5 mm ([FIGURE:1]A).

1.2.4 CT-FFR Assessment Based on Software 2

Calculations were performed using specialized software (DeepVessel FFR, Korray Adaptive Medical Technology), which utilizes an algorithm evaluated through a trained deep learning framework. CT-FFR analysis is conducted via a Tree-structured Recurrent Neural Network (TreeVes-Net). This model was trained on a database containing 13,000 synthetic coronary artery trees—generated based on the geometric parameters of coronary anatomy—and subsequently validated using 180 real-world coronary artery trees.

Within the training database, geometric parameter values were randomly assigned within appropriate physiological ranges. The input for TreeVes-Net consists of geometric feature vectors related to fluid dynamics, including local vascular features, local stenosis features, and global features, all of which are extracted from the synthetic coronary input images. By applying the finite element method to solve the Navier-Stokes equations, the FFR values are calculated at various points along the coronary artery centerlines.

To address the issue of long-term spatial dependencies between fluid states at different points within the TreeVes-Net, a bidirectional recurrent neural network with Long Short-Term Memory (LSTM) units was employed. As a result of this process, three-dimensional color-coded coronary maps were generated to visualize the outputted CT-FFR values. The calculation of CT-FFR was performed by a core laboratory using a blinded protocol, and the subsequent results were returned to the researchers for blinded analysis. CT-FFR values were measured at a point 20 mm distal to the final stenosis. In cases involving multiple lesions within the same vessel, the lowest CT-FFR value was recorded.

1.2.5 ICA and Invasive FFR Assessment

Invasive coronary angiography (ICA) was performed using standard clinical protocols. All coronary arteries and their major branches were evaluated by a team of interventional cardiologists who remained blinded to the clinical information and CT-FFR values. FFR was measured using a 0.014-inch pressure wire (Prime Wire Prestige PLUS, Volcano Corporation) during continuous intravenous infusion of adenosine ($140\text{ mg}\cdot\text{kg}^{-1}\cdot\text{min}^{-1}$) to induce maximal hyperemia.

The measurements were performed under a state of induced hyperemic congestion. Functional significance was defined as a coronary artery stenosis degree $>90.0\%$ as determined by invasive coronary angiography (ICA), or an invasive FFR $\le 0.80$ when the coronary artery stenosis degree ranged between $30.0\%$ and $90.0\%$. Conversely, functional non-significance was defined by coronary artery stenosis less than 30.0%, or as invasive FFR $> 0.80$ when the degree of coronary artery stenosis was between 30.0% and 90.0%.

Statistical Methods

Statistical analysis was performed using the MedCalc software package (version 20; MedCalc Software, Ostend, Belgium). The normality of data distributions was assessed using the Kolmogorov-Smirnov test. Quantitative data following a normal distribution are expressed as mean ± standard deviation ($\bar{x} \pm s$), and comparisons between two groups were conducted using the independent samples t-test. Quantitative data that did not follow a normal distribution are expressed as median (interquartile range), and comparisons between groups were performed using the rank-sum test. Categorical data are presented as frequencies (percentages), and comparisons between groups were conducted using the chi-square test or Fisher's exact test. A p-value of $< 0.05$ was considered statistically significant.

Scatter plots were generated to illustrate the correlation between CT-FFR and invasive FFR. Bland-Altman plots were used to evaluate the agreement between CT-FFR (derived from Software 1 and Software 2) and invasive FFR, with the corresponding 95% limits of agreement (LOAs) reported. The sensitivity, specificity, positive predictive value, negative predictive value, and accuracy, along with their respective 95% confidence intervals, were calculated. Receiver operating characteristic (ROC) curves were constructed to assess the diagnostic performance of all parameters in detecting functionally significant stenotic vessels. The areas under the curve (AUC) were compared using the DeLong test.

[FIGURE:1]
Note: Figures A and B display the CCTA images and reconstructed coronary artery trees for a 48-year-old male patient, processed using Software 1 and Software 2, respectively.

2.1 Patient Baseline Characteristics

This study included a total of 389 patients with suspected or known coronary artery disease (CAD), consisting of 181 males (46.5%) and 208 females (53.5%), with a mean age of $55.1 \pm 10.9$ years. A total of 1,161 coronary arteries were evaluated. Using Software 1-based CT-FFR, functionally significant stenosis was identified in 172 vessels (14.8%), while Software 2-based CT-FFR identified 114 such vessels (9.8%). The demographic and baseline characteristics of the patients are summarized in Table 1 [TABLE:1].

[TABLE:1] Baseline characteristics of the study population.
- Sex [n (%)]
- BMI $\ge 28$ kg/m$^2$
- Cardiovascular risk factors [n (%)]
- Agatston score [0 (0, 73.9)]
- CCTA effective radiation dose [mSv, 7.0 (5.7, 8.7)]
- CCTA voltage [kV, 100 (100, 100)]
- Coronary artery dominance type [n (%)]
- Software 1-based CT-FFR [n (%)]: LAD $\le 0.80$ (20.8%), LCX $\le 0.80$ (9.4%), RCA $\le 0.80$ (14.2%)
- Software 2-based CT-FFR [n (%)]: LAD $\le 0.80$ (17.5%), LCX $\le 0.80$ (4.4%), RCA $\le 0.80$ (7.5%)

Note: CAD = coronary artery disease; CCTA = coronary computed tomography angiography; CT-FFR = computed tomography-derived fractional flow reserve; LAD = left anterior descending artery; LCX = left circumflex artery; RCA = right coronary artery.

2.2 Diagnostic Performance of CT-FFR in Detecting Functional Significance

Scatter plots revealed that the correlation coefficient between CT-FFR based on Software 1 and invasive FFR was 0.44 (95% CI = 0.11–0.68, $P = 0.011$). Similarly, the correlation coefficient between CT-FFR based on Software 2 and invasive FFR was 0.53 (95% CI = 0.22–0.74, $P = 0.002$). Bland-Altman analysis demonstrated good agreement between Software 1-based CT-FFR and invasive FFR, with a mean difference of -0.03 (95% CI = -0.10 to 0.04, $P = 0.360$). For Software 2-based CT-FFR and invasive FFR, the mean difference was -0.06 (95% CI = -0.12 to -0.00; $P = 0.069$), as shown in [FIGURE:2]A–D.

Compared to CT-FFR derived from Software 1 and Software 2, CCTA exhibited the highest sensitivity (97.8%) and negative predictive value (98.5%). However, CCTA demonstrated lower specificity and positive predictive value compared to the CT-FFR results from both software packages. Overall, CT-FFR calculated via Software 1 and Software 2 demonstrated higher diagnostic accuracy than CCTA, as detailed in [TABLE:2].

[TABLE:2] Diagnostic performance of CCTA and CT-FFR in detecting functional significance of stenosis.
- CCTA (Stenosis $\ge 50\%$): Sensitivity 97.8%, Specificity 66.7%, PPV 57.7%, NPV 98.5%, Accuracy 76.6%
- Software 1 (CT-FFR $\le 0.80$): Sensitivity 89.1%, Specificity 80.8%, PPV 68.3%, NPV 94.1%, Accuracy 83.4%
- Software 2 (CT-FFR $\le 0.80$): Sensitivity 80.4%, Specificity 93.9%, PPV 86.0%, NPV 91.2%, Accuracy 89.7%

2.4 ROC Curve Analysis

The results of the ROC curve analysis demonstrate that the diagnostic performance of CT-FFR derived from Software 1 (AUC = 0.91, 95% CI: 0.85–0.96) and Software 2 (AUC = 0.89, 95% CI: 0.83–0.94) was significantly superior to that of CCTA alone (AUC = 0.82, 95% CI: 0.75–0.88). These differences were statistically significant ($P = 0.002$ and $P = 0.077$, respectively), as illustrated in [FIGURE:3].

Bland-Altman analysis for CT-FFR revealed that the mean difference between Software 1 and Software 2 across 1,161 vessels was 0.05 (95% CI: 0.04–0.06, $P < 0.001$). At the level of the left anterior descending (LAD) artery, the mean difference for 389 vessels was 0.05 ($P < 0.001$). For the 385 vessels in the left circumflex (LCX) artery, the mean difference was 0.04 ($P < 0.001$). Finally, at the level of the right coronary artery (RCA), which included 387 vessels, the mean difference was 0.05 ($P < 0.001$), as illustrated in [FIGURE:4].

[FIGURE:2]
Note: Panel A illustrates the correlation between CT-FFR derived from Software 1 and invasive FFR. Panel B shows a Bland-Altman plot indicating that the mean difference between Software 1-based CT-FFR and invasive FFR is -0.03. Panel C illustrates the correlation between CT-FFR derived from Software 2 and invasive FFR. Panel D shows a Bland-Altman plot indicating that the mean difference between Software 2-based CT-FFR and invasive FFR is -0.06.

[FIGURE:3] ROC curves for CCTA and CT-FFR based on Software 1 and Software 2.

[FIGURE:4] Bland-Altman plots comparing CT-FFR from Software 1 and Software 2 for all vessels and individual branches.

[TABLE:3] Distribution of CT-FFR in different ranges across the three epicardial coronary arteries.
The comparison between Software 1 and Software 2 showed statistically significant differences in the distribution of CT-FFR values across various ranges ($P < 0.05$ for most intervals), particularly in the 0.7–0.8 and 0.9–1.0 ranges.

3 Discussion

Coronary Computed Tomography Angiography (CCTA) possesses high sensitivity and a high negative predictive value, making it an effective tool for the anatomical assessment of coronary artery disease (CAD). However, it is difficult to directly reflect the functional significance of stenoses using CCTA alone \cite{7-11}. CT-derived fractional flow reserve (CT-FFR) addresses this limitation by calculating or learning coronary hemodynamic information to provide a non-invasive assessment.

The results of this study show that the overall mean difference in CT-FFR between Software 1 and Software 2 was 0.05. At the levels of the LAD, LCX, and RCA, the mean differences were 0.05, 0.04, and 0.05, respectively, suggesting that both algorithms maintain good consistency at both the individual vessel and global levels. This is largely consistent with findings in related research suggesting that different algorithms can achieve comparable consistency \cite{26-28}. A possible reason is that while the two algorithms differ in their implementation of coronary tree reconstruction and hemodynamic approximation, both employ anatomy-based multi-scale feature extraction and blood pressure approximation.

Regarding the comparison between the two deep learning CT-FFR methods and invasive FFR, our results show correlation coefficients of $r=0.44$ for Software 1 and $r=0.53$ for Software 2, indicating a moderate correlation with the gold standard. Bland-Altman analysis showed mean differences of -0.03 and -0.06, respectively, suggesting good agreement with invasive FFR. This aligns with most previous reports where CT-FFR slightly underestimates invasive FFR \cite{26-28}.

In terms of diagnostic performance, CCTA demonstrated the highest sensitivity (97.8%) and NPV (98.5%), highlighting its advantage in exclusion. However, both CT-FFR methods outperformed CCTA in specificity, PPV, and accuracy. This is consistent with previous literature indicating that the introduction of CT-FFR significantly improves specificity and overall accuracy. ROC analysis showed that the AUC for Software 1 (0.91) and Software 2 (0.89) were both higher than that of CCTA (0.82).

In summary, this study utilized a real-world cohort to perform a systematic evaluation of two commercially available deep learning CT-FFR tools. It provides evidence for cross-algorithm consistency and performance relative to the gold standard. The introduction of a reproducible post-processing workflow offers significant reference value for clinical implementation. However, limitations such as the single-center retrospective design and limited sample size for invasive FFR must be acknowledged. Future large-scale, multi-center studies are needed to refine the cross-platform calibration framework.

Author Contributions: Zhou Yichun proposed the primary research objectives and was responsible for the conception, design, and implementation of the study, as well as drafting the manuscript. Zhou Yichun, Han Yeming, Zhang Pengfei, Song Wenwen, Wan Xiaoyu, Li Qimou, Liu Quande, and Yang Wei performed data collection, organization, statistical processing, and the creation of figures and tables. Pan Jichen, Li Xinhao, Li Dumin, Yu Dexin, Dong Mei, Liang Yongfeng, Hu Shan-shan, Lyu Lijuan, and Zhang Mei performed the revision of the manuscript. Lyu Lijuan and Zhang Mei were responsible for quality control and review, overall accountability for the article, and supervision/management.

The authors declare no conflicts of interest.

References

1. WANG Ruochen, DAI Yuxiang, GE Junbo. Development and prospects of transradial approach for coronary angiography and intervention [J]. Chinese Journal of Clinical Medicine, 2024, 31(1): 3-11.
2. DOENST T, HAVERICH A, SERRUYS P, et al. PCI and CABG for treating stable coronary artery disease: JACC review topic of the week [J]. J Am Coll Cardiol, 2019, 73(8): 964-976.
3. Hu Xinyang, Wu Xianpeng. Basic principles and research progress of fractional flow reserve derived from coronary angiography and intravascular imaging [J]. Journal of Clinical Cardiology, 2023, 39(10).
4. Li Jiawei, Cheng Zhaoyun, Li Zhengyu, et al. Research and clinical application progress of quantitative flow ratio [J]. Journal of Clinical Cardiology, 2024, 40(4): 271.
5. HAKEEM A, URETSKY B F. Role of postintervention fractional flow reserve to improve procedural and clinical outcomes [J]. Circulation, 2019, 139(5): 694-706.
6. Du Tianjian, Gu Xiang, Zhu Ye. Research on the Value of Coronary CT-Derived Fractional Flow Reserve in Predicting Coronary Artery Disease [J]. Chinese Journal of Geriatric Cardiovascular and Cerebrovascular Diseases, 2024, 26(9).
7. Zeng Hua, Chen Ming, Song Banghao. Diagnostic Value of Coronary Computed Tomography Angiography for the Severity of Coronary Artery Stenosis [J]. Journal of Vascular and Endovascular Surgery.
8. JOSHI P H, DE LEMOS J A. Diagnosis and management of stable angina: a review [J]. JAMA, 2021, 325(17): 1765-1778.
9. KNUUTI J, WIJNS W, SARASTE A, et al. 2019 ESC Guidelines for the diagnosis and management of chronic coronary syndromes [J]. Eur Heart J, 2020, 41(3): 407-477.
10. VIRANI S S, NEWBY L K, ARNOLD S V, et al. 2023 AHA/ACC/ACCP/ASPC/NLA/PCNA guideline for the management of patients with chronic coronary disease [J]. Circulation, 2023, 148(9): e9-e119.
11. ABDELRAHMAN K M, CHEN M Y, DEY A K, et al. Coronary computed tomography angiography from clinical uses to emerging technologies: JACC state-of-the-art review [J]. J Am Coll Cardiol, 2020, 76(10): 1226-1243.
12. GAO Yongguang, XIA Ping, SHI Yibing, et al. Clinical preliminary study of CT-derived fractional flow reserve for risk stratification and functional evaluation of coronary artery stenosis [J]. Journal of Interventional Radiology, 2024, 33(9): 956-960.
13. MIN J K, B ERMAN D S, BUDOFF M J, et al. Noninvasive fractional flow reserve derived from computed tomography angiography for coronary lesions of intermediate stenosis severity: results from the DeFACTO study [J]. Circ Cardiovasc Imaging, 2013, 6(6): 881-889.
14. KOO B K, ERGLIS A, DOH J H, et al. Diagnosis of ischemia-causing coronary stenoses by noninvasive fractional flow reserve computed from coronary computed tomographic angiograms. Results from the prospective multicenter DISCOVER-FLOW study [J]. J Am Coll Cardiol, 2011, 58(19).
15. NØRGAARD B L, LEIPSIC J, GAUR S, et al. Diagnostic performance of noninvasive fractional flow reserve derived from coronary computed tomography angiography in suspected coronary artery disease: the NXT trial [J]. J Am Coll Cardiol, 2014.
16. PONTONE G, BAGGIANO A, ANDREINI D, et al. Stress computed tomography perfusion versus fractional flow reserve CT derived in suspected coronary artery disease: the PERFECTION study [J]. JACC Cardiovasc Imaging, 2019, 12(8 Pt 1).
17. SULAIMAN N, SOON J, LEIPSIC J. Coronary CT angiography-derived fractional flow reserve [J]. Curr Radiol Rep, 2016, 4(8): 46.
18. TANG C X, LIU C Y, LU M J, et al. CT FFR for ischemia-specific CAD with a new computational fluid dynamics algorithm: a Chinese multicenter study [J]. JACC Cardiovasc Imaging, 2020.
19. KO B S, CAMERON J D, MUNNUR R K, et al. Noninvasive CT-derived FFR based on structural and fluid analysis: a comparison with invasive FFR for detection of functionally significant stenosis [J]. JACC Cardiovasc Imaging, 2017, 10(6): 663.
20. COENEN A, LUBBERS M M, KURATA A, et al. Fractional flow reserve computed from noninvasive CT angiography data: diagnostic performance of an on-site clinician-operated computational fluid dynamics algorithm [J]. Radiology, 2015, 274(3): 674-683.
21. YANG J J, SHAN D K, WANG X, et al. On-site computed tomography-derived fractional flow reserve to guide management of patients with stable coronary artery disease: the TARGET randomized trial [J]. Circulation, 2023, 147(18): 1369-1381.
22. PEPER J, BECKER L M, VAN DEN BERG H, et al. Diagnostic performance of CCTA and CT-FFR for the detection of CAD in TAVR work-up [J]. JACC Cardiovasc Interv, 2022, 15(11).
23. PATEL M R, NØRGAARD B L, FAIRBAIRN T A, et al. 1-year impact on medical practice and clinical outcomes of FFRCT: the ADVANCE registry [J]. JACC Cardiovasc Imaging, 2020, 13(1).
24. CHINNAIYAN K M, SAFIAN R D, GALLAGHER M L, et al. Clinical use of CT-derived fractional flow reserve in the emergency department [J]. JACC Cardiovasc Imaging, 2020, 13(2 Pt 1).
25. LAN Z T, DING X Y, YU Y R, et al. CT-derived fractional flow reserve for prediction of major adverse cardiovascular events in diabetic patients [J]. Cardiovasc Diabetol, 2023, 22(1): 65.
26. TESCHE C, DE CECCO C N, BAUMANN S, et al. Coronary CT angiography-derived fractional flow reserve: machine learning algorithm versus computational fluid dynamics modeling [J]. Radiology, 2018, 288(1): 64-72.
27. COENEN A, KIM Y H, KRUK M, et al. Diagnostic accuracy of a machine-learning approach to coronary computed tomographic angiography-based fractional flow reserve: result from the MACHINE consortium [J]. Circ Cardiovasc Imaging, 2018, 11(6): e007217.
28. DRIESSEN R S, DANAD I, STUIJFZAND W J, et al. Comparison of coronary computed tomography angiography, fractional flow reserve, and perfusion imaging for ischemia diagnosis [J]. J Am Coll Cardiol, 2019, 73(2): 161-173.
29. Expert consensus on the prevention and treatment of obesity among Chinese residents [J]. Chinese Journal of Epidemiology, 2022, 43(5): 609-626.
30. MCEVOY J W, MCCARTHY C P, BRUNO R M, et al. 2024 ESC Guidelines for the management of elevated blood pressure and hypertension [J]. Eur Heart J, 2024, 45(38): 3912-4018.
31. American Diabetes Association Professional Practice Committee. 2. diagnosis and classification of diabetes: standards of care in diabetes-2025 [J]. Diabetes Care, 2025, 48(1 Suppl 1): S27-S49.
32. LI J J, ZHAO S P, ZHAO D, et al. 2023 Chinese guideline for lipid management [J]. Front Pharmacol, 2023, 14: 1190934.
33. Cardiothoracic Group of the Radiology Branch of the Chinese Medical Association. CT-Derived Fractional Flow Reserve: Chinese Expert Consensus on Operational Standards and Clinical Applications [J]. Chinese Journal of Radiology, 2023, 57(7): 711-722.
34. KRUK M, WARDZIAK L, DEMKOW M, et al. Workstation-based calculation of CTA-based FFR for intermediate stenosis [J]. JACC Cardiovasc Imag, 2016, 9(6): 690-699.