Using S-Detect to Improve Breast Ultrasound: The Different Combined Strategies Based on Radiologist Experience

doi:10.37015/AUDT.2022.220007

Abstract

Abstract:

Objective: To investigate the best combined method of S-Detect, a computer-aided diagnosis (CAD) system, with breast ultrasound (US) according to radiologists’ experience.

Methods: From March 2019 to June 2019, 259 breast masses in 255 women were included in this study. Ultrasonographic images of the target masses were prospectively analyzed by radiologists and CAD. Three combined methods, including method 1 [selective downgrading combination for Breast Imaging Reporting and Data System (BI-RADS) 4a lesions], method 2 (selective upgrading combination for BI-RADS 3 lesions) and method 3 (selective upgrading or downgrading combination for BI-RADS 3 or 4a lesions), were applied to interpret the CAD results. The sensitivity, specificity, the area under the receiver operating characteristic curve (AUC) of experienced or inexperienced radiologists before and after adding CAD results were compared using the histopathological results as a reference standard.

Results: In identifying breast malignancy, the AUC for CAD was similar to that of experienced radiologists (P= 0.410), but higher than that of inexperienced radiologists (P= 0.003). When combining CAD with experienced radiologists based on method 1 and combining CAD results with inexperienced radiologists based on method 3, the AUCs were significantly improved (P= 0.024 and 0.003, respectively) compared to US alone, with significantly increased specificity (P< 0.001 for both) and no significantly decreased sensitivity (P> 0.05 for both).

Conclusion: The combination of CAD system and conventional ultrasound can improve ultrasound diagnostic performance in determining breast malignancy. The method 1 and method 3 combinations are respectively recommended for experienced and inexperienced radiologists when CAD is combined with conventional breast ultrasound.

Key words: Ultrasound; Breast; Computer-assisted diagnosis; Neoplasms; BI-RAD classification

Ying Zhu, MD, Xiaohong Jia, MD, Yijie Dong, MD, Juan Liu, MD, Yilai Chen, MD, Congcong Yuan, MD, Weiwei Zhan, MD, Jianqiao Zhou, MD. Using S-Detect to Improve Breast Ultrasound: The Different Combined Strategies Based on Radiologist Experience. Advanced Ultrasound in Diagnosis and Therapy, 2022, 6(4): 180-187.

Figures/Tables 7

Figure 1

Table 1

Table 2

Table 3

Table 4

Figure 2

Table 5

References 20

[1]	DeSantis CE, Ma J, Goding SA, Newman LA, Jemal A.Breast cancer statistics, 2017, racial disparity in mortality by stat. CA Cancer J Clin 2017; 67:439-448. doi: 10.3322/caac.21412
[2]	D’Orsi CJ, Sickles EA, Mendelson EB, Morris E.ACR BI-RADS® Atlas: breast imaging reporting and data system. 5th ed. Reston, VA:American college of radiology; 2013.
[3]	Yoon JH, Kim MJ, Moon HJ, Kwak JY, Kim EK. Subcategorization of ultrasonographic BI-RADS category 4: positive predictive value and clinical factors affecting it. Ultrasound Med Biol 2011; 37:693-699. doi: 10.1016/j.ultrasmedbio.2011.02.009 pmid: 21458145
[4]	Wiratkapun C, Bunyapaiboonsri W, Wibulpolprasert B, Lertsithichai P. Biopsy rate and positive predictive value for breast cancer in BI-RADS category 4 breast lesion. J Med Assoc Thai 2010; 93:830-837. pmid: 20649064
[5]	Doi K. Computer-aided diagnosis in medical imaging: historical review, current status and future potentia. Comput Med Imaging Graph 2007; 31:198-211. doi: 10.1016/j.compmedimag.2007.02.002
[6]	Joo S, Yang YS, Moon WK, Kim HC. Computer-aided diagnosis of solid breast nodules: use of an artificial neural network based on multiple sonographic feature. IEEE Trans Med Imaging 2004; 23:1292-1300. doi: 10.1109/TMI.2004.834617
[7]	Huang YL, Chen DR. Support vector machines in sonography: application to decision making in the diagnosis of breast cance. Clin Imaging 2005; 29:179-184. doi: 10.1016/j.clinimag.2004.08.002
[8]	Singh S, Maxwell J, Baker JA, Nicholas JL, Lo JY. Computer-aided classification of breast masses: performance and interobserver variability of expert radiologists versus resident. Radiology 2011; 258:73-80. doi: 10.1148/radiol.10081308
[9]	Jalalian A, Mashohor SB, Mahmud HR, Saripan MI, Ramli AR, Karasfi B. Computer-aided detection/diagnosis of breast cancer in mammography and ultrasound: a revie. Clin Imaging 2013; 37:420-426. doi: 10.1016/j.clinimag.2012.09.024
[10]	Cho E, Kim EK, Song MK, Yoon JH. Application of computer-aided diagnosis on breast ultrasonography: evaluation of diagnostic performances and agreement of radiologists according to different levels of experienc. J Ultrasound Med 2018; 37:209-216. doi: 10.1002/jum.14332
[11]	Choi JH, Kang BJ, Baek JE, Lee HS, Kim SH. Application of computer-aided diagnosis in breast ultrasound interpretation: improvements in diagnostic performance according to reader experienc. Ultrasonography 2018; 37:217-225. doi: 10.14366/usg.17046
[12]	Di SM, de Soccio V, Cantisani V, Bonito G, Rubini A, Di Segni G, et al. Automated classification of focal breast lesions according to S-detect: validation and role as a clinical and teaching too. J Ultrasound 2018; 21:105-118. doi: 10.1007/s40477-018-0297-2
[13]	Choi JS, Han BK, Ko ES, Bae JM, Ko EY, Song SH, et al. Effect of a deep learning framework-based computer-aided diagnosis system on the diagnostic performance of radiologists in differentiating between malignant and benign masses on breast ultrasonograph. Korean J Radiol 2019; 20: 749-758. doi: 10.3348/kjr.2018.0530 pmid: 30993926
[14]	Han S, Kang HK, Jeong JY, Park MH, Kim W, Bang WC, et al. A deep learning framework for supporting the classification of breast lesions in ultrasound image. Phys Med Biol 2017; 62:7714-7728. doi: 10.1088/1361-6560/aa82ec
[15]	Landis JR, Koch GG. The measurement of observer agreement for categorical dat. Biometrics 1977; 33:159-174. pmid: 843571
[16]	Huang Q, Zhang F, Li X. Machine learning in ultrasound computer-aided diagnostic systems: a surve. Biomed Res Int 2018; 2018:5137904.
[17]	Zhao C, Xiao M, Jiang Y, Liu H, Wang M, Wang H, et al. Feasibility of computer-assisted diagnosis for breast ultrasound: the results of the diagnostic performance of S-detect from a single center in Chin. Cancer Manag Res 2019; 11:921-930. doi: 10.2147/CMAR.S190966
[18]	Kim K, Song MK, Kim EK, Yoon JH. Clinical application of S-Detect to breast masses on ultrasonography: a study evaluating the diagnostic performance and agreement with a dedicated breast radiologis. Ultrasonography 2017; 36:3-9. doi: 10.14366/usg.16012
[19]	Shan J, Alam SK, Garra B, Zhang Y, Ahmed T. Computer-aided diagnosis for breast ultrasound using computerized BI-RADS features and machine learning method. Ultrasound Med Biol 2016; 42:980-988. doi: 10.1016/j.ultrasmedbio.2015.11.016
[20]	Tan T, Platel B, Twellmann T, van Schie G, Mus R, Grivegnée A, et al. Evaluation of the effect of computer-aided classification of benign and malignant lesions on reader performance in automated three-dimensional breast ultrasoun. Acad Radiol 2013; 20:1381-1388. doi: 10.1016/j.acra.2013.07.013

Initial BI-RADS category	BI-RADS 3		BI-RADS 4a
Initial BI-RADS category	CAD (-)	CAD (+)	CAD (-)	CAD (+)
Reevaluation on Method 1	BI-RADS 3	BI-RADS 3	BI-RADS 3	BI-RADS 4a
Reevaluation on Method 2	BI-RADS 3	BI-RADS 4a	BI-RADS 4a	BI-RADS 4a
Reevaluation on Method 3	BI-RADS 3	BI-RADS 4a	BI-RADS 3	BI-RADS 4a

Pathological results	No. (%)
Benign
Fibroadenoma	96 (37.1%)
Papilloma	32 (12.4%)
ANDI	31 (1,2%)
Granulomatous lobular mastitis	4 (1.5%)
Nodular fasciitis	1 (0.4%)
Lobular CIS	1 (0.4%)
Benign phyllodes tumors	1 (0.4%)
Fat necrosis	1 (0.4%)
Hamartoma	1 (0.4%)
Malignant
Invasive ductal carcinoma	64 (24.7%)
Ductal carcinoma in situ	13 (5.0%)
Invasive lobular carcinoma	6 (2.3%)
Mucinous carcinoma	2 (0.8%)
Intracystic papillary carcinoma	2 (0.8%)
Solid papillary carcinoma	2 (0.8%)
Primary angiosarcoma	1 (0.4%)
AAC	1 (0.4%)

Items	Sensitivity	Specificity	PPV	NPV	Accuracy	AUC (95 % CI)
CAD	82.4 %	74.4 %	64.6 %	88.7 %	77.2 %	0.784(0.729-0.833)
Experienced radiologist
US alone	97.8 %^#	63.7 %^#	59.3 %	98.1 %	75.7 %	0.813(0.760-0.859)
CAD + US: Method 1	93.4 %	80.9 %^*	72.7 %^*	95.8 %	85.3 %^*	0.872(0.825-0.910)^*
CAD + US: Method 2	97.8 %	53.6 %^*	53.3 %	97.8 %	69.1 %	0.759(0.693-0.818) ^*
CAD + US: Method 3	93.4 %	70.8 %	63.4 %	95.2 %	78.8 %	0.832(0.772-0.881)
Inexperienced reader
US alone	90.1 %	42.9 %^#	46.1 %^#	88.9 %	59.5 %^#	0.665(0.604-0.722) ^#
CAD + US: Method 1	86.8 %	71.4 %^†	62.2 %^†	90.9 %	76.8 %^†	0.791(0.737-0.839) ^†
CAD + US: Method 2	95.6 %	37.5 %^†	45.3 %	94.0 %	57.9 %	0.666(0.604-0.723)
CAD + US: Method 3	92.3 %	66.1 %^†	59.6 %	94.1 %^†	75.3 %^†	0.792(0.737-0.840) ^†

BI-RADS category	Before adding CAD results				After adding CAD results
BI-RADS category	CAD	Benign	Malignant	M%	CAD	Benign	Malignant	M%
3	(-)	91	1	1.8	(-)	120	5	4.2
	(+)	16	1		(+)	16	1
4a	(-)	29	4^#	17.5	(-)	0	0	25.0
	(+)	18	6		(+)	18	6

BI-RADS category	Before adding CAD results				After adding CAD results
BI-RADS category	CAD	Benign	Malignant	M%	CAD	Benign	Malignant	M%
3	(-)	63	4	11.1	(-)	111	7	6.3
	(+)	9	5^*		(+)	0	0
4a	(-)	48	3^#	15.4	(-)	27	14	34.1
	(+)	18	9		(+)	0	0