Review of Swarm Fuzzy Classifier and a Convolutional Neural Network with VGG-16 Pre-Trained Model on Dental Panoramic Radiograph for Osteoporosis Classification

Usman Bello Abubakar¹, Moussa Mahamat Boukar² and Senol Dane^3*

^*Correspondence: Senol Dane, Department of Physiology, College of Health Sciences, Nile University of Nigeria, Nigeria, Email:

Author info »

Introduction

Every year, millions of people are affected by osteoporosis, the most common bone disease. Osteoporosis is a skeletal illness marked by a loss of bone mass and a deterioration of bone architecture [1]. It's linked to cortical bone thinning and increased porosity, as well as impaired connectivity of trabecular bone structures, all of which enhance the risk of fracture and fragility of bone. Bone Mineral Density (BMD), for osteoporosis classification are majorly evaluated by dual-energy X-ray absorptiometry (DXA) and it is identified as the gold standard for identifying osteoporosis. However, they are expensive and have limited availability in the population. Although bone mineral density (BMD) is an important metric for the prediction of fracture, it is a generic component that cannot tell the distinguish from trabecular and cortical bones neither can they forecast to certainty about the internal structure of bone [2]. Dental radiography, on the other hand, is an excellent tool for observing changes in the mandibular cortex and trabecular bone. Recently, a growing number of studies have established the viability of estimating BMD using relatively low-cost panoramic radiography [3,4] investigates the relationship between DXA measures and several densitometry and linear parameters such as mandibular cortical thickness and panoramic mandibular index.

Vlasiadis et al. and Leite et al. [5,6] confirm the possibility of employing mental index, mandibular cortical index, and visual estimation of cortical as osteoporosis predictors, and verify the associations of numerous panoramic radio morphometric indices with lumbar spine and hip BMDs. Another interesting option for osteoporosis evaluation is image feature information from trabecular bone.

The use of computer-assisted image processing to assess and evaluate bone architecture immediately from a dental panoramic radiograph (DPR) can save time and money [7]. Various regression models for osteoporosis have been developed previously [8]. However, in order to forecast the link between illness risk and each risk factor, these models require substantial assumptions. In recent years, classifier systems based on cortical or trabecular bone features have been developed such as Multilayer perceptron, Bayes classifier, Multilayer Feed-Forward Neural Network, Support Vector Machine (SVM).

Materials and Methods

Deep learning feature selection

Datasets collected come with a number of features. These features are necessary for a deep learning model to perform classification, each feature (independent variable) have some correlation to the target variable (dependent variable). However, situation may occur when the dataset consists of irrelevant features which will in turn hinder the classification performance of the machine learning model. Hence, the goal of feature selection process is given a dataset of size n which is described by m features (m dimensions), to find the minimum number of m dimensions that will describe the dataset as much as the original set of attributes do without losing the original semantics of the dataset [9].

Swarm fuzzy classifier approach

Between February 2007 and April 2012, 141 female patients aged 45 to 92 years old visited Kyungpook National University Hospital in Daegu, Korea, for the study proposed by [9]. During their visit, each participant had their digital DPR and BMD evaluated. Based on lumbar spine BMD, 120 patients were classified as osteoporosis negative while 21 patients were classified with a low BMD and thus, osteoporosis positive. On the other hand, femoral neck BMD classified 121 patients as osteoporosis positive and 20 patients to be osteoporosis negative and hence, having a low BMD. The same digital panorama equipment was used to capture all of the digital DPR. Using automatic exposure control, the voltage was changed between 60 and 70 kV. Images were saved in a matrix with a resolution of 2972 x 1536 pixels in a joint photographic expert’s group format.

According to World Health Organization, patients were classified based on standard T-scores:

• Normal: This indicates 1.0 as T-score,

• Osteopenia: This indicates between 1 and 2.5 as Tscore,

• Osteoporotic: This indicates 2.5 as T-score.

The Institutional Research Board of Kyungpook National University Hospital gave its approval to the study protocol [10].

As shown in Figure 1, the inferior mandibular cortical width (300 X 300 pixels) was measured constantly on both the right side of the mandibular cortex as well as the left side of mandibular cortex at every location between the top and bottom bounds of the cortical bone.

Figure 1 DPR Showing ROI [10].

On DPRs, a computer-aided diagnostic system was used to measure the trabecular bone pattern of the mandible [10]. Due to its precision, the region of interest was defined as the area to the left (250 x 150 pixels) of the mandible, inferior to the first premolar. In a nutshell, median filtering, Radon transformation, erosion, and dilation were used to recover the morphological skeleton of the trabecular bone from the original image. Finally, the trabecular bone was divided into one-pixel wide line segments using an average filter and the usual thinning procedure, as shown in Figure 2. On the segmented image, the trabecular fractal dimension was measured, which is a measure of the trabecular bone structure's complexity [10].

Figure 2 The original region of interest (a), Skeleton image (b), Trabecular segments (c) and Thinning of trabecular segments (d)” [10].

Based on structural anisotropy and mechanical capacities of the trabecular bone, the following properties were derived from the segmented image [10]:

• Trabecular width.

• Trabecular number.

• Trabecular separation.

• Trabecular Euler number.

• Trabecular angular orientation.

• Trabecular eccentricity.

In medical diagnostics, Fuzzy classifiers have been shown to be particularly useful for both quantitative and qualitative evaluation of medical data, with consistent findings. A fuzzy classifier is a classification system that consists of a set of rules and a collection of fuzzy membership functions (MF) (RS). It is made up of MFdefined if–then rules that are adopted and executed to generate the result via the inference process. Partitioning of the input variables to generate initial MF and RS, followed by classification using a fuzzy inference system based on the obtained MF and RS are the key steps in constructing a classifier for diagnosing a low BMD or osteoporosis using DPR (Figure 3).

Figure 3 The block diagram of proposed system [10].

Convolutional neural network with VGG

Deep learning techniques, particularly the design of deep convolutional neural networks (CNNs), have long been considered as a reliable way for learning feature categorization directly from medical images [11]. Deep CNNs, as opposed to ML techniques that rely on explicitly categorized features, are a sort of deep neural network that can learn high-dimensional features to improve the networks' ability to recognize abnormalities among images [12]. There are a variety of CNN architectures for picture categorization and recognition. In each of these topologies, the number and size of layers, the connections between these levels, and the overall network depth are all different. Because different network topologies are better suited for different difficulties and it is difficult to forecast which architecture is the best answer for a specific task in advance, empirical inquiry is widely acknowledged as the best method for making these decisions [13].

Although deep CNNs have been identified as effective image classification tools, they require a lot of training data, which might be challenging to apply to medical radiographic image data. Transfer learning is a strong technique for training deep CNNs without over fitting when the target dataset is much smaller than the basis dataset. Retrained models are used in a two-step manner to accomplish the general process of transfer learning:

• Copying the first n layers of pretrained model to first n layers of a custom target network

• Randomly initializing the layers of the custom target network and then training the network on the suitable end task. The concept of transfer learning shows that several state-of-the-art resulted in outperformance in classification of general images [14] as well as classification of medical image [15,16].

The study by [17] examined 680 DPR pictures from 680 distinct patients at Korea University Ansan Hospital. Between 2009 and 2018, the patients got skeletal BMD assessments and digital panoramic radiography evaluations at the same time. According to World Health Organization guidelines, the participants were divided into two groups: non-osteoporosis (T-score 2.5) and osteoporosis (T-score 2.5). The width and height of the dental X-ray images gathered ranged from 1348 to 2820 pixels wide and 685 to 1348 pixels high. The photos were down sampled to a uniform size of 1200 630 pixels using bilinear interpolation for consistency of image preprocessing, as shown in Figure 4. For an image size of 700 X 140 pixels, the final ROI was confined to the bottom region of the jaw, below the teeth-containing alveolar bone, as shown in Figure 3. This included the most ROI areas of previous studies [18,19] that used a variety of classification techniques to index the image feature properties of a small region of the mandible in great detail and specifically. By setting the ROI to include most of the mandible instead of the specific area of it, the study proposed by [17] assessed the area that plays the most distinctive function in osteoporosis classification.

Figure 4 Down-sampling and ROI cropping [17].

Classification results

The CNN model of this study was trained using a crossentropy loss function on the selected training image dataset. The architecture of the CNN model is shown in Figure 5 while the architecture of the CNN model combined with VGG-16 is shown in Figure 6. It was observed that the transfer learning, fine tuning VGG16 model with the custom CNN achieved accuracy of 0.840(84%).

Figure 5 CNN model [17].

Figure 6 CNN Model with VGG-16 [17].

The Genetic Swarm Fuzzy Classifier (GSF) based on the DPR built in [10] provides diagnostic knowledge and explanation ability in terms of the most relevant and interpretable rules with classification performance of 0.986(98%), for the diagnosis of a low BMD or osteoporosis. The ranges of values of each linguistic term for every input attribute derived from the MF were manually examined by oral radiologists [10]. The structural parameters such as thickness, number and separation of the trabecular bone were found to be related to osteoporotic subjects and the reported ranges of values for identifying a low BMD or osteoporosis [20]. Furthermore, several studies [21] suggested the cut-off threshold of cortical width as the most appropriate threshold for referral for bone densitometry and it is also similar to the ranges of values obtained from MF in the study by [10].

Conclusion

Even though the training dataset is restricted, this review study supports the effectiveness of transfer learning with a deep CNN for osteoporosis detection in dental panoramic radiographs (DPR) images. It also demonstrates how to use an automated screening strategy based on a hybrid genetic swarm fuzzy (GSF) classifier and digital dental panoramic radiographs to diagnose females with low bone mineral density (BMD) or osteoporosis.

Experiments revealed that using CNN with pre-trained transfer learning resulted in an accuracy of 84%. The findings suggest that the difficulty of a small training sample of photos was well handled utilizing the correct deep CNN architectures and transfer learning techniques, and that DPR images have the potential to be exploited for osteoporosis prescreening.

According to the results of the swarm fuzzy classifier, the fuzzy classifier's combination of high accuracy and interpretation ability allows it to detect a large percentage of previously undetected low BMD or osteoporosis at an early stage. With a classification performance of 0.986, this gives diagnostic knowledge for the diagnosis of low BMD or osteoporosis (98%).

References

1. Macdonald H, Nishiyama K, Kang J, et al. Age-related patterns of trabecular and cortical bone loss differ between sexes and skeletal sites: A population-based HR-pQCT study. J Bone Miner Res 2011; 26:50-62.

Indexed at, Google Scholar, Cross Ref

2. Zaia A. Fractal lacunarity of trabecular bone and magnetic resonance imaging: New perspectives for osteoporotic fracture risk assessment. World J Orthop 2015; 6:221-35.

Indexed at, Google Scholar, Cross Ref

3. Chu P, Bo C, Liang X, et al. Using octuplet siamese network for osteoporosis analysis on dental panoramic radiographs. Annu Int Conf Eng Med Biol Soc 2018; 2018:2579-2582.

Indexed at, Google Scholar, Cross Ref

4. Horner K, Devlin H. The relationship between mandibular bone mineral density and panoramic radiographic measurements. J Dent 1998; 26:337-343.

Indexed at, Google Scholar, Cross Ref

5. Vlasiadis K, Skouteris C, Velegrakis G, et al. Mandibular radiomorphometric measurements as indicators of possible osteoporosis in postmenopausal women. Maturitas 2007; 58:226-235.

Indexed at, Google Scholar, Cross Ref

6. Leite A, Figueiredo P, Guia C, et al. Correlations between seven panoramic radiomorphometric indices and bone mineral density in postmenopausal women. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 2010; 109:449-456.

Indexed at, Google Scholar, Cross Ref

7. Kavitha M, Samopa F, Asano A, et al. Computer-aided measurement of mandibular cortical width on dental panoramic radiographs for identifying osteoporosis. J Investig Clin Dent 2012; 3:36-44.

Indexed at, Google Scholar, Cross Ref

8. Sindeaux R, Figueiredo P, Melo N, et al. Fractal dimension and mandibular cortical width in normal and osteoporotic men and women. Maturitas 2014; 77:142-148.

Indexed at, Google Scholar, Cross Ref

9. Umar R, Boukar M, Adeshina S, et al. Machine learning approaches for optimal parameter selection for hepatitis disease classification. J Res Med Dent Sci 2021; 9:526-535.

Google Scholar

10. Kavitha M, Kumar P, Park S, et al. Automatic detection of osteoporosis based on hybrid genetic swarm fuzzy classifier approaches. Dentomaxillofac Radiol 2016; 45:20160076.

Indexed at, Google Scholar, Cross Ref

11. Litjens G, Kooi T, Ehteshami B, et al. A survey on deep learning in medical image analysis. Med Image Anal 2017; 42:60-88.

Indexed at, Google Scholar, Cross Ref

12. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521:436-444.

Indexed at, Google Scholar, Cross Ref

13. Baker B, Gupta O, Naik N, et al. Designing neural network architectures using reinforcement learning. Int Conf on Learning Representations 2016.
14. Krizhevsky A, Sutskever I, Hinton G. Image net classification with deep convolutional neural networks. Int Conf Neural Information Processing Systems 2017.

Google Scholar

15. Christopher M, Belghith A, Bowd C, et al. Performance of deep learning architectures and transfer learning for detecting glaucomatous optic neuropathy in fundus photographs. Sci Rep 2018; 8:1-3.

Indexed at, Google Scholar, Cross Ref

16. Shin H, Roth H, Gao M, et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 2016; 35:1285-1298.

Indexed at, Google Scholar, Cross Ref

17. Lee K, Jung S, Ryu J, et al. Evaluation of transfer learning with deep convolutional neural networks for screening osteoporosis in dental panoramic radiographs. J Clin Med 2020; 9:392.

Indexed at, Google Scholar, Cross Ref

18. Okabe S, Morimoto Y, Ansai T, et al. Assessment of the relationship between the mandibular cortex on panoramic radiographs and the risk of bone fracture and vascular disease in 80-year-olds. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 2008; 106:433-442.

Indexed at, Google Scholar, Cross Ref

19. Taguchi A, Ohtsuka M, Tsuda M, et al. Risk of vertebral osteoporosis in post-menopausal women with alterations of the mandible. Dentomaxillofac Radiol 2007; 36:143-148.

Indexed at, Google Scholar, Cross Ref

20. Shen Y, Zhang Y, Shen L. Postmenopausal women with osteoporosis and osteoarthritis show different microstructural characteristics of trabecular bone in proximal tibia using high-resolution magnetic resonance imaging at 3 tesla. BMC Musculoskelet Disord 2013; 14:136.

Indexed at, Google Scholar, Cross Ref

21. Hardanti S, Azhari, Oscandar F. Description of mandibular bone quality based on measurements of cortical thickness using Mental Index of male and female patients between 40-60 years old. Imaging Sci Dent 2011; 41:151-153.

Indexed at, Google Scholar, Cross Ref