scholarly journals Deterministic Annealing Approach to Fuzzy C-Means Clustering Based on Entropy Maximization

2011 ◽  
Vol 2011 ◽  
pp. 1-9 ◽  
Author(s):  
Makoto Yasuda

This paper is dealing with the fuzzy clustering method which combines the deterministic annealing (DA) approach with an entropy, especially the Shannon entropy and the Tsallis entropy. By maximizing the Shannon entropy, the fuzzy entropy, or the Tsallis entropy within the framework of the fuzzy c-means (FCM) method, membership functions similar to the statistical mechanical distribution functions are obtained. We examine characteristics of these entropy-based membership functions from the statistical mechanical point of view. After that, both the Shannon- and Tsallis-entropy-based FCMs are formulated as DA clustering using the very fast annealing (VFA) method as a cooling schedule. Experimental results indicate that the Tsallis-entropy-based FCM is stable with very fast deterministic annealing and suitable for this annealing process.

Author(s):  
Makoto Yasuda ◽  

The Tsallis entropy is a q-parameter extension of the Shannon entropy. By maximizing it within the framework of fuzzy c-means, statistical mechanical membership functions can be derived. We propose a clustering algorithm that includes the membership function and deterministic annealing. One of the major issues for this method is the determination of an appropriate values for q and an initial annealing temperature for a given data distribution. Accordingly, in our previous study, we investigated the relationship between q and the annealing temperature. We quantitatively compared the area of the membership function for various values of q and for various temperatures. The results showed that the effect of q on the area was nearly the inverse of that of the temperature. In this paper, we analytically investigate this relationship by directly integrating the membership function, and the inversely proportional relationship between q and the temperature is approximately confirmed. Based on this relationship, a q-incrementation deterministic annealing fuzzy c-means (FCM) algorithm is developed. Experiments are performed, and it is confirmed that the algorithm works properly. However, it is also confirmed that differences in the shape of the membership function of the annealing method and that of the q-incrementation method are remained.


2015 ◽  
Vol 2015 ◽  
pp. 1-7 ◽  
Author(s):  
Makoto Yasuda

Tsallis entropy is aq-parameter extension of Shannon entropy. By extremizing the Tsallis entropy within the framework of fuzzyc-means clustering (FCM), a membership function similar to the statistical mechanical distribution function is obtained. The Tsallis entropy-based DA-FCM algorithm was developed by combining it with the deterministic annealing (DA) method. One of the challenges of this method is to determine an appropriate initial annealing temperature and aqvalue, according to the data distribution. This is complex, because the membership function changes its shape by decreasing the temperature or by increasingq. Quantitative relationships between the temperature andqare examined, and the results show that, in order to changeuikqequally, inverse changes must be made to the temperature andq. Accordingly, in this paper, we propose and investigate two kinds of combinatorial methods forq-incrementation and the reduction of temperature for use in the Tsallis entropy-based FCM. In the proposed methods,qis defined as a function of the temperature. Experiments are performed using Fisher’s iris dataset, and the proposed methods are confirmed to determine an appropriateqvalue in many cases.


Author(s):  
Makoto Yasuda ◽  
◽  
Yasuyuki Orito

Tsallis entropy is aq-parameter extension of Shannon entropy. Based on the Tsallis entropy, we have introduced an entropy maximization method to fuzzyc-means clustering (FCM), and developed a new clustering algorithm using a single-qvalue. In this article, we propose a multi-qextension of the conventional single-qmethod. In this method, theqs are assigned individually to each cluster. Eachqvalue is determined so that the membership function fits the corresponding cluster distribution. This is done to improve the accuracy of clustering over that of the conventional single-qmethod. Experiments are performed on randomly generated numerical data and Fisher’s iris dataset, and it is confirmed that the proposed method improves the accuracy of clustering and is superior to the conventional single-qmethod. If the parameters introduced in the proposed method can be optimized, it is expected that the clusters in data distributions that are composed of clusters of various sizes can be determined more accurately.


Author(s):  
Makoto Yasuda ◽  

This paper considers a fuzzyc-means (FCM) clustering algorithm in combination with deterministic annealing and the Tsallis entropy maximization. The Tsallis entropy is aq-parameter extension of the Shannon entropy. By maximizing the Tsallis entropy within the framework of FCM, statistical mechanical membership functions can be derived. One of the major considerations when using this method is how to determine appropriate values forqand the highest annealing temperature,Thigh, for a given data set. Accordingly, in this paper, a method for determining these values simultaneously without introducing any additional parameters is presented, where the membership function is approximated using a series expansion method. The results of experiments indicate that the proposed method is effective, and bothqandThighcan be determined automatically and algebraically from a given data set.


2011 ◽  
Vol 07 (01) ◽  
pp. 155-171 ◽  
Author(s):  
H. D. CHENG ◽  
YANHUI GUO ◽  
YINGTAO ZHANG

Image segmentation is an important component in image processing, pattern recognition and computer vision. Many segmentation algorithms have been proposed. However, segmentation methods for both noisy and noise-free images have not been studied in much detail. Neutrosophic set (NS), a part of neutrosophy theory, studies the origin, nature, and scope of neutralities, as well as their interaction with different ideational spectra. However, neutrosophic set needs to be specified and clarified from a technical point of view for a given application or field to demonstrate its usefulness. In this paper, we apply neutrosophic set and define some operations. Neutrosphic set is integrated with an improved fuzzy c-means method and employed for image segmentation. A new operation, α-mean operation, is proposed to reduce the set indeterminacy. An improved fuzzy c-means (IFCM) is proposed based on neutrosophic set. The computation of membership and the convergence criterion of clustering are redefined accordingly. We have conducted experiments on a variety of images. The experimental results demonstrate that the proposed approach can segment images accurately and effectively. Especially, it can segment the clean images and the images having different gray levels and complex objects, which is the most difficult task for image segmentation.


1966 ◽  
Vol 1 (5) ◽  
pp. 415-421 ◽  
Author(s):  
A Esin ◽  
W J D Jones

The paper presents an outline of a theory of micro-inhomogeneity of stresses and strains resulting from the micro-structural properties of engineering materials. The problem is approached from a statistical point of view and it is experimentally shown that the degree of micro-inhomogeneity can be defined by normal distribution functions. Using the experimental results a general concept is postulated which takes into account the physical reality as completely as is practicable. It is shown that the suggested approach can be used to take into account the micro-plastic strains which exist while the material is nominally within the elastic limit.


Author(s):  
Aleksandra Noskova ◽  
◽  
Aleksander Alekseev

The motivation for this research was the result obtained earlier by the authors in the field of developing industry models for predicting bankruptcy with high prognostic ability. The article examines the prediction reliability of the financial position of companies in the case of introducing an additional category of financial position that reflects the position between financial solvency and insolvency (bankruptcy). The authors hypothesize that the reliability of models decreases if the requirements for their accuracy increase due to the introduction of an additional category of financial position. Hypothesis testing is performed using a non-entropic approach. This approach should reduce the measure of uncertainty in terms of the uncharacteristic nature of some of the identified features of financial position relative to the initial categories. At the same time, features of financial position are defined as ranges of specific weight of balance sheet items that have positive or negative information importance. Information importance is determined based on the methods of system-cognitive analysis, implemented automatically in the EIDOS X++ system, as well as by reproducing information models using MS Excel tools. Normalization of the informational importance values of features and their interpolation allowed us to obtain functions similar to the membership functions in the theory of fuzzy sets. When constructing membership functions relative to ranges of significant balance sheet items ("Fixed assets", "Inventory", "Accounts Receivable", "Short-Term financial investments", "Retained earnings (uncovered loss)", "Accounts payable"), ranges with zero or insignificant values of characteristic functions corresponding to the initial categories of financial position are identified. This actually meant a high level of uncertainty in the prediction. The authors propose to introduce additional linguistic variables and their corresponding fuzzy sets, whose carriers are the relative scales of the above balance items, this will reduce uncertainty. A total of 5 such fuzzy sets were identified, where the researchers used the concept of "gray zone" as a linguistic variable, which was actually used as a new category of financial position. All calculations are shown on the example of fixed assets. The prognostic ability of models based on an optimized sample, where the category of the position of companies that have at least 3 out of 5 features of the "gray zone" has been replaced, is reduced, as expected, but only slightly. And in the case of reproducing algorithms of system-cognitive analysis using MS Excel tools, there is even an increase in the prognostic ability of one of the models. In fact, the hypothesis that the reliability of models decreases if the requirements for their accuracy increase was not confirmed. From an economic point of view, the theoretical significance of the obtained result is that with the help of a non-entropic approach it was possible to show the need to introduce a new category of financial position. From a mathematical point of view, the theoretical significance lies in the fact that membership functions for linguistic variables are obtained based on real data on the financial position of almost two hundred Russian companies, these reduction functions can be used by specialists in the field of fuzzy set theory in the future. The results obtained are applicable at least for the construction industry, but can also be replicated relative to other sectors of the economy when forming the corresponding samples.


Entropy ◽  
2019 ◽  
Vol 21 (5) ◽  
pp. 522 ◽  
Author(s):  
Zhongfan Zhu ◽  
Jingshan Yu ◽  
Jie Dou ◽  
Dingzhi Peng

In the context of river dynamics, some experimental results have shown that particle velocity is different from fluid velocity along the stream-wise direction for uniform sediment-laden open-channel flows; this velocity difference has been termed velocity lag in the literature. In this study, an analytical expression for estimating the velocity lag in open-channel flows was derived based on the Tsallis entropy theory together with the principle of maximum entropy. The derived expression represents the velocity lag as a function of a non-dimensional entropy parameter depending on the average and maximum values of velocity lag from experimental measurements. The derived expression was tested against twenty-two experimental datasets collected from the literature with three deterministic models and the developed Shannon entropy-based model. The Tsallis entropy-based model agreed better with the experimental datasets than the deterministic models for eighteen out of the twenty-two total real cases, and the prediction accuracy for the eighteen experimental datasets was comparable to that of the developed Shannon entropy-based model (the Tsallis entropy-based expression agreed slightly better than the Shannon entropy-based model for twelve out of eighteen test cases, whereas for the other six test cases, the Shannon entropy-based model had a slightly higher prediction accuracy). Finally, the effects of the friction velocity of the flow, the particle diameter, and the particles’ specific gravity on the velocity lag were analyzed based on the Tsallis entropy-based model. This study shows the potential of the Tsallis entropy theory together with the principle of maximum entropy to predict the stream-wise velocity lag between a particle and the surrounding fluid in sediment-laden open-channel flows.


Sign in / Sign up

Export Citation Format

Share Document