This book is an updated version of the information theory classic, first published in 1990. About one-third of the book is devoted to Shannon source and channel coding theorems; the remainder addresses sources, channels, and codes and on information and distortion measures and their properties. New in this edition: Expanded treatment of stationary or sliding-block codes and their relations to traditional block codesExpanded discussion of results from ergodic theory relevant to information theoryExpanded treatment of B-processes -- processes formed by stationary coding memoryless sourcesNew mat.
AbstractGlobally, floods as dynamic hydraulic hazard have caused widespread damages to both socioeconomic conditions and environment at various scales. Managing flood and management of water resource is a global challenge under the changing climatic condition. This study assessed flood susceptibility in the Bhagirathi sub‐basin, India using entropy information theory and geospatial technology. Twelve flood susceptibility parameters such as land use/land cover, normalized difference vegetation index (NDVI), slope, elevation, geology, geomorphology, normalized difference water index (NDWI), soil, drainage density, average rainfall, maximum temperature, and humidity during monsoon season were utilized to examine flood susceptibility. Receiver operating characteristics (ROC) curve and Leave‐One‐Out Cross‐Validation (LOOCV) techniques were carried out to validate flood susceptibility map. Kappa statistics was also used to check the reliability of the flood susceptibility model. Findings of the study revealed that nearly 45% area of the sub‐basin was highly susceptible to flood followed by moderate (29.3%), very high (19%), low (6.9%), and very low (0.2%). These findings also revealed that nearly 92% area in the eastern, north‐eastern, and deltaic sub‐basin was susceptible to floods. ROC analysis indicated high success (0.932) and prediction (0.903) rates for the susceptibility map while LOOCV (R2 being 0.97) and Kappa (k = 0.934) have shown substantial prediction of the model. Hence, the susceptibility maps are useful for the local planners and government organization in designing the early flood warning system, and reducing the human and economic losses. The methodology used in this study is applicable for analyzing flood susceptibility at spatial scales in similar systems.
This paper presents a study and a comparison of the use of different information-theoretic measures for polygonal mesh simplification. Generalized measures from Information Theory such as Havrda–Charvát–Tsallis entropy and mutual information have been applied. These measures have been used in the error metric of a surface simplification algorithm. We demonstrate that these measures are useful for simplifying three-dimensional polygonal meshes. We have also compared these metrics with the error metrics used in a geometry-based method and in an image-driven method. Quantitative results are presented in the comparison using the root-mean-square error (RMSE) ; This work was supported by the Spanish Ministry of Science and Innovation (Project TIN2010-21089-C03-03 and TIN2010-21089-C03-01) and Feder Funds, Bancaixa (Project P1.1B2010-08), Generalitat Valenciana (Project PROMETEO/2010/028) and Project 2009-SGR-643 of Generalitat de Catalunya (Catalan Government)
This paper presents a study and a comparison of the use of different information-theoretic measures for polygonal mesh simplification. Generalized measures from Information Theory such as Havrda–Charvát–Tsallis entropy and mutual information have been applied. These measures have been used in the error metric of a surface simplification algorithm. We demonstrate that these measures are useful for simplifying three-dimensional polygonal meshes. We have also compared these metrics with the error metrics used in a geometry-based method and in an image-driven method. Quantitative results are presented in the comparison using the root-mean-square error (RMSE) ; This work was supported by the Spanish Ministry of Science and Innovation (Project TIN2010-21089-C03-03 and TIN2010-21089-C03-01) and Feder Funds, Bancaixa (Project P1.1B2010-08), Generalitat Valenciana (Project PROMETEO/2010/028) and Project 2009-SGR-643 of Generalitat de Catalunya (Catalan Government)
This paper presents a study and a comparison of the use of different information-theoretic measures for polygonal mesh simplification. Generalized measures from Information Theory such as Havrda–Charvát–Tsallis entropy and mutual information have been applied. These measures have been used in the error metric of a surface simplification algorithm. We demonstrate that these measures are useful for simplifying three-dimensional polygonal meshes. We have also compared these metrics with the error metrics used in a geometry-based method and in an image-driven method. Quantitative results are presented in the comparison using the root-mean-square error (RMSE) ; This work was supported by the Spanish Ministry of Science and Innovation (Project TIN2010-21089-C03-03 and TIN2010-21089-C03-01) and Feder Funds, Bancaixa (Project P1.1B2010-08), Generalitat Valenciana (Project PROMETEO/2010/028) and Project 2009-SGR-643 of Generalitat de Catalunya (Catalan Government)
Today, eye trackers are extensively used in studying human cognition. However, it is hard to analyze and interpret eye movement data from the cognitive comprehension perspective of poster reading. To find quantitative links between eye movements and cognitive comprehension, we tracked observers' eye movement for reading scientific poster publications. We model in this paper eye tracking fixation sequences between content-dependent Areas of Interests (AOIs) as a Markov chain. Furthermore, we use the fact that a Markov chain is a special case of information or communication channel. Then, the gaze transition can be modeled as a discrete information channel, the gaze information channel. Next, some traditional eye tracking metrics, together with the gaze entropy and mutual information of the gaze information channel are calculated to quantify cognitive comprehension for every participant. The analysis of the results demonstrate that the gaze entropy and mutual information from individual gaze information channel are related to participants' individual differences. This is the first study that eye tracking technology has been used to assess the cognitive comprehension of poster reading. The present work provides insights into human cognitive comprehension by using the novel gaze information channel methodology ; This work has been funded by the National Natural Science Foundation of China under grants No.61471261 and No.61771335, and by grants TIN2016-75866-C3-3-R from the Spanish Government
Cross entropy and Kullback–Leibler (K-L) divergence are fundamental quantities of information theory, and they are widely used in many fields. Since cross entropy is the negated logarithm of likelihood, minimizing cross entropy is equivalent to maximizing likelihood, and thus, cross entropy is applied for optimization in machine learning. K-L divergence also stands independently as a commonly used metric for measuring the difference between two distributions. In this paper, we introduce new inequalities regarding cross entropy and K-L divergence by using the fact that cross entropy is the negated logarithm of the weighted geometric mean. We first apply the well-known rearrangement inequality, followed by a recent theorem on weighted Kolmogorov means, and, finally, we introduce a new theorem that directly applies to inequalities between K-L divergences. To illustrate our results, we show numerical examples of distributions ; Mateu Sbert acknowledges the funding of National Natural Science Foundation of China under grants No.61471261 and No.61771335, and by grant TIN2016-75866-C3-3-R from Spanish Government, Jordi Poch and Anton Bardera acknowledge the funding of TIN2016-75866-C3-3-R from Spanish Government
This paper uses quantitative eye tracking indicators to analyze the relationship between images of paintings and human viewing. First, we build the eye tracking fixation sequences through areas of interest (AOIs) into an information channel, the gaze channel. Although this channel can be interpreted as a generalization of a first-order Markov chain, we show that the gaze channel is fully independent of this interpretation, and stands even when first-order Markov chain modeling would no longer fit. The entropy of the equilibrium distribution and the conditional entropy of a Markov chain are extended with additional information-theoretic measures, such as joint entropy, mutual information, and conditional entropy of each area of interest. Then, the gaze information channel is applied to analyze a subset of Van Gogh paintings. Van Gogh artworks, classified by art critics into several periods, have been studied under computational aesthetics measures, which include the use of Kolmogorov complexity and permutation entropy. The gaze information channel paradigm allows the information-theoretic measures to analyze both individual gaze behavior and clustered behavior from observers and paintings. Finally, we show that there is a clear correlation between the gaze information channel quantities that come from direct human observation, and the computational aesthetics measures that do not rely on any human observation at all ; This work is supported by the National Natural Science Foundation of China under grant No. 61702359, and by grant TIN2016-75866-C3-3-R from Spanish Government.
Information theory, and the concept of information channel, allows us to calculate the mutual information between the source (input) and the receiver (output), both represented by probability distributions over their possible states. In this paper, we use the theory behind the information channel to provide an enhanced interpretation to a Social Accounting Matrix (SAM), a square matrix whose columns and rows present the expenditure and receipt accounts of economic actors. Under our interpretation, the SAM's coefficients, which, conceptually, can be viewed as a Markov chain, can be interpreted as an information channel, allowing us to optimize the desired level of aggregation within the SAM. In addition, the developed information measures can describe accurately the evolution of a SAM over time. Interpreting the SAM matrix as an ergodic chain could show the effect of a shock on the economy after several periods or economic cycles. Under our new framework, finding the power limit of the matrix allows one to check (and confirm) whether the matrix is well-constructed (irreducible and aperiodic), and obtain new optimization functions to balance the SAM matrix. In addition to the theory, we also provide two empirical examples that support our channel concept and help to understand the associated measures ; M.S., M.F. and M.V. acknowledge support from the project PID2019-106426RB-C31 from the Spanish Government
Abstract A wide variety of techniques are used to assess the development of survey-based scales. The majority of these techniques focus on the quality of information characterized by the scale. Aside from very rudimentary measures such as response rates and sample sizes, very few empirical techniques are available to measure the quantity of information contained in a scale. This article conducts an exploratory empirical analysis to assess whether information entropy can be useful for measuring the quantity of information in a scale's development. If the quantity of information in the scale significantly increases (decreases) with the addition of the survey item, researchers may consider retaining (discarding) that item in the scale. The study was conducted within the context of a natural experiment that occurred at a major amateur sporting event in 2018. Customer satisfaction was assessed using a survey whose core questions have been assessed repeatedly over time. The most recent survey contained a previously validated empathy scale, with two items contained in the base measure. Six additional items were added to this base empathy measure. The quantity of information provided (as measured by information entropy) is calculated for each set of scale items. Statistical analysis indicates that, when adding the behavioral, cognitive, and affective scales to the two-item base scale, the quantity of information available increased. However, most of the increase in information quantity was attributable to three survey items, one item from each of the behavioral, cognitive, and affective domains. These findings suggest that information entropy may indeed be a useful quality control tool for survey scale development.
