Results 221 to 230 of about 105,297 (262)

The constrained entropy and cross-entropy functions

Physica A: Statistical Mechanics and its Applications, 2004
Abstract This study introduces the constrained forms of the Shannon information entropy and Kullback–Leibler cross-entropy functions. Applicable to a closed system, these functions incorporate the constraints on the system in a generic fashion, irrespective of the form or even the number of the constraints.
openaire   +1 more source

Cross-entropy motion planning

The International Journal of Robotics Research, 2012
This paper is concerned with motion planning for non-linear robotic systems operating in constrained environments. A method for computing high-quality trajectories is proposed building upon recent developments in sampling-based motion planning and stochastic optimization.
openaire   +2 more sources

Cross-Entropy Based Ensemble Classifiers

2016
Multiple classification rules are simultaneously identified by applying the Cross-Entropy method to the maximization of accuracy measures in a supervised learning context. Optimal ensembles of rules are searched through stochastic traversals of the rule space.
openaire   +2 more sources

A Cross-Entropy Scheme for Mixtures

ACM Transactions on Modeling and Computer Simulation, 2015
We discuss how to generalize the classic cross-entropy method in the case where a family of mixture distributions, such as the mixture of multiple Gaussian modes, is used as an importance sampling distribution. A new iterative cross-entropy scheme, based on the idea of the EM method, is proposed to overcome the challenge of deciding the optimal weights
Wang, Hui, Zhou, Xiang
openaire   +2 more sources

Cross entropy, dissimilarity measures, and characterizations of quadratic entropy

IEEE Transactions on Information Theory, 1985
In this paper the authors first define and study the directed divergence or cross entropy of a probability distribution, with respect to another one, induced by a probabilistic entropy. On the other hand, a canonical representation of an entropy in terms of the induced cross entropy is stated for the discrete case (where it is pointed out that with ...
Rao, C. Radhakrishna, Nayak, Tapan K.
openaire   +1 more source

Entropy and Cross Entropy: Characterizations and Applications

2010
The paper provides an axiomatic setup for an entropy function as a measure of diversity. A general definition of cross entropy is given and its use in solving a variety of stochastic and nonstochastic optimization problems is mentioned. A method of deriving a cross entropy function associated with a given entropy function is given.
openaire   +1 more source

LARGE LANGUAGE MODELS: COMPARISON OF CROSS-ENTROPY AND BINARY CROSS-ENTROPY LOSS

HUMAN. ENVIRONMENT. TECHNOLOGIES. Proceedings of the Students International Scientific and Practical Conference
The paper explores Large Language Model (LLM) training on custom datasets for classification microservice development. As training general purpose models for every possible situation is not feasible on smaller scale, because of limitations of computation power, usage of smaller model architectures, such as NanoGPT for training LLM model for specific ...
Apeinans, Ilmars, Kodors, Sergejs, Zarembo, Imants   +2 more
openaire   +2 more sources

Minimum cross-entropy threshold selection

Pattern Recognition, 1996
Thresholding is a common and easily implemented form of image segmentation. Many methods of automatic threshold selection based on the optimization of some discriminant function have been proposed. Such functions often take the form of a metric distance or similarity measure between the original image and the segmented result. A non-metric measure, the
A.D. Brink, N.E. Pendock
openaire   +1 more source

Cross-Entropy Minimization Model

2013
Cross-entropy is used to characterize the divergence between two fuzzy variables. Within the framework of credibility theory, Li and Liu defined the cross-entropy for fuzzy variable by using credibility function, and proposed a fuzzy cross-entropy minimization principle, which tells us that out of all credibility functions satisfying given moment ...
openaire   +1 more source

Prototype Softmax Cross Entropy: A New Perspective on Softmax Cross Entropy

2023
Qendrim Bytyqi, Nicola Wolpert, Elmar Schömer, Ulrich Schwanecke   +3 more
openaire   +1 more source