Results 261 to 270 of about 583,636 (288)

Parametrisation Independence of the Natural Gradient in Overparametrised Systems

Lecture Notes in Computer Science, 2021
In this paper we study the natural gradient method for overparametrised systems. This method is based on the natural gradient field which is invariant with respect to coordinate transformations.
Jesse Van Oostrum, Nihat Ay, Ay Nihat
exaly   +1 more source

Some of the next articles are maybe not open access.

Related searches:

Why natural gradient?

Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181), 2002
Gradient adaptation is a useful technique for adjusting a set of parameters to minimize a cost function. While often easy to implement, the convergence speed of gradient adaptation can be slow when the slope of the cost function varies widely for small changes in the parameters.
Shun-ichi Amari, Scott C. Douglas
openaire   +1 more source

Online natural gradient as a Kalman filter

Electronic Journal of Statistics, 2018
3rd version: expanded ...
exaly   +4 more sources

Electrofocusing in natural pH gradients formed by buffers: Gradient modification

Analytical Biochemistry, 1977
Abstract Natural pH gradients formed in buffers (1) can be shifted in slope or in parallel along the pH scale by addition or substitution of buffers to obtain, for each particular fractionation problem, a gradient in which the species of interest occupies the center position.
N Y, Nguyen, A, Salokangas, A, Chrambach
openaire   +2 more sources

Gradient direction dependencies in natural images

Spatial Vision, 2007
We have used information-theoretic measures to compute the amount of dependency which exists between two and three gradient directions at separate locations in an ensemble of natural images. Control experiments were performed on other image classes: phase randomized natural images, whitened natural images and Gaussian noise images.
Alexandre J, Nasrallah, Lewis D, Griffin
openaire   +2 more sources

Natural Gradient and Multiclass NLDA Networks

2002
Natural gradient has been recently introduced as a method to improve the convergence of Multilayer Perceptron (MLP) training [1] as well as that of other neural network type algorithms. The key idea is to recast the training process as a problem in quasi maximum log—likelihood estimation of a certain semipara-metric probabilistic model. This allows the
José R. Dorronsoro, Ana M. González
openaire   +1 more source

Natural Gradient for Minor Component Extraction

2005 IEEE International Symposium on Circuits and Systems, 2005
The paper proposes constrained optimization criteria for extracting in parallel multiple minor components using a weighted inverse Rayleigh quotient (WIRQ). This WIRQ is formulated into many constrained optimization problems which results in deriving many variations of MCA flows using the natural gradient concept.
openaire   +1 more source

Natural Gradient Learning in NLDA Networks

2001
Neural network training is usually formulated as a problem in function minimization. More precisely, if W are the weights defining a network’s architecture And e(W) is the weight depending error function, its gradient ∇e(W) is usually employed to arrive at the optimal weight set W*.
José R. Dorronsoro, Ana M. González, Carlos Santa Cruz   +2 more
openaire   +1 more source

Natural Gradient Works Efficiently in Learning

Neural Computation, 1998
When a parameter space has a certain underlying structure, the ordinary gradient of a function does not represent its steepest direction, but the natural gradient does. Information geometry is used for calculating the natural gradients in the parameter space of perceptrons, the space of matrices (for blind source separation), and the space of linear ...
openaire   +1 more source

Fast Global Convergence of Natural Policy Gradient Methods with Entropy Regularization

Operations Research, 2022
Shicong Cen, Yuxin Chen, Yuting Wei
exaly