Results 11 to 20 of about 6,478,030 (244)
Activated Gradients for Deep Neural Networks [PDF]
IEEE Transactions on Neural Networks and Learning Systems, 2023 Deep neural networks often suffer from poor performance or even training failure due to the ill-conditioned problem, the vanishing/exploding gradient problem, and the saddle point problem. In this paper, a novel method by acting the gradient activation function (GAF) on the gradient is proposed to handle these challenges.
Long Jin +2 more
exaly +6 more sources
Sequential Training of Neural Networks With Gradient Boosting
IEEE Access, 2023 This paper presents a novel technique based on gradient boosting to train the final layers of a neural network (NN). Gradient boosting is an additive expansion algorithm in which a series of models are trained sequentially to approximate a given function.
Seyedsaman Emami, Gonzalo Martinez-Munoz
doaj +5 more sources
GradINN: Gradient Informed Neural Network [PDF]
CoRRWe propose Gradient Informed Neural Networks (GradINNs), a methodology inspired by Physics Informed Neural Networks (PINNs) that can be used to efficiently approximate a wide range of physical systems for which the underlying governing equations are completely unknown or cannot be defined, a condition that is often met in complex engineering problems ...
Filippo Aglietti +3 more
core +5 more sources
Neural network gradient Hamiltonian Monte Carlo [PDF]
Computational Statistics, 2019 Hamiltonian Monte Carlo is a widely used algorithm for sampling from posterior distributions of complex Bayesian models. It can efficiently explore high-dimensional parameter spaces guided by simulated Hamiltonian flows. However, the algorithm requires repeated gradient calculations, and these computations become increasingly burdensome as data sets ...
Lingge Li +3 more
openaire +9 more sources
A feed forward neural network approach for matrix computations [PDF]
, 2001 This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.A new neural network approach for performing matrix computations is presented. The idea of this approach is to construct a feed-forward neural network (FNN)
Al-Mudhaf, Ali F
core +7 more sources
An Aeromagnetic Compensation Algorithm Based on a Residual Neural Network
Applied Sciences, 2022 Aeromagnetic compensation is a crucial step in the processing of aeromagnetic data. The aeromagnetic compensation method based on the linear regression model has poorer fitting capacity than the neural network aeromagnetic compensation algorithm.
Ping Yu +5 more
doaj +1 more source
On the computation of the gradient in implicit neural networks
The Journal of Supercomputing, 2022 Abstract Implicit neural networks and the related deep equilibrium models are investigated. To train these networks, the gradient of the corresponding loss function should be computed. Bypassing the implicit function theorem, we develop an explicit representation of this quantity, which leads to an easily accessible computational ...
Béla J. Szekeres, Ferenc Izsák
openaire +3 more sources
Non-Gradient Manifold Neural Network
CoRR, 2021 Deep neural network (DNN) generally takes thousands of iterations to optimize via gradient descent and thus has a slow convergence. In addition, softmax, as a decision layer, may ignore the distribution information of the data during classification. Aiming to tackle the referred problems, we propose a novel manifold neural network based on non-gradient
Rui Zhang 0017 +3 more
openaire +3 more sources
Neural Network Pruning by Gradient Descent
CoRR, 2023 21 pages, 5 ...
Zhang Zhang, Ruyi Tao, Jiang Zhang
openaire +3 more sources
Robustness of Adaptive Neural Network Optimization Under Training Noise
IEEE Access, 2021 Adaptive gradient methods such as adaptive moment estimation (Adam), RMSProp, and adaptive gradient (AdaGrad) use the temporal history of the gradient updates to improve the speed of convergence and reduce reliance on manual learning rate tuning, making ...
Subhajit Chaudhury, Toshihiko Yamasaki
doaj +1 more source