Results 41 to 50 of about 3,000 (250)
Catastrophic Importance of Catastrophic Forgetting
CoRR, 2018 This paper describes some of the possibilities of artificial neural networks that open up after solving the problem of catastrophic forgetting. A simple model and reinforcement learning applications of existing methods are also proposed.
openaire +2 more sources
How catastrophic can catastrophic forgetting be in linear regression?
CoRR, 2022 To better understand catastrophic forgetting, we study fitting an overparameterized linear model to a sequence of tasks with different input distributions. We analyze how much the model forgets the true labels of earlier tasks after training on subsequent tasks, obtaining exact expressions and bounds. We establish connections between continual learning
Itay Evron +4 more
openaire +3 more sources
Neural modularity helps organisms evolve to learn new skills without forgetting old skills. [PDF]
PLoS Computational Biology, 2015 A long-standing goal in artificial intelligence is creating agents that can learn a variety of different skills for different problems. In the artificial intelligence subfield of neural networks, a barrier to that goal is that when agents learn a new ...
Kai Olav Ellefsen +2 more
doaj +1 more source
Continual Learning Objective for Analyzing Complex Knowledge Representations
Sensors, 2022 Human beings tend to incrementally learn from the rapidly changing environment without comprising or forgetting the already learned representations. Although deep learning also has the potential to mimic such human behaviors to some extent, it suffers ...
Asad Mansoor Khan +4 more
doaj +1 more source
Localizing Catastrophic Forgetting in Neural Networks
CoRR, 2019 Artificial neural networks (ANNs) suffer from catastrophic forgetting when trained on a sequence of tasks. While this phenomenon was studied in the past, there is only very limited recent research on this phenomenon. We propose a method for determining the contribution of individual parameters in an ANN to catastrophic forgetting. The method is used to
Felix Wiewel, Bin Yang 0009
openaire +2 more sources
Overcoming Catastrophic Forgetting by Generative Regularization
CoRR, 2019 In this paper, we propose a new method to overcome catastrophic forgetting by adding generative regularization to Bayesian inference framework. Bayesian method provides a general framework for continual learning. We could further construct a generative regularization term for all given classification models by leveraging energy-based models and ...
Patrick H. Chen +3 more
openaire +2 more sources
Reducing catastrophic forgetting with learning on synthetic data [PDF]
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2020 Catastrophic forgetting is a problem caused by neural networks' inability to learn data in sequence. After learning two tasks in sequence, performance on the first one drops significantly. This is a serious disadvantage that prevents many deep learning applications to real-life problems where not all object classes are known beforehand; or change in ...
Wojciech Masarczyk, Ivona Tautkute
openaire +2 more sources
On the role of neurogenesis in overcoming catastrophic forgetting
CoRR, 2018 Lifelong learning capabilities are crucial for artificial autonomous agents operating on real-world data, which is typically non-stationary and temporally correlated. In this work, we demonstrate that dynamically grown networks outperform static networks in incremental learning scenarios, even when bounded by the same amount of memory in both cases ...
German Ignacio Parisi +2 more
openaire +2 more sources
Label-Guided relation prototype generation for Continual Relation Extraction [PDF]
PeerJ Computer ScienceContinual relation extraction (CRE) aims to extract relations towards the continuous and iterative arrival of new data. To address the problem of catastrophic forgetting, some existing research endeavors have focused on exploring memory replay methods by
Shuang Liu +3 more
doaj +2 more sources
Advanced Functional Materials, EarlyView.Electroforming‐free TiN/SiOx/Cu/SiOx/TiN memristive devices exploit pancake‐like Cu nanoparticles embedded in a SiOx double layer to create a heterogeneous Schottky‐barrier landscape. Under bias, oxygen‐vacancy redistribution progressively lowers local barriers and recruits initially inactive Cu‐PC pathways into a parallel conduction ensemble, enabling
Rouven Lamprecht +10 more
wiley +1 more source