Abstract
Modeling animal decision-making requires mathematical rigor and computational analysis to capture underlying cognitive mechanisms. This study presents a cognitive model for rat decision-making behavior in \(\mathbb {T}\)-mazes by combining stochastic methods with deep neural architectures. The model adapts Wyckoff’s stochastic framework, originally grounded in Bush’s discrimination learning theory, to describe probabilistic transitions between directional choices under reinforcement contingencies. The existence and uniqueness of solutions are demonstrated via fixed-point theorems, ensuring the formulation is well-posed. The asymptotic properties of the system are examined under boundary conditions to understand the convergence behavior of decision probabilities across trials. Empirical validation is performed using Monte Carlo simulations to compare expected trajectories with the model’s predictive output. The dataset comprises spatial trajectory recordings of rats navigating toward food rewards under controlled experimental protocols. Trajectories are preprocessed through statistical filtering, augmented to address data imbalance, and embedded using t-SNE to visualize separability across behavioral states. A hybrid convolutional-recurrent neural network (CNN-LSTM) is trained on these representations and achieves a classification accuracy of 82.24%, outperforming conventional machine learning models, including support vector machines and random forests. In addition to discrete choice prediction, the network reconstructs continuous paths, enabling full behavioral sequence modeling from partial observations. The integration of stochastic dynamics and deep learning develops a computational basis for analyzing spatial decision-making in animal behavior. The proposed approach contributes to computational models of cognition by linking observable behavior to internal processes in navigational tasks.
Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.Avoid common mistakes on your manuscript.
Introduction
Mathematical models are essential tools for advancing scientific knowledge, especially once comprehensive descriptive statistics have been collected. These models help structure and interpret scientific findings while guiding future experimental research (see Tsibulsky and Norman 2007; An et al. 2024). In educational psychology, where data collection is abundant, the development of robust numerical models is highly feasible (see Tedeschi 2023; Clark 2018).
Research on spatial learning and decision-making in animals has long been a central focus in cognitive neuroscience and psychology (see Ma et al. 2023; Majhi et al. 2024). The \(\mathbb {T}\)-maze experiment is one of the most widely used frameworks for investigating these cognitive processes (see Ngoc Hieu et al. 2020; Gammeri et al. 2022). Despite its simple design, the \(\mathbb {T}\)-maze offers a sophisticated platform for exploring complex behaviors such as learning and memory (see Sharma et al. 2010; Wenk 1999).
In behavioral psychology, key concepts such as associative learning, categorization, reward, and inhibition are foundational for real-world applications. Despite a wealth of virtual studies on training and extinction, there remains a lack of critical research offering precise, quantifiable models to direct experimental work (see Couzin and Heins 2023; Babenko and Romanov 2024; Turab et al. 2022a). Decision-making research has yielded mixed results, particularly in light-guessing experiments and coalition games. While some models accurately predict behavior, others prompt further questions rather than solutions (see Turab et al. 2022b; Ofshe and Ofshe 1970). Theoretical learning experiments suggest that stochastic mechanisms, supported by occurrence theory, can be applied to basic learning studies in probability-learning contexts (see Estes and Straughan 1954; Grant et al. 1951). However, conflicting findings highlight the need for further research into functional equations that analyze animal behavior in two-choice scenarios (see Mosteller 2006; Turab and Sintunavarat 2019, 2020).
The study of animal behavior, particularly in rats, attracts interest from fields such as psychology, neuroscience, ethology, and pharmacology (see Luo et al. 2019; Li et al. 2022). The \(\mathbb {T}\)-maze test is one of the oldest and most trusted experimental setups for studying spatial cognition and behavior in rats (see Tolman 1948; O’keefe and Nadel 1979; Bush and Wilson 1956). While the maze may seem straightforward, it serves as an effective tool for investigating cognitive processes like memory retention and decision-making (see Oberto et al. 2023; d’Isa et al. 2021). On the other hand, Smith et al., in Smith et al. (2005), highlight a growing trend toward using mathematical models, rather than solely empirical or statistical methods, to study rats’ behavior in \(\mathbb {T}\)-mazes. Techniques such as Bayesian statistical theory and neural spike train analysis are increasingly employed to offer deeper insights into decision-making mechanisms (see Brown et al. 1998).
Examining how rats navigate spatial environments necessitates an interdisciplinary approach that bridges cognitive neuroscience and computational modeling. The hippocampal-entorhinal system plays a central role in this process, constructing an internal representation of space that facilitates memory integration and flexible learning. Recent theoretical advancements, such as the Tolman-Eichenbaum Machine (TEM), have provided a structured framework for understanding these cognitive processes. According to this model, medial entorhinal neurons generate abstract spatial representations, which hippocampal circuits then associate with sensory input, enabling adaptive decision-making across varied environments (see Whittington et al. 2020). This conceptualization aligns with empirical findings demonstrating that the hippocampus encodes spatial and relational knowledge through distinct neuronal firing patterns, including those of grid and place cells, allowing rodents to generalize learned information to novel contexts.
Advances in network science have deepened our understanding of the structural organization in biological systems by uncovering fundamental connectivity patterns that govern the emergence of complex morphological architectures (see Gosak et al. 2022). Transformer-based architectures, when modified with recurrent positional encoding, have been observed to spontaneously develop spatial representations resembling those found in the mammalian hippocampus, reinforcing the notion that neural-inspired deep learning models can approximate cognitive mapping mechanisms (Whittington et al. 2021). These insights contribute to a broader understanding of how the brain organizes and retrieves spatial information. In an effort to unify disparate perspectives on hippocampal function, recent studies have synthesized various models, illustrating how structural knowledge and past experiences interact to form coherent, flexible cognitive maps (see Whittington et al. 2022).
Beyond encoding spatial structures, the hippocampus also plays a predictive role in guiding behavior through dynamic processes such as replay. This mechanism allows neural circuits to reconstruct past experiences, rearranging familiar spatial elements to generate potential future pathways. Such recombination of stored information enables efficient decision-making, particularly in novel or uncertain scenarios, supporting the idea that hippocampal processing extends beyond mere memory storage to active inference and planning (for more detail, see Bakermans et al. (2023)).
On the other hand, fixed point methods are instrumental in establishing the existence and uniqueness of solutions across various complex mathematical problems. In the realm of differential equations, for instance, Banach’s fixed point theorem has been applied to demonstrate the existence and uniqueness of solutions to differential equations with boundary conditions by transforming the problem into a fixed-point equation in an appropriate function space (see Turab et al. 2024a, b; Hammad et al. 2021). Similarly, in the study of nonlinear integral equations, fixed point theorems have been utilized to prove the existence and uniqueness of solutions, as seen in recent research on multidimensional fixed-point theorems and their applications (Rashid et al. 2024; Sazaklioglu 2024). Moreover, in the context of functional equations within Banach algebras, the approximation of fixed points of the product of two operators has been explored to address complex equations. These examples underscore the versatility of fixed point methods in providing rigorous solutions to intricate problems across diverse mathematical disciplines (for more detail, see Combettes and Pesquet (2021); Turab and Sintunavarat (2023); Turab et al. (2023)).
This study aims to develop and assess a mathematical model that captures the complexity of rat behavior in a \(\mathbb {T}\)-maze environment. Our objective is to establish criteria for the existence and uniqueness of solutions to the functional equations that describe these behaviors using fixed-point methods. We will design and validate these mathematical models with data obtained from \(\mathbb {T}\)-maze experiments. Additionally, we utilize deep learning techniques, such as Convolutional Neural Networks (CNNs) combined with Long Short-Term Memory (LSTM) units and Gated Recurrent Units (GRUs), to analyze rat navigation patterns. These advanced methods provide a comprehensive understanding of rats’ decision-making processes within the maze.
This research is significant because it advances scientific and mathematical knowledge of rat behavior in \(\mathbb {T}\)-mazes. The models we develop will enhance experimental design, data interpretation, and the development of computational methods for behavioral analysis. Furthermore, these mathematical frameworks could have broader applications, serving as foundational tools for research in behavioral psychology and neuroscience.
The key contributions of this study include:
-
Demonstrating the effectiveness of fixed-point methods in creating a solid mathematical framework for predicting rat behavior in \(\mathbb {T}\)-maze experiments.
-
Establishing clear criteria for the existence and uniqueness of solutions to specific stochastic functional equations.
-
Validating the mathematical models through alignment with empirical data obtained from \(\mathbb {T}\)-maze experiments, confirming accuracy and practical relevance.
-
Highlighting the potential of mathematical models to improve scientific understanding and solve real-world challenges in the analysis of animal behavior.
This paper delves into the intricate field of behavioral science, specifically examining rats’ cognitive and decision-making processes within the \(\mathbb {T}\)-maze framework. It begins with an introduction that outlines the study’s background, objectives, and significance. This is followed by a comprehensive literature review detailing prior research on \(\mathbb {T}\)-mazes, the role of mathematical modeling in behavioral studies, and the application of probability and learning theory. The methodology section explains the \(\mathbb {T}\)-maze setup and the experimental design. A thorough mathematical analysis is presented, incorporating advanced techniques like fixed-point theory and behavior modeling. The study then validates the proposed models using deep learning methods and conducts an in-depth analysis of the results. Finally, the paper summarizes the main findings and identifies potential areas for future research in this rapidly evolving field.
Literature review
The study of rat behavior in \(\mathbb {T}\)-maze experiments has been instrumental in advancing our understanding of cognitive processes, spatial learning, and decision-making. Early research by Tolman introduced the concept of cognitive maps, suggesting that rats develop internal representations of their environment rather than relying solely on stimulus–response associations (Tolman 1948). This foundational work laid the groundwork for neuroscientific explorations, leading to the discovery of place cells in the hippocampus by O’Keefe and Nadel, which provided direct neurophysiological evidence of spatial encoding (O’keefe and Nadel 1979; Bush and Wilson 1956). Over time, the \(\mathbb {T}\)-maze has remained a key apparatus for studying memory retention, reward-based learning, and the influence of pharmacological agents on cognitive function (Wang and Salmaniw 2023; Danieli et al. 2023). Its versatility has enabled researchers to implement modifications such as delayed alternation tasks and spatial navigation challenges, further broadening its applicability in behavioral research (see Knowlton and Castel 2022).
Mathematical modeling has emerged as a crucial tool for formalizing hypotheses and generating quantitative predictions in animal behavior studies. The integration of stochastic models has allowed researchers to capture the probabilistic nature of decision-making, particularly in structured experimental settings like \(\mathbb {T}\)-mazes (Smith et al. 2005; Tuqan and Porfiri 2021). These models, often formulated using Markov decision processes and stochastic differential equations, provide insights into how behavioral choices evolve over time (see Wijeyakulasuriya et al. 2020; Ghanbari and Djilali 2020). More recently, fixed-point methods have been applied to establish the existence and uniqueness of solutions in these models, strengthening the computational foundations of behavioral analysis (Luxem et al. 2022; Shi et al. 2020). The intersection of probability theory and learning mechanisms further supports this approach, aligning with Bayesian inference models that describe how prior knowledge is updated based on new observations (Conway 2020). In this context, neural computation and synaptic plasticity have been shown to reflect statistical regularities in the environment, consistent with Hebbian learning principles (Mazzucato 2022).
The reinforcement learning paradigm has also been instrumental in understanding decision-making processes in both artificial and biological systems (see Hao et al. 2025). Inspired by behavioral psychology, models such as the Rescorla-Wagner framework and temporal difference learning algorithms capture the principles of adaptive learning through reward-driven mechanisms (Ernst and Louette 2024; Navarro et al. 2024). Neuropsychological theories, including the free-energy principle, further emphasize how organisms minimize uncertainty and prediction errors by dynamically updating their internal models (Sánchez-Cañizares 2021; Cook et al. 2022). These approaches have provided a unified perspective on learning and cognition, bridging the gap between theoretical modeling and empirical observations (see Bai et al. 2024).
Advancements in computational methods have further enhanced our ability to analyze complex behavioral patterns (see Gao et al. 2021). The integration of deep learning with traditional stochastic models has enabled the study of hierarchical structures in large-scale animal movement data, improving behavioral classification and predictive accuracy (Dehghani and Trojovský 2022; Torney et al. 2021). In particular, neural networks have demonstrated remarkable efficacy in capturing nonlinear dependencies in behavioral datasets, contributing to more explainable and interpretable models of decision-making (Goodwin et al. 2022). The synergy between machine learning and mathematical modeling has resulted in hybrid approaches that provide a comprehensive framework for decoding animal cognition, particularly in controlled experimental environments such as \(\mathbb {T}\)-mazes (Turab et al. 2024). Recent studies have further emphasized the role of neural mechanisms in structuring naturalistic animal behavior, reinforcing the importance of computational techniques in behavioral ecology and cognitive neuroscience (see Mazzucato (2022)).
Methodology
Setting of \(\mathbb {T}\)-maze for spatial learning
The \(\mathbb {T}\)-maze is a crucial experimental tool commonly used in neuroscience and psychology to study spatial learning and decision-making in animals, particularly rodents such as mice and rats. This maze consists of a long, linear corridor called the “stem”, which ends at a junction resembling the shape of a “\(\mathbb {T}\)”. The junction branches into two arms positioned at right angles to the stem, offering the animal a choice between two distinct paths, typically referred to as the “left” and “right” arms (see Fig. 1 below).
Researchers use the \(\mathbb {T}\)-maze to investigate a variety of cognitive and behavioral processes, including memory retention, decision-making patterns, and the effects of pharmacological interventions. The maze’s design can be modified with rewards, barriers, or cues, allowing for a wide range of experimental setups. Its simplicity and adaptability make the \(\mathbb {T}\)-maze a popular tool in behavioral research (see Fig. 2 below).
Experimental design
Bush’s linearization of Wyckoff’s theory provides the foundation for the model being presented (see Luce et al. 1963; Bush 1959). The experimental setup includes two lights, L and \(\tilde{\textbf{L}}\), located at the choice point (see Fig. 3). A rat starts at the initial position \(\textbf{s}\) and moves to a decision point, \(\textbf{w}\). From there, it selects one of two containers, \(\textbf{A}\) or \(\textbf{B}\), where food may be available. On each trial, one of the lights is randomly lit, with food offered either on the left (A) or right (B), depending on whether L or \(\tilde{\textbf{L}}\) is illuminated-each occurring randomly in 50% of the trials. Over time, most rats will learn to make the correct choice consistently during such experiments.
Let \(x_{n}\) represent the probability that the rat notices which light is on during trial n (event \(\tilde{{\textbf {T}}_{n}}\)). Let \(u_{1,n}\) and \(u_{2,n}\) denote the probabilities of obtaining food (event \({\textbf {O}}_{n}\)) on L and \(\tilde{\textbf{L}}\) trials, assuming the rat is attentive to the lights. If the rat ignores the lights, the probability of receiving food is \(\frac{1}{2}\). The likelihood \(u_{i}\) of getting food increases if a rewarded response follows attention, or if a nonrewarded response follows inattention. These are the only conditions under which \(u_{i}\) changes. The probability x of attending to the lights increases if attention is followed by reward, or if inattention is followed by nonreward. Conversely, it decreases in opposite cases. All changes are assumed to occur through linear functions.
These response patterns are systematically represented using a probability tree diagram, a graphical structure that illustrates sequential decision-making processes. In this model, each branching point represents a decision or event, with branches corresponding to possible outcomes and their associated probabilities. The probability tree diagram (see Figs. 4, 5 and 6 below) provides a structured visualization of how response probabilities evolve based on prior experiences.
Possible response outcomes in the \(\mathbb {T}\)-maze task when a light cue is presented. In both scenarios, the rat is exposed to an illuminated cue signaling the availability of reward. Scenario 1 (top) illustrates a correct choice toward the reward side, while Scenario 2 (bottom) shows an incorrect decision despite identical cue conditions
Possible response outcomes in the \(\mathbb {T}\)-maze task when the light cue is absent. Scenario 3 (top) shows a correct choice toward the reward side without an external cue. Scenario 4 (bottom) illustrates an incorrect choice made under the same cue-absent condition, indicating behavioral variability without stimulus guidance
Probability tree diagram representing possible outcomes in the \(\mathbb {T}\)-maze task. Starting from the initial point, the rat encounters a light cue with probability x or no cue with probability \(1 - x\). In each condition, it chooses between the left and right arms with probabilities \(u_1\) and \(u_2\), respectively. Each directional choice results in a reward or no reward with equal probability
The summary of these assumptions is provided in Table 1.
The chain of actions presented above describes a single experimental test. This procedure is typically repeated multiple times with the same rat. The overall activity of the rat in a maze experiment is complex. When the rat reaches the choice point, it is in one of two stimulus conditions. These conditions can be categorized, and specific metrics can be used to analyze various aspects of the rat’s behavior. For example, one might measure the latency from the starting point \(\textbf{s}\) or analyze the momentum between \(\textbf{s}\) and \(\textbf{w}\). However, the rat’s decision at the choice point \(\textbf{w}\) is the most critical aspect of this experiment.
In the following research, we focus solely on the rat’s route choice during each trial rather than analyzing other behaviors. Repeated experimental trials, with consistent stimuli, lead the rat to make a decision at the choice point. In Table 1, the \(\mathbb {T}\)-maze presents two options, each corresponding to one of the two possible target boxes, \(\textbf{A}\) or \(\textbf{B}\). Only one of these options is selected in each trial. In contrast to an individual trial, experimental research allows the rat to choose between two mutually exclusive and exhaustive alternatives.
Theoretical framework
Mathematical foundation and assumptions
The following fixed point outcome is required in the progression (to obtain further information regarding fixed point theory, one may refer (Jiang 2022; Hazarika et al. 2024)).
Theorem 4.1
(Banach 1922) Let \((\mathcal {A},d )\) be a complete metric space and \(\mathcal {W}:\mathcal {A}\rightarrow \mathcal {A}\) be a Banach contraction mapping (shortly as, BCM) given by
for \(\wp <1\). Then \(\mathcal {A}\) has one fixed point. In addition, the iteration \(\{\ell _{1}^{(n)}\}\) in \(\mathcal {A}\) defined as \(\ell _{1}^{(n)}=\mathcal {W}\ell _{1}^{(n-1)}\) for all \(n\in \mathbb {N}\), where \(\ell _{1}^{(0)}\in \mathcal {A}\), converges to the unique fixed point of \(\mathcal {W}\).
For the computational convenience, we use the following notations:
Model formulation
In this experiment, both the movement of the rat towards a particular compartment (\(\textbf{A}\) or \(\textbf{B}\)) and the location of the food serve as critical cues. The rat’s behavior is monitored to see whether it moves to the left or right of where the food is placed. If the rat chooses the food side, event \(\mathbf {O_{1}}\) occurs; otherwise, event \(\mathbf {O_{2}}\) arises when the rat moves to the opposite side. Based on the rat’s movement, the light conditions (\({\textbf {L}}\) and \(\tilde{\textbf{L}}\)), and the location of the food, there are eight possible outcomes, as shown in Table 2.
The responses \(\textbf{T}\) and \(\tilde{\textbf{T}}\) occur with probabilities x and \((1-x)\), respectively. The experimental design assigns probabilities to the outcomes, determining whether the rat receives food in the presence of lights \(\textbf{L}\) or \(\tilde{\textbf{L}}\), with the probabilities \(u_{1}\) and \(u_{2}\), where \(u_{1}, u_{2} \in [0,1]\). Table 3 presents the probabilities for each of the eight possible outcomes.
Let \(\vartheta _{1},\vartheta _{2}, \ldots , \vartheta _{8}\in (0,1)\) represent learning-rate parameters, which reflect the effect of the outcomes \(\mathbf {E_{1}}-\mathbf {E_{8}}\) on modifying response probabilities. Additionally, let \(\lambda _{k}\in [0,1]\), where \(k=1,2,...,8\), be the fixed point of the associated events. For instance, if \(\frac{1}{2}u_{1}x\) represents the probability of a response \(\textbf{LT}\) with an outcome \(\mathbf {O_{1}}\), and \(\textbf{A}\) occurs, then the updated probability of \(\textbf{LT}\) resulting in \(\mathbf {O_{1}}\) will be \(\vartheta _{1}x+(1-\vartheta _{1})\lambda _{1}\). Similarly, for a response \(\textbf{LT}\) with outcome \(\mathbf {O_{2}}\), the new probability becomes \(\vartheta _{2}x+(1-\vartheta _{2})\lambda _{2}\).
To formalize these probability updates, we introduce the transition operators \(P_{1}, P_{2}, \ldots , P_{8}\), which are mathematical mappings from \([0,1] \rightarrow [0,1]\). These operators define how response probabilities evolve over time-based on past events, ensuring that the decision-making process remains consistent with the model’s stochastic framework. The transition operators are defined as follows:
for all \(x\in [0,1].\)
These transition probabilities reflect the weighted sum of x and \(\lambda _k\), with the learning rate parameters ensuring adaptation in response to the outcomes. If \(\vartheta _{1}, \vartheta _{2}, \ldots , \vartheta _{8}\) all equal 1, the system would stabilize, ceasing further change in response probabilities. Given x, \(\vartheta _{1},\vartheta _{2},\ldots ,\vartheta _{8}\), and \(\lambda _{1},\lambda _{2},\ldots ,\lambda _{8}\), the probability of the rat ceasing to respond to one choice while consistently responding to the other can be expressed as:
This equation accounts for the response to various cues such as the pathway, light, and outcome, updating the probabilities according to the transitions described in (4.6). The final probability after each trial is computed by aggregating the event probabilities and the parameters \(\vartheta _{i}\) and \(\lambda _{i}\) for each event, as shown below:
Mathematical analysis
Throughout this work, we let \(\mathcal {A}=[0,1]\), \(\bar{\mathcal {A} }\) be a nonempty convex subset of a normed space \(\mathcal {E}\), \(\mathbb {N}\) denotes the set of all positive integers, and the class of all continuous functions \(\mathcal {W}:\mathcal {A} \rightarrow \mathbb {R}\) is represented by \(\mathcal {B}\) such that \(\mathcal {W}(0)=0\) and
Here, the Banach space associated with \(\left( \mathcal {B},\left\| \cdot \right\| \right)\) is evident with
Furthermore, we rewrite (4.7) as
where \(0<\vartheta _{1},\vartheta _{2},\vartheta _{3},\vartheta _{4},\vartheta _{5},\vartheta _{6}, \vartheta _{7}, \vartheta _{8}<1\), \(\lambda _{k}\ (k=1,2,...,8), u_{1},u_{2}\in \mathcal {A}\), and \(\mathcal {W}:\mathcal {A}\rightarrow \mathbb {R}\) is an unknown function.
In the context of analyzing the movement of a rat within a \(\mathbb {T}\)-maze setup, the parameters described in the functional Eq. (5.1) are interpreted as follows:
-
1.
State space \(\mathcal {A} = [0,1]\): This parameter defines the range of possible states, where each state \(x \in \mathcal {A}\) reflects the probability that the rat will choose a specific direction (left or right) within the maze. A state value near 1 indicates a strong preference for one direction, while a value closer to 0 suggests the opposite.
-
2.
Learning rate expressions \(\vartheta _{\ell }x + (1 - \vartheta _{\ell }) \lambda _{\ell } \ (\ell = 1, 2, \dots , 8)\): These expressions capture the learning dynamics of the rat. The parameter \(\vartheta _{\ell }\) governs how the rat adjusts its behavior based on previous outcomes. A higher value of \(\vartheta _{\ell }\) suggests a greater responsiveness to recent experiences, while \(\lambda _{\ell }\) represents a baseline fixed point that influences the rat’s preference during the decision-making process.
-
3.
Decision probability function \(u_{j}x \ (j = 1, 2)\): This function models the probability distribution between the two available choices in the maze: left (\(u_1\)) or right (\(u_2\)). It quantifies the likelihood of the rat selecting one of these options. The probabilities evolve over time as part of a Markov decision process, where the state transitions are influenced by the rat’s prior choices and learned behaviors.
-
4.
Outcome probability function \(\mathcal {W}\): The function \(\mathcal {W}\) represents the long-term probability that the rat will consistently choose a particular direction. It reflects how the rat’s decision-making stabilizes over time as it learns from repeated trials, capturing the eventual preference for one path over the other. This function essentially models the culmination of the rat’s learning process, providing insight into its final behavioral patterns after multiple decision-making instances.
Before moving to the main objectives, we define the following key assumptions here, which we shall use in the later section.
- (\(\mathfrak {P}^{\star }\)):
-
For an operator \(\psi\), there exists a closed subset \(\mathcal {O}\) of \(\mathcal {B}\) such that \(\mathcal {O}\) is \(\psi\)-invariant, that is, \(\psi (\mathcal {O} )\subseteq \mathcal {O}\).
Theorem 5.1
Let \(0<\vartheta _{1},\vartheta _{2},\dots ,\vartheta _{8}<1\) and \(\lambda _{k}\ (k=1,2,...,8),u _{1},u _{2} \in \mathcal {A}\) such that \(\mathfrak {K}_{1}^{\star } <1,\) where \(\mathfrak {K}_{1}^{\star }\) is given in (4.2). If an operator Z from \(\Lambda\) defined for each \(\mathcal {W}\) \(\in\) \(\Lambda\) and for all \(x\in \mathcal {A}\) by
and satisfies property (\(\mathfrak {P}^{\star }\)), then Z is a BCM associated with the metric d imposed by \(\left\| \cdot \right\|\).
Proof
Let \(\mathcal {W}_{1},\mathcal {W}_{2}\in \Lambda\). For all \(x,y\in \mathcal {A}\) with \(x \ne y\), we have
This gives that
As \(0<\mathfrak {K}_{1}^{\star } <1,\) hence Z is a BCM with the metric d imposed by \(\left\| \cdot \right\|\). \(\square\)
From Theorem 5.1, we may deduce the following regarding the unique solution of a stochastic Eq. (5.1).
Theorem 5.2
The functional Eq. (5.1) has a unique solution provided that \(\mathfrak {K}_{1}^{\star }<1,\) where \(\mathfrak {K}_{1}^{\star }\) is defined by (4.2), and there exists an operator Z from \(\Lambda\) defined for each \(\mathcal {W}\) \(\in \Lambda\) and for all \(x\in \mathcal {A}\) in (5.2) and satisfies property (\(\mathfrak {P}^{\star }\)). Furthermore, the iteration \(\{\mathcal {W}_{n}\}\) (\(\forall n\in \mathbb {N}\)) in \(\Lambda\), where \(\mathcal {W}_{0}\in \Lambda\), is given by
converges to the unique solution of the stochastic Eq. (5.1).
Proof
We conclude the proof by coupling Theorem 5.1 with the Banach fixed point theorem. \(\square\)
Asymptotic behavior and numerical approximation
The numerical behavior of \(\mathcal {W}(x)\) is analyzed through a combination of iterative approximations and asymptotic evaluations. Figure 7, observed with specific parameter settings (\(\theta\) and \(\lambda\) values), illustrates the function’s convergence properties and numerical stability. The iterative method demonstrates a consistent reduction in the maximum change between successive approximations, affirming the reliability of the numerical scheme employed. Additionally, the function exhibits a characteristic trend across its domain, initially rising before reaching a peak and subsequently declining, reinforcing its nonlinear structure under the defined parameter constraints.
The asymptotic properties of \(\mathcal {W}(x)\) as \(x\) approaches the boundaries provide further insights into its stability. Figure 8 reveals that near \(x=0\), the function experiences steady growth, stabilizing without divergence, while at \(x=1\), it remains relatively constant, suggesting minimal variation. These observations align with theoretical expectations, supporting the existence and uniqueness of solutions derived via the Banach fixed-point theorem. Such graphical insights validate the robustness of the functional equation and its sensitivity to initial conditions.
To further evaluate the numerical approach, we compare the fixed-point iteration method with the Monte Carlo approximation and their error analysis (see Fig. 9). The fixed-point method exhibits smooth deterministic convergence, ensuring well-defined solutions, while the Monte Carlo approach, leveraging stochastic sampling, introduces slight deviations. The latter closely follows the deterministic trajectory but tends to underestimate peak values due to its inherent randomness. Error analysis indicates that while absolute error remains within an acceptable range, fluctuations occur around mid-domain values before gradually diminishing. This trade-off highlights the precision of fixed-point methods in solving functional equations while acknowledging the probabilistic variability introduced by Monte Carlo approximations.
Some specific features
In this portion, certain situations of the Wyckoff stochastic model have been examined.
Situation with identical lambda constraints
This condition, also known as the commutative condition, states that our transition operators \(P_{1}\) through \(P_{6}\) (none of which are identity operators) have identical lambda conditions \((\lambda _{1}=\lambda _{2}=\lambda _{3}=\lambda _{4}=\lambda _{5}=\lambda _{6}=\lambda _{7}=\lambda _{8}=\lambda )\). Due to these constraints, our transition operators (4.6) are reduced to
By ensuring that all transition operators share the same lambda value, this condition maintains the model’s stability across different maze structures and configurations. This uniformity allows the model to generalize beyond a specific experimental setting, making it adaptable to alternative maze layouts without requiring extensive parameter adjustments. Now, our functional Eq. (5.1) can be expressed as
where \(0<\vartheta _{1},\vartheta _{2},\vartheta _{3},\vartheta _{4},\vartheta _{5},\vartheta _{6}, \vartheta _{7}, \vartheta _{8}<1\), \(\lambda ,u_{1},u_{2}\in \mathcal {A}\) and \(\mathcal {W}:\mathcal {A}\rightarrow \mathbb {R}\) is an unknown function. The Theorem 5.1 has the following findings as a result.
Corollary 6.1
Let \(0<\vartheta _{1},\vartheta _{2},\dots ,\vartheta _{8}<1\) and \(\lambda ,u_{1},u_{2}\in \mathcal {A}\) with \(\mathfrak {K}_{2}^{\star } <1\), where \(\mathfrak {K}_{2}^{\star }\) is defined in (4.3). If an operator Z from \(\Lambda\) associated for every \(\mathcal {W}\) \(\in\) \(\Lambda\) and for all \(x\in \mathcal {A}\) by
and satisfies property (\(\mathfrak {P}^{\star }\)), then Z is a BCM.
Corollary 6.2
The Eq. (6.2) has a unique solution having that \(\mathfrak {K}_{2}^{\star } <1\), where \(\mathfrak {K}_{2}^{\star }\) is given in (4.3), and there exists an operator Z defined in (6.3) satisfies property (\(\mathfrak {P}^{\star }\)). Furthermore, the iteration \(\{\mathcal {W}_{n}\}\) (\(\forall n\in \mathbb {N}\)) in \(\Lambda\), where \(\mathcal {W}_{0}\in \Lambda\), is given by
converges to the unique solution of (6.2).
Elimination of a behavioral reflex
It’s possible that the mouse’s frequent right- or left-turning side reactions might reduce an event’s probability to the point of asymptote to zero. Such a situation necessitates the assumption that \(\lambda _{1}=\lambda _{2}=\dots =\lambda _{8} =0\). These constraints decrease our four operators (4.6) to
By setting lambda values to zero in cases where habitual side preferences emerge, this condition prevents the model from being influenced by fixed behavioral reflexes. As a result, decision-making remains dynamically responsive to environmental cues rather than being constrained by preconditioned biases, thereby improving the model’s generalizability across different testing conditions. So, we can write (5.1) as
where \(0<\vartheta _{1},\vartheta _{2},\vartheta _{3},\vartheta _{4},\vartheta _{5},\vartheta _{6}, \vartheta _{7}, \vartheta _{8}<1\), \(u_{1},u_{2}\in \mathcal {A}\) and \(\mathcal {W}:\mathcal {A}\rightarrow \mathbb {R}\) is an unknown function. The following are the Theorem 5.1’s corollaries.
Corollary 6.3
Let \(0<\vartheta _{1},\vartheta _{2},\vartheta _{3},\vartheta _{4},\vartheta _{5},\vartheta _{6}, \vartheta _{7}, \vartheta _{8}<1\) and \(u_{1},u_{2}\in \mathcal {A},\) with \(\mathfrak {K}_{3}^{\star } <1\), where \(\mathfrak {K}_{3}^{\star }\) is defined in (4.4). If an operator Z from \(\Lambda\) associated for every \(\mathcal {W}\) \(\in\) \(\Lambda\) and for all \(x\in \mathcal {A}\) by
and satisfies property (\(\mathfrak {P}^{\star }\)), then Z is a BCM.
Corollary 6.4
The stochastic Eq. (6.6) has a unique solution with \(\mathfrak {K}_{3}^{\star } <1\), where \(\mathfrak {K}_{3}^{\star }\) is given in (4.4), and there exists an operator Z defined in (6.7) satisfies property (\(\mathfrak {P}^{\star }\)). Furthermore, the iteration \(\{\mathcal {W}_{n}\}\) (\(\forall n\in \mathbb {N}\)) in \(\Lambda\), where \(\mathcal {W}_{0}\in \Lambda\), is given by
converges to the unique solution of (6.6).
In the same way, if the rat frequently selects the food side, the chance of that event occurring increases. Therefore, we have
In this situation, our four operators will be
Now, our functional Eq. (5.1) may be expressed as
where \(0<\vartheta _{1}, \vartheta _{2}, \vartheta _{3}, \vartheta _{4},\vartheta _{5}, \vartheta _{6}, \vartheta _{7}, \vartheta _{8} <1\), \(u_{1}, u_{2} \in \mathcal {A}\) and \(\mathcal {W}:\mathcal {A}\rightarrow \mathbb {R}\) is an unknown function. The following outcomes are the findings of Theorem 5.1.
Corollary 6.5
Let \(0<\vartheta _{1}, \vartheta _{2}, \vartheta _{3}, \vartheta _{4}, \vartheta _{5}, \vartheta _{6}, \vartheta _{7}, \vartheta _{8}<1\) and \(u_{1}, u_{2} \in \mathcal {A}\) with \(\mathfrak {K}_{4}^{\star } <1\), where \(\mathfrak {K}_{4}^{\star }\) is defined in (4.5). If an operator Z from \(\Lambda\) associated for every \(\mathcal {W}\) \(\in\) \(\Lambda\) and for all \(x\in \mathcal {A}\) by
and satisfies property (\(\mathfrak {P}^{\star }\)), then Z is a BCM.
Corollary 6.6
The Eq. (6.10) has a unique solution claiming that \(\mathfrak {K}_{4}^{\star } <1\), where \(\mathfrak {K}_{4}^{\star }\) is defined in (4.5), and there exists an operator Z defined in (6.11) satisfies property (\(\mathfrak {P}^{\star }\)). Furthermore, the iteration \(\{\mathcal {W}_{n}\}\) (\(\forall n\in \mathbb {N}\)) in \(\Lambda\), where \(\mathcal {W}_{0}\in \Lambda\), is given by
converges to the unique solution of (6.10).
Performance validation with deep learning methods
To analyze and classify rat movements (left or right), we applied two widely recognized deep learning approaches: Convolutional Neural Networks-Long Short-Term Memory (CNN-LSTM) and Convolutional Neural Networks-Gated Recurrent Unit (CNN-GRU) (see Wu et al. 2021; Ullah et al. 2024 for more detail).
Data collection and processing
The dataset initially consisted of video recordings capturing rat movements toward rewards in a controlled environment. We developed a method to extract frames from the videos, resulting in 153 images of left movements and 105 images of right movements. However, this dataset was insufficient for training deep learning models, which require a larger volume of data to generalize effectively during training.
To overcome this limitation, we applied data augmentation techniques to generate additional images while preserving the characteristics of the original dataset. This method enhanced the diversity of the training set, improving model generalization. The augmentation techniques included rotation, flipping, zooming, cropping, brightness and contrast adjustment, and noise injection:
-
Rotation: Rotating images to improve the model’s resilience to varying perspectives of movement.
-
Flip: Flipping images vertically or horizontally to make the model more robust against different movement orientations.
-
Zoom: Cropping and resizing images to simulate zooming, aiding the model in tracking movement.
-
Crop: Randomly cropping portions of the images to help the model focus on different areas of the objects.
-
Brightness and Contrast Adjustment: Modifying brightness and contrast to simulate different lighting conditions.
-
Noise Injection: Adding random noise to make the model more resistant to variations in input data.
Statistical analysis
To address the high dimensionality of the dataset, a strategic approach was adopted to ensure the analysis remained concise and focused. Including all features (Feature_0, Feature_1, Feature_2, \(\dots\), Feature_127), which are extracted from the images, would have resulted in an overly complex and lengthy report. Instead, the top five features exhibiting the highest variability-Feature_0, Feature_8, Feature_16, Feature_96, and Feature_104-were selected for detailed examination based on their standard deviation. A t-test was employed to compare these features between the left and right turn datasets. The analysis revealed that Feature_8 and Feature_16 demonstrated statistically significant differences \((p < 0.001)\), highlighting their potential as robust discriminators between the two movement types.
To further validate these findings, the Kolmogorov-Smirnov (KS) test was applied to compare the cumulative distributions of the left and right turn movements across the top five features. The results confirmed that Feature_0, Feature_8, and Feature_16 exhibited the highest KS statistics and the lowest p-values \((p < 0.001)\), indicating substantial differences in their distributions between the two movement types. This further underscores their discriminative potential for classification tasks. Table 4 represents a summary of the key statistical outcomes for these features.
To explore the separability of the datasets further, t-Distributed Stochastic Neighbor Embedding (t-SNE) was employed to visualize high-dimensional data in lower-dimensional space. Figures 10, 11 and 12 illustrate the clustering of left and right movements under different perplexity values \((p=30, 40, 50)\). The visual separation between the movements indicates the extracted features’ effectiveness in distinguishing the two types. The distinct boundaries of the clusters demonstrate the robustness of the feature extraction process, confirming the validity of the machine learning models used.
The dataset is shown using t-SNE with perplexity \(p = 30\). The t-SNE approach transforms high-dimensional data into lower-dimensional space, exposing cluster structures and interactions between data points. This visualization aids in determining class separability and the efficacy of feature representation
In addition, Linear Discriminant Analysis (LDA) was performed to project the data onto a single discriminative axis. Figure 13 illustrates the distribution of the left and right turn datasets with distinct color coding: light/dark blue represents the left turn movement, while green represents the right turn movement. The overlapping regions in the plot indicate areas where the two classes are less separable using a linear boundary, while the distinct peaks highlight regions with clearer differentiation. Unlike PCA, which maximizes variance, LDA focuses on maximizing class separation by identifying an optimal linear boundary. The LDA plot demonstrated superior separation compared to the PCA projection, underscoring the contribution of specific features to class differentiation.
The Cumulative Distribution Function (CDF) plots illustrate that greater separation between the curves corresponds to stronger feature discriminability (see Fig. 14). While Feature_96 and Feature_104 also showed some separation, their differences were less pronounced compared to Feature_0, Feature_8, and Feature_16. These findings emphasize the importance of feature selection in movement classification and suggest that leveraging these key features can significantly enhance the predictive accuracy of machine learning models.
Results analysis
The augmented dataset was processed using the designed deep learning models, CNN-LSTM and CNN-GRU. Figure 15 presents the epoch curves for training and testing data. In Part (a), training accuracy is depicted in blue, while testing accuracy is shown in red. These graphs illustrate the model’s performance over time. Initially, the training curve starts at 15%, and the testing curve at 63%. The testing accuracy dips to 40% before gradually increasing with each epoch. The training curve follows a steady progression, while the testing curve fluctuates, dipping to 70% at epoch 20 before rising again. After a few more epochs, both curves stabilize around 84%, indicating consistent model performance.
Part (b) shows the training and testing losses for each epoch. Training loss is shown in yellow, and testing loss is shown in green. Both curves begin at around 70% and steadily decrease over the epochs. Although the testing curve briefly rises to 55% by the 35th epoch, it quickly decreases. Ultimately, both curves converge to a loss of around 19%. These loss curves complement the accuracy curves, confirming that the proposed approach achieves strong performance with minimal loss.
Similarly, Fig. 16 displays the training and testing curves for the CNN-GRU model. The training curve follows a regular, increasing pattern in line with the number of epochs. The testing curve, while occasionally showing rapid fluctuations, stabilizes and mirrors the behavior of the training curve. Overall, both curves show performance between 50% and 82%. The loss curves demonstrate a reduction from 70% to 20%, further supporting the model’s effectiveness.
Five metrics were used to evaluate model performance: precision, recall, F1-score, accuracy, and confusion matrices. True Positives (TP) and True Negatives (TN) represent correct classifications for left and right movements, while False Positives (FP) and False Negatives (FN) represent misclassifications. Accuracy was calculated by dividing correctly classified instances by the total number of instances, with the formulas provided in Eqs. (7.1)–(7.4).
Table 5 displays the performance metrics for the CNN-LSTM model, where precision, recall, and F1-score for left movement are 81.18%, 81.51%, and 82.24%, respectively. For right movement, the corresponding values are 83.12%, 83.14%, and 81.76%, with an overall accuracy of 82.24%. Table 6 shows the CNN-GRU model performance, with left movement values of 81.23%, 80.82%, and 80.44%, and right movement values of 77.28%, 77.12%, and 80.61%, yielding an overall accuracy of 80.52%. These results demonstrate the proposed method’s capability to classify rat movements accurately.
Confusion matrices provide further insight into classification performance, detailing TP, TN, FP, and FN rates. Figure 17 shows that the CNN-LSTM model correctly classifies 81% of left movements and 83% of right movements, with respectively 19% and 17% misclassification rates. For CNN-GRU, left movements are correctly classified 84% of the time, with an error rate of 16%, while right movements show a 77% accuracy and a 23% error rate. These results confirm the models’ robustness in tracking and classifying rat movements.
Training and testing performance curves for the CNN-LSTM model. a Accuracy curves show the variation in training and testing accuracy over epochs, demonstrating the model’s learning progress and convergence. b The training and testing loss changes over epochs are represented by loss curves, which illustrate how well the model reduces errors and generalizes to unseen data
Training and testing performance curves for the CNN-GRU model. a Accuracy curves show the variation in training and testing accuracy over epochs, demonstrating the model’s learning progress and convergence. b The training and testing loss changes over epochs are represented by loss curves, which illustrate how well the model reduces errors and generalizes to unseen data
Confusion matrices for the classification performance of the CNN-LSTM and CNN-GRU models. a CNN-LSTM: The confusion matrix depicts the CNN-LSTM model’s performance by showing the amount of true positives, false positives, true negatives, and false negative predictions in each class. b CNN-GRU: The confusion matrix for the CNN-GRU model, similarly illustrating the model’s classification performance with the respective counts of true and false predictions. These matrices present a comprehensive understanding of the models’ accuracy, class balance, and potential misclassifications
Table 7 shows the evaluation metrics for a model that employs a transformer encoder to categorize Left Move and Right Move. The model obtained a Left Move precision score of 82.07% and a Right Move precision score of 82.14%. These results are quite similar, indicating that the model is equally effective at accurately projecting positive events for both motions. The model showed a higher recall for Left Move (84.05%) than for Right Move (80.31%). This demonstrates that the model identified more positive instances of Left Move than Right Move. The F1-measure for Left Move was 82.05%, while Right Move had an F1 of 81.36%. This measure combines precision and recall. Although the Left Move F1-measure is slightly lower, both values are close, demonstrating that the model’s overall performance remains stable over the two movements. Finally, the model achieved an Accuracy of 82.15%, which means that around 82.15% of all predictions provided by the model throughout both movements are correct.
A comparison of the Transformer Encoder and CNN-LSTM models reveals that each model has distinct advantages in certain areas. The Transformer Encoder has a high left move recall rate (84.05%) and a slightly higher precision rate (82.07%). While both models achieve better F1-measures, the CNN-LSTM model has an outperformance in Precision for Right Move (83.12%). The Transformer Encoder consistently obtains an accuracy of 82.15%, whereas the CNN-LSTM model is slightly more accurate for Left Move (82.24%) but less accurate for Right Move (81.76%). In general, both models exhibit exceptional performance, with each model excelling in distinct areas based on the movement style.
We used cross-validation (KFold with 5 splits) to assess the models’ generalization performance, ensuring that they did not overfit the training data. Cross-validation evaluates the performance of models using previously unknown data by dividing the dataset into multiple training and testing segments. The experiment tests for overfitting by comparing training and test accuracy. If training accuracy is significantly higher than test accuracy, the model could be overfitting. This comparison demonstrates that the models can accurately generalize to new data. Table 8 compares the performance of two machine learning models, Decision Tree (DT) and Support Vector Machine (SVM), in detecting mouse movements. The Decision Tree model outperforms the Right Move in terms of recall and precision, yielding a total accuracy of 81.39%. The Decision Tree obtains an accuracy of 80.83%, whereas the SVM model has slightly lower recall and precision scores for both motions. The F1-measure is quite evenly distributed between the two models, implying that they have similar classification efficiency.
The presented experiments demonstrate the CNN-LSTM model’s contribution when compared to other models such as the Transformer Encoder, Decision Tree, and SVM. The CNN-LSTM model is particularly effective in encoding spatial and temporal dependencies in the data, which enhances its performance, particularly in terms of precision for Right Move, by combining convolutional and LSTM layers. The CNN-LSTM outperforms the Transformer Encoder in precision for Right Move, despite its strong recall performance, particularly for Left. Deep learning models outperform DT and SVM because they are better able to detect sequential patterns, which leads to more accurate classifications.
Conclusion and open problems
This study proposed a cognitive framework for modeling rat decision-making behavior in \(\mathbb {T}\)-mazes by combining stochastic processes with deep learning methods. The model builds on Wyckoff’s stochastic formulation to represent probabilistic response shifts across trials under reinforcement contingencies. The existence and uniqueness of solutions were established through the Banach fixed-point theorem, ensuring the mathematical consistency of the system. A comparative analysis between Picard iteration and Monte Carlo simulations demonstrated close agreement, supporting the numerical stability of the model.
We employed deep neural architectures, such as CNN-LSTM and CNN-GRU models, which were trained on trajectory sequences derived from experimental recordings to examine behavioral data. These models achieved notable classification performance, outperforming standard approaches. Statistical preprocessing and nonlinear dimensionality reduction using t-SNE facilitated the analysis of feature distributions across behavioral states, offering interpretable structure within high-dimensional data.
The results suggest that integrating stochastic modeling with data-driven neural methods can effectively capture the probabilistic structure and temporal dynamics of navigational behavior. This approach enables the analysis of individual learning patterns without relying on strong parametric assumptions or oversimplified behavioral rules.
Several open problems remain for further investigation.
-
Problem 1:
How does the decision-making process evolve in trial k if the rat does not move toward the left or right compartment?
-
Problem 2:
A fundamental aspect of functional equations is their stability, particularly within the Ulam-Hyers and Ulam-Hyers-Rassias frameworks (see Brzdek 2023; Choubin and Javanshiri 2021). The stability properties of the following equation remain unresolved and warrant further analysis.
$$\begin{aligned} \mathcal {W}(x)= & \frac{1}{2}u_{1}x\mathcal {W}(\vartheta _{1}x+(1-\vartheta _{1})\lambda _{1})\\ & +\frac{1}{2}(1-u_{1})x\mathcal {W}(\vartheta _{2}x+(1-\vartheta _{2})\lambda _{2}) \\ & +\frac{1}{2}u_{2}x\mathcal {W}(\vartheta _{3}x+(1-\vartheta _{3})\lambda _{3})\\ & +\frac{1}{2}(1-u_{2})x\mathcal {W}(\vartheta _{4}x+(1-\vartheta _{4})\lambda _{4}) \\ & +\frac{1}{2}(1-x)u_{1}\mathcal {W}(\vartheta _{5}x+(1-\vartheta _{5})\lambda _{5})\\ & + \frac{1}{2}(1-x)(1-u_{1}) \mathcal {W}(\vartheta _{6}x+(1-\vartheta _{6})\lambda _{6})\nonumber \\ & + \frac{1}{2}(1-x)u_{2} \mathcal {W}(\vartheta _{7}x+(1-\vartheta _{7})\lambda _{7})\\ & + \frac{1}{2}(1-x)(1-u_{2}) \mathcal {W}(\vartheta _{8}x+(1-\vartheta _{8})\lambda _{8}), \end{aligned}$$where \(\mathcal {W}:\mathcal {A}\rightarrow \mathbb {R}\), \(0<\vartheta _{1},\vartheta _{2},\vartheta _{3},\vartheta _{4},\vartheta _{5},\vartheta _{6},\vartheta _{7},\vartheta _{8}<1\) and \(\lambda _{k}\ (k=1,2,...,8),u_{1},u_{2}\in \mathcal {A}\).
Data Availability
The data that support the findings of this study can be accessed from here: https://github.com/Farhankhancs/AliTurab.
References
An XK, Du L, Jiang F, Zhang YJ, Deng ZC, Kurths J (2024) A few-shot identification method for stochastic dynamical systems based on residual multipeaks adaptive sampling. Chaos 10(1063/5):0209779
Babenko Y, Romanov V (2024) Intelligent methods in behavioral studies on animal models. Proceedings http://ceur-ws.org ISSN 16130073
Bai Y, Shao S, Zhang J, Zhao X, Fang C, Wang T, Zhao H (2024) A review of brain-inspired cognition and navigation technology for mobile robots. Cyborg Bionic Syst 5:0128. https://doi.org/10.34133/cbsystems.0128
Bakermans JJ, Warren J, Whittington JC, Behrens TE (2023) Constructing future behaviour in the hippocampal formation through composition and replay. Biorxiv. https://doi.org/10.1101/2023.04.07.536053
Banach S (1922) Sur les opérations dans les ensembles abstraits et leur application aux équations intégrales. Fundam Math 3(1):133–181
Brown EN, Frank LM, Tang D, Quirk MC, Wilson MA (1998) A statistical paradigm for neural spike train decoding applied to position prediction from ensemble firing patterns of rat hippocampal place cells. J Neurosci 18(18):7411–7425. https://doi.org/10.1523/JNEUROSCI.18-18-07411.1998
Brzdek J (2023) On Ulam stability with respect to 2-norm. Symmetry 15(9):1664. https://doi.org/10.3390/sym14071365
Bush RR (1959) Sequential properties of linear models. Studies in mathematical learning theory. Stanford Univ. Press, Stanford, pp 215–227
Bush RR, Wilson TR (1956) Two-choice behavior of paradise fish. J Exp Psychol 51(5):315. https://doi.org/10.1037/h0044651
Choubin M, Javanshiri H (2021) A new approach to the Hyers–Ulam–Rassias stability of differential equations. RM 76:1–14. https://doi.org/10.1007/s00025-020-01318-w
Clark CW (2018) Modelling the behaviour of fishers and fishes. ICES J Mar Sci 75(3):932–940. https://doi.org/10.1093/icesjms/fsx212
Combettes PL, Pesquet JC (2021) Fixed point strategies in data science. IEEE Trans Signal Process 69:3878–3905. https://doi.org/10.1109/TSP.2021.3069677
Conway CM (2020) How does the brain learn environmental structure? Ten core principles for understanding the neurocognitive mechanisms of statistical learning. Neurosci Biobehav Rev 112:279–299. https://doi.org/10.1016/j.neubiorev.2020.01.032
Cook BJ, Peterson AD, Woldman W, Terry JR (2022) Neural field models: a mathematical overview and unifying framework. Math Neurosci Appl. https://doi.org/10.46298/mna.7284
Couzin ID, Heins C (2023) Emerging technologies for behavioral research in changing environments. Trends Ecol Evolut 38(4):346–354. https://doi.org/10.1016/j.tree.2022.11.008
Danieli K, Guyon A, Bethus I (2023) Episodic Memory formation: a review of complex Hippocampus input pathways. Prog Neuropsychopharmacol Biol Psychiatry 126:110757. https://doi.org/10.1016/j.pnpbp.2023.110757
Dehghani M, Trojovský P (2022) Hybrid leader based optimization: a new stochastic optimization algorithm for solving optimization applications. Sci Rep 12(1):5549. https://doi.org/10.1038/s41598-022-09514-0
d’Isa R, Comi G, Leocani L (2021) Apparatus design and behavioural testing protocol for the evaluation of spatial working memory in mice through the spontaneous alternation T-maze. Sci Rep 11(1):21177. https://doi.org/10.1038/s41598-021-00402-7
Ernst D, Louette A (2024) Introduction to reinforcement learning. Feuerriegel S, Hartmann J, Janiesch C, Zschech P, pp 111–126
Estes WK, Straughan JH (1954) Analysis of a verbal conditioning situation in terms of statistical learning theory. J Exp Psychol 47(4):225. https://doi.org/10.1037/h0060989
Gammeri R, Léonard J, Toupet M, Hautefort C, Van Nechel C, Besnard S, Lopez C (2022) Navigation strategies in patients with vestibular loss tested in a virtual reality T-maze. J Neurol 269(8):4333–4348. https://doi.org/10.1007/s00415-022-11069-z
Gao Z, Dang W, Wang X, Hong X, Hou L, Ma K, Perc M (2021) Complex networks and deep learning for EEG signal analysis. Cogn Neurodyn 15(3):369–388
Ghanbari B, Djilali S (2020) Mathematical analysis of a fractional-order predator-prey model with prey social behavior and infection developed in predator population. Chaos Solitons Fractals 138:109960. https://doi.org/10.1016/j.chaos.2020.109960
Goodwin NL, Nilsson SR, Choong JJ, Golden SA (2022) Toward the explainability, transparency, and universality of machine learning for behavioral classification in neuroscience. Curr Opin Neurobiol 73:102544. https://doi.org/10.1016/j.conb.2022.102544
Gosak M, Milojević M, Duh M, Skok K, Perc M (2022) Networks behind the morphology and structural design of living systems. Phys Life Rev 41:1–21. https://doi.org/10.1016/j.plrev.2022.03.001
Grant DA, Hake HW, Hornseth JP (1951) Acquisition and extinction of a verbal conditioned response with differing percentages of reinforcement. J Exp Psychol 42(1):1. https://doi.org/10.1037/h0054051
Hammad HA, Aydi H (2021) De la Sen M (2021) Solutions of fractional differential type equations by fixed point techniques for multivalued contractions. Complexity 1:5730853. https://doi.org/10.1155/2021/5730853
Hao J, Chen P, Chen J, Li X (2025) Effectively detecting and diagnosing distributed multivariate time series anomalies via Unsupervised Federated Hypernetwork. Inf Process Manag 62(4):104107. https://doi.org/10.1016/j.ipm.2025.104107
Hazarika B, Acharjee S, Djordjević DS (2024). Advances in functional analysis and fixed-point theory. https://doi.org/10.1007/978-981-99-9207-2
Jiang Z (2022) Banach contraction principle, q-scale function and ultimate ruin probability under a Markov-modulated classical risk model. Scand Actuar J 2022(3):234–243. https://doi.org/10.1080/03461238.2021.1958917
Knowlton BJ, Castel AD (2022) Memory and reward-based learning: a value-directed remembering perspective. Annu Rev Psychol 73(1):25–52. https://doi.org/10.1146/annurev-psych-032921-050951
Li H, Tan Y, Cheng X, Zhang Z, Huang J, Hui S, Peng W (2022) Untargeted metabolomics analysis of the hippocampus and cerebral cortex identified the neuroprotective mechanisms of Bushen Tiansui formula in an \(a\beta 25-35\)-induced rat model of Alzheimer’s disease. Front Pharmacol 13:990307. https://doi.org/10.3389/fphar.2022.990307
Luce R, Bush RR, Galanter EE (1963) Handbook of mathematical psychology, Chapters 15–21, vol III. Wiley, New York
Luo H, Xiang Y, Qu X, Liu H, Liu C, Li G, Qin X (2019) Apelin-13 suppresses neuroinflammation against cognitive deficit in a streptozotocin-induced rat model of Alzheimer’s disease through activation of BDNF-TrkB signaling pathway. Front Pharmacol 10:395. https://doi.org/10.3389/fphar.2019.00395
Luxem K, Mocellin P, Fuhrmann F, Kürsch J, Miller SR, Palop JJ, Bauer P (2022) Identifying behavioral structure from deep variational embeddings of animal motion. Commun Biol 5(1):1267. https://doi.org/10.1038/s42003-022-04080-7
Ma S, Chen Y, Yang S, Liu S, Tang L, Li B, Li Y (2023) The autonomous pipeline navigation of a cockroach bio-robot with enhanced walking stimuli. Cyborg Bionic Syst 4:0067. https://doi.org/10.34133/cbsystems.0067
Majhi S, Ghosh S, Pal PK, Pal S, Pal TK, Ghosh D, Perc M (2024) Patterns of neuronal synchrony in higher-order networks. Phys Life Rev. https://doi.org/10.1016/j.plrev.2024.12.013
Mazzucato L (2022) Neural mechanisms underlying the temporal organization of naturalistic animal behavior. Elife 11:e76577. https://doi.org/10.7554/eLife.76577
Mazzucato L (2022) Neural mechanisms underlying the temporal organization of naturalistic animal behavior. Elife 11:e76577. https://doi.org/10.7554/eLife.76577
Mosteller F (2006) Stochastic models for the learning process. In: Selected papers of Frederick Mosteller. Springer, New York, pp 295–307. https://doi.org/10.1007/978-0-387-44956-2_16
Navarro V, Dwyer DM, Honey RC (2024) Variation in the effectiveness of reinforcement and nonreinforcement in generating different conditioned behaviors. Neurobiol Learn Mem 211:107915. https://doi.org/10.1016/j.nlm.2024.107915
Ngoc Hieu BT, Ngoc Anh NT, Audira G, Juniardi S, Liman RAD, Villaflores OB, Hsiao CD (2020) Development of a modified three-day t-maze protocol for evaluating learning and memory capacity of adult zebrafish. Int J Mol Sci 21(4):1464. https://doi.org/10.3390/ijms21041464
Oberto V, Gao H, Biondi A, Sara SJ, Wiener SI (2023) Activation of prefrontal cortex and striatal regions in rats after shifting between rules in a T-maze. Learn Memory 30(7):133–138. https://doi.org/10.1101/lm.053795.123
Ofshe R, Ofshe SL (1970) Choice behavior in coalition games. Behav Sci 15(4):337–349. https://doi.org/10.1002/bs.3830150406
O’keefe J, Nadel L (1979) Précis of O’Keefe & Nadel’s The hippocampus as a cognitive map. Behav Brain Sci 2(4):487–494. https://doi.org/10.1017/S0140525X00063949
Rashid M, Saleem N, Bibi R (2024) George R (2024) Some multidimensional fixed point theorems for nonlinear contractions in C-distance spaces with applications. J Inequal Appl 1:13. https://doi.org/10.1186/s13660-024-03079-4
Sánchez-Cañizares J (2021) The free energy principle: Good science and questionable philosophy in a grand unifying theory. Entropy 23(2):238. https://doi.org/10.3390/e23020238
Sazaklioglu AU (2024) An iterative numerical method for an inverse source problem for a multidimensional nonlinear parabolic equation. Appl Numer Math 198:428–447. https://doi.org/10.1016/j.apnum.2024.02.001
Sharma S, Rakoczy S, Brown-Borg H (2010) Assessment of spatial memory in mice. Life Sci 87(17–18):521–536. https://doi.org/10.1016/j.lfs.2010.09.004
Shi J, Wang C, Wang H, Yan X (2020) Diffusive spatial movement with memory. J Dyn Differ Equ 32:979–1002. https://doi.org/10.1007/s10884-019-09757-y
Smith AJ, Becker S, Kapur S (2005) A computational model of the functional role of the ventral-striatal D2 receptor in the expression of previously acquired behaviors. Neural Comput 17(2):361–395. https://doi.org/10.1162/0899766053011546
Tedeschi LO (2023) The prevailing mathematical modeling classifications and paradigms to support the advancement of sustainable animal production. Animal 17:100813. https://doi.org/10.1016/j.animal.2023.100813
Tolman EC (1948) Cognitive maps in rats and men. Psychol Rev 55(4):189. https://doi.org/10.1037/h0061626
Torney CJ, Morales JM, Husmeier D (2021) A hierarchical machine learning framework for the analysis of large scale animal movement data. Mov Ecol 9:1–11. https://doi.org/10.1186/s40462-021-00242-0
Tsibulsky VL, Norman AB (2007) Mathematical models of behavior of individual animals. Curr Pharm Des 13(15):1571–1595. https://doi.org/10.2174/138161207780765873
Tuqan M, Porfiri M (2021) Mathematical modeling of zebrafish social behavior in response to acute caffeine administration. Front Appl Math Stat 7:751351. https://doi.org/10.3389/fams.2021.751351
Turab A, Sintunavarat W (2019) On analytic model for two-choice behavior of the paradise fish based on the fixed point method. J Fixed Point Theory Appl 21:1–13. https://doi.org/10.1007/s11784-019-0694-y
Turab A, Sintunavarat W (2020) On the solution of the traumatic avoidance learning model approached by the Banach fixed point theorem. J Fixed Point Theory Appl 22:1–12. https://doi.org/10.1007/s11784-020-00788-3
Turab A, Sintunavarat W (2023) On the solution of the generalized functional equation arising in mathematical psychology and theory of learning approached by the Banach fixed point theorem. Carpathian J Math 39(2):541–551. https://doi.org/10.37193/CJM.2023.02.14
Turab A, Bakery AA, Mohamed OKS (2022) Ali W (2022) On a unique solution of the stochastic functional equation arising in gambling theory and human learning process. J Funct Spaces 1:1064803. https://doi.org/10.1155/2022/1064803
Turab A, Bakery AA, Mohamed OKS (2022) Ali W (2022) On a unique solution of the stochastic functional equation arising in gambling theory and human learning process. J Funct Spaces 1:1064803. https://doi.org/10.1155/2022/6081250
Turab A, Rosli N, Ali W, Nieto JJ (2023) The existence and uniqueness of solutions to a functional equation arising in psychological learning theory. Demonstratio Math 56(1):20220231. https://doi.org/10.1515/dema-2022-0231
Turab A, Montoyo A, Nescolarde-Selva JA (2024) Computational and analytical analysis of integral-differential equations for modeling avoidance learning behavior. J Appl Math Comput 70(5):4423–4439. https://doi.org/10.1007/s12190-024-02130-3
Turab A, Montoyo A, Nescolarde-Selva JA (2024) Stability and numerical solutions for second-order ordinary differential equations with application in mechanical systems. J Appl Math Comput 70(5):5103–5128. https://doi.org/10.1007/s12190-024-02175-4
Turab A, Sintunavarat W, Ullah F, Zaidi SA, Montoyo A, Nescolarde-Selva JA (2024) Computational modeling of animal behavior in T-mazes: insights from machine learning. Eco Inform 81:102639. https://doi.org/10.1016/j.ecoinf.2024.102639
Ullah F, Ullah S, Srivastava G, Lin JCW (2024) IDS-INT: intrusion detection system using transformer-based transfer learning for imbalanced network traffic. Digital Commun Netw 10(1):190–204. https://doi.org/10.1016/j.dcan.2023.03.008
Wang H, Salmaniw Y (2023) Open problems in PDE models for knowledge-based animal movement via nonlocal perception and cognitive mapping. J Math Biol 86(5):71. https://doi.org/10.1007/s00285-023-01905-9
Wenk GL (1999) Assessment of spatial memory. Curr Protoc Toxicol 1:11–3. https://doi.org/10.1002/0471140856.tx1103s00
Whittington JC, Warren J, Behrens T E (2021) Relating transformers to models and neural representations of the hippocampal formation. arXiv preprint arXiv:2112.04035. https://doi.org/10.48550/arXiv.2112.04035
Whittington JC, Muller TH, Mark S, Chen G, Barry C, Burgess N, Behrens TE (2020) The Tolman-Eichenbaum machine: unifying space and relational memory through generalization in the hippocampal formation. Cell 183(5):1249–1263. https://doi.org/10.1016/j.cell.2020.10.024
Whittington JC, McCaffary D, Bakermans JJ, Behrens TE (2022) How to build a cognitive map. Nat Neurosci 25(10):1257–1272. https://doi.org/10.1038/s41593-022-01153-y
Wijeyakulasuriya DA, Eisenhauer EW, Shaby BA, Hanks EM (2020) Machine learning for modeling animal movement. PLoS ONE 15(7):e0235750. https://doi.org/10.1371/journal.pone.0235750
Wu D, Wang Y, Han M, Song L, Shang Y, Zhang X, Song H (2021) Using a CNN-LSTM for basic behaviors detection of a single dairy cow in a complex environment. Comput Electron Agric 182:106016. https://doi.org/10.1016/j.compag.2021.106016
Acknowledgements
This research is supported by the University of Alicante, Spain, the Spanish Ministry of Science and Innovation, the Generalitat Valenciana, Spain, and the European Regional Development Fund (ERDF) through the following funding sources: At the national level, this work was funded by the following projects: TRIVIAL (PID2021-122263OB-C22) and CORTEX (PID2021-123956OB-I00), granted by MCIN/AEI/10.13039/501100011033 and, as appropriate, co-financed by “ERDF A way of making Europe”, the “European Union”, or the “European Union Next Generation EU/PRTR”. At the regional level, the Generalitat Valenciana (Conselleria d’Educació, Investigació, Cultura i Esport), Spain, provided funding for NL4DISMIS (CIPROM/2021/21). The authors also sincerely thank Universitas Airlangga (UNAIR), Surabaya, Indonesia, for supporting this research through the APD program under contract No. 1019/B/UN3.AGE/HK.07.01/2024. This work was also supported by the Thailand Science Research and Innovation Fundamental Fund Fiscal year 2023, Thammasat University.
Funding
Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. Not applicable.
Author information
Authors and Affiliations
Contributions
All authors contributed equally to this work
Corresponding authors
Ethics declarations
Conflict of interest
The authors state that they do not have any conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Turab, A., Nescolarde-Selva, JA., Ullah, F. et al. Deep neural networks and stochastic methods for cognitive modeling of rat behavioral dynamics in \(\mathbb {T}\)-mazes. Cogn Neurodyn 19, 66 (2025). https://doi.org/10.1007/s11571-025-10247-9
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11571-025-10247-9
Keywords
Mathematics Subject Classification
Profiles
- Ali Turab View author profile