Stability estimation of some Markov controlled processes
Open Mathematics, 2022 We consider a discrete-time Markov controlled process endowed with the expected total discounted reward. We assume that the distribution of the underlying random vectors is unknown and that it is approximated by an appropriate known distribution.
Gordienko Evgueni, Chavez Juan Ruiz de
doaj +1 more source
Open Mathematics, 2023 This article presents an approximation of discrete Markov decision processes with small noise on Borel spaces with an infinite horizon and an expected total discounted cost by the corresponding deterministic Markov process.
Portillo-Ramírez Gustavo +3 more
doaj +1 more source
Quickest Online Selection of an Increasing Subsequence of Specified Size [PDF]
, 2015 Given a sequence of independent random variables with a common continuous distribution, we consider the online decision problem where one seeks to minimize the expected value of the time that is needed to complete the selection of a monotone increasing ...
Arlotto, Alessandro +2 more
core +3 more sources
Observations of interplanetary scintillation during the 1998 Whole Sun Month: a comparison between EISCAT, ORT and Nagoya data [PDF]
, 2000 International audienceObservations of interplanetary scintillation (IPS) allow accurate solar wind velocity measurements to be made at all heliographic latitudes and at a range of distances from the Sun.
Ananthakrishnan, S. +8 more
core +2 more sources
Renewal characterization of Markov modulated Poisson processes
International Journal of Stochastic Analysis, Volume 2, Issue 1, Page 53-70, 1989., 1989 A Markov Modulated Poisson Process (MMPP) M(t) defined on a Markov chain J(t) is a pure jump process where jumps of M(t) occur according to a Poisson process with intensity λi whenever the Markov chain J(t) is in state i. M(t) is called strongly renewal (SR) if M(t) is a renewal process for an arbitrary initial probability vector of J(t) with full ...
Marcel F. Neuts +2 more
wiley +1 more source
Optimal Online Selection of a Monotone Subsequence: a Central Limit Theorem [PDF]
, 2014 Consider a sequence of $n$ independent random variables with a common continuous distribution $F$, and consider the task of choosing an increasing subsequence where the observations are revealed sequentially and where an observation must be accepted or ...
Arlotto, Alessandro +2 more
core +5 more sources
Controllability Metrics on Networks with Linear Decision Process-type Interactions and Multiplicative Noise [PDF]
, 2015 This paper aims at the study of controllability properties and induced controllability metrics on complex networks governed by a class of (discrete time) linear decision processes with mul-tiplicative noise.
Diallo, Tidiane, Goreac, Dan
core +4 more sources
Continuous-time Markov decision processes under the risk-sensitive average cost criterion
, 2015 This paper studies continuous-time Markov decision processes under the risk-sensitive average cost criterion. The state space is a finite set, the action space is a Borel space, the cost and transition rates are bounded, and the risk-sensitivity ...
Chen, Xian, Wei, Qingda
core +1 more source
On time-inhomogeneous controlled diffusion processes in domains
, 2007 Time-inhomogeneous controlled diffusion processes in both cylindrical and noncylindrical domains are considered. Bellman's principle and its applications to proving the continuity of value functions are investigated.Comment: Published at http://dx.doi ...
Dong, Hongjie, Krylov, N. V.
core +1 more source
Average optimality for continuous-time Markov decision processes under weak continuity conditions
, 2014 This article considers the average optimality for a continuous-time Markov decision process with Borel state and action spaces and an arbitrarily unbounded nonnegative cost rate.
Zhang, Yi
core +1 more source