A Faster-Than Relation for Semi-Markov Decision Processes [PDF]
Electronic Proceedings in Theoretical Computer Science, 2020 When modeling concurrent or cyber-physical systems, non-functional requirements such as time are important to consider. In order to improve the timing aspects of a model, it is necessary to have some notion of what it means for a process to be faster ...
Mathias Ruggaard Pedersen +2 more
doaj +8 more sources
Optimal maintenance of deteriorating equipment using semi-Markov decision processes and linear programming [PDF]
International Journal of Industrial Engineering and ManagementThis paper considers a mathematical model analysing the deterioration of system equipment and available maintenance options. Under specific conditions on costs and transition probabilities of the model, the issue of ideal maintenance of the equipment by ...
Giannis Kechagias +3 more
doaj +3 more sources
SEMI-MARKOV DECISION PROCESSES WITH COUNTABLE STATE SPACE AND COMPACT ACTION SPACE [PDF]
Bulletin of Mathematical Statistics, 1978 We shall be concerned with the optimization problem of semi-Markov decision processes with countable state space and compact action space. Defined is the generalized reward function associated with the semi-Markov decision processes which include the ordinary discounted Markov decision processes of discrete time parameter and also the continuous time ...
Masami Yasuda
openaire +4 more sources
SEMI-MARKOV DECISION PROCESSES AND THEIR APPLICATIONS IN REPLACEMENT MODELS
Journal of the Operations Research Society of Japan, 1985 We consider the problem of minimizing the long-run average expected cost per unit time in a semi-Markov decision process with arbitrary state and action space. Using the idea of successive approximations, sufficient conditions for the existence of an optimal stationary policy are given.
Masami Kurano
openaire +4 more sources
Maximal Average-Reward Policies for Semi-Markov Decision Processes With Arbitrary State and Action Space [PDF]
The Annals of Mathematical Statistics, 1971 We consider the problem of maximizing the long-run average (also the long-run average expected) reward per unit time in a semi-Markov decision processes with arbitrary state and action space. Our main result states that we need only consider the set of stationary policies in that for each $\varepsilon > 0$ there is a stationary policy which is ...
Steven A. Lippman
openaire +3 more sources
Learning to maximize reward rate: a model based on semi-Markov decision processes [PDF]
Frontiers in Neuroscience, 2014 When animals have to make a number of decisions during a limited time interval, they face a fundamental problem: how much time they should spend on each decision in order to achieve the maximum possible total outcome.
Arash eKhodadadi +2 more
doaj +2 more sources
Solving the non-preemptive two queue polling model with generally distributed service and switch-over durations and Poisson arrivals as a Semi-Markov Decision Process [PDF]
, 2021 The polling system with switch-over durations is a useful model with several practical applications. It is classified as a Discrete Event Dynamic System (DEDS) for which no one agreed upon modelling approach exists. Furthermore, DEDS are quite complex.
Dylan Solms
openaire +3 more sources
Possibility of estimating the reliability of diesel engines by applying the theory of semi-Markov processes and making operational decisions by considering reliability of diagnosis on technical state of this sort of combustion engines [PDF]
Combustion Engines, 2015 The paper presents semi-Markov models of technical state transitions for diesel engines, useful for determining the reliability of engines. A possibility of application of a three-state model with a simplified matrix function, or even a two-state model, to determine reliability of the engines, has been described herein on examples of known from ...
Jerzy Girtler
openaire +2 more sources
Journal of Advanced Computational Intelligence and Intelligent Informatics, 2009 k-certainty exploration method, an efficient reinforcement learning algorithm, is not applied to environments whose state space is continuous because continuous state space must be changed to discrete state space. Our purpose is to construct discrete semi-Markov decision process (SMDP) models of such environments using growing cell structures to ...
Takeshi Tateyama +2 more
openaire +2 more sources
Discounted semi-Markov decision processes : linear programming and policy iteration
, 1974 For semi-Markov decision processes with discounted rewards we derive the well known results regarding the structure of optimal strategies (nonrandomized, stationary Markov strategies) and the standard algorithms (linear programming, policy iteration). Our analysis is completely based on a primal linear programming formulation of the problem.
Wessels, J., van Nunen, J.A.E.E.
openaire +3 more sources