Parametric ltl on markov chains souymodip chakraborty, joostpieter katoen to cite this version. Ltl model checking of timeinhomogeneous markov chains. Markov chains, that is, discrete time markov chains where each transition is associated. Foundations of software science and computation structure fossacs 2008, roberto m. Accordingy, for example, a discretetime markov chain m can be mode. Datadriven probabilistic modeling and verification of. Approximate verification of the symbolic dynamics of markov. Ltl model checking of timeinhomogeneous markov chains taolue chen1, tingting han2,3, joostpieter katoen2,3, and alexandru mereacre2 1 design and analysis of communication systems, university of twente, the netherlands 2 software modelling and verification, rwth aachen university, germany 3 formal methods and tools, university of twente, the netherlands abstract. Box 217, 7500 ae enschede, the netherlands abstract. Pdf modelchecking omegaregular properties of interval. On ltl model checking for lowdimensional discrete linear.
Sketch the initial probability density function and the probability density functions in parts a, b, and c on a. An idtmc is a classical discretetime markov chain with ranged probabilities, where a transition between two states is associated with an interval in which the actual transition probability must lie. Model checking for discretetime markov chains dtmcs. Approximate abstractions of markov chains with interval. Smoothed model checking for uncertain continuoustime markov. Instead in idtmcs, each transition is associated with an interval in which the actual transition probability.
Idtmcs are discretetime finite markov chains for which the. In this paper, we fill the longstanding gap in the field of the verification of continuoustime systems by proposing a lineartime temporal logic, named continuous linear logic cll and the corresponding model checking algorithm for continuoustime markov chains. Thereby, we provide a formal framework to answer a very wide range of questions about imprecise markov chains, in a systematic and computationally e cient way, whilst at the same time improving and simplifying model checking for a fairly broad class of markov decision processes. Markov chain models a markov chain model is defined by a set of states some states emit symbols other states e. Regular properties of interval markov chains krishnendu chatterjee 1,koushiksen, and thomas a. We study the problem of model checking interval valued discretetime markov chains idtmc. Ltl model checking for dtmcs ltl model checking for dtmc dand ltl formula. A finitestate markov chain m can be regarded as a linear transform operating on the set of probability distributions over its node set. Since classic model checking approaches cannot be used to prove the correctness of these systems, probabilistic models and model checking techniques were developed. Modelchecking omegaregular properties of interval markov.
Verifying pctl model checking chair for logic and verification. Citation krishnendu chatterjee, tom henzinger, koushik sen. Probabilistic verification and synthesis prism model checker. Our approach leverages the semantics of interval valued labelled markov chains imcs and markov decision processes mdps. Ctl, a logic widely used in the verification community, and rewrite these. They are useful for modeling systems in which some transition probabilities depend on an unknown environment, are only approximately known. Interval markov chains imcs generalize ordinary markov chains by having interval valued transition probabilities. Importance sampling of interval markov chains core. In the probabilistic ltl model checking problem that we consider here, we are given an ltl formula f and a discretetime. Counterexamples in probabilistic ltl model checking for. Apr 01, 2016 ltl model checking of interval markov chains proc. Ltl pnu77 nonprobabilistic lineartime temporal logic. Markov chains provide a good model to analyze such beha vior. Model checking quantum markov chains sciencedirect.
Model checking markov chains with actions and state labels. Approximate verification of the symbolic dynamics of markov chains. On statistical model checking of stochastic systems. Pctl 14 is a probabilistic extension of ctl in which. Only nonzero probabilities and weights in boldareshown. In another setting, the model checking problem for interval markov chains has been addressed in 10,3,8. Approximate verification of the symbolic dynamics of.
We study the problem of model checking interval valued. In this semantics, the transitions are fixed onceandforall1. We show that for markov chains, the model checking problem for qltl reduces to the one for the ltl case, and that it has no additional cost. Idtmcs are discretetime fi nite markov chains for which the exact transition probabilities are not known. In 10,3, the authors propose algorithms and complexity bounds for checking respectively. There is a simple test to check whether an irreducible markov chain is aperiodic. They are useful for modeling systems in which some transition probabilities depend on an unknown environment, are only approximately known, or are parameters that can be controlled.
The uncertain semantics treats interval valued markov chains as families of markov chains with the same state. They then consider ltl model checking over the resulting formalism, but observe that the set of infinite words arising as symbolic trajectories of a given markov chain can fail to be. Discrete time markov chain reachability pctl complexity linearprogramming we investigate the complexity of model checking. Idtmcs are discretetime finite markov chains for which the exact transition probabilities are not known.
Abstract we study the problem of model checking interval valued discretetime markov chains idtmc. Two semantics have been introduced for interval valued markov chains. Regular properties of interval markov chains krishnendu chatterjee 1, koushik sen, and thomas a. Speci cally, we consider all the states of the concrete model belonging to each partition. We demonstrate the utility of the techniques developed in this paper by examples of model checking the correctness of a simple quantum loop program as well as the basic bb84 protocol. Many of the examples are classic and ought to occur in any sensible course on markov chains.
Pdf ltl model checking of timeinhomogeneous markov chains. Model checking continuoustime markov chains joostpieter katoen software modeling and veri. Instead in idtmcs, each transition is associated with an interval in which the actual transition probability must lie. The problems of computing reachability probabilities and expected total reward for imcs and imdps were rst investigated in 8,35. On the complexity of model checking intervalvalued. Index termscontinuoustime markov chain, lumping, model checking, temporal logic. On the complexity of model checking intervalvalued discrete time. Symbolic model checking for probabilistic processes. Markov chains these notes contain material prepared by colleagues who have also presented this course at cambridge, especially james norris. Modelchecking omegaregular properties of interval markov chains krishnendu chatterjee, tom henzinger, koushik sen. Marta kwiatkowska gethin norman dave parker university of. These tools work together to verify iltl properties of dtmcs.
Probabilistic motion planning under temporal tasks and soft. Ltl model checking of interval markov chains university of oxford. A markov chain model a c t g begin state transition. Nov 01, 20 based on this, we propose a quantum extension of pctl and develop an algorithm for model checking quantum markov chains. Their analysis most often concerns the computation of steadystate and transientstate probabilities. On the complexity of model checking interval valued discrete time markov. Pdf ltl model checking of timeinhomogeneous markov. If there is a state i for which the 1 step transition probability pi,i 0, then the chain is aperiodic. Modelchecking markov chains in the presence of uncertainties. From the side of view of compositional specification, interval markov chains 18. Instead in idtmcs, each transition is associated with an interval in. Checking ltl properties of recursive markov chains,mihalis yannakakis,department of computer science,columbia university,kousha etessami,lfcs, school of informatics,university of edinburgh,abstract,we present algorithms for the qualitative and,quantitative model checking of linear temporal logic ltl,properties for recursive markov chains rmcs. Ltl model checking of interval markov chains michael benedikt, rastislav lenhardt, and james worrell department of computer science, university of oxford, united kingdom abstract. Following from the idea of structured markov chains, we considered the markovian sir susceptibleinfectiousrecovered epidemic model and looked to extend the.
In particular, queueing systems and network protocols are often analyzed using mark ov chains 14, 4. Markov chains ctmc 17, semi markov chains smc 7,14, which are a generalization of ctmc, meet the above requirements. Parameter synthesis for parametric interval markov chains. Modeling systems as discretetime markov chains is a popular technique to an.
Monte carlo based statistical model checking of cyber. Ltl model checking on interval markov chains michael benedikt, rastislav lenhardt, and james worrell department of computer science, university of oxford, united kingdom introduction interval markov chains imcs generalise ordinary markov chains by having variables that represent undetermined transition probabilities. Thus, a set of initial distributions will induce a set of trajectories. Model checking quantitative linear time logic sciencedirect. Citeseerx document details isaac councill, lee giles, pradeep teregowda. Uncertain markov chains umcs and interval markov decision. Continuoustime markov chains ctmcs have been widely used to determine system performance and dependability characteristics. The corresponding model checking problem has so far been studied mainly for. Datadriven probabilistic modeling and verification of human. This paper introduces a branching temporal logic for expressing realtime probabilistic properties on ctmcs and presents approximate model checking algorithms for this. It is an interesting and useful task to analyze the. It relies on creating a set of representatives of each partition that are not necessarily states of the concrete model. Pdf continuoustime markov chains ctmcs have been widely used to determine system performance and.
Mature probabilistic model checking tools such as prism 22 support veri. Dtmc were used for model checking of dtmcs for abstraction purposes to overcome the state space explosion problem 6,15. The iterative applications of m to an initial probability distribution. With their logic, we can specify such characteristics as the expected value. Markovestimator a tool to estimate a markov transition matrix, and iltlchecker, a tool to model check iltl properties of dtmcs. Ctl formulas over finitestate automata can be established. Probabilistic bisimulations for pctl model checking of interval mdps.
Ltl formula and maximize the accumulated reward in 12. Markov chain is irreducible, then all states have the same period. The ehrenfest chains, named for paul ehrenfest, are simple, discrete models for the exchange of gas molecules between two containers. Modelchecking omegaregular properties of interval markov chains. Interval markov chains imcs generalize ordinary markov chains by having intervalvalued transition probabilities. Following from the idea of structured markov chains, we considered the markovian sir susceptibleinfectiousrecovered epidemic model. Pdf modelchecking algorithms for continuoustime markov. Ltl model checking of interval markov chains springerlink. An ltl model checker for markov chains has been developed by vardi which checks whether the probabilitymeasure of the paths that do not satisfy a speci. In classical ltl model checking, we can exploit the fact that a periodic counterexample always exists see e. Branchingtime modelchecking of probabilistic pushdown. Smoothed model checking for uncertain continuoustime.
This motivated the introduction of markov chains where intervals are attached to the transitions rather than concrete transition probabilities 24,30,9,14,7,5. Probabilistic model checking with prism prism model checker. While we believe our algorithm will work for any model that satis. Model checking smc is a simulationbased approach that holds the. However, they can be formulated as simple ball and urn models. Ltl model checking of timeinhomogeneous markov chains taolue chen1, tingting han2,3, joostpieter katoen 2,3, and alexandru mereacre 1 design and analysis of communication systems, university of twente, the netherlands 2 software modelling and veri. Then, several of their pctl and ltl model checking algorithms were introduced in 2,6,8 and 22,32,34, respectively. A robust control policy for mdps with uncertain transition probabilities is proposed in 8. On the complexity of model checking intervalvalued discrete. We investigate the problem of model checking interval valued discretetime markov chains idtmc. Verification of continuoustime markov chains deepai. Discrete abstractions of nonlinear systems based on error.
1614 826 548 382 1355 545 321 1173 1047 1257 1042 1256 1219 1260 1076 189 1557 858 234 532 1198 1360 1689 247 610 1761 171 1185 728 153 19 986 518 267 733 893 394 618