China 
Africa 
Explore chapters and articles related to this topic
Monte Carlo Molecular Simulations
Published in Mihai V. Putz, New Frontiers in Nanochemistry, 2020
Bogdan Bumbăcilă, Mihai V. Putz
A Markov chain (discrete-time Markov chain or DTMC), named after Andrey Markov (1856–1922, a Russian mathematician knew for his work in the stochastic statistics field), is a random process that undergoes transitions from one state to another on a state space (a set of values which a process can adopt). It must possess a property that is usually characterized as “memorylessness”: the probability distribution of the next state depends only on the current state and not on the sequence of events that preceded it, or the properties of random variables related to the future depend only on relevant information about the current time, not on information from further in the past. This specific kind of “memorylessness” is called the Markov property. In a Markov process, the evolution is determined only from what happened at the preceding time step. So at each time step, the system loses the memory of its previous evolution (Everitt, 200; Serfozo, 2009; Durett, 2012).
Finally, after a long wait ... Queueing Theory
Published in Alan R. Jones, Risk, Opportunity, Uncertainty and Other Random Models, 2018
This state of memorylessness in queues leads us to the Exponential Distribution, the only continuous distribution with this characteristic (Liu, 2009) and the Geometric Distribution is the only Discrete Probability Distribution with this property.
Deep Q learning-based traffic signal control algorithms: Model development and evaluation with field data
Published in Journal of Intelligent Transportation Systems, 2023
Hao Wang, Yun Yuan, Xianfeng Terry Yang, Tian Zhao, Yang Liu
The underlying assumptions for applying DRL are illustrated explicitly, and the mathematical notations and formulation are presented. DRL well fits the traffic signal timing problem because the problem accepts assumptions: (a) the Markovian (i.e. memoryless) assumption that only the total future measure of effectiveness (MOE) of the traffic light timing is concerned; (b) the quick-responsive assumption that the output of the signal timing change is observable in a limited time. In addition, the environment assumption is used that the transition from the state and action at time t to that at time t + 1 is modeled with a trained deep model. The Markov property means the decision process is memoryless, which is a common implicit assumption in real-time control problems. Memorylessness means that a given probability distribution is independent of its history. If a probability distribution has the memoryless property the likelihood of something happening in the future has no relation to whether or not it has happened in the past. The history of the traffic state is irrelevant to the future. Considering the Markov property, the decision marking in the real-time control system only concerns the future signal plan, does not involve the previous signal plans and treat the corresponding traffic delay as sunk cost.