China
Africa
Explore chapters and articles related to this topic
Machine Learning-Based Optimal Consensus Networked Control with Application to Van der Pol Oscillator Systems
Published in Nishu Gupta, Srinivas Kiran Gottapu, Rakesh Nayak, Anil Kumar Gupta, Mohammad Derawi, Jayden Khakurel, Human-Machine Interaction and IoT Applications for a Smarter World, 2023
The objective of optimal control for a nonlinear system is to seek optimal control strategies deriving from the Pontryagin maximum principle. However, the principle may be only a necessary condition. It is required that a sufficient condition of Hamilton-Jacobi-Bellman (HJB). Unfortunately, there do not exist analytical HJB solutions [13, 14] due to differential nonlinear. Recently, reinforcement learning (RL) techniques [14], a basic core of the machine learning theory [15], has emerged as one of the most well-known methods being employed to approximate the HJB solutions for MAS problems [16]. For example, RL can learn the solutions of differential, stochastic, and Markov games, two-player or multiplayer games, or Nash Q-learning. A branch of RL, adaptive dynamic programming (ADP) [5], is widely researched for optimal control designs [5, 17–24]. Inspired by naturally behavioral psychology, almost ADP-based algorithms use policy iteration techniques, which use two or three neural networks (NN) for control structures [17–24], called actor-disturber-critic (ADC). The critic approximates a value function while the others tune the optimal control strategy and disturbance rejection policy via the critic. The NN-weight training process is executed in two sequential steps: evaluation of actor's policy and improvement of disturber's policy. Unfortunately, with the ADC structure, the training algorithms have the disadvantages of sequential update, and hence, they require stabilized initial weights [18].
INTRODUCTION
Published in Gennadii E. Kolosov, Optimal Design of Control Systems, 2020
From the mathematical viewpoint, all problems of optimal control are related to finding a conditional extremum of a functional (the optimal-ity criterion), i.e., are problems of calculus of variations [28, 58, 64, 137]. However, a distinguishing feature of many optimal control problems is that they are “nonclassical” due restrictions imposed on the admissible values of controlling actions u(1). For instance, this often leads to discontinuous extremals inadmissible in the classical theory [64]. Therefore, problems of optimal control are usually solved by contemporary mathematical methods, the most important being the Pontryagin maximum principle [156] and the Bellman dynamic programming approach [14]. These methods develop and generalize two different approaches to variational problems in the classical theory: the Euler method and the Weierstrass variational principle used for constructing a separate extremal and the Hamilton-Jacobi method based on the consideration of the entire field of extremals, which leads to partial differential equations for controlled systems with lumped parameters or to equations with functional derivatives for controlled systems with distributed parameters.
The Use of On-line Sensors in Bioprocess Control
Published in John V. Twork, Alexander M. Yacynych, Sensors in Bioprocess Control, 2020
Michael T. Reilly, Marvin Charles
One control method which has received a great deal of attention for use in process optimization is optimal control. Optimal control owes its mathematical basis to Bellman’s dynamic programming [53], and Pontryagin’s maximum principle [54]. Early applications of optimal control were in the areas of navigation and guidance [8], but it has been applied increasingly to the control of bioprocesses [55,56,57]. One of the advantages of optimal control is that no a priori assumptions of control structure are required. The problem now becomes one of finding a function which, when applied to the process inputs, will optimize the process performance, relative to a predefined cost function [8]. The details of the method of optimal control may be found in the references, along with information on general features of controller design and process optimization [4,5,7,8,58] and specific features of bioprocess optimization [55,56,57,59,60,61,62].
Regional optimal control for a class of semilinear systems with distributed controls
Published in International Journal of Control, 2019
El Hassan Zerrik, Nihale El Boukhari
To characterise an optimal control, the main tool used throughout the literature is Pontryagin maximum principle. For instance, in Bonnans and Casas (1991), elliptic semilinear systems were considered and optimality conditions of Pontryagin-type were developed. Control problems with box constraints were studied in Barbu (1993) for a class of elliptic and parabolic semilinear equations, using the generalised maximum principle. In Li and Yong (1995), the maximum principle was extended to abstract infinite dimensional systems, with assumptions on the reachable set. Besides, Raymond and Zidani (1999) derived optimality conditions in the form of Pontryagin's minimum principle for semilinear parabolic equations, etc.
Cooperative CAVs optimal trajectory planning for collision avoidance and merging in the weaving section
Published in Transportmetrica B: Transport Dynamics, 2021
Shoucai Jing, Xiangmo Zhao, Fei Hui, Asad J. Khattak, Lan Yang
In order to obtain an analytical solution to the optimization problem and achieve the real-time trajectory planning, the Pontryagin’s Maximum Principle is used to solve the optimization problem. Here, we consider only the unconstrained problem; the constrained problem formulation was discussed in (Malikopoulos, Cassandras, and Zhang 2016, 2018). By combining the optimal equation and the state given by Equation (2), the Hamiltonian function for a vehicle can be formulated as
First-order and second-order necessary optimality conditions concerning components for discrete-time stochastic systems
Published in International Journal of Control, 2022
Teng Song, Bin Liu, Qinglong Zhou
Nevertheless, discrete-time optimal control problems are more relevant to biomedical, engineering, economic, operation research problems, optimising complex technological systems, etc. It is well accepted that Pontryagin maximum principle in continuous-time framework cannot be extended to discrete-time counterpart, except for some very special cases (Rozonoer, 1959), due to the nature of the admissible control variations. Naturally, it inspired us to formulate discrete analogue and even some uncorrected results were derived. Butkovskii (1963) and Holtzman (1966) clearly showed some errors in previous works. The fundamental reason for the errors is that the importance of convexity has been ignored. In general, the discrete-time maximum principle is invalid unless a certain convexity precondition is imposed on the control system. Pshenichnyi (1971) elaborated why discrete-time systems require a certain convexity assumption for the effectiveness of the necessary condition while continuous-time systems enjoy it automatically owing to the so-called hidden convexity. Ever since the pioneering work (Butkovskii, 1963), the researchers have been committed to the necessary optimality conditions under diverse types of weakened convexity assumptions, for example, quasi-convexity (see Gamkrelidze, 1965), local convexity (see Vinter, 1988), directional convexity (see Holtzman, 1966), a tangent cone to a set relative at a point (see Aubin & Frankowska, 2009) and so on. Observe that the above results were derived in deterministic systems. To the best of our knowledge, there are only two literature dealing with the discrete-time stochastic necessary optimality conditions. By virtue of a pair of backward stochastic difference equations, Lin and Zhang (2015) established a stochastic maximum principle for discrete-time optimal control problems over the convex control domain. Mahmudov (2019) derived the first-order and second-order necessary optimality conditions for discrete-time stochastic systems under the assumption of the set being convex. It should be highlighted that the convexity of the sets and does not always imply the convexity of the set where , (see Example 3.1).