On adaptive control processes

Abstract

One of the most challenging areas in the field of automatic control is the design of automatic control devices that 'learn' to improve their performamce based upon experience, i.e., that can adapt themselves to circumstances as they find them. The military and commercial implications of such devices are impressive, and interest in the two main areas of research in the field of control, the USA and the USSR, runs high. Unfortunately, though, both theory and construction of adaptive controllers are in their infancy, and some time may pass before they are commonplace. Nonetheless, development at this time of adequate theories of processes of this nature is essential. The purpose of our paper is to show how the functional equation technique of a new mathematical discipline, dynamic programming, can be used in the formulation and solution of a variety of optimization problems concerning the design of adaptive devices. Although, occasionally, a solution in closed form can be obtained, in general, numerical solution via the use of high-speed digital computers is contemplated. We discuss here the closely allied problems of formulating adaptive control processes in precise mathematical terms and of presenting feasible computational algoritbms for determining numerical solutioms. To illustrate the general concepts, consider a system which is governed by the inhomogeneous Van der Pol equation\ddot{x} + \mu(x^{2} - 1) \dot{x} + x = r(t), 0 \leq t \leq T, wherer(t)is a random function whose statistical properties are only partially known to a feedback control device which seeks to keep the system near the unstable equilibrium statex = 0, \dot{x} = 0. It proposes to do this by selecting the value of μ as a function of the state of the system at timet, and the timetitself. By observing the random processr(t), the controller may, with the passage of time, infer more and more concerning the statistical properties of the functionr(t)and thus may be expected to improve the quality of its control decisions. In this way the controller adapts itself to circumstances as it finds them. The process is thus an interesting example of adaptive control, and, conceivably, with some immediate applications. Lastly, some areas of this general domain requiring additional research are indicated.