by
A H Ahmad Kamil
Heung Wing Chui
Information Systems Engineering II
Imperial College
Control engineering is a well developed subject and it is crucial to our daily life. Nowadays even a microwave oven has a complex logic decision unit to cook different kind of food deliciously or a complex chemical processing plant uses highly sophisticated control systems in order to produce something efficiently. Conventional approaches to control systems are no longer sufficient to cope with a current large complex system. As digital computers became widely available, a new kind of system emerged, which is a mixture of discrete and continuous system.
If you look at the electric appliances around, you could hardly find one without control systems. Refrigerator, washing machine and electric kettle all include control systems of one kind or other. So what is a control system? It is an interconnection of components forming a system that provides a desired system response. Take refrigerator from the above examples: the control system of the refrigerator will tries to keep the temperature inside the refrigerator according to the temperature setting.
Control systems are not limited to household uses. In fact, we need reliable and accurate control systems for rocket launch and putting a satellite to the required orbit. A fast response control system is necessary for a computer hard disk driver to work properly while an intelligent control system is essential for robot control.
If we look at control systems more closely, we will find that they can be grouped into two categories, with or without feedback[1]. The system with feedback is called the closed-loop system, which is shown in figure 2.1, and the one without feedback is called the open-loop system, which is shown in figure 2.2. An open-loop system utilises an actuating device to control the process directly without feedback. On the other hand, a closed-loop system, the difference between the process under control and the reference input is used to control the process so that the difference is continuously reduced.
We usually think of a control system as a system with feedback today, since the use of feedback enables us to control the desired output and can improve accuracy. One of the classical systems using feedback control is the water-level float regulator and a very common example of open-loop control system is the electric toaster.
The first recorded use of feedback control is the float regulator mechanisms of the water clock of Kitesibios( shown in figure 3.1,). It was invented roughly in the period between 300 to 1 B.C. in Greece.
James Watt's flyball governor[2], developed in 1769 for controlling the speed of a system engine, is shown in figure 3.2. The problems of instability of control systems was raised as people tried to get accurate control of the governor. When, in 1868, J.C. Maxwell formulated a mathematical theory to tackle this problem, he demonstrated the importance and usefulness of mathematical models and methods in understanding complex phenomena. The era of development of automatic control systems through intuition and invention had ended.
Many control theories were developed over the last hundred years. Most of the theories originating from the United States were based on the frequency-domain, while the theories developed in the former Soviet Union were based on the time-domain[1]. Frequency-domain and time-domain are just different scales. For example the measurement of sound waves is in time but the frequency represents the rate of vibration of a wave. Although the frequency-domain approach dominated the control field, the advances of optimal control methods brought the time-domain to attention again. It is now clear that the understanding and use of both domains in design and analysis are essential in control engineering.
With the availability of fast, low-priced and small-sized microprocessors, many of the control industrial and commercial processes are moving toward the use of computers within control systems. It is the result of the rapid development of computers over the last two decades. Signal processing engineering has been playing an important role in dealing with the problems of digital control systems.
Digital control systems, or discrete-time systems, as they are usually called, sample a signal and take the discrete values which are then processed in digital form(0 and 1).
A digital computer in a control system is considered to be a logical
decision-making component since the programme inside the computer
control the system[3]. This is called the logic system. Systems without
decision logic which are modelled by differential or difference equations
are called continuous system, because mathematical equations can process
continuous signals. Those equations are usually derived from the physical world
In general, discrete control systems have advantages over continuous control
systems which use analog hardware. Some of the advantages are :
You may now ask whether there are any systems which are mixtures of both systems, namely logic and continuous. The answer is yes. This kind of system is called the hybrid control system.
Hybrid systems are generally understood as reactive systems that intermix discrete and continuous components[4]. The discrete part of the system makes the decision for the whole system to switch to another set of control rules if conditions are favourable. The continuous part as a result works according to the new rules. As to make the above idea more concrete, let us discuss the case of an aircraft control system.The system may have climbing, descending and level flight modes, in which different control laws are used. The logic decision-making unit chooses the mode automatically (the pilot can override this). There are a lot more examples such as computers, manufacturing production and power stations which are designed to select, control and supervise the behaviour of the continuous components. Also the potential applications for hybrid systems are vast, as most of today's control systems use computers, and even consumer electronics use software to control physical processes. Therefore it raises a lot of interest in the academic communities as well as industry.
A hybrid systems can be viewed as a large collection of systems of various classes. A hierarchical structure arises when a logical control unit governs such a system by issuing logic decisions[4]. This leads to the system framework shown below in figure 5.2.1 which clearly illustrates this architecture.
The top layer, which is a discrete event system, can use different types of description language such as finite state machines, fuzzy logic, Petri nets, etc. The bottom layer is a continuous system, and is usually the physical system[5]. The interface plays the role of facilitating communication between the two different layers by means of translating signals between them. As the techniques for control design and analysis are well developed for the continuous and discrete systems, the design of the interface is of utmost importance because it determines the way in which the combined system behaves. This control architecture which has just been described appears in a wide variety of applications and forms the heart of most hybrid system designs. For most system designs, the logic unit and continuous unit are usually designed separately and then combined together by an interface. Another approach is converting the whole system to be purely logic or continuous for design. As you may realize, the modelling of a hybrid system is not an easy job.
Hybrid systems model the interactions between logical elements and continuous systems. This involves a variety of mathematical and engineering disciplines such as differential geometry, differential and difference equations, optimal control, automata(programs) theory, discrete event systems, data structures and computation[4]. This is a completely new phenomenon for control engineers and computer scientists. Hybrid systems are not only hard to model but also hard to analyse and simulate. In fact a unified theory did not exist at all 10 years ago. Currently, it is an active area of research in control theory and computer science but a fully mature theory may yet take more years.
There are many people working on this problem, for example researchers, such as W.Kohn, A.Nerode and others, who came up with a new approach[6] to Computer-Aided Control Engineering(CACE). Their approach is briefly explained in the following. The CACE architecture is based on multiple views. These are shown in figure 5.4.1. This relationship between user, design engineer and system engineer is needed to avoid the delay of development of a system which is caused by the consistent disagreement between them than by specifying each player's responsibilities.
The level of abstraction modelling is supported by the scenario-based requirement analysis. It is because scenario-based modelling is an easily understood and readily modified way of discussing complex processes. The multiple-agent hybrid control architecture(MAHCA) consists of a variable network of control agents. MAHCA is the main architecture to support the integration of the above multiple views(see figure 5.4.2 and 5.4.3)
Each agent is formally structured. That is to say, its behaviour is characterised by a model encoded in logic clauses(IF-THEN-ELSE)[3] in hierarchy. Logic failures occur when the model and behaviour of the agent do not agree with each other. They are triggered by events affecting the processes under control. This model enables us to detect logic failures and hence we can develop structural adaptation processes to rectify them. Structural adaptation is accomplished by modifying the logic clauses according to a set of rules, or by creating or deleting agents in the network.
The Maruti Operating System was developed at the University of Maryland. The requirement of the real-time, distributed operating system in their approach is implemented by Maruti. The commands generated by the agent(logic unit) programs are sent to a system's controllers and actuators in a timely manner, and the data collected by the sensors are transmitted to the agent programs in order to generate well-accounted commands on time.
Discrete systems may have a set of rules and the behaviour of a system will be determined by those rules. As the system becomes more complex, more rules may be added and this kind systems may be modelled as a hybrid system. Let's look at the case of a volume control of an ordinary radio. Assuming the volume change is decided by the logic, from one particular volume level to next level, the current allowed to flow to the speaker may increase but there should be no volume change in between. If we include sufficient logic for enough levels, it will be just the same as a continuous system.The whole system become a hybrid, the top layer you have decision logic and the bottom layer you have a system that behaves as a continuous system. As a result the performance of the system increases. Performance is another important factor in control engineering. It has been well-developed over the last two decades as a branch of control called optimal control.
A control system performance is measured by the steady state error, gain margin
and phase margin. These are essentially criteria of optimality. In optimal
control problem, the system measure of performance, or performance index is
not fixed beforehand. A system in considered as an optimum control system when
the system parameters are adjusted so that the index is either maximised
or minimised.
Generally, the performance index is a function of error between the actual and
ideal responses. The best system is then defined as the system that minimised
this index.Control systems are optimised mainly by applying Bellman's
Optimality Principle which states
An optimal policy has the property that no matter what the previous decision (i.e. controls) have been, the remaining decisions must constitute an optimal policy with regard to the state resulting from those previous decisions.For hybrid systems, the applications of Bellman's principle leads to Hamilton Jacobi Bellman (HJB) relaxed variational form. An alternative approach to optimal control is by using dynamic programming which based on the above principle.
Dynamic Programming was developed by Richard E Bellman in the later 1950s.
It can be used to solve control problems for non-linear, time-varying systems
and it is straightforward to program.
Dynamic programming is a recursive method for obtaining optimal control as a
function of the state in multistage systems. The procedure first determines
the optimal control when there is only one stage left in the life of the system.
Then it determines the optimal control when there are two stages left and it
continues onwards.
The recursion proceeds backwards in time. An example of finding the shortest
path illustrates the basic idea.
A student wishes to determine the shortest path from Imperial College to other
colleges in the east of London travelling by car. Figure 7.1 shows the possible
paths.
If the students goes north, he will reach the college NA
after travelling for 6 km. If he goes center, he reaches CA
after travelling for 4.5 km and if he goes south he reaches SA
after travelling 5 km. Therefore he will either be in
NA,
CA or SA at stage A. Then he has to
decide which college he will travel to. The journey ends in stage C
in either NC, CC or SC.
The distances between all the colleges are shown in figure 7.1.
First and foremost, we have to determine the optimal decision when there is
only one stage left. If the student is in NB, he has 3
choices; north, center or south, leading to NC,
CC or SC respectively. The best choice is
to go to NC with a distance of 4.5 km. Similarly, if the
students is in CB, the best choice would be to go to
CC with a distance of 5.5 km. Again, if the student is in
SB, the best choice is to go to SC with
a distance of 4.0 km. Figure7.2 summarised the optimal decision with only
one stage left.
The next step is to decide the optimal decision when there are 2 stages left.
If the student is in NA and goes north, he will reach
NB after travelling for 6.0 km. In addition, he needs to
travel a further 4.5 km to reach the colleges in the east of London
making a total journey of 10.5 km. On the other hand, if he goes to
the center, he has to travel 6.5 km to reach CB and a
further 5.5 km to reach his destinations making a total journey
of 12.0 km. For choosing south from NA, he has to travel
for a total distance of 11.5 km to reach the colleges in the east of
London. The consequences of each of these 3 choices are shown in figure 7.3.
Similarly, we also determine the optimal path from CA and
SA as well as the shortest distance from these places to the
east of London. The optimal solution when there are only two stages left is
shown in figure 7.4.
Finally, we determine the optimal solution when there are three stages
remaining. The student is in Imperial College. If he goes north, he travels
6.0 km to reach NA and then a further 10.5 km to
reach the east of London. Thus the total distance will be 16.5 km. Similarly,
if he goes to the centre, he will travel a total distance of 14.5 km and
a distance of 14.0 km if he goes south. Therefore the best choice is to travel
south from Imperial College and the total distance of his journey will be
14 km (may as well avoiding traffic congestions). The final result is shown
in figure 7.5.
A few important properties[7] are observed.
In addition, dynamic programming is also the name of a mathematical theory of
multistage decision processes. These solve problems which cannot usually be
solved by methods of classical calculus. In general, the dynamic programming
procedure[8] is
Discrete-time, deterministic, multistage decision processes are characterised by
The problem is to find the series of decisions that maximises the total return
from the process over some specified operating time. Processes with a finite
number of stages usually give rise to functional recurrence equations which,
in principle, can always be solved numerically.
Infinite numbers of stages in stationary processes give rise to functional
equations that cannot generally be solved by direct methods. An alternative
method would be successive approximation.
7.5.2 Continuous Decision Process
The dynamic programming procedure for continuous-time decision processes is to
define a corresponding discrete-time processes which approximates
more and more closely to the original continuous-time processes as the discrete
time interval goes to zero. Then the discrete-time processes may be solved as
briefly explained earlier. When it can be assumed that the continuous-time
solution is differentiable, the discrete-time functional equation goes over to
a partial differential expressions as the discrete time interval goes to zero.
If the functions defining the problem are also differentiable and the required
maximum, or minimum, can be found by calculus, then the partial differential
expression reduces to a pair of simultaneous partial differential equations.
Hence, a solution might be obtainable. If however the analytic equations are
insoluble, an approximate solution may be found by solving the discrete-
time process using the numerical procedures.
We have already seen that hybrid systems are models for networks of digital(logic) and continuous devices, in which digital control programs sense and supervise continuous and discrete systems governed by differential or difference equations. Modern industrial society is filled with hybrid systems used for varied purposes, such as aircraft control or industrial process control. In addition, hybrid system theory provides the backbone for the formulation and implementation of learning control policies. The importance of hybrid system will grow, as the demand for more, larger and more complex automatic control system is ever increasing. A new area of research is open to various disciplines such as control engineering and computer science. Although we have developed different methods to deal with specific problems, further research is needed to bring about a fully reliable and well-developed theory that can solve all problems. The issue of optimal control cannot be ignored in design of control systems with today's requirements. Examples include the necessity to minimise the weight of satellites and to control them very accurately, or the fuel consumption of a car, etc. This emphasises the importance of optimal control.