Markov Chains and
Dynamic Models
The refinery and economic supplydemand models of Section 1.2 were static
in the sense that we solved them once and that was it. There was one set of
production levels required, not a sequence of levels that would be needed
to describe an economy changing over time. A model that tries to predict
the behavior of a system over a period of time is called a dynamic model.
In this section we examine two dynamic linear models.
The first dynamic model we consider involves probability. This model,
called a Markov chain, will arise over and over in this book, so it is important
to understand the model well. The concepts of probability we need for this
model are simple and intuitive.
A Markov chain is a probabilistic model that describes the random
movement over time of some activity. At each period of time, the activity
is in one of several possible states. States might be amounts won in gam
yield
heating oil = 520,
diesel oil = 890,
gasoline = 1040
This is getting quite close to our production goals. The overproduction
of 20 to 40 gallons in each product might be a reasonable safety margin
that is actually desirable.
To get closer, we should decrease x_{2} and x_{3} a little.
yield
heating oil = 504,
diesel oil = 852,
gasoline = 1006
This is an excellent fit. We have been a bit lucky. To do better, we
would probably have to use fractional values. (In the Exercises the
reader may need more tries to get this close.)
We next consider a slightly more complicated supplydemand model.
This model has the balancing advantage that trialanderror calculations to
estimate a solution are easier. The reader is warned that it takes a little
while
to get a feel for all the numbers in this model.
Example 2. A Model of General Economic
SupplyDemand
We present a linear model due to W. Leontief, a Nobel Prizewinning
economist. The model seeks to balance supply and demand throughout
a whole economy. For each industry, there will be one supplydemand
equation. In practical applications, Leontief economic models can have
hundreds or thousands of specific industries. We consider an example
with four industries.
The lefthand side of each equation is the supply, the
amount
produced by the ith industry. Call this quantity x_{i}; it is measured
in
dollars. On the righthand side, we have the demand for the product
of the ith industry. There are two parts to the demand. The first part
is demand for the output by other industries (to create other products
requires some of this product as input). The second part is consumer
demand for the product.
For a concrete instance, let us consider an economy of
four general
industries: energy, construction, transportation, and steel. Suppose
that the supplydemand equations are
Industrial Demands  
Supply  Transport.  Energy  Construct.  Steel 
Consumer Demand 

Energy:  x_{1}=  
Construct.:  x_{2}=  
Transport. :  x_{3}=  
Steel:  x_{4}= 
(6)
The first equation, for energy, has the supply of energy x_{1}
on the
left. The terms on the right of this equation are the various demands
that this supply must meet. The first term on the right, 4x_{1} is the
input of energy required to produce our x_{1}dollars of energy (A units
of energy input for one unit of energy output). Also, the second term
of .2x_{2 }is the input of energy needed to make x_{2 } dollars
of construction.
Similarly, terms .2x_{3} and .2x_{4} are energy inputs required
for transportation
and steel production. The final term of 100 is the fixed consumer
demand.
Each column gives the set of input demands of an industry. For
example, the third column tells us that to produce the x_{3} dollars of
transportation requires as input .2x_{3} dollars of energy, .2x_{3}
dollars of
construction, and .1x_{3} dollars of steel. In the previous refinery
model,
the demand for each product was a single constant quantity. In the
Leontief model, there are many unknown demands that each industry's
output must satisfy. There is an ultimate consumer demand for each
output, but to meet this demand industries generate input demands on
each other. Thus the demands are highly interrelated: Demand for
energy depends on the production levels of other industries. and these
production levels depend in turn on the demand for their outputs by
other industries, and so on.
When the levels of industrial output satisfy these supplydemand
equations, economists say that the economy is in equilibrium.
As in the refinery model, let us try to solve this system of equations
by trialanderror. As a first guess, let us set the production levels
at twice the consumer demand (the doubling tries to account for the
interindustry demands). So , and
x_{4} = 0; these are our supplies. Given these production levels, we can
compute the demands from (6).
Supply  Demand  
Energy:  200  
Construct.:  100  
Transport.:  200  
Steel:  0 
(7)
For our next approximations, let us try supply levels
halfway
between the supply and demand values in (7). That is,
, and similarly,
, and
Supply  Demand  
Energy:  220  
Construct.:  140  
Steel:  165  
Steel:  15 
(8)
The second approximation is only moderately better. The
interaction
effects between different industries are hard to predict. Adjusting production
levels was much easier in the refinery problem, where the
demand for each product was constant.
Let us stop trying to be clever and just use the
simpleminded
approach of setting production levels (i.e., supply levels) equal to the
previous demand levels. So from (8), we try
Supply  Demand  
Energy:  252  
Construct. :  192  
Transport. :  139  
Steel:  30 
(9)
The demand values here have been rounded to whole numbers.
The
supplies and demands are getting a little closer together in (9).Repeating
the process of setting the new supply levels equal to the previous
demand levels (i.e., the demands on the right side in (9)) yields
Supply  Demand  
Energy:  273  
Construct. :  214  
Transport. :  150  
Steel:  33 
(10)
Repeating this process again, we have
Supply  Demand  
Energy:  289  
Construct. :  229  
Transport. :  155  
Steel:  36 
(11)
Observe that in successive rounds (9), (10), (11),
supplies are
rising. This is because as we produce more, we need more input which
requires us to produce still more, and so on. It may be that this iteration
will go on forever, and no equilibrium exists. On the other hand, the
gap between supplies and demands is decreasing.
Leontief proposed a constraint on the input costs that we
shall
show (in Section 3.4) guarantees that an equilibrium exists. The constraint
is
Input Constraint. Every industry is profitable:
Every industry must
require less than $1 of inputs to produce $1 of output.
In mathematical terms, this means that the sum of the
coefficients
in each column must be less than I. Our data in (6) satisfy this constraint,
so an equilibrium does exist for this fourindustry economy.
Moreover, the iteration process of repeatedly setting production levels
equal to the previous demands will converge to this equilibrium. The
reader should check that the following numbers are equilibrium values
(rounded to the nearest integer).
Equilibrium: energy = 325, construction = 265,
transportation = 168, steel = 43
Note that any system of linear equations can be rewritten
in the form
of supplydemand equations with x_{i} appearing alone on the left side of the
ith equation, as in the Leontief supplydemand model (6). It is standard
practice to solve large systems of linear equations by some sort of iterative
method. The nature of the supplydemand equations suggested the iterative
scheme we used here, letting the demands from one round be the production
levels of the next round .