# State Transition Dynamic Models

State Models and Kalman Filters

## Ouch!

The Kalman filter model assumes the true state at time
*k* is evolved from the state at *(k − 1)* according to
`x`

_{k} = F_{k}x_{k-1} + B_{k}u_{k} +
w_{k} ... ^{[1]}

```
``` Well sure. *You were just about to assume that weren't you.*

Have you ever wondered why any sane person would
"out of the blue" jump into assumptions of that sort? There are
reasons behind the assumptions, however, even if nobody bothers to
discuss *why*. Until now.

We have described
qualitatively the requirements and complications of modeling a discrete
dynamic system using a state model. Now we need to fill in some details
about what that model is, and why things *get assumed* as they do,
as notations become more mathematical and explicit.

## State dynamics produce a sequence

Because the system is dynamic, the internal state variables representing
its behavior will change over time. In discrete variable models, the
state variables are considered to assume values at a sequence of time intervals,
indexed by integer variable `k`

. The actual changes might be
happening continuously in the small intervals along the way, but especially
in the case of computer automation, the observations are only available
at discrete times. Since we will be considering linear systems, we won't
need a superscript notation to indicate *variable raised to
a power*, so we will borrow that notation to indicate *the advance
of time.*

$$\phantom{\rule{1em}{0ex}}{x}^{k}\phantom{\rule{1em}{0ex}}\to \phantom{\rule{1em}{0ex}}\mathrm{past,}\phantom{\rule{1em}{0ex}}{x}^{0},\phantom{\rule{1em}{0ex}}{x}^{1},\phantom{\rule{1em}{0ex}}{x}^{2},\phantom{\rule{1em}{0ex}}{x}^{3},\phantom{\rule{1em}{0ex}}\mathrm{future,}\phantom{\rule{0.5em}{0ex}}...$$
*Subscript notations* are reserved for indexing the terms from a
numerical state vector with several terms.

## State depends on state

We'll use the notation `x`

to represent the vector
containing the numerical values of the current model state.

The most useful state transition models represent the dynamic progress
of the system over very small time intervals, so that changes are "seen"
at a relatively high resolution before any big changes can occur. Since we are
looking for a linear model, we can express the linear relationship using matrix
notations, where the matrix term contains all of the constant multiplier parameters.

$${x}^{k+1}\phantom{\rule{1em}{0ex}}\leftarrow \phantom{\rule{1em}{0ex}}A\xb7{x}^{k}\phantom{\rule{1em}{0ex}}...$$
Since state `x`

does not change very much in one time
increment, this means that the *state transition matrix* `A`

will not be very much different from an *identity matrix*.
Terms along the main diagonal will be reasonably close to 1.0 in value,
and off-diagonal terms will be small.

## State depends on inputs

Inputs also affect the next state. Over one time increment, the
state changes driven by the inputs should be quite small. The terms of
* input coupling matrix* `B`

indicate the degree to
which each input variable in the input list `u`

, represented
as a vector, will effect changes in each of the state variables
during one time interval. We are again looking for a linear relationship,
so it can be represented in matrix form something like the following.

$${x}^{k+1}\phantom{\rule{1em}{0ex}}\leftarrow \phantom{\rule{1em}{0ex}}B\xb7{u}^{k}\phantom{\rule{1em}{0ex}}...$$
In non-pathological models, we expect *both* state dependent and input
dependent changes in state. Because of the additive property of linearity,
we know that the combination of the two effects is their sum.

$${x}^{k+1}\phantom{\rule{1em}{0ex}}\leftarrow \phantom{\rule{1em}{0ex}}A\xb7{x}^{k}\phantom{\rule{1em}{0ex}}+\phantom{\rule{1em}{0ex}}B\xb7{u}^{k}\phantom{\rule{1em}{0ex}}$$

## Outputs depends on state

The outputs that happen *right now* depend on the state as it
exists *right now*. That is a little different from the state transition
equations, which show what state values will be *at the next step*. The
linear relationship between the state variables and the observable outputs
can be represented using an *output observation matrix* `C`

.

$${y}^{k}\phantom{\rule{1em}{0ex}}=\phantom{\rule{1em}{0ex}}C\xb7{x}^{k}\phantom{\rule{1em}{0ex}}$$
We generally preclude any effects of current inputs `u`

that
couple directly and immediately to the outputs `y`

. That sort
of thing is allowable, but direct and instant connections from the system
inputs to the system outputs, before the system dynamics can see them,
can produce some anomalous behavior. Seek to avoid
this complication.^{[2]}

Some useful special cases for observation matrix `C`

:

If the output is a single variable, the `C`

matrix
has a single row.

In the special case that all of the state variables are easy to
observe directly, the `C`

matrix turns into an identity matrix
the same size as the state transition matrix.

If a subset of the state variables can be observed directly,
the C matrix consists of selected rows from an identity matrix.

Outputs can consist of a combination of multiple internal state
variables, so an output does not necessarily correspond to one single state
variable.

As an example, for the following observation matrix

$${y}^{k}\phantom{\rule{1em}{0ex}}\leftarrow \phantom{\rule{1em}{0ex}}\left[\begin{array}{c}1.0\phantom{\rule{1em}{0ex}}0.0\phantom{\rule{1em}{0ex}}0.0\phantom{\rule{1em}{0ex}}\\ \phantom{\rule{1em}{0ex}}0.0\phantom{\rule{1em}{0ex}}0.8\phantom{\rule{1em}{0ex}}-0.8\phantom{\rule{1em}{0ex}}\end{array}\right]\phantom{\rule{1em}{0ex}}\xb7{x}^{k}$$
there are two output variables. The first output is a direct observation
of the current value of the first state variable. The second output
is proportional to the difference between the last two state variables.
Neither the second nor the third state variable can be observed directly
in isolation.

## Putting it together

Summarizing, the discrete time state model used by Kalman filters
^{[3]}
consists of two parts, a state transition equation to predict what the
*next* state will be, and an output observation equation to
expose what the *current* observable model outputs are.

$$\begin{array}{c}{x}^{k+1}=A\xb7{x}^{k}+B\xb7{u}^{k}\\ {y}^{k}=C\xb7{x}^{k}\end{array}$$
The problem of obtaining a suitable system model Kalman
filtering is to determine the parameter values in the `A`

,
`B`

, and `C`

matrices such that the state trajectories
indicated by the state transition equation result in output values
`y`

that are consistent with the behavior of the actual system.

## Shaking it apart

We will reserve some notations for random influences. Detailing what these
are all about must wait for later. The following kinds of random disturbances
are included:

Random disturbances that are introduced into the system along
with each new input, and that disrupt the next state variables directly,
are indicated by random vector `v`

.

Random disturbances that disrupt the observations of the
output variables are indicated by random vector `r`

.

A notation `p`

is reserved for the random variations
that accumulate and propagate in the state variables as a result of the
random noise.

System equations augmented with the additional noise input
vectors will be:

$$\begin{array}{c}{x}^{k+1}=A\xb7{x}^{k}+B\xb7{u}^{k}+{v}^{k}\\ {y}^{k}=C\xb7{x}^{k}+{r}^{k}\end{array}$$
We have no way to predict what the random variables will do, so the
best we can hope for is quantifying the statistics of the random effects.
Randomness will interfere with the process of fitting a good state
transition model.

## What's next

Sometimes it isn't possible to operate the system in a pristine and
isolated manner suggested by the dynamic model equations. Next time, we will
consider how stabilization affects the model.

Footnotes:

[1] The Wikipedia article on
Kalman Filter.

[2] An example of this sort of bad idea is
the impulsive response to setpoint changes from the "D term"
in a simplistic PID controller implementation. The "tracking error"
is the difference between the current input level and the current
output level, so any change in the reference input level is perceived
as an instantaneous step-change, causing an artificial and severe
jolt in the produced feedback signal.

[3] This particular dynamic model form was the
"regulator problem" and already well established in *"the literature"* when
Kalman published his famous papers; Kalman's papers reference it. Kalman's
contribution was related to establishing the connection to *optimal Wiener
filtering*, to determine the theoretical conditions for optimality.

```
```