State Transition Dynamic Models

State Models and Kalman Filters


The Kalman filter model assumes the true state at time k is evolved from the state at (k − 1) according to xk = Fkxk-1 + Bkuk + wk ... [1]

Well sure. You were just about to assume that weren't you.

Have you ever wondered why any sane person would "out of the blue" jump into assumptions of that sort? There are reasons behind the assumptions, however, even if nobody bothers to discuss why. Until now.

We have described qualitatively the requirements and complications of modeling a discrete dynamic system using a state model. Now we need to fill in some details about what that model is, and why things get assumed as they do, as notations become more mathematical and explicit.


State dynamics produce a sequence

Because the system is dynamic, the internal state variables representing its behavior will change over time. In discrete variable models, the state variables are considered to assume values at a sequence of time intervals, indexed by integer variable k. The actual changes might be happening continuously in the small intervals along the way, but especially in the case of computer automation, the observations are only available at discrete times. Since we will be considering linear systems, we won't need a superscript notation to indicate variable raised to a power, so we will borrow that notation to indicate the advance of time.

x k past, x 0 , x 1 , x 2 , x 3 , future, ...

Subscript notations are reserved for indexing the terms from a numerical state vector with several terms.


State depends on state

We'll use the notation x to represent the vector containing the numerical values of the current model state.

The most useful state transition models represent the dynamic progress of the system over very small time intervals, so that changes are "seen" at a relatively high resolution before any big changes can occur. Since we are looking for a linear model, we can express the linear relationship using matrix notations, where the matrix term contains all of the constant multiplier parameters.

x k + 1 A · x k ...

Since state x does not change very much in one time increment, this means that the state transition matrix A will not be very much different from an identity matrix. Terms along the main diagonal will be reasonably close to 1.0 in value, and off-diagonal terms will be small.


State depends on inputs

Inputs also affect the next state. Over one time increment, the state changes driven by the inputs should be quite small. The terms of input coupling matrix B indicate the degree to which each input variable in the input list u, represented as a vector, will effect changes in each of the state variables during one time interval. We are again looking for a linear relationship, so it can be represented in matrix form something like the following.

x k + 1 B · u k ...

In non-pathological models, we expect both state dependent and input dependent changes in state. Because of the additive property of linearity, we know that the combination of the two effects is their sum.

x k + 1 A · x k + B · u k


Outputs depends on state

The outputs that happen right now depend on the state as it exists right now. That is a little different from the state transition equations, which show what state values will be at the next step. The linear relationship between the state variables and the observable outputs can be represented using an output observation matrix C.

y k = C · x k

We generally preclude any effects of current inputs u that couple directly and immediately to the outputs y. That sort of thing is allowable, but direct and instant connections from the system inputs to the system outputs, before the system dynamics can see them, can produce some anomalous behavior. Seek to avoid this complication.[2]

Some useful special cases for observation matrix C:

  • If the output is a single variable, the C matrix has a single row.

  • In the special case that all of the state variables are easy to observe directly, the C matrix turns into an identity matrix the same size as the state transition matrix.

  • If a subset of the state variables can be observed directly, the C matrix consists of selected rows from an identity matrix.

  • Outputs can consist of a combination of multiple internal state variables, so an output does not necessarily correspond to one single state variable.

As an example, for the following observation matrix

y k [ 1.0 0.0 0.0 0.0 0.8 - 0.8 ] · x k

there are two output variables. The first output is a direct observation of the current value of the first state variable. The second output is proportional to the difference between the last two state variables. Neither the second nor the third state variable can be observed directly in isolation.


Putting it together

Summarizing, the discrete time state model used by Kalman filters [3] consists of two parts, a state transition equation to predict what the next state will be, and an output observation equation to expose what the current observable model outputs are.

x k + 1 = A · x k + B · u k y k = C · x k

The problem of obtaining a suitable system model Kalman filtering is to determine the parameter values in the A, B, and C matrices such that the state trajectories indicated by the state transition equation result in output values y that are consistent with the behavior of the actual system.


Shaking it apart

We will reserve some notations for random influences. Detailing what these are all about must wait for later. The following kinds of random disturbances are included:

  • Random disturbances that are introduced into the system along with each new input, and that disrupt the next state variables directly, are indicated by random vector v.

  • Random disturbances that disrupt the observations of the output variables are indicated by random vector r.

  • A notation p is reserved for the random variations that accumulate and propagate in the state variables as a result of the random noise.

System equations augmented with the additional noise input vectors will be:

x k + 1 = A · x k + B · u k + v k y k = C · x k + r k

We have no way to predict what the random variables will do, so the best we can hope for is quantifying the statistics of the random effects. Randomness will interfere with the process of fitting a good state transition model.


What's next

Sometimes it isn't possible to operate the system in a pristine and isolated manner suggested by the dynamic model equations. Next time, we will consider how stabilization affects the model.



[1] The Wikipedia article on Kalman Filter.

[2] An example of this sort of bad idea is the impulsive response to setpoint changes from the "D term" in a simplistic PID controller implementation. The "tracking error" is the difference between the current input level and the current output level, so any change in the reference input level is perceived as an instantaneous step-change, causing an artificial and severe jolt in the produced feedback signal.

[3] This particular dynamic model form was the "regulator problem" and already well established in "the literature" when Kalman published his famous papers; Kalman's papers reference it. Kalman's contribution was related to establishing the connection to optimal Wiener filtering, to determine the theoretical conditions for optimality.


Contents of the "Developing Models for Kalman Filters" section of the website, including this page, are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License unless otherwise specifically noted. For complete information about the terms of this license, see The license allows usage and derivative works for individual or commercial purposes under certain restrictions. For permissions beyond the scope of this license, see the contact information page for this site.