Continuous-Time HMMs: When to Use Them

This article gives an overview of a tool called the “continuous-time hidden markov model” (CT-HMM), an analytical tool for decoding signal from a stream of noisy data. I wrote another article on the technical details of CT-HMMs. This article, in contrast, is about the problems they can solve. My goal is to help you identify situations where they might be the tool for the job.

A lot of situation in analytics fit the following description:

• A system is in some state that varies over time. For example, a person can be healthy or sick, and they typically stay that ways on the order of weeks or days.
• You have a series of measurements over time that hint at the state, but are not definitive. Like a person’s temperature measured at different points in time.

Usually your business goal is to figure out what the state was at every point in time. You could try training a model to classify every measurement in isolation, but often the signal just isn’t strong enough. What you need is a way to combine a given measurement with the context of the measurements before/after it. Enter the CT-HMM.

Relation to HMMs

You may have heard of hidden markov models (HMMs) in the past. They solve the same basic problem: given a sequence of imperfect measurements they tell you what’s going on at every state. But the issue is this: they assume that time is broken into discrete, evenly-spaced steps. If your measurements are taken at irregular intervals this isn’t going to work.

CT-HMMs are the generalization of normal HMMs to the situation where time flows continuously. Both measurements and state changes can happen at any moment, rather than just at regular intervals.

Applications of CT-HMMs

CT-HMMs have been used extensively in medicine to study the progression of various diseases. Given the ambiguity of any one measurement, and the irregular intervals at which patients get seen by their doctors, it’s the perfect use case!

But there is nothing medicine-specific about CT-HMMs. Other applications that I have seen include:

• Modeling the degradation of software systems
• Monitoring the congestion in computer networks

When to Use CT-HMMs

When people are at liberty to take measurements whenever they want, they will generally do so as regular intervals (and hence traditional HMMs are usually used). But CT-HMMs apply to situations when we cannot take measurements at will. For example:

• A disease’s progression, as noted
• In e-commerce the tastes of customers can evolve over time, but you can only gauge their tastes when they choose to make a purchase
• The health of a machine’s internal components can often only be measured when the machine is turned off for some reason and opened up

What Goes into a CT-HMM

A CT-HMM (like a traditional HMM) has two key components:

• Which measurements are how likely when the system is in a particular state.
• How long the system tends to stay in a given state, and how likely it is to go to each other state when it does switch.

Collectively I will call these the “parameters” of the CT-HMM. They are mathematical objects, largely dependent on the application at hand, and I won’t go into the gory details here (that’s in my other article).

What’s important to understand is that if we know these parameters, we can use them to decode any particular sequence of measurements and figure out (probabilistically) what state the system was in at those times.

What Problems You Can Solve

The classic problem is to infer, at each measurement, what state the system was in. The inference is inherently probabilistic: confidence will be higher if the measurement itself is clearer, or if there are many other nearby measurements taken shortly before/after.

Once you have the likelihood of the system’s state at each measurement, these likelihood can be interpolated to any point in time. From this you can estimate how long the system spent in a given state. You can also interpolate forward into the future, predicting the system’s future state based on its history so far.

Often the state of the system at a particular point in time is incidental. What we are really interested in is the parameters of the CT-HMM, because those tell us how the system operates. In this case the goal is to take a corpus of measurements and infer what the underlying CT-HMM is (this is called “training” the CT-HMM).

What Libraries are Available

Traditional HMMs are widely used, and supported by a wide range of libraries. Unfortunately, though, CT-HMMs have comparatively little support in open-source libraries.

I am only familiar with the following options:

• cthmm: a python library that supports CT-HMMs, including features that make it easier to integrate with machine learning models. Full disclosure: it was written by the author of this article.
• hhms: a python library that supports both traditional HMMs and CT-HMMs
• cthmm: an R package supporting CT-HMMs