Algorithms Q&A - Recent questions and answers in HMM

Answered: MLE for the given MDP and Observation Sequence (RGB Color Cycler)

Mon, 27 Apr 2026 13:30:50 +0000

Hello,

Explanation:

1. Deterministic Transitions: The states follow a strict cycle: R -> G -> B -> R -> G. You cannot stay in a state or skip one.

2. Observation Check:

    *   Observation 4 appears at index 4. Only state Blue can emit 4 (probability 0.4). Therefore, the state at step 4 must be Blue.

    *   Working backward and forward from Step 4 (Blue) using the cycle R -> G -> B:

        *   Step 4: Blue

        *   Step 3: Red (precedes Blue)

        *   Step 2: Green (precedes Red)

        *   Step 1: Blue (precedes Green)? Let's check the start.



    Let's test the three possible starting positions for the cycle against the observations:

    *   Case 1: Start with Red (Sequence: R, G, B, R, G, B, R, G, B)

        *   Obs 1 (State R): P(1|R) = 0.4

        *   Obs 2 (State G): P(2|G) = 0.3

        *   Obs 3 (State B): P(3|B) = 0.1

        *   Obs 4 (State R): P(4|R) = 0 (Impossible)

    *   Case 2: Start with Green (Sequence: G, B, R, G, B, R, G, B, R)

        *   Obs 1 (State G): P(1|G) = 0.3

        *   Obs 2 (State B): P(2|B) = 0.1

        *   Obs 3 (State R): P(3|R) = 0.1

        *   Obs 4 (State G): P(4|G) = 0 (Impossible)

    *   Case 3: Start with Blue (Sequence: B, R, G, B, R, G, B, R, G)

        *   Obs 1 (State B): P(1|B) = 0.4

        *   Obs 2 (State R): P(2|R) = 0.5

        *   Obs 3 (State G): P(3|G) = 0.4

        *   Obs 4 (State B): P(4|B) = 0.4

        *   Obs 5 (State R): P(1|R) = 0.4

        *   Obs 6 (State G): P(3|G) = 0.4

        *   Obs 7 (State B): P(1|B) = 0.4

        *   Obs 8 (State R): P(2|R) = 0.5

        *   Obs 9 (State G): P(3|G) = 0.4

    Only Case 3 is possible because it is the only one that allows Observation 4 to be emitted (by Blue) and Observation 4 to be emitted again? No, Obs 4 is only at index 4. But wait, in Case 1 and 2, the state at index 4 was R and G respectively, neither of which can emit 4. Only Blue emits 4. In the cycle $R \to G \to B$, Blue appears at positions 3, 6, 9 if starting at R; positions 2, 5, 8 if starting at G; and positions 1, 4, 7 if starting at B.

    Since Observation 4 is at index 4, the state at index 4 MUST be Blue. This aligns perfectly with Case 3 (Start with Blue).

The sequence is Blue, Red, Green, Blue, Red, Green, Blue, Red, Green.

Answered: MLE for given HMM and Observation Sequence

Mon, 27 Apr 2026 13:24:27 +0000

Hello,

The most likely explanation is that any sequence of states is equally likely.

Reasons:

1. Uniform Emissions: Every state (Red, Green, Blue) emits every observation (1, 2, 3) with the exact same probability 1/3. Therefore, the observations provide no information to distinguish which state you are in.

2. Uniform Transitions: From any state, you transition to any other state (including staying) with the exact same probability 1/3.

3. Uniform Initial State: You start in any state with equal probability 1/3.

Because every possible path through the states has the exact same mathematical probability, there is no single "most likely" sequence. All 3^13 possible state sequences are equally probable.

Answered: MLE for given HMM and Observation Sequence

Mon, 27 Apr 2026 13:21:53 +0000

Hello,

The most likely explanation (state sequence) is that the system starts in Red, transitions to Green at some point, and then eventually transitions to Blue, where it stays for the remainder of the sequence.

However, looking closely at the specific probabilities:

1. Emissions are uniform 1/3 for all symbols in all states. This means the observations themselves provide no information to distinguish between states. The likelihood depends entirely on the transition probabilities.

2. Transitions:

    *   From R: You can go to R, G, or B with equal probability 1/3.

    *   From G: You must go to G or B 1/2 each. You cannot go back to R.

    *   From B: You must stay in B p=1. This is an absorbing state.

Since emissions don't help, we want the path with the highest transition probability.

*   Staying in R forever has probability (1/3)^(T-1).

*   Moving to G and staying there has probability involving 1/3 for the first step and 1/2 for subsequent steps. Since 1/2 > 1/3, staying in G is more likely than staying in R.

*   Moving to B and staying there has probability involving 1/3 or 1/2 for the entry step, and 1 for all subsequent steps. Since 1 > 1/2 > 1/3, once you enter B, the probability cost for remaining steps is zero (log-probability is 0).

Therefore, the single most likely specific path is the one that enters the absorbing state Blue as early as possible, because staying in Blue has a probability of 1 per step, which is higher than staying in Red 1/3 or Green 1/2.

The earliest you can enter Blue is at step 2 R -> B.

Path: R, B, B, B, ..., B

Let's compare:

*   Path R, R, R...: Prob = (1/3)^24

*   Path R, G, G... Prob = 1/3 * (1/2)^23

*   Path R, B, B...: Prob = 1/3 * 1^23 = 1/3

Clearly, 1/3 is much larger than the others.

The most likely state sequence is:

Red, Blue, Blue, Blue, Blue, Blue, Blue, Blue, Blue, Blue, Blue, Blue, Blue, Blue, Blue, Blue, Blue, Blue, Blue, Blue, Blue, Blue, Blue, Blue, Blue

Answered: Heads - 2 in a row

Sun, 26 Apr 2026 13:13:29 +0000

Hello Professor,

Let F be the fair coin and B be the biased coin.

At the start:

P(F) = 1/2

P(B) = 1/2

The fair coin gives heads with probability 1/2, while the biased coin always gives heads.

First, we find the probability of getting heads on the first toss:

P(H1) = P(H1 | F)P(F) + P(H1 | B)P(B)

P(H1) = (1/2)(1/2) + (1)(1/2)

P(H1) = 1/4 + 1/2 = 3/4

Now, since we already saw heads on the first toss, we update the probability that the coin was biased:

P(B | H1) = [(1)(1/2)] / (3/4)

P(B | H1) = 2/3

So the probability that it was the fair coin is:

P(F | H1) = 1/3

Now we calculate the probability that the second toss is also heads:

P(H2 | H1) = P(H2 | B)P(B | H1) + P(H2 | F)P(F | H1)

P(H2 | H1) = (1)(2/3) + (1/2)(1/3)

P(H2 | H1) = 2/3 + 1/6

P(H2 | H1) = 5/6

Final answer:

5/6, or about 83.33%

So after seeing one head, it becomes more likely that the coin is the biased coin, which makes the probability of getting heads again higher than 1/2

Robot States and Sensors

Mon, 08 May 2023 21:05:20 +0000

Suppose we have a robot that can be in one of two states: state 1 or state 2. The robot moves between states according to the following rules:

If the robot is in state 1, it has a 70% chance of staying in state 1 and a 30% chance of transitioning to state 2.

If the robot is in state 2, it has a 60% chance of staying in state 2 and a 40% chance of transitioning to state 1.

The robot has two sensors: sensor 1 and sensor 2. The sensors are not perfect and sometimes give incorrect readings. The emission probabilities for each sensor are as follows:

If the robot is in state 1, sensor 1 gives the correct reading with a probability of 0.8 and an incorrect reading with a probability of 0.2. If the robot is in state 2, sensor 1 gives the correct reading with a probability of 0.4 and an incorrect reading with a probability of 0.6.

If the robot is in state 1, sensor 2 gives the correct reading with a probability of 0.3 and an incorrect reading with a probability of 0.7. If the robot is in state 2, sensor 2 gives the correct reading with a probability of 0.9 and an incorrect reading with a probability of 0.1.

Assume that the robot starts in state 1 with equal probability and generates the following sensor readings over time:

sensor 1: correct, correct, incorrect

sensor 2: incorrect, correct, correct

What is the most likely sequence of states that the robot went through to generate these observations?

Newsletter optimization using HMM

Sun, 02 May 2021 12:59:28 +0000

You are trying to sell a product by using an email newsletter (delivered weekly). A visitor can be interested in the product due to 2 different interests. If they are interested due to interest 1 (for example “soccer”), they may click a link (link 1) you have in the newsletter. If they are interested due to interest 2 (for example “tennis”), they may click a different link (link 2). Here is the probability table that shows the likelihood of viewer clicking these two links:

Interest	Link 1	Link 2
No Interest	0.01	0.05
Interest 1	0.3	0.6
Interest 2	0.4	0.8
Interest 1 and Interest 2	0.8	0.95

We do not know the initial state of the visitor. If the visitor has no interest in interest 1 or interest 2, she a 10% probability of developing either of those interests (independently) during the week. When she has one of the interests, there is a 20% probability of her developing the other interest as well. Once an interest is developed, that interest is never lost.

Draw and explain an HMM to model this problem.
You make the following observations in 8 weeks of newsletter delivers:

No Click, Link 1, Link 1, No Click, Link 1, Link 1 and Link 2, Link 1 and Link 2, Link 2, Link 1 and Link What is the best explanation for the interest level of the visitor during those 8 weeks?