Algorithms Q&A - Recent questions in Artificial Intelligence

MLE for given HMM and Observation Sequence

Sat, 25 Apr 2026 15:40:44 +0000

We are given an MDP with 3 states: Red, Green, Blue (RGB).

Emission probabilities:

R --> 1 (p=1/3), 2 (p=1/3), 3 (p=1/3)

G --> 1 (p=1/3), 2 (p=1/3), 3 (p=1/3)

B --> 1 (p=1/3), 2 (p=1/3), 3 (p=1/3)

Transition Matrix

R --> R (p=1/3), G (p=1/3), B (p=1/3)

G --> G (p=1/2), B (p=1/2)

B --> B (p=1)

These are the observations: 1, 2, 3, 1, 1, 2, 1, 2, 3, 1, 1, 2, 2, 1, 3, 2, 2, 1, 1, 2, 1, 1, 1, 2, 1

What is the most likely explanation for this observation sequence?
Initial state is R.

MLE for given HMM and Observation Sequence

Sat, 25 Apr 2026 13:26:54 +0000

We are given an MDP with 3 states: Red, Green, Blue (RGB).

Emission probabilities:

R --> 1 (p=1/3), 2 (p=1/3), 3 (p=1/3)

G --> 1 (p=1/3), 2 (p=1/3), 3 (p=1/3)

B --> 1 (p=1/3), 2 (p=1/3), 3 (p=1/3)

Transition Matrix

R --> R (p=1/3), G (p=1/3), B (p=1/3)

G --> R (p=1/3), G (p=1/3), B (p=1/3)

B --> R (p=1/3), G (p=1/3), B (p=1/3)

These are the observations: 1, 2, 3, 1, 1, 2, 1, 2, 3, 1, 1, 2, 2

What is the most likely explanation for this observation sequence?
Initial state is not known and is equally likely from all states.

MLE for the given MDP and Observation Sequence (RGB Color Cycler)

Sat, 25 Apr 2026 01:09:37 +0000

We are given an MDP with 3 states: Red, Green, Blue (RGB).

Emission probabilities:

R --> 1 (p=0.4), 2, (p=0.5), 3 (p-0.1)

G --> 1 (p=0.3), 2 (p=0.3), 3 (p=0.4)

B --> 1 (p=0.4), 2 (p=0.1), 3 (p=0.1), 4 (p=0.4).

Transition Matrix: R -> G (p = 1.0), G -> B (p = 1.0), B -> R (p = 1.0).

These are the observations: 1, 2, 3, 4, 1, 3, 1, 2, 3

What is the most likely explanation for this observation sequence?
Initial state is not known and is equally likely from all states.

Heads - 2 in a row

Tue, 14 Apr 2026 21:06:07 +0000

I have two coins, Fair coin - gives H and T with 0.5 probability each, and a Biased coin, always gives H.

I pick one of the two coins randomly and toss it. It comes up as heads. I toss it again. What is the probability that it comes up as heads again?

Why is it useful to compute the marginal distribution instead of working with the full joint distribution?

Tue, 07 Apr 2026 22:16:23 +0000

Suppose we model weather using two variables:

X: Weather condition (Sunny, Cloudy, Rainy)
Y: Whether people carry an umbrella (Yes, No)

You are given the joint distribution:

	Umbrella (Yes)	Umbrella (No)	Total
Sunny	0.05	0.35	0.40
Cloudy	0.15	0.15	0.30
Rainy	0.25	0.05	0.30
Total	0.45	0.55	1.00

Core Question

If you are only interested in predicting whether people carry umbrellas, regardless of the actual weather: Why is it useful to compute the marginal distribution of umbrella usage instead of working with the full joint distribution? What do you gain and what do you lose by marginalizing out the weather variable?

Some Discussion Points

Relevance to Modeling
- In what situations would an AI system care only about P(Y) rather than P(X,Y)?
- Is marginalization equivalent to “ignoring causes” in this context?
Hidden Variables / Latent State
- If weather were unobserved, how would marginalization help in modeling behavior?
- How does this relate to hidden state models (e.g., HMMs in AI)?
Decision-Making
- Suppose a city planner wants to estimate umbrella demand.
- Is P(Y) sufficient, or do they need P(Y/X)?
Information Loss
- What important causal relationship disappears when we marginalize out weather?
- Could two very different weather patterns produce the same marginal umbrella usage?

Solve this grid world.

Tue, 06 May 2025 15:13:38 +0000

Solve the final V* values for the given grid world. All given values are exit states. Fill in all missing cells. Don’t use value iteration. Show all your work/logic/steps, etc.

Assume: Gamma = 0.9. Noise Model: [0.8, 0.1, 0.1], that is, 80% chance of going in the intended direction and 10% in each of perpendicular ones.

Using pen and paper is preferable.

Admissible Heuristic for Chutes and Ladders

Tue, 06 May 2025 15:12:07 +0000

You are a player of Chutes and Ladders. You are going from cell 1 to cell 100, using standard 6 faced dice. There are “chutes” and “ladders” that hurt and help, respectively.
Explain an algorithm to calculate an admissible heuristic starting square i.

Weird N Puzzle

Tue, 04 Mar 2025 16:03:53 +0000

You are given a weirdly shaped n puzzle (sliding puzzle).

The best way to imagine the shape is that think of 10 x 10 n puzzle (a rather large one), and that some tiles in the center are just fixed - they don't move. (See Slack for an example).

For this problem, what is a good heuristic, from an A Star algorithm perspective? Clearly Manhattan Distance still works, but is that the best?

I have all the (unsorted) cards

Tue, 04 Mar 2025 15:50:53 +0000

You are given 52 playing cards, organized in 4 columns and 13 rows. However, these are all Aces, in 4 different colors: red, blue, green and yellow. These are all jumbled up. You are supposed to organize them such that each column has one color of Aces only. You can decide which color you want to go in which column. The only move that you can move is to swap/exchange two cards that horizontally or vertically adjacent. That is, just vertical or horizontal movement, like the n-puzzle, and no diagonal moves.

Analyze the problem, and present an algorithm for this. Present your heuristic. No need to explain how A-Star works itself. Describe without using any pseudocode.

What is the most likely weather estimate?

Sat, 13 May 2023 20:49:35 +0000

Consider the classic Viterbi example problem of determining the weather based on whether an individual carried an umbrella. The weather on a particular day matches the previous day 70% of the time. The individual will carry an umbrella on 90% of rainy days and 20% of non-rainy days.

Given an umbrella sequence of [+u, +u, -u, +u, +u], what is the most likely explanation for the weather?

Evaluate an MDP given several observed episodes

Thu, 11 May 2023 12:58:02 +0000

Let's say you are given a '+'-shaped MDP with five states and a gamma (discount rate) of 1:

Given MDP

_ A _
B C D
_ E _

The input policy \Pi is as follows:

A -> Terminal
B -> C
C -> D
D -> Terminal
E -> C

Let's say you have the following observed episodes (training) though:

Episode 1:
B, east, C, -1
C, east, D, -1
D, exit, x, +10

Episode 2:
B, east, C, -1
C, east, D, -1
D, exit, x, +10

Episode 3:
E, north, C, -1
C, east, D, -1
D, exit, x, +10

Episode 4:
E, north, C, -1
C, north, A, -1
A, exit, x, -10

What are the output values based on these episodes?

Robot States and Sensors

Mon, 08 May 2023 21:05:20 +0000

Suppose we have a robot that can be in one of two states: state 1 or state 2. The robot moves between states according to the following rules:

If the robot is in state 1, it has a 70% chance of staying in state 1 and a 30% chance of transitioning to state 2.

If the robot is in state 2, it has a 60% chance of staying in state 2 and a 40% chance of transitioning to state 1.

The robot has two sensors: sensor 1 and sensor 2. The sensors are not perfect and sometimes give incorrect readings. The emission probabilities for each sensor are as follows:

If the robot is in state 1, sensor 1 gives the correct reading with a probability of 0.8 and an incorrect reading with a probability of 0.2. If the robot is in state 2, sensor 1 gives the correct reading with a probability of 0.4 and an incorrect reading with a probability of 0.6.

If the robot is in state 1, sensor 2 gives the correct reading with a probability of 0.3 and an incorrect reading with a probability of 0.7. If the robot is in state 2, sensor 2 gives the correct reading with a probability of 0.9 and an incorrect reading with a probability of 0.1.

Assume that the robot starts in state 1 with equal probability and generates the following sensor readings over time:

sensor 1: correct, correct, incorrect

sensor 2: incorrect, correct, correct

What is the most likely sequence of states that the robot went through to generate these observations?

Coin - Real or Biased

Tue, 25 Apr 2023 17:40:07 +0000

There are two coins: Fair Coin: Heads and Tails with 50% Probability, and Biased Coin: Always lands on Heads. Dealer uses either coin with equal probability.

We see that the dealer tosses the coin, and it comes in a sequence as H, H, T. We can immediately infer that the dealer is using the fair coin (because biased coin never lands on Tails).

In next run, the dealer tosses the coin, and it comes in a sequence as H, H, H. What is the probability that the dealer is using the biased coin.

The dealer does not switch coins during a single run.

Comparison of Lottery Choices and MEU

Sun, 16 Apr 2023 19:53:02 +0000

Given these two lottery choices:

A: [0.75, $3k; 0.25, $0]

B: [1.0, $2k; 0.0, $0]

and

C: [0.3, $3k; 0.7, $0]

D: [0.4, $2k; 0.6, $0]

People seem to prefer B over A, and C over D

What are the implications of this observation?

What is the contradiction here? What assumption did you use to arrive at your contradiction?

Snake - A through Y on a 5 x 5 grid

Sun, 16 Apr 2023 19:50:15 +0000

You are given a 5 x 5 grid, and this grid should hold the letters A through Y. Some of the cells in the grid are already filled out. The others can go anywhere, but the constraint is that adjacent letters (for example F and G) must be adjacent to each other, either horizontally or vertically.

- - - - Y
R A - - -
- - - - -
- E - - -
- - - - K

Design a solution for this constraint satisfaction problem. (Solution should explain the variables, the domain values, the constraints, and how the constraint propagation would work.)

Solve the V* values for this grid world MDP - 3 x 4

Sun, 16 Apr 2023 19:47:18 +0000

Estimate the final V* values for a given grid world. Discount, noise, terminal states are all given. You are given a 3x4 grid world and you want to find V* values. There is a discount rate of 0.9 and noise of 0.8, 0.1, 0.1. The first column has all terminal states with value -1 and the last column has terminal states with value 1. There is no living reward.

-1			1
-1			1
-1			1

A* algorithm for fruit sorting

Sun, 19 Mar 2023 16:44:14 +0000

You are given 10 apples of different sizes, 10 bananas of different sizes and 10 oranges of different sizes, organized in a 3x10 array. You want to organize them so that fruits go from top to bottom in ascending order of size. Any fruit can be used in any column that you like. The only move allowed is to swap two fruits horizontally or vertically. You want to use A* algorithm to minimize the number of moves for this. Explain your approach.

Searching Problem with Numbers

Wed, 19 Jan 2022 00:57:10 +0000

Start state = Number "4"
Target state = Number "5"
Successor function:

factorial, cost = 3
square root, cost = 2
floor, cost 1.

1. Frame this problem as a regular AI "search" problem.
2. Find the shortest cost path from start node to target node.

Disco Fever and Two Kinds of Tests

Mon, 10 May 2021 00:52:23 +0000

25% of the people in a city have Disco Fever. There are two tests for Disco Fever, Rapid Antigen Test (RAT) and Full Scan Test (FST). When a person goes to a doctor to test for Disco Fever, with probability 2/3 the doctor prescribes RAT, and with probability 1/3 the doctor prescribes FST.

When RAT is done on a person, the outcome is as follows: If the person has the disease, the result is positive with probability 3/4. If the person does not have the disease, the result is positive with probability 1/4.

When FST is done on a person, the outcome is as follows: If the person has the disease, the result is positive with probability 1 (that is, FST has no false negatives). If the person does not have the disease, the result is positive with probability 1/3.

A person is picked uniformly at random from the town and is sent to a doctor to test for Disco Fever. The result comes out positive. What is the probability that the person has Disco Fever?

Newsletter optimization using HMM

Sun, 02 May 2021 12:59:28 +0000

You are trying to sell a product by using an email newsletter (delivered weekly). A visitor can be interested in the product due to 2 different interests. If they are interested due to interest 1 (for example “soccer”), they may click a link (link 1) you have in the newsletter. If they are interested due to interest 2 (for example “tennis”), they may click a different link (link 2). Here is the probability table that shows the likelihood of viewer clicking these two links:

Interest	Link 1	Link 2
No Interest	0.01	0.05
Interest 1	0.3	0.6
Interest 2	0.4	0.8
Interest 1 and Interest 2	0.8	0.95

We do not know the initial state of the visitor. If the visitor has no interest in interest 1 or interest 2, she a 10% probability of developing either of those interests (independently) during the week. When she has one of the interests, there is a 20% probability of her developing the other interest as well. Once an interest is developed, that interest is never lost.

Draw and explain an HMM to model this problem.
You make the following observations in 8 weeks of newsletter delivers:

No Click, Link 1, Link 1, No Click, Link 1, Link 1 and Link 2, Link 1 and Link 2, Link 2, Link 1 and Link What is the best explanation for the interest level of the visitor during those 8 weeks?

Solve the V* values for this MDP - 5x5

Tue, 30 Mar 2021 14:38:01 +0000

Estimate the final V* values for a given grid world.

Discount (gamma) is 0.9.
Noise (probability is 0.8, 0.1, 0.1. That is, when moving in a direction, there is 80% probablity of going in that direction and 10% probability of going in one of two perpendicular directions.
Terminal states are given in the table. All non terminal states are left blank and need to be solved.
Living reward (R(s,a,s')) is 0.

Instead of iteration, use the solving approach.

1	0	1

0		0

1	0	1

Algorithm for Power Plant CSP

Wed, 24 Mar 2021 16:43:15 +0000

Design a solution for the given constraint satisfaction problem. Solution should explain the variables, the domain values, the constraints, the heuristics, and how the constraint propagation would work.

In a country, there are 15 power plants. They produce different amount of power, that is given by the array P[i]. The total power produced must equal at least X (for example 1000), but should not exceed 1500 to maintain a decent price. For adequate workforce planning and employment reasons, at least one of the power plant near each city must operate each day.

Partial example..

Power Plant, i	P[i]	Associated Cities
P1	200	C1,C2
P2	200	C2, C3
P3	180	C2, C4, C6
P4	210	C5,C6
……	…..	……….
i-th power plant	P[i]	List of cities

Jumpy Car - Estimate/Solve the V* values for the following MDP

Mon, 01 Mar 2021 23:11:54 +0000

You are traveling on a straight road, but have a jumpy car. The car sometimes "jumps" (moves double). At other times, it doesn't move at all. The following MDP has been created to model this behavior and the landscape.

Estimate the V* values (optimal values for the states) for this MDP:

S1	S2	S3	S4	S5	S6	S7	S8	S9
sqrt(3)				100				sqrt(3)

MDP is defined as follows: There are two actions L (Left) and R (Right). When moving left, there is a 40% chance moving left, 50% chance of moving DOUBLE (2 spots left), and 10% chance of not moving at all. Similarly, when moving right, there is a 40% chance moving right, 50% chance of moving DOUBLE (2 spots right), and 10% chance of not moving at all.

S1, S5 and S9 are terminal states with values sqrt(3), 100 and sqrt(3) respectively.

Discount factor is 0.9 (so, gamma = 0.9).

There is no living reward, that is R(s,a,s’) = 0.

Solve the V* values for this MDP

Tue, 23 Feb 2021 22:40:41 +0000

For the given MDP, find the values for the states S2, S3 and S4. States S1 and S5 are terminal states with values 0 and 1 respectively. Living Reward (R) is 0. Transition function is defined as follows: When going Left or Right, there is a 90% probability that the move goes as planned and 10% probability that no move occurs. Discount rate gamma is 0.9

S1	S2	S3	S4	S5
0	x2	x3	x4	1

Probability of rain in desert when a reliable weatherperson predicts one

Sun, 21 Feb 2021 18:38:53 +0000

Antonio has an exciting soccer game coming up. In recent years, it has rained only 5 days each year in the city where they live.

Unfortunately, the weatherperson has predicted rain for that day. When it actually rains, she correctly forecasts rain 90% of the time. When it doesn't rain, she incorrectly forecasts rain only 10% of the time.

What is the probability that it will rain on the day of Antonio’s soccer game?

Stationary Distribution for this conditional probability table

Sat, 04 Apr 2020 13:35:39 +0000

What is the stationary distribution for the following conditional probability table?

What is the code level difference between graph search and tree search?

Mon, 03 Feb 2020 16:54:34 +0000

What is the code level difference between graph search and tree search?

Solve this GridWorld

Mon, 13 May 2019 20:24:23 +0000

Calculate the final V* values for the given grid world. Fill in all missing cells. All given values are terminal states. Use value iteration/policy iteration, any method that you like.

Assume 50% discount (that is, gamma = 0.5). Assume 0.8, 0.1, 0.1 noise, that is, probability of going to the intended direction is 0.8, and probability of going left/right is 0.1 each.

GridWorld
10	10	10	10	10
10				10
10				10
10				10
10	10	10	10	10

Admissible Heuristic for n-puzzle if multiple tiles can be slided at once

Tue, 07 May 2019 10:51:24 +0000

In the context of n-puzzle, consider a variation that we can slide multiple tiles in one row or column, if they are touching each other. The cost of this slide operation is the same as that of sliding one tile.

Design an admissible and useful heuristic for this problem.

8 puzzle that can be solved or not?

Tue, 30 Apr 2019 22:49:15 +0000

Some of 8 puzzles can be solved if it is transferred from the #12345678. But there are 9! states. How to rule out many of the wrong states use rules that 8 puzzles have potential, like odd and even, the right rule is #12345678, every line only three place and so on.

Is Harmonic Mean of admissible heuristics an admissible heuristic?

Tue, 09 Apr 2019 02:25:06 +0000

Given a family of admissible heuristic functions h1, h2, .. hn. Is the Harmonic Mean, H = n / (1/h1 + 1/h2 + ... 1/hn) an admissible function? How would you define this function if one of the h functions is zero?

Converting ternary constraint into binary constraints

Sat, 09 Feb 2019 15:59:04 +0000

Show how a single ternary constraint such as "x + y = z" can be turned into three binary constraints by using an auxiliary variable.

Manhattan Distances, but Linear Interference...

Sat, 02 Feb 2019 16:11:04 +0000

Manhattan Distance does not take into account linear conflicts. For example:

- - 3,1 needs to be changed to - 1, -, 3

[Only first row is being shown here, and the blank is assumed to be at the top left corner.]

Manhattan Distance is 4, but tiles 1 and 3 interfere with each other while moving.

So, how can we improve this heuristic?

Give an example of a task environment that is..

Fri, 25 Jan 2019 18:48:48 +0000

Give an example of a task environment that is..

Episodic and Discrete
Continuous and Episodic
Continuous, Non-Deterministic and Non-Episodic.

For each of the combinations above, give an example of a task environment.

Admissible Heuristic for n-Queens Problem

Sat, 24 Mar 2018 11:44:41 +0000

Given all queens are in their own columns, and a move is defined as picking up a queen and moving to any other row in the same column, what is an admissible heuristic for this problem?

[We note that by the structure of the problem, we always have a queen in each column.]

In the context of informed search, what is the difference between complete, optimal and optimally efficient algorithm?

Mon, 19 Mar 2018 02:35:55 +0000

In the context of informed search, what is the difference between complete, optimal and optimally efficient algorithm?

Is A* complete?

Is A* optimal?

Is A* optimally efficient?

N-Queens Count all conflicts in O(n) time

Sat, 03 Mar 2018 02:26:23 +0000

Count all the conflicts in a given n-Queen state. It is easy to do in O(n²) time. Give an O(n) time algorithm to count all conflicts.