Reinforcement Learning — DU B.Sc. (Hons) Computer Science Previous Year Question Papers

Your Roll N............... 4. Parts of a question must be answered together. 1. (a) Wies is Reinforement Leaming? How is it difinest tom Supervised learning? (b) Diferentiate between a Reward Signal and Value Function. Which one of (3) them represents the long-tem poratial of a state? ( A robot's nest position dependa only on its current coordinates and its curent velocity, Does this sysem sausty the Markov Property? Justify your answer (d) At agent receives a seguence of rewards Ey = 2, E. = 4, R = 10. If the discaunt facior y = 03, calculate the total discounted retum Our {e) Consider a 4x4 grid world. The top-left and bottom-right are the tarimal states, while the rest are non-terminal states. Initial state values v are all Os and corresponding greedy policy is shown using anows for all directions achieving the macionm. After a series of policy iteration steps, the new state-values are shown as below. Show the greedy policy (using arrows) with respect to these given state values: (f) An agent is using e-gréedy policy with e = 0.1. If there are 5 possible (3) actions, compute the probabilities of choosing the greedy (best-known) action and the non-greedy (random) actions. (g) In the Actor-Critic algorithm, which component is responsible for the (3) policy, and which is responsible for value function computation? (h) Using a suitable example, describe the trade-off between exploration and (3) exploitation. What are the consequences of favoring one over the other? (i) In:a TD()) algorithm, the eligibility trace is updated as: (3) Given y = 0.9, initial trace zo = 0, compute z, and zz when 2 = 1. i) What is the difference between n-step TD and TD() in terms of how they (3) use future rewards? 2 (a) Describe the Agent-Environment interaction in a Reinforcement Learning (5) Task. Draw a diagram showing the interaction loop of S, Ay and R, and explain how the time step progresses from one interaction to the next. (b) Consider a state so with two possible actions a, and as: (5) - Action a, leads to state s, with reward = 4 : - Action az leads to state sz with reward = 2 The agent follows a policy: Given y= 09, W(6) = 5, Vs) = 1, compute Vs) wing the Bellman Expeciation Equation. (e) Considera Reinforcement learning agent baing trained to play the game of (5) Chess, Identify and briefly destibe the following components of this EL system: Agent, Environment, State, Action, Reward. 3{a) Define Value Functious in Reinforcement Learning. Write the Bellmen (3) Equation for the state-value fanstion vs). Identify and explain the meaning of each mathematical term used in the equation such as 1, 5, 1, E, and y- : (b) Explain the difference between Policy Iteration and Value Iteration (5) algorithms. Describe the primary steps involved in each Which one of the two is generally faster in temus of the momber of iterations to converge? (c) Consider a state s where taking action fa' leats to state s, with probability 15)