cons_news.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "<a id='information-consumption-smoothing-v3'></a>\n",
    "<div id=\"qe-notebook-header\" align=\"right\" style=\"text-align:right;\">\n",
    "        <a href=\"https://quantecon.org/\" title=\"quantecon.org\">\n",
    "                <img style=\"width:250px;display:inline;\" width=\"250px\" src=\"https://assets.quantecon.org/img/qe-menubar-logo.svg\" alt=\"QuantEcon\">\n",
    "        </a>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Information and Consumption Smoothing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contents\n",
    "\n",
    "- [Information and Consumption Smoothing](#Information-and-Consumption-Smoothing)  \n",
    "  - [Overview](#Overview)  \n",
    "  - [Two Representations of the **Same** Nonfinancial Income Process](#Two-Representations-of-the-**Same**-Nonfinancial-Income-Process)  \n",
    "  - [State Space Representations](#State-Space-Representations)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition to what’s in Anaconda, this lecture employs the following libraries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": true
   },
   "outputs": [],
   "source": [
    "!pip install --upgrade quantecon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "This lecture studies two consumers who have exactly the same\n",
    "nonfinancial income process and who both conform to the linear-quadratic\n",
    "permanent income of consumption smoothing model described in the\n",
    "[quantecon lecture](https://python.quantecon.org/perm_income_cons.html).\n",
    "\n",
    "The two consumers  have different information about\n",
    "future nonfinancial incomes.\n",
    "\n",
    "One consumer each period receives **news** in the form of a shock that simultaneously\n",
    "affects both **today’s** nonfinancial income and  the\n",
    "present value of **future** nonfinancial incomes in a particular way.\n",
    "\n",
    "The other, less well informed, consumer each period receives a shock\n",
    "that equals  the part of today’s nonfinancial income that could not be\n",
    "forecast from all  past values of nonfinancial income.\n",
    "\n",
    "Even though they receive exactly the same nonfinancial incomes each\n",
    "period, our two consumers behave differently because they have different information about\n",
    "their future nonfinancial incomes.\n",
    "\n",
    "The second consumer receives less information about future nonfinancial\n",
    "incomes in a sense that we shall make precise below.\n",
    "\n",
    "This difference in their information sets manifests itself in their responding differently to what they regard as time $ t $ information\n",
    "shocks.\n",
    "\n",
    "Thus, while they receive exactly the same histories of\n",
    "nonfinancial income, our two consumers receive different **shocks** or **news** about\n",
    "their **future** nonfinancial incomes.\n",
    "\n",
    "We compare behaviors of our two consumers as a way to learn about\n",
    "\n",
    "- operating characteristics of the linear-quadratic permanent income\n",
    "  model  \n",
    "- how the Kalman filter introduced in [this lecture](https://python.quantecon.org/kalman.html)\n",
    "  and/or the theory of optimal\n",
    "  forecasting introduced in [this lecture](https://python.quantecon.org/classical_filtering.html)\n",
    "  embody lessons\n",
    "  that can be applied to the **news** and **noise** literature  \n",
    "- various ways of representing and computing optimal decision rules in\n",
    "  the linear-quadratic permanent income model  \n",
    "- a **Ricardian equivalence** outcome describing effects on optimal\n",
    "  consumption of a tax cut at time $ t $ accompanied by a foreseen\n",
    "  permanent increases in taxes that is just sufficient to cover the interest\n",
    "  payments used to service the risk-free government bonds that are issued to finance\n",
    "  the tax cut  \n",
    "- a simple application of alternative ways to factor a covariance\n",
    "  generating function along lines described in [this lecture](https://python.quantecon.org/classical_filtering.html)  \n",
    "\n",
    "\n",
    "This lecture can be regarded as an introduction to some of the **invertibility** issues that take center stage in\n",
    "the analysis of **fiscal foresight** by Eric Leeper, Todd Walker, and Susan Yang [[LWY13]](https://python.quantecon.org/zreferences.html#leeper-walker-yang)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Two Representations of the **Same** Nonfinancial Income Process\n",
    "\n",
    "Where $ \\beta \\in (0,1) $, we study consequences of endowing a\n",
    "consumer with one of the two alternative representations for the change\n",
    "in the consumer’s nonfinancial income $ y_{t+1} - y_t $.\n",
    "\n",
    "The first representation, which we shall refer to as the **original representation**, is\n",
    "\n",
    "\n",
    "<a id='equation-eqn-1'></a>\n",
    "$$\n",
    "y_{t+1} - y_t  =  \\epsilon_{t+1} - \\beta^{-1}  \\epsilon_t  \\quad \\tag{1}\n",
    "$$\n",
    "\n",
    "where $ \\{\\epsilon_t\\} $ is an i.i.d. normally distributed scalar\n",
    "process with means of zero and contemporaneous variances\n",
    "$ \\sigma_\\epsilon^2 $.\n",
    "\n",
    "This representation of the process is used by a consumer who at time\n",
    "$ t $ knows both $ y_t $ and the original shock\n",
    "$ \\epsilon_t $ and can use both of them to forecast future\n",
    "$ y_{t+j} $’s.\n",
    "\n",
    "Furthermore, as we’ll see below, representation [(1)](#equation-eqn-1) has the peculiar property that a positive shock\n",
    "$ \\epsilon_{t+1} $ leaves the discounted present value of the consumer’s financial income at time $ t+1 $\n",
    "unaltered.\n",
    "\n",
    "The second representation of the **same** $ \\{y_t\\} $ process is\n",
    "\n",
    "\n",
    "<a id='equation-eqn-2'></a>\n",
    "$$\n",
    "y_{t+1} - y_t =  a_{t+1} - \\beta  a_t \\quad \\tag{2}\n",
    "$$\n",
    "\n",
    "where $ \\{a_t\\} $ is another i.i.d. normally distributed scalar\n",
    "process, with means of zero and now variances $ \\sigma_a^2 $.\n",
    "\n",
    "The two i.i.d. shock variances are related by\n",
    "\n",
    "$$\n",
    "\\sigma_a^2 = \\beta^{-2} \\sigma_\\epsilon^2 >  \\sigma_\\epsilon^2\n",
    "$$\n",
    "\n",
    "so that the variance of the innovation exceeds the variance of the\n",
    "original shock by a multiplicative factor $ \\beta^{-2} $.\n",
    "\n",
    "The second representation is the **innovations representation** from\n",
    "Kalman filtering theory.\n",
    "\n",
    "To see how this works, note that equating representations [(1)](#equation-eqn-1)\n",
    "and [(2)](#equation-eqn-2) for $ y_{t+1} - y_t $ implies\n",
    "$ \\epsilon_{t+1} - \\beta^{-1} \\epsilon_t = a_{t+1} - \\beta a_t $,\n",
    "which in turn implies\n",
    "\n",
    "$$\n",
    "a_{t+1} = \\beta a_t + \\epsilon_{t+1} - \\beta^{-1} \\epsilon_t .\n",
    "$$\n",
    "\n",
    "Solving this difference equation backwards for $ a_{t+1} $ gives,\n",
    "after a few lines of algebra,\n",
    "\n",
    "\n",
    "<a id='equation-eqn-3'></a>\n",
    "$$\n",
    "a_{t+1} = \\epsilon_{t+1} + (\\beta - \\beta^{-1}) \\sum_{j=0}^\\infty \\beta^j \\epsilon_{t-j} \\quad \\tag{3}\n",
    "$$\n",
    "\n",
    "which we can also write as\n",
    "\n",
    "$$\n",
    "a_{t+1} = \\sum_{j=0}^\\infty \\epsilon_{t+1 -j} \\equiv h(L) \\epsilon_{t+1}\n",
    "$$\n",
    "\n",
    "where $ L $ is the one-period lag operator, $ I $ is the\n",
    "identity operator, and\n",
    "\n",
    "$$\n",
    "h(L) = \\frac{ I -\\beta^{-1} L}{ I - \\beta L}\n",
    "$$\n",
    "\n",
    "Let $ c_j \\equiv E z_t z_{t-j} $ be the $ j $th autocovariance\n",
    "of the $ \\{y_t - y_{t-1}\\} $ process.\n",
    "\n",
    "Using calculations in the [quantecon lecture](https://python.quantecon.org/classical_filtering.html), where\n",
    "$ z \\in C $ is a complex variable, the covariance generating\n",
    "function $ g (z) =\n",
    "\\sum_{j=-\\infty}^\\infty c_j z^j $\n",
    "of the $ \\{(y_t - y_{t-1})\\} $ process equals\n",
    "\n",
    "$$\n",
    "g(z) = \\sigma_\\epsilon^2  h(z) h(z^{-1}) = \\beta^{-2} \\sigma_\\epsilon^2 > \\sigma_\\epsilon^2 ,\n",
    "$$\n",
    "\n",
    "which confirms that $ \\{a_t\\} $ is a **serially uncorrelated**\n",
    "process with variance\n",
    "\n",
    "$$\n",
    "\\sigma_a^2 = \\beta^{-1} \\sigma_\\epsilon^2 .\n",
    "$$\n",
    "\n",
    "To verify these claims, just notice that\n",
    "$ g(z) = \\beta^{-2} \\sigma_\\epsilon^2 $ implies that the coefficient\n",
    "$ g_0 = \\beta^{-2} \\sigma_\\epsilon^2 $ and that $ g_j = 0 $ for\n",
    "$ j \\neq 0 $.\n",
    "\n",
    "Alternatively, if you are uncomfortable with covariance generating\n",
    "functions, note that we can directly calculate $ \\sigma_a^2 $ from\n",
    "formula [(3)](#equation-eqn-3) according to\n",
    "\n",
    "$$\n",
    "\\sigma_a^2 = \\sigma_\\epsilon^2 + [ 1 + (\\beta - \\beta^{-1})^2 \\sum_{j=0}^\\infty \\beta^{2j} ] = \\beta^{-1} \\sigma_\\epsilon^2 .\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Application of Kalman filter\n",
    "\n",
    "We can also obtain representation [(2)](#equation-eqn-2) from representation [(1)](#equation-eqn-1) by using\n",
    "the **Kalman filter**.\n",
    "\n",
    "Thus, from equations associated with the **Kalman filter**, it can be\n",
    "verified that the steady-state Kalman gain $ K = \\beta^2 $ and the\n",
    "steady state conditional covariance\n",
    "$ \\Sigma = E [(\\epsilon_t - \\hat \\epsilon_t)^2 | y_{t-1}, y_{t-2}, \\ldots ] = (1 - \\beta^2) \\sigma_\\epsilon^2 $.\n",
    "\n",
    "In a little more detail, let $ z_t = y_t - y_{t-1} $ and form the\n",
    "state-space representation\n",
    "\n",
    "$$\n",
    "\\begin{aligned} \\epsilon_{t+1} & = 0 \\epsilon_t +  \\epsilon_{t+1} \\cr\n",
    "           z_{t+1} & = - \\beta^{-1} \\epsilon_t + \\epsilon_{t+1} \\end{aligned}\n",
    "$$\n",
    "\n",
    "and assume that $ \\sigma_\\epsilon = 1 $ for convenience\n",
    "\n",
    "Compute the steady-state Kalman filter for this system and let $ K $\n",
    "be the steady-state gain and $ a_{t+1} $ the one-step ahead\n",
    "innovation.\n",
    "\n",
    "The innovations representation is\n",
    "\n",
    "$$\n",
    "\\begin{aligned} \\hat \\epsilon_{t+1} & = 0 \\hat \\epsilon_t + K a_{t+1} \\cr\n",
    "           z_{t+1} & = - \\beta^{-1} a_t + a_{t+1} \\end{aligned}\n",
    "$$\n",
    "\n",
    "By applying formulas for the steady-state Kalman filter, by hand we\n",
    "computed that\n",
    "$ K = \\beta^2, \\sigma_a^2 = \\beta^{-2} \\sigma_\\epsilon^2 = \\beta^{-2}, $\n",
    "and $ \\Sigma = (1-\\beta^2) \\sigma_\\epsilon^2 $.\n",
    "\n",
    "We can also obtain these formulas via the classical filtering theory\n",
    "described in [this lecture](https://python.quantecon.org/classical_filtering.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### News Shocks and Less Informative Shocks\n",
    "\n",
    "Representation [(1)](#equation-eqn-1) is cast in terms of a **news shock**\n",
    "$ \\epsilon_{t+1} $ that represents a shock to nonfinancial income\n",
    "coming from taxes, transfers, and other random sources of income changes\n",
    "known to a well-informed person having all sorts of information about\n",
    "the income process.\n",
    "\n",
    "Representation [(2)](#equation-eqn-2) for the **same** income process is driven by shocks\n",
    "$ a_t $ that contain less information than the news shock\n",
    "$ \\epsilon_t $.\n",
    "\n",
    "Representation [(2)](#equation-eqn-2) is called the **innovations** representation for the\n",
    "$ \\{y_t - y_{t-1}\\} $ process.\n",
    "\n",
    "It is cast in terms of what time series statisticians call the\n",
    "**innovation** or **fundamental** shock that emerges from applying the\n",
    "theory of optimally predicting nonfinancial income based solely on the\n",
    "information contained solely in **past** levels of growth in\n",
    "nonfinancial income.\n",
    "\n",
    "**Fundamental for the** $ {y_t} $ **process** means that the shock\n",
    "$ a_t $ can be expressed as a square-summable linear combination of\n",
    "$ y_t, y_{t-1}, \\ldots $.\n",
    "\n",
    "The shock $ \\epsilon_t $ is **not fundamental** and has more\n",
    "information about the future of the $ \\{y_t - y_{t-1}\\} $ process\n",
    "than is contained in $ a_t $.\n",
    "\n",
    "Representation [(3)](#equation-eqn-3) reveals the important fact that the **original\n",
    "shock** $ \\epsilon_t $ contains more information about future\n",
    "$ y $’s than is contained in the semi-infinite history\n",
    "$ y^t = [y_t, y_{t-1}, \\ldots ] $ of current and past $ y $’s.\n",
    "\n",
    "Staring at representation [(3)](#equation-eqn-3) for $ a_{t+1} $ shows that it consists\n",
    "both of **new news** $ \\epsilon_{t+1} $ as well as a long moving\n",
    "average $ (\\beta - \\beta^{-1})\\sum_{j=0}^\\infty \\beta^j\\epsilon_{t-j} $ of **old news**.\n",
    "\n",
    "The **better informed** representation [(1)](#equation-eqn-1) asserts that a shock\n",
    "$ \\epsilon_{t} $ results in an impulse response to nonfinancial\n",
    "income of $ \\epsilon_t $ times the sequence\n",
    "\n",
    "$$\n",
    "1, 1- \\beta^{-1}, 1- \\beta^{-1}, \\ldots\n",
    "$$\n",
    "\n",
    "so that a shock that **increases** nonfinancial income $ y_t $ by\n",
    "$ \\epsilon_t $ at time $ t $ is followed by an **increase** in\n",
    "future $ y $ of $ \\epsilon_t $ times $ 1 - \\beta^{-1} < 0 $\n",
    "in **all** subsequent periods.\n",
    "\n",
    "Because $ 1 - \\beta^{-1} < 0 $, this means that a positive shock of\n",
    "$ \\epsilon_t $ today raises income at time $ t $ by\n",
    "$ \\epsilon_t $ and then **decreases all** future incomes by\n",
    "$ (\\beta^{-1} -1)\\epsilon_t $.\n",
    "\n",
    "This pattern precisely describes the following mental experiment:\n",
    "\n",
    "- The consumer receives a government transfer of $ \\epsilon_t $ at\n",
    "  time $ t $.  \n",
    "- The government finances the transfer by issuing a one-period bond on\n",
    "  which it pays a gross one-period risk-free interest rate equal to\n",
    "  $ \\beta^{-1} $.  \n",
    "- In each future period, the government **rolls over** the one-period\n",
    "  bond and so continues to borrow $ \\epsilon_t $ forever.  \n",
    "- The government imposes a lump-sum tax on the consumer in order to pay\n",
    "  just the current interest on the original bond and its successors\n",
    "  created by the roll-over operation.  \n",
    "- In all future periods $ t+1, t+2, \\ldots $, the government levies\n",
    "  a lump-sum tax on the consumer of $ \\beta^{-1} -1 $ that is just\n",
    "  enough to pay the interest on the bond.  \n",
    "\n",
    "\n",
    "The **present value** of the impulse response or moving average\n",
    "coefficients equals $ d_\\epsilon(L) = \\frac{0}{1 -\\beta } =0 $, a fact that we’ll see again\n",
    "below.\n",
    "\n",
    "Representation [(2)](#equation-eqn-2), i.e., the innovation representation, asserts that a\n",
    "shock $ a_{t} $ results in an impulse response to nonfinancial\n",
    "income of $ a_t $ times\n",
    "\n",
    "$$\n",
    "1, 1 -\\beta, 1 - \\beta, \\ldots\n",
    "$$\n",
    "\n",
    "so that a shock that increases income $ y_t $ by $ a_t $ at time\n",
    "$ t $ can be expected to be followed by an **increase** in $ y_{t+j} $\n",
    "of $ a_t $ times $ 1 - \\beta > 0 $ in all future periods $ j=1, 2, \\ldots $.\n",
    "\n",
    "The present value of the impulse response or moving average coefficients\n",
    "for representation [(2)](#equation-eqn-2) is\n",
    "$ d_a(\\beta) = \\frac{1 -\\beta^2}{1 -\\beta } = (1 + \\beta) $, another\n",
    "fact that will be important below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Representation of $ \\epsilon_t $ in Terms of Future $ y $’s\n",
    "\n",
    "Notice that reprentation [(1)](#equation-eqn-1), namely, $ y_{t+1} - y_t = -\\beta^{-1} \\epsilon_t + \\epsilon_{t+1} $\n",
    "implies the linear difference equation\n",
    "\n",
    "$$\n",
    "\\epsilon_t = \\beta \\epsilon_{t+1} - \\beta (y_{t+1} - y_t ).\n",
    "$$\n",
    "\n",
    "Solving forward we eventually obtain\n",
    "\n",
    "$$\n",
    "\\epsilon_t = \\beta ( y_t - (1-\\beta) \\sum_{j=0}^\\infty \\beta^j y_{t+j+ 1} )\n",
    "$$\n",
    "\n",
    "This equation shows that $ \\epsilon_t $ equals\n",
    "$ \\beta $ times the one-step-backwards error in\n",
    "optimally  **backcasting** $ y_t $ based on\n",
    "the **future** $ y^t_+ \\equiv y_{t+1}, y_{t+2}, \\ldots ] $ via the optimal backcasting formula\n",
    "\n",
    "$$\n",
    "E [ y_t | y^t_+] = (1-\\beta) \\sum_{j=0}^\\infty \\beta^j y_{t+j+ 1}\n",
    "$$\n",
    "\n",
    "Thus, $ \\epsilon_t $ contains **exact** information about an\n",
    "important linear combination of **future** nonfinancial income."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Representation in Terms of $ a_t $ Shocks\n",
    "\n",
    "Next notice that representation [(2)](#equation-eqn-2), namely, $ y_{t+1} - y_t = -\n",
    "\\beta a_t + a_{t+1} $ implies the linear difference\n",
    "equation\n",
    "\n",
    "$$\n",
    "a_{t+1} = \\beta a_t + (y_{t+1} - y_t)\n",
    "$$\n",
    "\n",
    "Solving this equation backward establishes that the one-step-prediction\n",
    "error $ a_{t+1} $ is\n",
    "\n",
    "$$\n",
    "a_{t+1} = y_{t+1} - (1-\\beta) \\sum_{j=0}^\\infty \\beta^j y_{t-j}\n",
    "$$\n",
    "\n",
    "and where the information set is $ y^t = [y_t, y_{t-1}, \\ldots ] $,\n",
    "the one step-ahead optimal prediction is\n",
    "\n",
    "$$\n",
    "E [ y_{t+1} | y^t ] = (1-\\beta) \\sum_{j=0}^\\infty \\beta^j y_{t-j}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Permanent Income Consumption-Smoothing Model\n",
    "\n",
    "When we computed optimal consumption-saving policies for the two\n",
    "representations using formulas obtained with the difference equation\n",
    "approach described in the [quantecon lecture](https://python.quantecon.org/perm_income_cons.html),\n",
    "we obtain:\n",
    "\n",
    "**for a consumer having the information assumed in the news\n",
    "representation** [(1)](#equation-eqn-1):\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "c_{t+1} - c_t & = 0 \\cr\n",
    "b_{t+1} - b_t &  = - \\beta^{-1} \\epsilon_t\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "**for a consumer having the more limited information associated with the\n",
    "innovations representation** [(2)](#equation-eqn-2):\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "c_{t+1} - c_t & = (1-\\beta^2) a_{t+1} \\cr\n",
    "b_{t+1} - b_t & = - \\beta a_t\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "These formulas agree with outcomes from the Python programs to be\n",
    "reported below using state-space representations and dynamic\n",
    "programming.\n",
    "\n",
    "Evidently the two consumers behave differently though they receive\n",
    "exactly the same histories of nonfinancial income.\n",
    "\n",
    "The consumer with information associated with representation [(1)](#equation-eqn-1)\n",
    "responds to each shock $ \\epsilon_{t+1} $ by leaving his consumption\n",
    "unaltered and **saving** all of $ a_{t+1} $ in anticipation of the\n",
    "permanently increased taxes that he will bear to pay for the addition\n",
    "$ a_{t+1} $ to his time $ t+1 $ nonfinancial income.\n",
    "\n",
    "The consumer with information associated with representation [(2)](#equation-eqn-2)\n",
    "responds to a shock $ a_{t+1} $ by increasing his consumption by\n",
    "what he perceives to be the **permanent** part of the increase in\n",
    "consumption and by increasing his **saving** by what he perceives to be\n",
    "the temporary part.\n",
    "\n",
    "We can regard the first consumer as someone\n",
    "whose behavior sharply illustrates the behavior assumed in a classic\n",
    "Ricardian equivalence experiment."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## State Space Representations\n",
    "\n",
    "We can cast our two representations in terms of the following two state\n",
    "space systems\n",
    "\n",
    "$$\n",
    "\\begin{aligned}  \\begin{bmatrix} y_{t+1} \\cr \\epsilon_{t+1} \\end{bmatrix}  &=\n",
    "      \\begin{bmatrix} 1 & - \\beta^{-1} \\cr 0 & 0 \\end{bmatrix} \\begin{bmatrix} y_t \\cr \\epsilon_t \\end{bmatrix}\n",
    "       + \\begin{bmatrix} \\sigma_\\epsilon \\cr \\sigma_\\epsilon \\end{bmatrix}\n",
    "       v_{t+1} \\cr\n",
    "y_t & =  \\begin{bmatrix} 1 & 0 \\end{bmatrix}    \\begin{bmatrix} y_t \\cr \\epsilon_t \\end{bmatrix} \\end{aligned}\n",
    "$$\n",
    "\n",
    "and\n",
    "\n",
    "$$\n",
    "\\begin{aligned}  \\begin{bmatrix} y_{t+1} \\cr a_{t+1} \\end{bmatrix}  &=\n",
    "      \\begin{bmatrix} 1 & - \\beta \\cr 0 & 0 \\end{bmatrix} \\begin{bmatrix} y_t \\cr a_t \\end{bmatrix}\n",
    "       + \\begin{bmatrix} \\sigma_a \\cr \\sigma_a \\end{bmatrix}\n",
    "       u_{t+1} \\cr\n",
    "y_t & =  \\begin{bmatrix} 1 & 0 \\end{bmatrix}    \\begin{bmatrix} y_t \\cr a_t \\end{bmatrix} \\end{aligned}\n",
    "$$\n",
    "\n",
    "where $ \\{v_t\\} $ and $ \\{u_t\\} $ are both i.i.d. sequences of\n",
    "univariate standardized normal random variables.\n",
    "\n",
    "These two alternative income processes are ready to be used in the\n",
    "framework presented in the section “Comparison with the Difference\n",
    "Equation Approach” in the [quantecon  lecture](https://python.quantecon.org/perm_income_cons.html).\n",
    "\n",
    "All the code that we shall use below is presented in that lecture."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Computations\n",
    "\n",
    "We shall use Python to form **both** of the above two state-space\n",
    "representations, using the following parameter values\n",
    "$ \\sigma_\\epsilon = 1, \\sigma_a = \\beta^{-1} \\sigma_\\epsilon = \\beta^{-1} $\n",
    "where $ \\beta $ is the **same** value as the discount factor in the\n",
    "household’s problem in the LQ savings problem in the [lecture](https://python.quantecon.org/perm_income_cons.html).\n",
    "\n",
    "For these two representations, we use the code in the\n",
    "[lecture](https://python.quantecon.org/perm_income_cons.html) to\n",
    "\n",
    "- compute optimal decision rules for $ c_t, b_t $ for the two types\n",
    "  of consumers associated with our two representations of nonfinancial\n",
    "  income  \n",
    "- use the value function objects $ P, d $ returned by the code to\n",
    "  compute optimal values for the two representations when evaluated at\n",
    "  the following initial conditions $ x_0 = $  \n",
    "\n",
    "\n",
    "$$\n",
    "\\begin{bmatrix} 10 \\cr 0 \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "for each representation.\n",
    "\n",
    "- create instances of the\n",
    "  [LinearStateSpace](https://github.com/QuantEcon/QuantEcon.py/blob/master/quantecon/lss.py)\n",
    "  class for the two representations of the $ \\{y_t\\} $ process and\n",
    "  use them to obtain impulse response functions of $ c_t $ and\n",
    "  $ b_t $ to the respective shocks $ \\epsilon_t $ and\n",
    "  $ a_t $ for the two representations.  \n",
    "- run simulations of $ \\{y_t, c_t, b_t\\} $ of length $ T $\n",
    "  under both of the representations (later I’ll give some more details\n",
    "  about how we’ll run some special versions of these)  \n",
    "\n",
    "\n",
    "We want to solve the LQ problem:\n",
    "\n",
    "$$\n",
    "\\min\\ \\sum_{t=0}^{\\infty}\\beta^{t}\\left(c_{t}-\\gamma\\right)^{2}\n",
    "$$\n",
    "\n",
    "subject to the sequence of constraints\n",
    "\n",
    "$$\n",
    "\\quad c_{t}+b_{t}=\\frac{1}{1+r}b_{t+1}+y_{t}, \\quad t \\geq 0\n",
    "$$\n",
    "\n",
    "where $ y_t $ follows one of the representations defined above.\n",
    "\n",
    "Define the control as $ u_t \\equiv c_t - \\gamma $.\n",
    "\n",
    "(For simplicity we can assume $ \\gamma=0 $ below because\n",
    "$ \\gamma $ has no effect on the impulse response functions that\n",
    "interest us.)\n",
    "\n",
    "The state transition equations under our two representations for the\n",
    "nonfinancial income process $ \\{y_t\\} $ can be written as\n",
    "\n",
    "$$\n",
    "\\left[\\begin{array}{c}\n",
    "y_{t+1}\\\\\n",
    "\\epsilon_{t+1}\\\\\n",
    "b_{t+1}\n",
    "\\end{array}\\right]=\\underset{\\equiv A_{1}}{\\underbrace{\\left[\\begin{array}{ccc}\n",
    "1 & -\\beta^{-1} & 0\\\\\n",
    "0 & 0 & 0\\\\\n",
    "-\\left(1+r\\right) & 0 & 1+r\n",
    "\\end{array}\\right]}}\\left[\\begin{array}{c}\n",
    "y_{t}\\\\\n",
    "\\epsilon_{t}\\\\\n",
    "b_{t}\n",
    "\\end{array}\\right]+\\underset{\\equiv B_{1}}{\\underbrace{\\left[\\begin{array}{c}\n",
    "0\\\\\n",
    "0\\\\\n",
    "1+r\n",
    "\\end{array}\\right]}}\\left[\\begin{array}{c}\n",
    "c_{t}\\end{array}\\right]+\\underset{\\equiv C_{1}}{\\underbrace{\\left[\\begin{array}{c}\n",
    "\\sigma_{\\epsilon}\\\\\n",
    "\\sigma_{\\epsilon}\\\\\n",
    "0\n",
    "\\end{array}\\right]}}\\nu_{t+1},\n",
    "$$\n",
    "\n",
    "and\n",
    "\n",
    "$$\n",
    "\\left[\\begin{array}{c}\n",
    "y_{t+1}\\\\\n",
    "a_{t+1}\\\\\n",
    "b_{t+1}\n",
    "\\end{array}\\right]=\\underset{\\equiv A_{2}}{\\underbrace{\\left[\\begin{array}{ccc}\n",
    "1 & -\\beta & 0\\\\\n",
    "0 & 0 & 0\\\\\n",
    "-\\left(1+r\\right) & 0 & 1+r\n",
    "\\end{array}\\right]}}\\left[\\begin{array}{c}\n",
    "y_{t}\\\\\n",
    "a_{t}\\\\\n",
    "b_{t}\n",
    "\\end{array}\\right]+\\underset{\\equiv B_{2}}{\\underbrace{\\left[\\begin{array}{c}\n",
    "0\\\\\n",
    "0\\\\\n",
    "1+r\n",
    "\\end{array}\\right]}}\\left[\\begin{array}{c}\n",
    "c_{t}\\end{array}\\right]+\\underset{\\equiv C_{2}}{\\underbrace{\\left[\\begin{array}{c}\n",
    "\\sigma_{a}\\\\\n",
    "\\sigma_{a}\\\\\n",
    "0\n",
    "\\end{array}\\right]}}u_{t+1}.\n",
    "$$\n",
    "\n",
    "As usual, we start by importing packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import quantecon as qe\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "# Set parameters\n",
    "β, σϵ = 0.95, 1\n",
    "σa = σϵ / β\n",
    "\n",
    "R = 1 / β\n",
    "\n",
    "# Payoff matrices are the same for two representations\n",
    "RLQ = np.array([[0, 0, 0],\n",
    "                [0, 0, 0],\n",
    "                [0, 0, 1e-12]]) # put penalty on debt\n",
    "QLQ = np.array([1.])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "# Original representation state transition matrices\n",
    "ALQ1 = np.array([[1, -R, 0],\n",
    "                 [0, 0, 0],\n",
    "                 [-R, 0, R]])\n",
    "BLQ1 = np.array([[0, 0, R]]).T\n",
    "CLQ1 = np.array([[σϵ, σϵ, 0]]).T\n",
    "\n",
    "# Construct and solve the LQ problem\n",
    "LQ1 = qe.LQ(QLQ, RLQ, ALQ1, BLQ1, C=CLQ1, beta=β)\n",
    "P1, F1, d1 = LQ1.stationary_values()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "# The optimal decision rule for c\n",
    "-F1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Evidently optimal consumption and debt decision rules for the consumer\n",
    "having news representation [(1)](#equation-eqn-1) are\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "c_{t}^{*} &= y_{t}-\\epsilon_{t}-\\left(1-\\beta\\right)b_{t}, \\\\\n",
    "b_{t+1}^{*} &=\\beta^{-1}c_{t}^{*}+\\beta^{-1}b_{t}-\\beta^{-1}y_{t} \\\\\n",
    "    & =\\beta^{-1}y_{t}-\\beta^{-1}\\epsilon_{t}-\\left(\\beta^{-1}-1\\right)b_{t}+\\beta^{-1}b_{t}-\\beta^{-1}y_{t} \\\\\n",
    "    & =b_{t}-\\beta^{-1}\\epsilon_{t}.\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "# Innovations representation\n",
    "ALQ2 = np.array([[1, -β, 0],\n",
    "                 [0,  0, 0],\n",
    "                 [-R, 0, R]])\n",
    "BLQ2 = np.array([[0, 0, R]]).T\n",
    "CLQ2 = np.array([[σa, σa, 0]]).T\n",
    "\n",
    "LQ2 = qe.LQ(QLQ, RLQ, ALQ2, BLQ2, C=CLQ2, beta=β)\n",
    "P2, F2, d2 = LQ2.stationary_values()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "-F2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For a consumer having access only to the information associated with the\n",
    "innovations representation [(2)](#equation-eqn-2), the optimal decision rules are\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "c_{t}^{*} &= y_{t}-\\beta^{2}a_{t}-\\left(1-\\beta\\right)b_{t}, \\\\\n",
    "b_{t+1}^{*} &= \\beta^{-1}c_{t}^{*}+\\beta^{-1}b_{t}-\\beta^{-1}y_{t} \\\\\n",
    "    &=\\beta^{-1}y_{t}-\\beta a_{t}-\\left(\\beta^{-1}-1\\right)b_{t}+\\beta^{-1}b_{t}-\\beta^{-1}y_{t} \\\\\n",
    "    &=b_{t}-\\beta a_{t}.\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "Now we construct two Linear State Space models that emerge from using\n",
    "optimal policies $ u_t =- F x_t $ for the control variable.\n",
    "\n",
    "Take the original representation case as an example,\n",
    "\n",
    "$$\n",
    "\\left[\\begin{array}{c}\n",
    "y_{t+1}\\\\\n",
    "\\epsilon_{t+1}\\\\\n",
    "b_{t+1}\n",
    "\\end{array}\\right]=\\left(A_{1}-B_{1}F_{1}\\right)\\left[\\begin{array}{c}\n",
    "y_{t}\\\\\n",
    "\\epsilon_{t}\\\\\n",
    "b_{t}\n",
    "\\end{array}\\right]+C_{1}\\nu_{t+1}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\left[\\begin{array}{c}\n",
    "c_{t}\\\\\n",
    "b_{t}\n",
    "\\end{array}\\right]=\\left[\\begin{array}{c}\n",
    "-F_{1}\\\\\n",
    "S_{b}\n",
    "\\end{array}\\right]\\left[\\begin{array}{c}\n",
    "y_{t}\\\\\n",
    "\\epsilon_{t}\\\\\n",
    "b_{t}\n",
    "\\end{array}\\right]\n",
    "$$\n",
    "\n",
    "To have the Linear State Space model of the innovations representation\n",
    "case, we can simply replace the corresponding matrices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "# Construct two Linear State Space models\n",
    "Sb = np.array([0, 0, 1])\n",
    "\n",
    "ABF1 = ALQ1 - BLQ1 @ F1\n",
    "G1 = np.vstack([-F1, Sb])\n",
    "LSS1 = qe.LinearStateSpace(ABF1, CLQ1, G1)\n",
    "\n",
    "ABF2 = ALQ2 - BLQ2 @ F2\n",
    "G2 = np.vstack([-F2, Sb])\n",
    "LSS2 = qe.LinearStateSpace(ABF2, CLQ2, G2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following we compute the impulse response functions of\n",
    "$ c_t $ and $ b_t $."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "J = 5 # Number of coefficients that we want\n",
    "\n",
    "x_res1, y_res1 = LSS1.impulse_response(j=J)\n",
    "b_res1 = np.array([x_res1[i][2, 0] for i in range(J)])\n",
    "c_res1 = np.array([y_res1[i][0, 0] for i in range(J)])\n",
    "\n",
    "x_res2, y_res2 = LSS2.impulse_response(j=J)\n",
    "b_res2 = np.array([x_res2[i][2, 0] for i in range(J)])\n",
    "c_res2 = np.array([y_res2[i][0, 0] for i in range(J)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "c_res1 / σϵ, b_res1 / σϵ"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "plt.title(\"original representation\")\n",
    "plt.plot(range(J), c_res1 / σϵ, label=\"c impulse response function\")\n",
    "plt.plot(range(J), b_res1 / σϵ, label=\"b impulse response function\")\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above two impulse response functions show that when the consumer has\n",
    "the information assumed in the original representation, his response to\n",
    "receiving a positive shock of $ \\epsilon_t $ is to leave his\n",
    "consumption unchanged and to save the entire amount of his extra income\n",
    "and then forever roll over the extra bonds that he holds.\n",
    "\n",
    "To see this notice, that starting from next period on, his debt\n",
    "permanently **decreases** by $ \\beta^{-1} $"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "c_res2 / σa, b_res2 / σa"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "plt.title(\"innovations representation\")\n",
    "plt.plot(range(J), c_res2 / σa, label=\"c impulse response function\")\n",
    "plt.plot(range(J), b_res2 / σa, label=\"b impulse response function\")\n",
    "plt.plot([0, J-1], [0, 0], '--', color='k')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above impulse responses show that when the consumer has only the\n",
    "information that is assumed to be available under the innovations\n",
    "representation for $ \\{y_t - y_{t-1} \\} $, he responds to a positive\n",
    "$ a_t $ by permanently increasing his consumption.\n",
    "\n",
    "He accomplishes this by consuming a fraction $ (1 - \\beta^2) $ of\n",
    "the increment $ a_t $ to his nonfinancial income and saving the rest\n",
    "in order to lower $ b_{t+1} $ to finance the permanent increment in\n",
    "his consumption.\n",
    "\n",
    "The preceding computations confirm what we had derived earlier using\n",
    "paper and pencil.\n",
    "\n",
    "Now let’s simulate some paths of consumption and debt for our two types\n",
    "of consumers while always presenting both types with the same\n",
    "$ \\{y_t\\} $ path, constructed as described below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "# Set time length for simulation\n",
    "T = 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "x1, y1 = LSS1.simulate(ts_length=T)\n",
    "plt.plot(range(T), y1[0, :], label=\"c\")\n",
    "plt.plot(range(T), x1[2, :], label=\"b\")\n",
    "plt.plot(range(T), x1[0, :], label=\"y\")\n",
    "plt.title(\"original representation\")\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "x2, y2 = LSS2.simulate(ts_length=T)\n",
    "plt.plot(range(T), y2[0, :], label=\"c\")\n",
    "plt.plot(range(T), x2[2, :], label=\"b\")\n",
    "plt.plot(range(T), x2[0, :], label=\"y\")\n",
    "plt.title(\"innovations representation\")\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Simulating the Income Process and Two Associated Shock Processes\n",
    "\n",
    "We now describe how we form a **single** $ \\{y_t\\}_{t=0}^T $ realization\n",
    "that we will use to simulate the two different decision rules associated\n",
    "with our two types of consumer.\n",
    "\n",
    "We accomplish this in the following steps.\n",
    "\n",
    "1. We form a $ \\{y_t, \\epsilon_t\\} $ realization by drawing a long\n",
    "  simulation of $ \\{\\epsilon_t\\}_{t=0}^T $ where $ T $ is a big\n",
    "  integer $ \\epsilon_t = \\sigma_\\epsilon v_t $, $ v_t $ is a\n",
    "  standard normal scalar, $ y_0 =100 $, and  \n",
    "\n",
    "\n",
    "$$\n",
    "y_{t+1} - y_t =-\\beta^{-1} \\epsilon_t + \\epsilon_{t+1} .\n",
    "$$\n",
    "\n",
    "1. We take the **same** $ \\{y_t\\} $ realization generated in step 1\n",
    "  and form an innovation process $ \\{a_t\\} $ from the formulas  \n",
    "\n",
    "\n",
    "$$\n",
    "\\begin{aligned} a_0 & = 0 \\cr\n",
    "a_t & = \\sum_{j=0}^{t-1} \\beta^j (y_{t-j} - y_{t-j-1}) + \\beta^t a_0, \\quad t \\geq 1 \\end{aligned}\n",
    "$$\n",
    "\n",
    "1. We throw away the first $ S $ observations and form the sample\n",
    "  $ \\{y_t, \\epsilon_t, a_t\\}_{S+1}^T $ as the realization that\n",
    "  we’ll use in the following steps.  \n",
    "1. We use the step 3 realization to **evaluate** and **simulate** the\n",
    "  decision rules for $ c_t, b_t $ that Python has computed for us\n",
    "  above.  \n",
    "\n",
    "\n",
    "The above steps implement the experiment of comparing decisions made by\n",
    "two consumers having **identical** incomes at each date but at each date\n",
    "having **different** information about their future incomes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculating Innovations in Another Way\n",
    "\n",
    "Here we use formula [(3)](#equation-eqn-3) above to compute $ a_{t+1} $ as a function\n",
    "of the history\n",
    "$ \\epsilon_{t+1}, \\epsilon_t, \\epsilon_{t-1}, \\ldots $\n",
    "\n",
    "Thus, we compute\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "a_{t+1} &=\\beta a_{t}+\\epsilon_{t+1}-\\beta^{-1}\\epsilon_{t} \\\\\n",
    "    &=\\beta\\left(\\beta a_{t-1}+\\epsilon_{t}-\\beta^{-1}\\epsilon_{t-1}\\right)+\\epsilon_{t+1}-\\beta^{-1}\\epsilon_{t} \\\\\n",
    "    &=\\beta^{2}a_{t-1}+\\beta\\left(\\epsilon_{t}-\\beta^{-1}\\epsilon_{t-1}\\right)+\\epsilon_{t+1}-\\beta^{-1}\\epsilon_{t} \\\\\n",
    "    &=\\cdots \\\\\n",
    "    &=\\beta^{t+1}a_{0}+\\sum_{j=0}^{t}\\beta^{j}\\left(\\epsilon_{t+1-j}-\\beta^{-1}\\epsilon_{t-j}\\right) \\\\\n",
    "    &=\\beta^{t+1}a_{0}+\\epsilon_{t+1}+\\left(\\beta-\\beta^{-1}\\right)\\sum_{j=0}^{t-1}\\beta^{j}\\epsilon_{t-j}-\\beta^{t-1}\\epsilon_{0}.\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "We can verify that we recover the same $ \\{a_t\\} $ sequence\n",
    "computed earlier."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Another Invertibility Issue\n",
    "\n",
    "This [quantecon lecture](https://python.quantecon.org/hs_invertibility_example.html) contains another example of a shock-invertibility issue that is endemic\n",
    "to the LQ permanent income or consumption smoothing model.\n",
    "\n",
    "The technical issue discussed there is ultimately the source of the shock-invertibility issues discussed by\n",
    "Eric Leeper, Todd Walker, and Susan Yang [[LWY13]](https://python.quantecon.org/zreferences.html#leeper-walker-yang) in their  analysis of **fiscal foresight**."
   ]
  }
 ],
 "metadata": {
  "date": 1584334736.208945,
  "filename": "cons_news.rst",
  "kernelspec": {
   "display_name": "Python",
   "language": "python3",
   "name": "python3"
  },
  "title": "Information and Consumption Smoothing"
 },
 "nbformat": 4,
 "nbformat_minor": 2
}