Well, as I write this late in the Kyoto afternoon, Donald Trump has just made…
The reality of Germany and the buffoons in Brussels intervenes …
This week, I seem to have been focused on central banking this week, which is not my favourite topic, but is all the rage over the last several days given the decision of the Bank of Japan to use negative interest rates on any new bank reserves and then continue to pump reserves into the system via its so-called QQE policy (swapping public and corporate bonds for bank reserves), and then imposing a tax on the reserves so created. Crazy is just one euphemism which comes to mind. So still on that theme and remembering that the Bank of Japan explicitly stated that the combination of QQE and the tax on reserves (they call it a negative interest rate – same thing) was introduced to increase the inflation rate back up towards its target of 2 per cent per annum, I thought the following paper was interesting. The paper from the Research Division of the Federal Reserve Bank of St Louis (published July 2015) – Current Federal Reserve Policy Under the Lens of Economic History: A Review Essay – considers the unconventional monetary monetary policy interventions taken by the US Federal Reserve Bank between 2007 and 2009 and comes to the conclusion that “there is no work, to my knowledge, that establishes a link from QE to the ultimate goals of the Fed inflation and real economic activity”. Maybe the Bank of Japan and the ECB bosses should sent this researcher an E-mail and request his evidence. They don’t seem to have been able to escape from the straitjacket of their neo-liberal Groupthink.
The Federal Reserve Bank of St Louis Research Paper initially considers a brief history of central banking and makes the point that:
… the Bank of England … was a central bank, which had a symbiotic relationship with one of its key borrowers, the British Crown. The Crown, by granting the Bank monopoly respect to circulating currency, was able to finance its spending more cheaply than would otherwise be the case. Further, the Bank’s monopoly, as well as its size, created the possibility that its actions would matter for asset prices and aggregate economic activity.
Interestingly, while the Bank of England was also a “typical private” bank, it “actually had a lot to lose from excessive risk-taking – its monopoly on currency issue – and this tended to reinforce actual conservative behaviour”.
The author (Stephen D. Williamson) contended that historical experience “tells us that”:
(i) a key element of central banking is the relationship between the central bank and the fiscal authority; (ii) the central bank’s power comes from its monopoly with respect to the liabilities it issues …
There is no financial crisis that a central bank cannot alleviate given this power. It also has the obvious power to fund any necessary fiscal plans that a sovereign government might consider appropriate to implement.
The paper then considers the “Great Recession” in the US.
We learn that in the early days:
A key problem early in the financial crisis was that the Fed wished to lend, but there appeared to be a reluctance of banks to borrow through conventional means at the Fed’s discount window.
In 2008, there was a lot of hype about bank’s not having sufficient liquidity to allow for loans to be made. The actual problem was not that financial institutions were illiquid in terms of making loans and then being able to access reserves from the central bank to ensure checks drawn on deposits cleared.
The problem was that many financial institutions relied on wholesale funds borrowed at maturities which were coming due and could not re-finance their own debt.
The overwhelming reality, though, was that the banks were not so reluctant to lend but they were struggling to attract borrowers from the household and corporate sector.
The demand from credit-worthy customer declined dramatically in 2008, which was no surprise given the collapsing housing market and the massive debt overhang that the past decade or so had left in the private domestic sector.
The paper considers that the Term Auction Facility (TAF) that was introduced by the US central bank in December 2007 was essentially a broader type of open market operation was the main difference being the type of “eligible collateral” that the financial institution can offer the central bank in return for a loan.
In other words, the US central bank “purchased some troubled assets” in addition to “commercial paper” as it sought to reduce financial fragility.
He concludes that the main impact of this form of intervention was to prevent the “runs on commercial bank deposits and disruption of retail payments that occurred during the Great Depression”, which were absent during the GFC. He realises that the other major difference in comparing the outcomes during the Great Depression and the more recent financial crisis was the introduction of the “credit deposit insurance” scheme in 1933.
But even taking that into consideration he believes that “one could also make the case that the Fed’s support of Önancial institutions gave the non-insured holders of bank liabilities the confidence not to run”.
He then moved on to consider the other unconventional monetary policy interventions following 2009, when the US economy had resumed growth.
There were three major interventions:
1. The “zero-interest-rate policy”.
2. The “large-scale asset purchases” (QE).
3. The “forward guidance” where the Federal Reserve makes public statements about its thinking and the possible future policy shifts to shape the public’s expectations. The author considers the US central bank failed in this regard because its statements were too vague and ill-defined.
I will only focus on the second element, given that has largely been of interest in the discussions of inflation targets etc. The Bank of Japan has just added a second dimension to the QE – negative rates. But, in effect, they are seen as part of the same deal.
Pump in reserves to give the bank’s ‘cash’ to loan out. Then impose a tax to create an additional incentive to make the loans.
The Research Paper notes that with excess reserves in the banking system “the interest rate on reserves will determine short-term interest rates”. Why? Because it becomes the point of indifference between holding the reserves within the bank’s central bank account and trying to loan those reserves to other banks in the interbank market.
Of course, when there is a system-wide excess, the loans between banks in the interbank market may help a bank get rid of their reserve excess and put reserves elsewhere to shore up a specific reserve shortage within another bank, but the surplus, ultimately cannot be offloaded this way.
The introduction of a interest payment on excess reserves from the central bank effectively stops the banks trying to off-load their excesses in the interbank market and therefore sets the short-term interest rate floor.
The paper claims that:
… so long as the interest rate on reserves in greater than zero, the Fed can ease in a conventional way by reducing the interest rate on reserves. But, if the interest rate on reserves cannot go below zero, then the Fed cannot ease conventionally.
Which suggests that the author considers open market operations (conventional interventions), which exchanges cash (from the central bank) for bonds is a stimulatory measure – that is, stimulates the real economy.
We need to be careful here because the same logic is often applied to QE. The stimulus does not come from the extra reserves that an open market purchase by a central bank makes (buying bonds for reserves).
The stimulus, if there is any, comes from the fact that by pushing up the demand for bonds, the central bank drives down the yields on those assets at the relevant maturity, which then influences other returns on financial assets within that maturity range.
So, the presumption is that lower interest rates then stimulate private sector investment spending and/or reduce the value of the currency (financial flows move out in search of higher relative returns), which stimulates real GDP growth.
The uninformed version of the story is that the increase in bank reserves arising from the open market operation provides them with more money to loan out.
Of course, anyone holding that view, including a substantial number of mainstream macro and monetary economists, disclose their lack of understanding of how the banking system works. Banks do not loan out reserves. They shuffle them between themselves to facilitate the efficiency of the so-called payments system (the ‘clearing house’) so that checks don’t bounce.
But bank lending is not reserve constrained.
Please read the following blog – Building bank reserves will not expand credit – for more detail on this.
And please read Monday’s blog – The folly of negative interest rates on bank reserves.
Once the Federal Reserve Bank had driven the short-term interest rate down to near zero, it decided:
… to engage in QE – purchases of long-maturity Treasury securities and mortgage-backed securities …
The logic was clearly expressed at the time:
1. “The basic idea is that finnancial markets are posited to be segmented by asset maturity.” Some institutions match long-term assets with long-term liabilities, while others operate at different ‘maturity segments’.
2. “… if the Fed issues short-term liabilities (reserves) in exchange for long-maturity assets, then this should, according to the theory, increase the price of long-maturity assets, and therefore reduce long-term bond yields. With the nominal short-term interest rate fixed at zero, this should flatten the yield curve.
The yield curve is just the term structure of interest rates from short-term out to the very long-term investment rates. A flatter yield curve is theoretically posited to act as a stimulus for investment-type borrowing (capital formation) because the longer-term borrowing rates become cheaper.
So the stimulus comes, if at all, from increased borrowing because of cheaper finance, rather than the banks having more cash to loan out.
The research paper then assesses the evidence as to whether the QE had any impact on inflation or real GDP growth.
He concludes that:
… there is no work, to my knowledge, that establishes a link from QE to the ultimate goals of the Fed inflation and real economic activity. Indeed, casual evidence suggests that QE has been ineffective in increasing inflation. For example, in spite of massive central bank asset purchases in the U.S., the Fed is currently falling short of its 2% inflation target. Further, Switzerland and Japan, which have balance sheets that are much larger than that of the U.S., relative to GDP, have been experiencing very low inflation or deflation.
He further considered the inflation issue – a key focus of the Bank of Japan plan.
The Research Paper notes that the inflation fears arise directly from the mainstream quantity theory of money which says that a rising money supply will transmit directly into higher prices in the economy.
Too much money chasing too few goods – is one of those awful (false) aphorisms that conservatives and smug economists like to say to summarise their position as if it is meaningful and insightful.
Stephen Williamson writes:
A conventional quantity-theory-of-money view of the world would tell us that, if commercial banks lend out excess reserves, that this will lead to an increase in monetary aggregates – through the “money multiplier” process – and therefore to an increase in the price level.
Please read my blog – Money multiplier and other myths – for more discussion on why this theory is deeply flawed.
The Federal Reserve Research Paper agrees with my assessment here.
It concludes that:
The available theory and empirical evidence says no … QE can actually lead to lower inflation. When government debt is in short supply as collateral, this imparts a liquidity premium to safe assets, and the real interest rate is low, just as has been the case in the United States since the beginning of the Great Recession. Under these circumstances, if the central bank conducts a swap of short-maturity government debt for long-maturity debt, as it did under Operation Twist, this acts to increase the effective stock of collateral. As a result, the liquidity premium falls, and the real interest rate rises. If the nominal interest rate is zero, then an increase in the real rate implies a decrease in the inflation rate, i.e. QE causes inflation to fall.
The “informal empirical evidence on QE and inflation” aka facts shows that most central banks have “undershot” their inflation targets over the last several years.
And, “in some cases, particularly Japan, Switzerland, and the Euro area, there have been substantial increases in the quantity of high-powered money and the central bank’s balance sheet, which have accompanied this low inflation experience.”
But excuse me, says the mainstream economist – these facts must be wrong. Our theory predicts heavy inflation arising from the expansion of central bank balance sheets.
Then all sorts of conspiracy theories emerge about how the national statistical agencies are rigging the books and deliberately under-reporting the national inflation statistics to avoid acknowledging how bad things are.
There have been many such conspiracy articles since the GFC started.
The reality is binding. There has been no cooking of the statistics. QE was never going to cause inflation. Inflation is highly unlikely when there are elevated levels of mass unemployment and weak spending – and then, on top of that, the fiscal authority embarks on a manic, ill-conceived austerity campaign.
Try getting inflation out of that.
Conclusion
In the Eurozone case, it is time to acknowledge that all the sophistry being pumped out of Frankfurt on a regular basis by Mario Draghi and his ‘executives’ about stimulus measures such as negative interest rates and continued QE is just hot air.
The serious research evidence backs the logic of Modern Monetary Theory (MMT) in allowing us to conclude that all these unconventional monetary policy interventions will neither push the inflation rate up nor stimulate real GDP in any significant way.
Refresh yourself about yesterday’s blog – The ECB could stand on its head and not have much impact.
The reality tells us that Japan and the Eurozone, which are both facing chronic deflation and slow growth, require substantial demand-side stimulus.
What allowed the US to recover relatively quickly was not the unconventional monetary policy interventions from the Federal Reserve Bank, but rather, the massive fiscal stimulus in 2009 (of the order of $US830 billion) which has been estimated to have pushed real GDP up by around 1 per cent per annum between 2009 and 2014 (Source).
Japan clearly has the capacity to do that. The dysfunctional Eurozone would need to abandon its neo-liberal Groupthink that is obsessed with austerity and use what flexibility there is in the Treaty to override the Stability and Growth Pact fiscal constraints.
In my latest book – Eurozone Dystopia: Groupthink and Denial on a Grand Scale – I show how that could, in theory, be facilitated.
But then, the reality of Germany and the collection of buffoons in Brussels masquerading as key European Commission official, intervenes.
That is enough for today!
(c) Copyright 2016 William Mitchell. All Rights Reserved.
In the meantime, let us not forget that Jeroen Dijsselbloem & colleagues are “ordering” Spain to introduce cutbacks worth 9 billion euros, or close to 1% of our GDP.
Is it correct to say that when someone deposits money into a savings account that the bank uses the money to buy Treasuries and collects the interest above what is paid to the saver and when the bank creates the money for a loan that it collects all of the interest on the loan except the interest it pays on the reserves?
Utter morons. How did such incompetent people get to positions of power? And how do we get rid of them!
http://www.reuters.com/article/us-ecb-praet-idUSKBN0UK0T820160106
“Executive Board member Peter Praet said various factors, notably low oil prices and less buoyant emerging economies, meant it was taking longer to reach the goal of inflation of close to but below 2 percent. “We need to be attentive that this shifting horizon does not damage the credibility of the ECB,” he added. “There is no plan B, there is just one plan. The ECB is ready to take all measures necessary to bring inflation up to 2 percent. If you print enough money, you get inflation. Always. If, as is happening now, the prices of oil and commodities are tumbling, then it’s more difficult to drive up inflation,” he said.”
“The European Central Bank’s money-printing plan has so far failed to drive up inflation but the bank does not have an alternative “plan B”, ECB Executive Board member Peter Praet said in a magazine interview published on Wednesday.
Praet said he remained confident that the stimulus would drive up inflation however, adding: “If you print enough money, you always get inflation. Always.””
“”I accept that our policy has not yet been successful: inflation in Europe has for a long time been at a very low level of almost zero,” Praet, the ECB’s chief economist, told Belgian weekly magazine Knack.”
“”We need to be attentive that this shifting horizon does not damage the credibility of the ECB,” he added.”
“The problem was that many financial institutions relied on wholesale funds borrowed at maturities which were coming due and could not re-finance their own debt”
“The demand from credit-worthy customer declined dramatically in 2008, which was no surprise given the collapsing housing market and the massive debt overhang that the past decade or so had left in the private domestic sector.”
PDS was squeezed by a reducing fiscal deficit and a current account deficit at 3%.But also the private debt overhang caused a peak of debt servicing cost as a percentage of personal income at 13% (source=http://www.federalreserve.gov/releases/housedebt/).
Prime borrowers defaults/forclosure increased in 2008 http://www.nber.org/digest/aug15/w21261.html.
So the US (private sector) was servicing debt to foreigners(current account deficit) not locals (because a country´s private sector being in debt to it self shouldn’t cause a housing market to collapse).Then again those USD would have stayed in the US Financial system (unless the gulf counrties and Chinese wanted to ship out cash!)
And so banks couldn’t lend to anyone,because the amount of creditworthy borrowers had dried up.
Except for the banks which couldn’t refinance their maturity debt and become insolvent,and therefore could not lend.Why couldn’t they refinance from the Central Banks?
So all this was caused by a current account deficit and debt service to the abroad?Debt service to domestic capital shouldn’t cause crisis as the money gets recycled.
But a lot of the banks had borrowed foreign money (global savings glut) through maturity instruments.The with drawl of which may not of affected their ability to originate loans (create demand deposit liabilities and borrow reserves from CB’s to settle interbank payments) but did affect their solvency.(Although I’m not sure why they couldn’t just rollover their debts through the central banks.(which is what Northern Rock did I think).
yesterdays blog showed that cost of borrowing doesn’t influence borrowing as much as business sales probably does(necessitating the need for a fiscal initiative to boost demand).But what causes housing markets to collapse,and highly indebted households and businesses to be credit worthy in 2006 to suddenly not be credit worthy in 2007/08 (when levels were marginally higher)(my point is what triggers everyone to realise “okay we are no longer credit worthy-lets stop borrowing” why doesn’t spending just continue non-stop?
Is a current account deficit the cause of a negative private Domestic balance…..but what does that even mean when USD doesn’t leave US banking systems?
The “inability” of modern governments and their controlling elites to run the economy better – to reduce inequality and improve efficiency in that order – is not located in any misunderstanding of the monetary system. They are not “unable” to run the economy better. They are unwilling to do so. The governing and oligarchic elites understand the monetary, financial market and labour market operations of this system very well. All of these operations, taken as a system, are intentionally designed to;
1. Maintain high unemployment.
2. Reduce wages.
3. Enable labour arbitrage to be used against workers.
4. Increase capitalist and rentier profits.
5. Keep populations servile (pacified or repressed).
6. Inflate asset prices (paper wealth).
7. Maintain the value of the money in which paper wealth is denominated.
Goals 6 and 7 might appear to be in conflict. Generally speaking though, a system is encouraged where the value of unproductive assets is continually inflated while standard goods and services inflation is kept low. Of course, these processes, which can enormously enrich the 1% and are doing so right now, are not sustainable long term. A crisis has to occur sooner or later.
The above processes, in combination, equate to a condition of overaccumulation of capital.
Keynesian and Marxian theories would both recognise that; “Accumulation can reach a point where the reinvestment of capital no longer produces returns. When a market becomes flooded with capital, a massive devaluation occurs. This over-accumulation is a condition that occurs when surpluses of devalued capital and labor exist side by side with seemingly no way to bring them together. The inability to procure adequate value stems from a lack of demand.” – Wikipedia.
Today, the capital which tends to get devalued is old productive capital, that is to say superseded productive capital. The process is enabled by rapid technological progress and planned obsolescence. Another word for planned obsolescence is waste. The other “thing” which gets devalued of course is workers’ labour. However, the FIRE sector (Finance, Insurance, Real Estate) is operated as a hedge against the devaluation of all capital. Financial capital is protected against the general devaluation of capital. Current policy, especially current monetary policy, is expressly designed to maintain the value of financial capital. In other words, the fictitious is valued over the real; paper wealth and non-producing assets are valued over productive assets.
The inflation of real estate and share prices is used to soak up the overaccumulation of financial capital. Real capital assets are either productive or non-productive. Productive assets are under-utilised (run under capacity) and non-productive assets are expanded. The system maintains integrity and stability (for the time being) by over-valuing assets, both productive and unproductive.
What we are dealing with here is the issue of fictitious capital. Again from Wikipedia;
“Fictitious capital could be defined as a capitalisation on property ownership. Such ownership is real and legally enforced, as are the profits made from it, but the capital involved is fictitious; it is “money that is thrown into circulation as capital without any material basis in commodities or productive activity”.
“Fictitious capital could also be defined as “tradeable paper claims to wealth”, although tangible assets may themselves under certain conditions also be vastly inflated in price.”
“Effectively, fictitious capital represents “accumulated claims, legal titles, to future production” and more specifically claims to the income generated by that production.”
“In terms of mainstream financial economics, fictitious capital is the net present value of future cash flows.”
All of the above tells us that if the future income does not eventuate (realise itself from real production) then the asset values will collapse at some point. It is hard, nay it is impossible, to see how adequate future production will eventuate from a process of expanding the financialisation of assets, underutilising real productive assets and misdirecting resources to unproductive ventures, for example constructing expensive unoccupied real estate.
The high productivity of our factories and technology, even when under-utilised, is one reason this process can now continue so long. The other reason is that there are still several billions of “peasants” or subsistence level people in the third world still to be brought into the capitalist system. This system could yet run a long time, in human terms, before being brought down by its internal contradictions.
Fortunately or unfortunately, there is an external contradiction which will bring the system down. This is its contradiction with the environment and environmental limits. We are very close to these limits. The real problem of course is that this system, as an article of ideological faith, completely denies that it has internal or external contradictions. Eventually, physics and biology refute all faith ideologies which are incongruent with objective physical reality. Our only real hope is to make predictions from validated theory (both Marxian and ecological theory) of the salutary disasters which will inevitably occur under this system. When predictions of theory are born out and validated by actual events (the salutary disasters) then the theory gains widespread credibility and acceptance. Finally, the people may realise the real and dire facts and be motivated to change the system. Before then “words are wind” in the words of lowbrow author George R.R. Martin.
“Is it correct to say that when someone deposits money into a savings account that the bank uses the money to buy Treasuries”
Not really. When a bank creates a loan it creates the equivalent deposit at the same time. The majority of a bank’s assets are loans with the deposits providing an offset for those loans.
If government spends then it creates deposits in commercial banks, but it also forces the bank to lend to the government (that’s what bank reserves are – the commercial bank lending to the central bank).
In that case the bank is likely to take their loan to the central bank and swap it for a loan to the Treasury – because the Treasury pays a higher interest.
But it may also be the case that the person holding the deposit decides to swap it for a loan to the Treasury – because the Treasury pays a higher interest. In which case the bank deposit *and* the equivalent bank reserves disappear from the bank.
So it’s more the case that the interest on commercial loans pays the interest on deposits. The ‘loan’ to the Treasury in the form of bonds held doesn’t pay much at all – and increasingly is becoming a cost.
Neil Wilson
Thank you.
Neil,
What is your view on the following:
I think by now the vast majority of observers (although not the Minneapolis Fed it seems) have caught on to the fact that banks don’t lend out reserves, so that’s barely even a point of contention any more…Far more interesting I think is the “wealth effect” view that is briefly summarized in the middle of Bill’s post:
“The stimulus, if there is any, comes from the fact that by pushing up the demand for bonds, the central bank drives down the yields on those assets at the relevant maturity, which then influences other returns on financial assets within that maturity range.
So, the presumption is that lower interest rates then stimulate private sector investment spending and/or reduce the value of the currency (financial flows move out in search of higher relative returns), which stimulates real GDP growth”
I believe Bill, and MMT in general, are skeptical of such a view.
Bernanke himself, as far as I could tell, in his last few speeches on the subject, made no mention whatsoever of the quantity of reserves being relevant in any way and explained the usefulness of QE purely in terms of its effect on asset prices (the “portfolio channel” as he calls it): basically, the Fed drives down yields across the Treasury curve causing a “reach for yield” in the market as a result of which risk premia across assets (equities, real estate, corporate bonds) go down and then two things happen: 1) risk asset owners feel wealthier and start spending more and 2) companies’ have cheaper access to capital, and, other things equal, a stronger incentive to invest (Note that increased bank lending need not be part of this)
But one can extend this logic further. If one believes that QE indeed forces a “reach for yield” (it absolutely did in financial markets i think) then, other things constant, it should also be increasing banks’ willingness to lend (if that’s important to you for whatever reason). Not because QE increases reserves, but via its pricing effect in the Treasuries market. A loan is just an asset on a bank’s balance sheet the same way a Treasury bond is. If the expected return on a Tsy holding becomes unattractive enough, the bank might shift the composition of its assets towards fewer treasuries and more loans (and other risky assets). This is completely analogous to how a fund manager might sell his Treasuries if the yield becomes unattractive and replace them with, say, corporate bonds.
I have no doubt at all that banks want to lend more.
But they are up against the only limiting factor of banks – enough creditworthy customers able to pay the price of money.
You have to remember that banks make money on the margin. They don’t actually care about the price charged or the price they have to pay for liabilities as long as they maintain their margin. And they have to maintain their margin because that is the insurance premium against bad debts.
So if you squeeze bank’s margins which is what happens when there is no demand for loans, then the bank has to be ever more cautious about who it lends to so that the insurance of the margin covers the risk of bad debts.
Banks are having to be careful of securitisation of mortgages. The ‘toxic mix’ approach is no longer acceptable to those funds buying the securitised mortgage bonds. There is much greater scrutiny of the quality of the mix. Any bank found to be polluting the water here would have their securitised mortgages red-lined – with disastrous impacts on profits.
The problem is that the Bernankes of the world, and half the economists, think that the world works on interest rates and only interest rates. It doesn’t.
Now interest is so low, the dominating factor is uncertainty. Banks and financiers are acting more like insurance underwriters than anything else.
There is a verry clear and interesting paper from Richard Koo on this subjekt:
Central Banks in Balance Sheet Recessions: A Search for Correct Response Richard C. Koo
http://ineteconomics.org/uploads/papers/Koo-Paper.pdf
quote:
In acts of desperation, central banks in the developed world have
flooded the financial system with liquidity in a policy known as
quantitative easing or QE. In spite of massive injection of
liquidity, however, credit growths in all of these countries, the key
indicator of the amount of funds that was able to leave the
financial system and enter the real economy, have been
absolutely dismal. p. 4
In particular, central banks can only increase monetary base via
QE. It cannot directly increase money supply which is the actual
amount of money the private sector has to spend (or play in the
foreign exchange market). For the money supply to increase,
someone has to borrow and spend the liquidity provided by the
central bank in the process known as money multiplier. But if
the private sector as a group is saving money or paying down debt,
money multiplier is negative at the margin, and a growth in
monetary base will not necessarily translate into a growth in
money supply. p. 6
With monetary policy largely ineffective, the only policy left to
keep the economy away from a deflational spiral in this type of
recession is for the government to borrow and spend the
unborrowed savings in the private sector. p. 10
When the effectiveness of monetary policy depends on the size of
fiscal stimulus, it should be the responsibility of the central bank
to inform the public and policy makers that the government
should not move toward fiscal consolidation when the private
sector is repairing balance sheets. Such persuasion is essential
because the average public is still unaware of the disease called
balance sheet recession. They are unaware because the schools
have never taught them about the possibility of such recessions. p. 16