Central Banks And Policy Communication: How Emerging Markets Have Outperformed The Fed And ECB | A Workshop Series On Using Text As Data In Policy Analysis

>> Steven Davis: Welcome to the Hoover Institution workshop on using text as data in policy analysis. My name is Steven Davis, I'm here today with my co conspirator, Justin Grimmer. Both of us are senior fellows at the Hoover Institution. Betsy Phillips and Kelsey Kimball are managing the technical side of things in the background, and Cecilia Chin is our organizer.

Today's workshop features a recent study titled overtaking the how emerging markets have outperformed the Fed and ECB. And we're delighted to be joined by the full complement of the four authors on this study. They are Alina Rybakova, senior fellow at the Peterson Institute, Perozhka Nej Mohakshti, a visiting professor at the London School of Economics and Political Science.

Tatiana senior economist at the Joint Vienna Institute, and Olga Ponomarenko, who is head of quantitative analytics at Caplite. So before I turn it over to Alina, who will start the presentation, let me just say a few words about our format. So Alina will take about 30 minutes to present, and then we'll turn to a discussion of the paper, kind of a back and forth Q and A.

If you have a question or comment, please put it into the Q and A box so that Justin and I can review them and decide who goes next. We will run for about 60 minutes in this recorded session. After that, we'll turn off the recording and continue with a more informal discussion for anyone who wants to stick around, okay?

So, Alina, please take it away.

>> Elina Ribakova: Thank you so much, it's a fantastic series. I'm really delighted to be part of this series because you see here the team of economists that have worked with this data. This is a new way of looking at the same data. And the fact that Taixt is data, I think, is completely transforming our area of research.

It is transforming also political science and many other areas, which I think is fantastic. And although there is sometimes resistance in the corners of our profession, but I think as this becomes more mainstream, I think we'll all have new research questions and new answers with this fantastic data.

I will be presenting, I also asked my colleagues to present as well. I know we only have 30 minutes, so I will dive right into it. And I also look forward to Q and A. So let's start with, what is this paper about and what is not about.

This is not a market paper. We're not trying to say what is the best way of forecasting the data. This is a way of presenting sterilized facts and some relationships, using new analytical techniques and analyzing over 22 emerging markets and developed markets, central banks. Although of course the definitions are becoming increasingly blurred between developed and emerging markets.

So what we do, we look at the statements. Not every central bank will have that statement specifically as we're used to it, but a great majority of them will. There is an announcement, the statement comes out, explains why the certain policy decisions have been made, where the rates maybe changes.

Talks about economic outlook inflation, and also sometimes talks about the expectations for future central bank action, maybe even forward guidance. So we collect the statements, we collect them from the central bank websites and then we put them into analysis. And therefore, if, for example, you or me, we might have felt before, emerging market, central banks talk more about exchange rate.

Well, we can put that to the test. We can look at the numbers and see are they really talking more about exchange rates or are they really talking more about macro prudential and compare them across central banks. Of course, emerging markets is a very diverse group of central banks.

But for the sake of this paper, and I do recommend everybody to go and read in the paper and also the code and the data is also available, so we can have a fantastic exchange afterwards. Well, there is a diverse group of central banks, but this is still relatively new area of research and that's why we put them together as one group, and the subsequent research will also present them by region.

So as I said, this is about policy analysis, it's not about trying to predict central bank action. Probably the key message here is that emerging markets have done a lot to catch up with the developed markets. It's not to undermine the work by the developed bankers, by the feed, the Fed and the ECB, which all, many of them have done their own policy reviews recently.

And we're also seeing dramatic improvements in the quality of communication, for example, the clarity of ECB statements after their policy review. But what we want to say is that among these 22 emerging markets, outside certain, I don't want to call them basket cases, but Turkey's, Argentinas and whatnot.

But many of the core central banks have caught up in the way that they transparent about communication. They're clear in their statement and also better at saying that they see inflation, they plan to act and then acting upon that. So that's what this presentation is about. I don't want to spend too much time on it, we all noticed inflation.

And no matter where you live, especially if you're a student of research, I'm sure you have noticed inflation. And we've also noticed that a lot of central banks, especially in emerging markets, seem to have reacted sooner to inflation. And it's not just because the inflation has picked up more.

I think the pickup inflation has been rather uniform. It just seems that they have indeed reacted faster in this statement and maybe sometimes in the action. Brief review of the previous literature, we have seen a fantastic paper by the BIS. There are a few others we have seen also certain papers maybe look at individual central banks.

But we think that this is the first study, to our knowledge, of looking at the comprehensive data set. It goes for many years, covers more than 20 central banks, and it compares developed and emerging markets. I think probably the closest to it is the BIS. Most of other papers tend to look maybe just at Brazil or just at Mexico or many studies look at the, at the Fed, on the ECB.

They don't have this comprehensive analysis. The range of policy or sort of technical approaches. Hold on, I'll just go very, yes, the range of them, of methodological approaches. There is a wide range. We look at the transparency, we'll look at the readability, we'll look at the certain topics which are covered.

We'll look at the fact, whether they notice inflation before inflation happens or actually they respond to inflation, we'll look at the policy response. And then we also look at the very specific emerging markets versus developed market topics. Maybe emerging markets indeed talk more about exchange rate, they talk more about the supply side factors.

They didn't use as much forward guidance. And actually it was very helpful for them not to use so much forward guidance. And then also maybe they refer more to reliance to th relationship between fiscal and monetary policy and the coordination between these two policies. In terms of the technical analysis or technical tools that we apply here.

We do start from the very simple that many of us seen, doing the sentiment analysis, doing the topic decomposition. And then we go to the most recent techniques, which are available now with chat GPT, using the Bert model on the pre trained model that some of these tools provide, and also trying to use better prompts.

So basically here you have the the full range of the techniques, and we're happy to discuss that. I think the key takeaway, and I'll start with the transparency and then pass it on to Tatiana to do more. And Peroshka and Olga to make some comments, is that we have seen big improvement in the transparency index and the clarity of the communication.

And you can see that the composition of the improvement in transparency score. So the overall transparency of emerging markets has improved and we also see what is contributing to that decomposition. We have noticed, of course, emerging markets are not all the same. We have seen, not surprisingly, maybe some central banks like the Czech Republic, also Hungary, South Africa and a few others have improved more than other central banks.

Then there are others that say Nigeria, that might need much more to catch up with the developed markets. But we see some central banks no longer really becoming emerging market central banks. And indeed in case of South Korea, it's more tends to be requalified into neutral category or developed markets as well.

So with that, I will pass it on to Tiana to make comments.

>> Steven Davis: Can somebody just say how you measure transparency?

>> Elina Ribakova: Let's sum, so the trans.

>> Steven Davis: Coming up that's fine, but.

>> Elina Ribakova: Yes, it is coming up.

>> Steven Davis: Okay.

>> Elina Ribakova: Yeah, so I'll pass it on to Tatiana for the next slide.

 

>> Tatiana Evdokimova: So maybe let me first answer this question. So basically, we do this for transparency we just use the well known index developed by Jinser and Eichengreen. So this is not our invention, this is something that is for a long while on the cards. And they have a database where they basically collect a lot of information from websites of the central banks in terms of what exactly they disclose.

Whether they disclose models that they use for forecasting, how well they are explaining the motivation for their decision making. So this is an index which we just used to start and to show that.

>> Steven Davis: To checklist, and how many do you disclose your forecasting.

>> Tatiana Evdokimova: Exactly?

>> Steven Davis: Do you state your objectives clearly?

And so on okay, got it.

>> Tatiana Evdokimova: Exactly, so like dummy variable. So with that, we just saw, it was a starting point for our paper where we saw that indeed emerging markets caught up really well in terms of transparency with advanced economist central banks. But then once we get hold of our huge database of central bank statements, we start applying different techniques of text analysis.

So we start with the simplest things like readability, where basically you measure the length of sentences and the length of words that are used for communication. And we see that indeed the advanced economies central banks, in particular, the Fed & ECB, etcetera point, got really hard to understand.

So here you see that these two central banks require a very high level of education to understand what exactly they are talking about. And this is in contrast to emerging markets central banks, which are depicted here with the red line. So on average, for understanding emerging markets central banks communication, you need a little bit less education, so these central banks are more readable.

But this is gradually changing, and the Fed & ECB actually got to, they understood that they are becoming more and more complicated for understanding for the general public. And since the recent reviews, they made big effort to improve the quality of their communication, so this change, this gap has closed lately.

And on average, we don't have no longer see such a huge difference between readability. So when we dealt with readability, we moved to sentiment analysis, and there we use quite standard techniques. So basically what sentiment analysis is an analysis that allows you to figure out the tone of the speech of the central banks.

And in particular, we were interested know whether central banks are becoming more whole. Start with a very simple technique of sentiment analysis using dictionaries approach. So basically, we have a list of words that we consider keywords like prices, inflation, unemployment. And once you identify this keyword in the sentence, then we look for so called modifiers, which are usually adjectives or verbs that accompany this keyword.

And with this modifiers, we can understand whether the sentence is more hawkish or more dovish, so we give some examples at the bottom here. And you can see, for example, downward pressure on prices. So prices would be the keyword, downward would be a modifier. And basically you could see that in this combination, this was viewed as a dovish sentence, in a sense that it creates higher chances for the central bank to cut interest rates.

So once we graded all the sentences in the statements as dovish, hawkish or neutral, we also calculated an average over the statement, and we came up with the level of this sentiment index. So, if you can go to the previous slide just for a second. Yeah, so here we gave a few examples for this workshop on what results, hawkish or dovish, we got for different types of sentences.

So, for instance, job gains have been solid, this was classified as relatively hawkish, implying that it increases chances for rate hikes in future. The opposite is true for muted inflation pressures and so on. So, in the paper, we give all the dictionaries, and actually this is the main part of this analysis.

The quality of the analysis crucially depends on the quality of the dictionary. If the dictionary is not full, then you can hardly come up with good results. So once we had these dictionaries in place, we analyzed all the statements and tried to figure out the tone, the sentiment, and of course, we went deep into history, from 2006 or even 2005.

But we were mostly interested in and how the tone of central banks was evolving around the pandemic, and in particular in the face of the post pandemic inflation surge. So here you see our chart of sentiment, the higher the number the more hawkish the tone of the central bank, and the lower the number, the more dovish the tone is.

And what we figured out is that emerging markets turned hawkish much earlier than advanced economies central banks. So this is this red line which goes into positive territory, actually one year ahead of the advanced economies central banks. So this is clearly a more successful outcome for emerging markets who managed to react more timely to inflation surge.

And they had good reasons for that, because they have less well anchored inflation expectations, they have lower credibility, they cannot waste their time. They have to actually react and prove that they are very focused on inflation. So this was not something that was observed just for a couple of central banks, but for different regions.

So here you can see very high synchronization in terms of different central banks from different regions. All of them had a turning point towards more hawkish communication early on. And they remained in the synchronized mode, because the shock was truly global and we were all confronted with supply chain disruptions and higher commodity prices.

And also, if just briefly, Aline, you could go to the previous slide 1 second there, interestingly, the previous one. So we sometimes get this question of whether maybe emerging markets just turned more hawkish earlier because they saw inflation earlier. Or inflation just materialized earlier in these countries, but in fact not.

And here you see that sentiment. So the tone of central banks in emerging markets, the black line, turned positive ahead of inflation. And in case of the Fed, it was vice versa. So inflation was accelerating for quite a while, and only then the Fed caught up with more hawkish communication.

So then we go further and we try to figure out what exactly was driving this sentiment and what were the key features that were on the radar of central banks. So we decomposed the sentiment into different topics, and with that we basically can read what is on the central banker's mind.

In case of the Fed, we see that labor. So communication on the labor market plays a very important role in determining the stance of monetary policy. So prior to the pandemic, we see that a lot of hawkishness in the United States came from a very tight labor market.

And during the pandemic, the Fed had to resort to forward guidance, which was dovish, and also it had to continue talking about quantitative easing. And even though the Fed spotted inflation early, so inflation is the red zones here. It spotted it in early 2021. But the fact that it was so dovish on many other topics, it was still dovish about economic activity.

It was caught in the very dovish forward guidance. All that factors combined led to the fact that that overall, the Fed remained very dovish, even though inflation was already on the radar. We look at the same charts for all central banks, but here we just show ECB. So same story.

Inflation was spotted early on. So the red part of the chart becomes positive in early 2021. But again, the central bank was so caught in the dovish forward guidance and still had a lot of worries about the sustainability of recovery, that overall it remained dovish for too long.

And also here we can see what kept the ECB awake at night throughout this period. For instance, after the sovereign debt crisis between 2012 and 2018, there is so much worry about very weak economic activities. These dark blue parts here show exactly this on the chart. When we look at the emerging markets, their communication is more one sided.

So the message is clear and it goes in one direction. Both economic activity and inflation are sending similar signals in terms of what the central banks should do. So here we see that inflation became visible for the central banks early, and they acted upon this decisively, and they used much less of a forward guidance.

Only the central and eastern European countries did so. And then, so once we were done with this part of the analysis, we also looked at what we call See-Say-Act when we compare what central banks talk about and what is actually materializing. So we compared inflation in communication of central banks.

We call it inflation sentiment, and we compared it with actual inflation that is observed in the United States and in all other countries as well. So what we notice, a bit surprisingly, is that central banks, on average, react to inflation when it already materializes and they are not able to warn us about upcoming inflation.

We show it, we find when the correlation between what central banks tell and what we actually observe is highest, and it appears that correlation is highest when we lag communication a little bit. So there are a few exceptions, but mostly we see that central banks are not yet very good at forewarning the markets about inflation.

And this is one of the policy lessons that we see a lot of room for improvement on that. And we also look at how sentiment is correlated. Yes, just one slide back. Elina, yeah, thanks. So we look at the sentiment and the rates. So how correlated are central bank actions with what they promise actually in their statements?

And here we see that indeed, most central banks warn the markets six to ten months ahead about their future actions. And the correlation between these signals and actual actions is very high. In the bottom chart, we show that it's frequently above 60%. In particular, that's the case for United States and the ECB.

And emerging markets, on average, are less good at walking the talk. So they follow what signals they showed earlier, but less precisely than the Fed and the ECB. And to some degree it's understandable, because emerging markets are exposed to much higher volatility, and maybe they have to change the course of action more frequently.

But we also see this as a bit of a risk for credibility. And one of the conclusions is that central banks in emerging markets should follow more closely in their actions what they promise in their communication. With that, I think I'll pass the floor to Olga.

>> Olga Ponomarenko: Sure, thanks, Tatiana.

So, yeah, when we were working on crunching the data, we were obviously aware of the large language model revolution that was happening at the time after the release of ChatGPT. And we were also very, very curious as to how those methods and techniques would show up in the analysis, what conclusions there would be.

Even though the main method we're using is this dictionary approach that Tatiana was talking about. Where we have hawkish and dovish words and we try to balance and estimate the sentiment of central banks. We also wanted to see how do the machine learning or LLM methods work and perform here.

And the two that we're considering is first, embeddings, and the other is processing through Chat GPT. And I'm gonna share some interesting findings. So what is an embedding? Embedding is a vector representation of the meaning of a word or sentence or a paragraph, any body of text. So vector is a numeric object that is very, very easy to analyze and correlate, unlike words, right?

And embeddings are an internal representation within all those sophisticated models. So we used Bert and OpenAI. The conclusions were very similar. And originally, when we approached this, we thought, well, embedding will be just a robustness check that probably will confirm whatever we find through dictionary approach. But we're actually quite surprised to see that there was a whole spectrum of how embeddings correlate with our dictionary metric.

So dictionary has been the golden standard in communication research for probably decades from now. And what we find was very interesting, that for some countries, the correlation between the sentiment metric that we get from dictionary. And the sentiment metric that we get from embedding were not always very, very correlated, and United States were interestingly, an exception.

And you can see on the chart on the left that red and gray lines sometimes are correlated. Sometimes they diverge a lot, especially in the latest episode of inflation, such that overall correlation is very, very low. They're almost unrelated to each other. And we, of course, started thinking, okay, why is this happening?

And our conclusion with this was that it is about this ambiguity of communication, or how complex the communication is. So embeddings famously don't do really well with negations. So if you're constructing a sentence and you say inflation is high versus inflation is nothing high or low can throw off the metric a little bit.

And on the previous charts, we showed that fads communication. It is pretty complex when we do the topic decomposition, that sometimes Fed can be hawkish about one thing and dovish about the other. And overall, this results in this low correlation, and that embeddings still need to be used sparingly.

And researchers probably should continue relying on the dictionary approach, even though there is a bit of subjectivity, potential subjectivity with dictionary. You just trying to guess what words the central bank is usually using when they're preparing the statements. And of course, every person has their preference of the words, adjectives, nouns and verbs that they apply.

But then the hope was for ChatGPT. If we use ChatGPT with a prompt, and by the way, prompts prompt engineering is still an evolving research area, but broadly, I think there's a consensus that when you construct a prompt. When you talk to ChatGPT, you want to assign it a role so that it knows how educated it needs to be in a certain topic.

So here we asked it to be an expert of central bank communication, and we basically asked it to simplify the statement, do you think inflation is high and the federal high crates, or do you think it's not gonna happen, broadly speaking. And once we pass the Fed statements through this pre processing with ChatGPT.

And then apply dictionary metric to those outputs, then the correlation goes up significantly from 13% to 40% and almost goes in line with other countries. So it is about the complexity of communication that we find results in on this, I think I will pass to Peroshko to continue with topics.

 

>> Piroska Nagy Mohácsi: I'm just mindful of time and then we also should go over the policy conclusions as well. So, Peroshka, let me know which slide you would like me to jump to at the moment. Yeah, I'm happy to take it up and really go very quickly through the specific topics, that particular matter for emerging markets.

So exchange rate, that is something that in theory, for inflation target countries. You wouldn't expect to appear very often in statements, and indeed in advanced countries, as you see it on the left hand side with the gray and black light, hardly any reference to that. But emerging markets do refer to the exchange rate.

And the reason is that in the global financial cycle, the dollar led, or a little bit euro led global financial cycle, and even the size of the capital flows and vulnerabilities. It is very normal that they refer to exchanges, even though they are kind of formally inflation targeters.

On the next one, supply side factors. This is an area where adverse countries have changed a lot. Traditionally, emerging markets would look quite a lot at supply side factors. It shows in the statement, and we know in explaining decisions that it is important factor. And we believe that perhaps this was one of the reasons, their focus on supply side factors, one of the reasons why they picked up the inflation signals that reacted faster than advanced countries, ECB and the Fed.

So there has been a change, both the Fed, the ECB, the bank of England particularly, that came out Governor Bailey, with a big speech a few months ago that with the title supply matters. So they took the lesson at heart and they look at supplies and stuck it, because in the end, inflation is a balance, however generated between supply and demand.

Okay, and the next one forward guidance. There's something that Alina already has mentioned and we very strongly believe that forward guidance that of course was developed after the global financial crisis being close to the lower interest rate bond. That might have had some positive role sort of giving additional assurances of policy action beyond what the data would have in flight.

That was the essence of it, right? But it certainly didn't serve well. Countries in the run up to inflation and to the reaction of that Rat Reserve bank of Australia was stuck with this statement made I think in September 2020 the governor did that they would keep interest rate law until 2024.

Well we know what happened and the governor was replaced and the framework revised. Luckily emerging markets never really, never were in that deflationary context but never really used forward guidance in the specific way as advanced countries did and even though they did more than before never in a fully committed way.

Next one is macroprudential policy. Now this of course, we know again supposed global financial crisis, this has been a. Focus of central banks everywhere and the focus on financial stability was enhanced through the development of macro potential policy. Obviously, the jury is asked whether this has worked or not.

In many respects, yes, but in some respects we see the mid sized bank crisis in the US just a few months ago. It may not have been enough. It is a good thing that emerging markets focus more on macro potential policy. We started to look at this, but of course we are constrained by the fact that monetary policy statements may not include all, or may not include at all reference to macro potential policy particularly in countries where the monetary policy central bank is separated from the Financial Services Agency.

It's an ongoing research for us. And finally, fiscal policy. How much references is there? There is a reference to fiscal policy in the ECB statements, that's the gray line as well as emerging markets red again, very little in the case of the Fed and nothing on coordination. And we believe, and we will come to the conclusions.

Really this is not a good thing, because what has happened already during the global financial crisis, but really in reality, big time during the pandemic, rightly so, that there was a very, very close collaboration between the monetary and fiscal authority. Everybody knew about it, but it was not communicated in the statements interestingly.

So, there was a lack of transparency, and we believe that's not a good thing. In the end, what really matters is the policy mix, and this is where we will come for the lessons. So, in sum, where does it take us? Emerging market central banks have adopted many of the principal advanced country central banks, both in terms of policy conduct and their communications, but very rightly with modification that would reflect their specific circumstances.

And these definitely concern exchange rate volatility, given that they remain vulnerable to capital flows, financial dollarization, and they use the forward guidance accordingly much more vaguely in the context of still weaker institutional credibility than advanced country counterparts. We've demonstrated that emerging markets have improved transparency, some of them fully caught up.

It's a big area of improvement readability also that the gap has closed. It was a different type of gap, but not central banks are on the same page across the globe. So overall, emerging markets, we hope to show it through communication, but also through policy, that they outperformed the Fed and the ECB in fighting post-Covid inflation, even though inflation broadly picked up at the same time, global inflation, it was, they reacted faster and communicated better.

And they, remarkably, even at the time when the Fed and ECB was already tightening, remarkably, they had no banking sector stability. This is at first in emerging market history. And then, we have highlighted, given the audience here, that the Fed policy and communication has had weaknesses. And obviously there is a lot of soul searching going on.

We just want to highlight, as we did with the dictionary method and others, that the dual mandate, why obviously this is a political mandate, it's fine. But at a time when there is a big stress and inflation shoots up, focusing the dual mandate and focusing too much on the labor market, maximum employment may have undermined the capacity of the Fed to communicate effectively on inflation.

We also wonder with many others.

>> Steven Davis: We should wrap up Roshka, because we're already well over time.

>> Piroska Nagy Mohácsi: Okay, then we talked about forward guidance and let's move further. Next one, Elena. So inflation projection, as we have shown, should be improved significantly for everyone. Markets are not that, central banks are not that good to follow up consistently on what they signal, so they need to work the talk better.

Forward guidance is detrimental at the times of rapid change. That reduced policy effectiveness in the post-Covid inflation and emerging markets have been better using it less strictly. Central banks with multiple mandates, and it's not only the Fed, there are some central bank with three targets, some four. If you add karma change, the multiple mandates really have to communicate it much more clearly during a time of stress when the calls may conflict.

And finally, I will be very fast because we have already mentioned that maybe I just wrap it up. There is a lot of more details if we want to touch upon during the discussion. Thank you.

>> Steven Davis: Great, thanks so much. So let me start off. When I think about what you've done, I kind of divide your work up into two categories.

And I'm talking about the text based analysis. There's the characterization of things like readability, transparency, and what topics central banks are communicating about. Those are relatively straightforward to analyze with text based methods. But then when you move on to the efforts to make assessments of performance, that's much, much trickier and I'm less persuaded by what you've done.

And, I think it would be helpful to be a little more put it on the table, exactly the extent of the challenge that you face in that regard. So, in particular, you have few macro episodes within your sample per country, okay? You can have unforeseen shocks. You can have state contingent aspects of monetary policy that aren't well captured by your text based metrics.

You rightly are considering heterogeneity across countries. So even the ability to limit to pool across countries is limited for the purposes of your analysis. So we know, for example, that in any kind of machine learning setting for example, the models typically need a lot of data in the sense of specific instances that are relevant to the performance evaluation in order to accurately assess some high dimensional objective and whether that's being met.

That's certainly the case, the task facing central banks is very complicated. So I just think I didn't see enough of like, look, even the best machine learning model I might possibly bring to bear here to evaluate the success of the central bank and predicting inflation. Or giving advance warning to its interest rate policy.

It's a very difficult undertaking, cuz we only got a handful of big shocks during this period. And so we can't tell very well whether it's just a truly unforeseen event outside the scope of previous experience. Or we should have used state contingent forward guidance when we didn't use state contingent.

So I just find all of that and there's not really a very, I didn't see, maybe I missed it, like when you made these claims about. Whether central banks are giving advance warning to the future conduct of monetary policy. Well, maybe they're giving advance warning in the form of state contingent statements about what they promised to do, and I don't think you're measuring that.

So there's just a lot, I don't know if I'm making it clear, there's this straight what are they talking about versus assessing their performance? And the latter is just a very difficult task with the data we have available. And in that respect, I find some of your claims overly strong relative to the evidentiary foundation.

 

>> Elina Ribakova: I really like that comment, maybe I'll start. And I think that's why we sort of try to start in the beginning by saying that I should have, I talked a little bit more about the markets, but also about the policy conclusions. We don't want to jump to policy conclusions too fast.

The contribution of this paper is more about stylized facts using the available hammut of all possible methods which we have applied here. That takes us beyond the, this is somebody did the sentiment analysis on the Fed because they want to forecast it, because they work in Goldman Sachs.

This is somebody who did some analysis on Brazilian central bank because they're good at that and they're a few, or Turkish central bank. Ironically, there actually is somehow some analysts who are very skilled at these techniques and they look at the Turkish central bank, which is, as you can imagine, that's a long stretch to do.

So the idea here was trying to put this methods, put a whole range of different central banks and see what we can see, rather than trying to then do deep analysis yet, and then make deep conclusions. So here the policy conclusions is from coming from the sterilized facts rather than, say a deep learning model that shows you the Fed rate translates like this to the market rate.

And translates like this to potential inflation. And there are some papers, of course, trying to do that specifically. So your comment is very valid, the field is vast. And that's why also we put the codes out there so people can see what parts are done and then maybe take out a part of the code and then try to run with it.

And that's why also we felt like we needed to close this paper because otherwise we just, it will keep on growing and growing. We cut it substantially before publishing, and it's still a very long paper. So, yes, I think in the paper itself, we do try to give the legal disclaimers upfront saying that, look, this has stylized facts to the best of the programming ability that we have here and the methods.

But let's also not jump to big conclusions, we need to do more work, and I see Piroshka nodding, I don't know if she'd like to add more.

>> Piroska Nagy Mohácsi: No, I very much agree, and I fully subscribe to what Steve says, that the central banks have a very difficult task, but that's their job, that's their job.

So they have a mandate, and many of them just focusing on Inflation as opposed to the Fed, which has, of course, the dual mandate, and that's the job that they have to do. And honestly, whether this is this Analysis or one looks at other outcomes, Inflation was missed, I mean, there is a lot of source searching.

So I like very much your comment, and we have to think it's the state contingent forward guidance, but of course, then it's really regular communication. But in that already there's an implicit recognition of the problems that is inherent in the kind of initial type of forward guidance that was provided.

One more thing, know how you're absolutely right and put your finger on the right point, that assessment of performance, we have to improve on that, sort of make it richer. And we are thinking, and then these seminars like this one, really helps us to see what are the elements of effectiveness, so to speak, and the quality of communication that's kind of on our agenda.

Definitely improving on the CSI act analysis, for example, that we have.

>> Steven Davis: I got more questions, comments, but time's limited, so I'm gonna see what Justin wants to say here and then we'll open it up to the broader audience.

>> Justin Grimmer.: Thanks, and I'll apologize for definitely knowing the least about macroeconomics of anybody whose pictures on the Zoom call here.,.

But my question really is about measuring something like readability, and then applying that to the audience who is gonna be consuming these central bank statements. And so I think you're using flesh Kincaid, which it's been around for a while, but you can imagine, for a variety of reasons.

The target audience for central banker statements might be okay with longer words or more complex statements. But I would imagine something like readability for that audience would, would be something like, do two actors who need to make a private actor who need to make a decision based on central bank communication.

Reach the same conclusion about either the central bank's planned action or the current state of the economy, is that revealed in the same way? Or is the sentence so jumbled that it unintentionally obscures what the central banker was trying to say? And you could imagine, for lots of reasons, that could be very different than a flesh Kincaid score.

And I do think that that then sort of ties into some of Steve's points about performance, because, well, actually don't know anything about central banks. But I'm taking from the communication that it's good to be more clear, although it seems like sometimes maybe you wanna be less clear, but I don't know.

And so you can imagine that if you wanna speak clearly, we really want a measure that says actors working on this information all reach the same conclusion about the state of facts. Don't have to take the same action, but we all agree on what the, the state of affairs is based on this speech.

 

>> Elina Ribakova: I think it's a great suggestion, especially around the tape potential, right? When all the actors read very different things, and although actors seem to have read it the same way, which was wrong relative to what the Fed was trying to say. So, no, I think the reason we first went to the easier readability index is because the desire of the central banks to communicate about more things and to a broader audience.

And is their style of communication following that? And then also, now the central banks, of course, go beyond the statements, they make little videos, they make kids books, and other things. So we don't include that in our analysis. But if the objective is to reach broader audience, then the statement also should be a bit more accessible.

Or maybe the level of education of the market participants in emerging markets is not the same as in developed markets. So how does that compare? So that's how it went first to that. But I think your suggestion about whether to two people who are supposed to read would reach the same conclusion, I think it's a more sophisticated way of addressing this issue.

Thank you.

>> Steven Davis: Okay, great, we've got an interesting question here from an anonymous attendee. So I'm gonna paraphrase slightly, it starts out with the observation, which I think you'll all share, that the Fed is like the elephant among central banks around the world. That's my term, not the questioners term, but what that means is that monetary policy actions by the Fed have an outsized effect on countries around the world.

And can shock the environment facing especially emerging market central banks in a way that might undermine their performance or drive their performance. And so, how do you separate that out? So, I guess the Taper Tantrum would be an example, a very vivid example of where Fed monetary policy actions, which might have had minor consequences in the advanced economies.

Had huge consequences for some emerging market economies, might have made their performance look bad, but not because they did something, it's because they were reacting to the Fed. So, can you speak to that issue?

>> Piroska Nagy Mohácsi: Maybe I kick off and let the others add, it is absolutely correct, but we have seen a historical break in this, in how emerging markets were able to manage exactly this tension, right?

Sort of reacting to the fallout that comes from the global financial crisis cycle. And before COVID basically, that was the thing. So the Taper Tantrum was an example, but every single, even the global financial crisis is kind of a good example for that, how to manage hackroom. There are kind of more sophisticated research that shows that better performance manage better the Taper Tantrum, doesn't matter.

Fundamentally, the question is very radical. Interestingly, with the pandemic, we have seen something different. Emerging markets, first of all, were able to do countercyclical policies which never ever on a big scale happen. They even did, some of them, particularly central eastern Europe, but also later they did quantitative easing to help manage the pandemic.

And then in the downturn, as you see, when the US, the Fed finally tightened, the ECB tightened at the same time, so really squeezing emerging markets from both ends. They have been able to follow their own policies and reduce inflation, and in fact, some of them easing now.

So we have seen-

>> Steven Davis: On the comparison between the Taper Tantrum and the COVID response and the better performance, better outcomes in the emerging market, central banks and the latter. Can you trace that to their communication policies, or is it something else?

>> Piroska Nagy Mohácsi: Two differences, but the communication policy has really improved.

I know somebody is putting balloons on, but it's not me. So two things, one is that this whole thing, what we try to show, that their overall framework has really improved. And Alina has a very nice article back March 2020, I think, on that. But in addition to that, we have seen something new, and that is the Fed and the ECB's currency repo and swap operations, which really became global.

So unless you were the central bank of a politically unacceptable country, say, Iran, you would be able to access those. And the Fed very nicely introduced this with the spillback argument, that spillovers and spillbacks, and we are part of an integrated world. But it is the first time that ECB did very much the same for emerging Europe, but the first time that a global central bank, the Fed and the ECB, offered these crisis instruments which were used and then not used.

Because once you know that it is available, it's a preventative measure, nobody will go against the Fed in little Colombia or whatever that may be the case. So I think it's both the improved framework, communication is part of it, but also, very importantly, the support that emerging markets got through the currency swap operations.

 

>> Steven Davis: Okay, I'm just looking at the next question here. So, Sir Malvaro Ortiz, some of the Central banks use very short statements on monetary policy, two to three paragraphs, much shorter than the big central banks used, and sometimes even in their original language. How do you deal with this heterogeneity in the form and here, in particular, the length of these statements?

That's question one. Question two are the particular advantages to shorter or longer statements.

>> Elina Ribakova: First, we want to say hi to Alvaro, who does great work in this area. You've probably heard something about Turkey as well. I think this is the very typical question that we get in terms of the local language.

And then our thinking is that, look, we have increasingly emerging market central banks communicating to international audience. We have to assume that their translation, official translation, reflects accurately what they're trying to say. Even though from certain languages, the best domestic central bank translator might still translate longer sentences into English than not, right?

Because they will have local staff working in translating. So it's very hard for us to clean it out. I mean, we could try to translate it ourselves, but it doesn't produce better results. So we're just stuck with using the official translation, with the disclaimer that this is the best we can do, and that the central banks are trying to improve their communication that way.

In terms of the certain central banks, I think, that you have almost no statement. So then we went more to the next document, and then we explained that, just like some central banks will have a PDF while others will have it online, so we just work with that.

We didn't move on to inflation reports and we didn't move to the macro financial stability reports and whatnot. We're still staying with the monetary policy statement to preserve the comparability between the central banks. In terms of the shorter, longer statements, I don't know if my colleagues maybe want to add more.

As per somebody, now this is not coming from the analysis, I mean, generally, the shorter sentences, shorter paragraphs tend to Rank as more clear, as we know, on the text analysis methods. But from my personal experience, to me the predictability of statements has just helped a lot. That the fact that in certain part of the paragraph we expect to see this and another that it doesn't have to be particularly short of it, maybe a page and a half or two pages, the predictability, I think that helps.

And to the more that this central banks emerging market, central banks get integrated into the global market, then similarity across the different central banks also helps. But this is again, the more intuitive rather than relying on the index.

>> Steven Davis: Maybe it's in your paper. I didn't recall and I didn't hear much of it in your presentation today.

But can you quantify the rise and diffusion of any forward guidance across central banks around the world? And two, can you quantify the extent to which that forward guidance is couched with state contingent language or not? Because my impression is that we had a rise and then a fall of forward guidance and there was an experimentation with different forms of forward guidance.

Particularly whether or not to make it state contingent or time contingent or something else. Can you speak to that? I mean, you seem well positioned to offer comprehensive evidence on that, of a sort I haven't seen before.

>> Elina Ribakova: So we do have it in our topic analysis and we do have in an appendix or maybe now in the text we have a description of what we pick up as forward guidance across central banks.

So we do have it. It's just here for the sake of time, we try to really go a little bit over time. So basically we do see the forward guidance picking up. We didn't do the state contingent analysis properly so that we don't have. But we do have the forward guidance per se.

We do see the period of the spread and not spreading as much in emerging markets, and then we see sort of the narrowing of that or disappearance of that forward guidance. So we do have it in the paper, but we don't have it as much in this presentation.

 

>> Steven Davis: What's the basic pattern? When did it really start rising? When did it drop off?

>> Elina Ribakova: So here we do have emerging markets following the Fed, but many fewer. Few, I don't remember. Maybe my colleagues remember exactly the number. We see it in the paper, but it's not a large number of emerging markets that get into it.

They do get into it, actually. And I think it's a little bit around the time also. God, so many things happened. I think it's around the time of the QE when the emerging markets get also into QE and then they also get into forward guidance and then it starts disappearing.

So, sorry.

>> Piroska Nagy Mohácsi: No, basically they were following with a little lag, but the big central banks have been doing, and that became a standard after the global financial crisis, Bernanke and all that. And the Fed was using it extensively, and the ECB. So they started to use it from before the mid 1920s.

So that's a very interesting point to check for the state continuous time that we have to do because we have the data and we haven't thought about it. So thank you, Steve, for that suggestion. So that's what we have to do. We notice that the language is much more flexible.

We noticed that they dropped it to adopt the forward guidance once deflation was flaring up, because of course, you were stuck. And then particularly in the kind of extreme, very kind of naive version of the forward guidelines, when you say that you are no longer data dependent, that was very detrimental.

Would have been detrimental, obviously. So when inflation went up, emerging markets really started withdrawing altogether. And I think we have specific analysis. Right Tatana on this.

>> Steven Davis: I wanna give you guys a chance to wrap up here and I also want to see if Justin had another comment or question.

 

>> Justin Grimmer.: I like him questions, but I'll defer them to the post, post session.

>> Steven Davis: Okay, so if you want, we can take two minutes here to wrap up and then we'll turn off the recording and we'll go to the informal discussion.

>> Elina Ribakova: So maybe I'll do the quickly the wrapping up is there.

I think the key finding here is that again, and using the techniques that are available and have been available and I used maybe an individual central bank or individual technique or individual purpose. Trying to put them together and look at the stellar specs on the policy changes. Also to give people to where they could be continued working on and ourselves, we are working more into specific topics.

We are looking more closely into see, say, act. And there, as you can imagine, Washington, deep learning. The possibilities are endless. I think it's very interesting area. Thank you so much for the suggestions on exactly on the state contingent, for example, forward guidance, which combines the topic and more sophisticated quantitative analysis.

And then also we're looking much more into the region of analysis, sort of breaking down in region and seeing which countries are breaking away or staying with their region. So that's probably the key of our paper. There is definitely more that can be done in terms of using machine learning and trying to inform that analysis with machine learning, looking at the better prompts for ChatGPT.

So there is also technical work that can be done. But to summarize, I think it's trying to put together data that gives us better research questions and potentially better research answers. And as you just asked about the forward guidance, it's one thing is my or your perception of how frequently we hear it and how it spread.

But it's a different thing having a picture or exhibit in the paper that shows you exactly what happened, how it spread, and how quickly it disappeared. And so I don't know if my colleagues would like to add a few words here.

>> Piroska Nagy Mohácsi: Maybe one sentence that we can internalize that emerging markets, sort of the lead emerging markets have grown out of this very vulnerable country category into something much more mature.

Something that actually maybe can provide some positive examples in areas where they have society or not, but very good experience, and they perform very well. And it's good to recognize that institutional frameworks actually can work when applied appropriately, yes.

>> Steven Davis: All right, well, thanks so much. It was a fascinating talk, and it's really exciting work.

I share your enthusiasm for the possibilities of both research here, but also improvement to policy. I think they're quite rich in this area, and both the data that becomes available, the techniques and the research like yours. So thanks so much for joining us. And we're gonna turn off the recording now.

And anyone who wants to stick around, we'll have a little more informal discussion on an unrecorded basis. Thanks again. Really appreciate it.