The Power of Proximity: A Conversation with Emma Harrington On Remote Work

>> Steven Davis: Two questions, first, what happens to mentoring and to learning on the job when employees no longer work together in the same location? Second, and related, how does the rise of remote work affect productivity in the near term and over time?

>> Steven Davis: Welcome to Economics Applied, a podcast sponsored by the Hoover Institution.

My name is Steven Davis, and I host the show. Our guest today has some very insightful research that speaks to these two questions. She is Emma Harrington, an economics professor at the University of Virginia. Emma earned her PhD in economics at Harvard, and before her recent move to Virginia, she held a postdoc position at Princeton and was on the faculty at the University of Iowa.

Welcome, Emma, it's really great to have you on the show.

>> Emma Harrington: Thanks so much for having me.

>> Steven Davis: So you co authored a recent study titled the power of proximity to co workers, training for tomorrow or productivity today. So the title kind of sets forth what the paper is really after, and that's in this fundamental point that proximity to coworkers involved a trade off between productivity today and productivity in the future.

So before we get into the details of the study, just flesh out the nature of that trade off a little bit for our audience.

>> Emma Harrington: Yeah, definitely, so we're particularly interested in studying how remote work plays out for knowledge workers, where there's lots of things that we mean when we say the word productivity.

So some of it is what we mean is sort of how much are you churning out today? In our particular setting is software engineers. So you can think of that as how many programs are you writing today? But then there's also this other component of your job where you're thinking about mentoring other people or learning yourself.

And there we find that being out of the office is a little bit more challenging. People actually write more programs when they're not sitting together, but they seem to also be engaged in less mentorship.

>> Steven Davis: Sounds a lot like professors, too.

>> Emma Harrington: Yeah, we've heard a lot.

>> Steven Davis: Trade off is very pronounced for professors.

Okay, so, that's the issue, but so tell us, give us the context in which you tried to address this issue in a compelling way. I'll let you kind of lay it out as you wish, but it's both the context of your study, the data you had available, which is impressive in some respects, and then the research design itself.

So sit that out for us, how did you go about tackling this really important issue?

>> Emma Harrington: Yeah, definitely. So we have data from a particular Fortune 500 firm, and a setting has a few key advantages for us. One is that the firm had two office buildings of engineers on its main campus that were about a ten minute walk, apartheid.

And because of limited desk availability, some teams ended up split across those two different buildings. And that mattered for the teams dynamics because it's pretty industry standard for engineers to have daily short stand up meetings with the idea.

>> Steven Davis: These are all software engineers, right? That's what we're talking about here.

 

>> Emma Harrington: Yeah, so they all basically worked on creating the website of the firm and maintaining the databases for the firm, it's an online retail firm. So these are pretty standard tasks for engineers.

>> Steven Davis: Okay, so you got some of these guys working together all the time. Some of them are, some teams, I should say are split apart.

How many teams, how many engineers overall in your study?

>> Emma Harrington: So it's 1000 software engineers and there's just a few people on each team. So also lots of teams, and about 60% of them were all co located in one of the two buildings. And then the remaining 40% of them were split across the two office buildings.

 

>> Steven Davis: Okay, so you've got a lot of useful variation in the data, as economists would put it. And then the setting, the time, the temporal setting is also very important for your study, the period covered by the data, so just lay that out for us.

>> Emma Harrington: Yeah, so we span the period before the office closure of the pandemic to a few months after the office closures.

And so we can see what happens to the differences between these co located teams versus these distributed teams once everyone's remote for the pandemic, we expect if the things were causal to begin with, they should go away once everyone is in the same boat.

>> Steven Davis: Okay, and there are, as you suggested, there are differences between the co located teams and the other teams before the pandemic struck.

And then there's going to be maybe differences, maybe similarities, we'll talk about that afterwards. But you got this event, which is pretty, this is an event we can safely say was largely unanticipated and arrived with extreme abruptness. So you have a firm that is operating in a certain way with respect to its software engineers.

This big shock forces them to do something different in terms of co location of workers. But some teams were highly co located before and some were split across the two buildings.

>> Emma Harrington: Yeah, and so are already, in some senses, acting like remote teams. They held some of their meetings remotely, even when they were in the office.

 

>> Steven Davis: Before we turn to your results, tell us a little bit more about exactly what you can measure in this context. That's another very attractive feature of your study.

>> Emma Harrington: Yeah, I think the really special thing about our study is that software engineers create this sort of paper trail of mentorship, that it's really standard that if you're writing a program at one of these firms, you're gonna have to get it peer reviewed by another programmer who's gonna give you pretty detailed feedback on your program before it can actually be used in production.

These firms really don't want there to be any bugs or glitches in the code, and so they have this peer review process.

>> Steven Davis: Are you confident that all or most of the feedback actually takes this written form, or is there also feedback occurring that you're not measuring because it's just oral?

 

>> Emma Harrington: So, I think our setting means that a lot of it is captured in the code review process because part of it is that they want documentation for the reason why there were changes to the program. So there are some references to conversations that occur outside the system.

So we can directly see things like references to slack conversations. And those go in sort of the same direction as our main results.

>> Steven Davis: Okay, and one more thing about the data. You have some information on the quality of the feedback, as well as not just measuring the quantity, but you've got some information on the quality, the feedback as well.

So that's gonna be important for some aspects of your results. So how do you assess the quality of feedback? How do you know it's even useful?

>> Emma Harrington: Yeah, so, I mean, we can look at that in a few different ways. So we can, we have the text of all the feedback that is given and so we can look at measures of how substantive the feedback is.

Like, does it include a block of code, for example, or a link to other documentation? We also had a subsample of the comments reviewed by external reviewers who, by and large, said that the comments were impactful and likely to change the code and also often provided actionable steps to take to change the code.

Yeah, those are our main measures of the quality.

>> Steven Davis: So this is a data rich environment, an interesting physical setting, an obvious shock that hits, that causes things to change over time, and you're able to measure, follow individual engineers and teams before and after the shock. So very nice setting.

So before the shock hit, though, there were already some noticeable differences between the co-located teams and the other teams. Tell us about that.

>> Emma Harrington: Yeah, so the co-located teams received about 20% more comments on each of their programs. And as we've been talking about, that's a pretty notable difference because what we're measuring is feedback that it's occurring online.

So to the extent to which there was some feedback that was occurring face to face, presumably that would be easier if you were co located. So we're likely providing a lower bound on how big the differences in mentorship are between these co-located and distributed teams.

>> Steven Davis: Okay, and to what extent is the feedback going from senior people to junior people as opposed to just horizontal people at roughly the same experience level within the firm?

 

>> Emma Harrington: There's some of both. The effects of proximity seem to be really concentrated among feedback received by junior people and feedback given by senior people. So some of the sort of horizontal feedback that happens, but it doesn't seem to be as sensitive to whether you're co-located or not.

 

>> Steven Davis: Okay.

>> Emma Harrington: I think some of that may be because of the underlying mechanism that some of it is because it's important to be co-located in terms of feeling comfortable asking for additional feedback, and maybe if you feel like you're in a peer situation, you feel comfortable doing that even remotely.

 

>> Steven Davis: Or the flip side of that is the senior person may be more comfortable saying, not providing feedback when they don't have to see the person in the hallway.

>> Emma Harrington: Yeah.

>> Steven Davis: We'll get into later more whether this feedback is actually, on the margin, the best use of time for all parties involved.

So I wanna come back to that, but okay, so there's more feedback received primarily by junior people in co-located teams, and the feedback is largely given by senior people. So senior people, well, their time is pretty valuable, so there's an opportunity cost here to this feedback, but presumably there's also quite a bit of benefit for the junior people.

And so we can already see even in a kind of a static sense, there's a trade off here in attention. But then the dynamic one is, well, if the junior people don't get enough feedback, they won't improve rapidly enough over time. So then the pandemic hits played out for us, what happened?

 

>> Emma Harrington: So, feedback in general declined across the board, but it declined much more for engineers that had been co-located with their teammates than engineers who are already distributed, and in some senses, already acting a bit like remote teams. And so, basically, that whole difference between those co-located teams and those distributed teams when the offices were open disappears once every once.

 

>> Steven Davis: Okay, so it completely disappears. But how much does the overall decline, say, for the teams that were originally co-located? How big is that decline?

>> Emma Harrington: I mean, it's pretty enormous if you include. So, like, the gap goes away, and then also you just have a secular decline across the board.

So I think if you added it all together, it would be like something along the order of a third of the feedback disappears. Now, it's unclear exactly how much you wanna think of that as all being a causal effect of proximity, because you do have the pandemic going on.

And so you might think that there's other shocks to these engineers. It's not just because of the loss of proximity.

>> Steven Davis: Okay, so if they were, for example, highly stressed by the pandemic, their overall work effectiveness might have diminished, right? Is that a coming mind? Okay, but what happened to their overall code production, though, in the wake of the pandemic?

Did that go up or down?

>> Emma Harrington: So, it went down overall, but it went down by more for the people who had been previously across the two buildings than people who had been on one building teams. And so, relative terms, the engineers who had been co-located saw their productivity in terms of the number of programs, they won't go up relative to those that have been distributed.

 

>> Steven Davis: Okay, but as I understand it, both mentorship measured, as you described earlier, feedback might be a more accurate term. Feedback fell roughly a third for the co-located teams fell less, but from a smaller original base for the non co-located teams. But lines of code written also fell, is that correct?

Okay, so there was a general across the board negative response, which, as you pointed out, was a mix that we maybe can't fully unpack between the lost benefits of proximity and the pandemic itself, disrupting people's lives, maybe even their mental states in many respects. Okay, do I have it right there?

 

>> Emma Harrington: Yeah, no, that's right. I mean, I think that's why we focus mainly on the difference and differences of thinking about how the one building teams- Sure. Changed around the office closures need to be the multi building teams.

>> Steven Davis: Right, but I wanna put myself in the shoes of the executives at this firm, the ones overseeing the software engineers.

So they must have seen some of this in real time. And while for research purposes, the difference in differences, it's extremely interesting from a business perspective. They say, whoa, l there's a lot of lost outputs really down here. Now, how did they react? Or did they react to this in near real time?

What did they say? What did they do? I presume you've talked to them about this.

>> Emma Harrington: Yeah, we have talked to some engineers at the firm. Some of them said it was really resonant with their own experiences, that some senior engineers, at least, felt, in relative terms, like they were being pretty productive in terms of how many programs they were writing in the pandemic.

 

>> Steven Davis: Relative to their own past or relative to other I think they viewed it as relative to their own past.

>> Emma Harrington: I mean, I guess the data in an aggregate sense is inconsistent with that. Although on the individual level, I'm sure for some people it is. And so for the people we talked to, they felt subjectively like they were being more productive than they had been in the office.

But they had some sneaking suspicion that that was partly because they were devoting less time giving-

>> Steven Davis: Okay.

>> Emma Harrington: Feedback to other people.

>> Steven Davis: This is really interesting, because I've done a lot of work relying on subjective assessment Of performance, I'm just curious, and maybe you didn't look at this cuz I don't remember seeing it in the study.

What's the relationship between subjective assessments of own performance and your external measures, especially just code production? That's probably what they're answering about when you ask them, are you getting more done than you used to? Are those two things I'd be really curious to know how they relate to?

 

>> Emma Harrington: Yeah, I mean, unfortunately, we weren't able to, in this setting, do a sort of systematic survey of people to get their subjective expectations.

>> Steven Davis: Okay.

>> Emma Harrington: Natalia, my co-author on this, and I have another paper about call center workers where we were able to more systematically elicit those.

And find some correlation there, but I'd have to check on the exact numbers.

>> Steven Davis: Okay, well, let's go back to this study. Now, you also found some very interesting differences between men and women. So flesh that out for us, cuz it's quite interesting.

>> Emma Harrington: Yes, we find that the tradeoffs from proximity are much more acute for women.

So it seems to be more important for junior women's feedback, the feedback they receive to be co-located with their colleagues. And also more taxing for senior women's time to be sitting near their junior colleagues. So for senior women, it seems like they have a really big loss of output from sitting near their co-workers.

 

>> Steven Davis: But how does it work dyadically? So is it that junior women are getting a lot more feedback from senior men when they're in proximity, or from senior women, or both?

>> Emma Harrington: Both.

>> Steven Davis: Both, okay.

>> Emma Harrington: So he thought it was possible that it would only be across genders, but it looks like it's just across the board.

More important for junior women to be near their colleagues.

>> Steven Davis: Okay, and for the senior women who are giving, they're giving more feedback when they're co-located. Because they're both more approachable to the junior men and the junior women, or is it primarily from the junior women?

>> Emma Harrington: It's both.

So the only dyad that seems to be pretty unaffected by proximity is men giving feedback to men. Basically, all the other combinations are effective.

>> Steven Davis: Okay, do you have any sense from your conversations with these software engineers, why the differences between men and women take this form? What did they tell you?

Or maybe you didn't ask them about this, but again, it's a very interesting question with relevance beyond your study.

>> Emma Harrington: Yeah. So, I mean, we didn't ask the software engineers of the firm that much about this question. They expressed a lot of concern at the time that this was some form of mansplaining, which is a general concern that's been raised when we've given this paper.

And so we were able to do some things using the text of the code reviews to try to get at that concern. So, I mean, one thing is that when we have the external raters look at the comments. They actually rate comments that are received by women as being more helpful and more likely to cause changes to the code.

So it doesn't seem like necessarily women are just getting a bunch of really nitpicky comments. And that's true both when they're proximate to their co-workers.

>> Steven Davis: But even your basic patterns don't really support a mansplaining story, as I understand it, because the senior women are giving more feedback.

You wouldn't call that mansplaining.

>> Emma Harrington: Yeah, I guess, I mean.

>> Steven Davis: And, and the junior women are getting more feedback from both senior women and senior men, which doesn't really sound like a mansplaining story. Okay, so it's not a mansplaining story. So what do you think it is?

 

>> Emma Harrington: I think it may be something about comfort asking for feedback. So we particularly see that, you can sometimes get feedback from multiple people or you can get feedback from just one person. And when junior women are proximate to their co-workers, they receive feedback from more people than when they're remote.

And so because in this setting, the software engineer themselves is asking other people to review their code. It does seem consistent with a story where when you're remote, you just, and you're a woman, you just don't feel that comfortable asking for this additional feedback and advice. Whereas if you're in person, maybe you can see someone in the halls and be like, hey, do you mind taking a look at this?

And that just feels a little less pushy to you.

>> Steven Davis: Right, so an important issue, I think, I know the answer, but I want to get it on the table for the sake of the audience. Is there anything in your study that tells us whether there was too little or too much feedback either before the pandemic or afterwards?

 

>> Emma Harrington: Yeah, it's hard for us to sort of horse race the benefits and costs of the feedback within our particular data because it's kind of relatively short-time span. And so it's hard for us to see the returns to the mentorship fully. From the individual's perspective, it's useful to get the feedback and mentorship in terms of downstream pay raises and also-

 

>> Steven Davis: Sure, but it's feedback is by definition a bilateral transaction or not transaction exchange, it's a bilateral exchange. So if we're gonna come up with some notion of whether there's too much or too little. We got to add a minimum account for the two parties involved directly, the feedback.

We may wanna account for the firm as an overall organization as well, and external parties, right? So I just think it's important to understand, and correct me if I'm wrong, we don't know whether there was a too much or too little feedback. We don't know whether the men were closer or the women were closer to what some optimal notion of feedback is.

Or do you disagree with that?

>> Emma Harrington: No, I think I broadly agree with that. I mean, we can see some evidence from the firm's own policies about what it thought. Now, it might be.

>> Steven Davis: I was gonna come to that next, so go ahead, tell me, that was my next question.

 

>> Emma Harrington: Yeah. So, I mean, and it's unclear whether the firm was perfectly well calibrated on this. But pre-COVID, it did allow some people to work remotely, and those people tended to be sort of mid-level engineers. So they were not the ones receiving the most mentorship. They also were not the managers or super senior engineers who were giving the most mentorship.

And so that's consistent with the firm wanting the most junior, most senior people to be located.

>> Steven Davis: Okay, so, yeah, that's a useful piece of information, but let me come at this from a different direction. So I presume you've been in conversations with some of the senior people at the firm after your study.

I'm sure they've read your study or at least been briefed about it carefully. So how did they react? Cuz your study has information that might cause them to change their behavior or to modify their policies or try to reinstitute the old practices. What did they do?

>> Emma Harrington: They have tried to get people to come back in a hybrid format.

 

>> Steven Davis: A hybrid format? Okay, so-

>> Emma Harrington: A hybrid format, yeah.

>> Steven Davis: That's interesting, and that's typical of what I see in many firms, which is, They don't try to go all the way back, either because they don't think it's necessary or productive or often because there's resistance from their employees, but they feel that it's essential to have some FaceTime.

And, that's basically the reaction to your study?

>> Emma Harrington: Yeah, it's unclear to me whether it's coming from, they don't think it's helpful to be in person all the time, or whether they just think it's infeasible. I mean, even the three days a week back in the office has been a very hard sell.

I think so.

>> Steven Davis: Okay.

>> Emma Harrington: Yeah.

>> Steven Davis: We didn't talk about this earlier, but I know from reading your study that at least before the pandemic, feedback was not explicitly rewarded. Of course, it can feed into performance reviews and so on, but it's not like there was some formula that says, you get this much compensation for the quality and frequent and I volume of your feedback and this much for your code or so on.

But after, now we settled into this new norm where at this firm, at least it sounds like there's a lot of hybrid working arrangements, has the firm tried to explicitly incentivize or disincentivize, I suppose feedback, especially from senior to junior people?

>> Emma Harrington: Not to our knowledge. I think when we talked to people at the firm, they said that that would be very hard because basically, it's pretty hard to evaluate whether comments are helpful or not.

And so if you incentivize giving more feedback, you would probably get a lot of relatively low quality feedback.

>> Steven Davis: Okay, yeah, that makes sense, if you reward something explicitly that can distort behavior, in ways that's kind of a well known point in personnel economics. But still, what has the firm done anything, if not through explicit incentives, to try to ensure that there is an adequate level of feedback.

Now that they're in a world where many people are working in hybrid mode?

>> Emma Harrington: I mean, something kind of interesting we can see in the data is that it seems like there's a shift towards a greater proportion of the feedback you're receiving coming from teammates as opposed to from people outside your team.

And so I think that's really consistent with thinking that within a team, it's maybe a little bit easier for the manager to observe those interactions and potentially reward them in something like a performance review, even if they're not getting explicit incentives for it.

>> Steven Davis: The reason I'm probing these questions is because, one of the key things into my mind about the longer term effects of the rise of remote work, on productivity, on human capital development and mentorship and so on, is how firms adapt over time.

And it's pretty clear that in many settings, some adaptation is necessary. If you suddenly, there's a lot of things that happen automatically, your study kind of shows this, when people are co located, you get more feedback, even through this formalized, written manner, than you do when they're not co located.

And now if we go to a world where there's a lot less co location, than there was before, firms need to adapt to replace that which is lost, by the way, in this hybrid mode, maybe the answer to this. When people are working in hybrid mode now, are they encouraged to come to the office on a certain set number of days, or they're kind of spread out over the week, or they can come in whenever they want.

 

>> Emma Harrington: Last time we talked to people at the firm, it was at the discretion of the employee what days they came in, which our studies suggest would be potentially problematic.

>> Steven Davis: Yeah, that's right. Because then going from five days a week in the office, to only three days a week in the office.

Where it's not the same, three days a week for everybody, is not a 40% reduction in colocation. It's much bigger than a 40% reduction in colocation.

>> Emma Harrington: Yeah, and ours, anyone suggests much bigger, because we find that even having one teammate in a different building, in our setting or remote, it changes the interactions between coworkers who themselves are actually in the office together.

And we think that that's because if you just have one guy who didn't come in that day, you need to have your meeting online. And so then you don't have the schmoozing on the way to the meeting, on the way back from the meeting, and it just changes those dynamics.

 

>> Steven Davis: Yeah, So tell us a bit more about exactly what you found in this regard. So John's home today. His four or five teammates are in the office, but they got to get on a Zoom call with John. So what does that do, according to your analysis, to current code writing and the extent of mentorship?

 

>> Emma Harrington: Yeah, so we find, most of our analysis that have focused on the mentorship aspect of things. And we find that about a third of the effect of being on a team that's all in one building versus a team that's distributed across the two buildings, comes actually from the spillovers of having a distant teammate on the interactions between people who are in the same building.

We can see that basically, if Tom and Susie are in the same building, but John is in the other building, Tom and Susie are likely to exchange less feedback, than if they were on a similar team that was all co located.

>> Steven Davis: Right, so that's a pretty big effect and it suggested much of the informal interaction among co located people was actually taking place right before or right after or during the physical meetings themselves.

Cuz if they were just bumping into each other at the water cooler, the fact that John's in another location and we have to have meetings via Zoom with him, you wouldn't expect to get much of a negative effect on mentorship from that. Okay, so that sounds plausible to me, but then kind of reinforces a view I had already.

I wanted to get your reaction to it, which is, if you go from a world in which everybody's coming into the office five days a week to one where there's hybrid, especially if it's people choosing their own times to be in the office, you really got to replace the old informal social interactions that took place with something else.

And in your context, this would be a way to overcome those kind of. Those social inhibitions the junior person might feel, in asking a senior person for feedback. So does this firm, are they making more efforts? Like, is there a, Wednesday night drinks or something? I don't know what it might be, but some way to encourage more social interactions when people are actually in the office, so that these productive exchanges will flow more readily.

 

>> Emma Harrington: Yeah, we haven't talked to the software engineers at this firm in a while, so I'm not entirely sure what the state of the art is for them. The last time we talked to them, they were really struggling to get people back into the office, even three days a week, whichever days they chose, and so it was a ways away from them institutional.

 

>> Steven Davis: Okay, maybe there's a future study coming, it's got a future study that is focused on how does your organization in particular, adapt? It sounds like they might be settling into some kind of hybrid working arrangements, but how are they adapting other aspects of the work environment, including the extent of socializing, in order to mitigate some of the costs associated with less co location?

I think that's a really important question for the long term.

>> Emma Harrington: Yeah, I agree, I mean, I think something also, additionally that's interesting that we can see in the data that we have is that there's some suggestive evidence, the firm shifted its hiring practices. So pre Covid, it hired a lot of new grads, who didn't have a lot of experience as engineers and were quite young.

And then when the office is all closed, it shifted towards hiring more senior engineers. And there's lots of reasons that that may have happened, but I think one interpretation is that when it was hard to facilitate that proximity, it wanted to sort of buy talent rather than build it.

And so I think if you don't have these successful strategies to potentially facilitate those mentorship relationships, firms may just change who they wanna hire.

>> Steven Davis: Right, which could have implications for the wage structure and, the wage tenure profile and so on. So, anything else you want to tell us about your study, other key points that we've neglected or over overlooked so far in our discussion?

 

>> Emma Harrington: I think we've really covered the main takeaways of the study.

>> Steven Davis: Okay, great. Well, we will put a link to your study on the episode website at the Hoover institution so people can check it out. And Emma has lots of other interesting research, too, that we didn't have time to get into today.

You might want to check that out, too, including the call center study that she mentioned. It's one of the better known, studies in this area and rewards a careful look. So thanks so much for Emma, it was lots of fun to chat with you about this work, and we're really delighted you could join us today.

 

>> Emma Harrington: Yeah, thanks so much for having me.

>> Steven Davis: Okay, take care.

>> Emma Harrington: Bye.