Richard Owen 
Yeah. And I wonder if it varies a lot by functions and organizations. So, you know, I certainly, I traditionally used to think, okay, manufacturers understood data. I mean, if you were running, if you were running a manufacturing plant, right, you had a good deal of understanding of data, because if you didn’t understand to the nth degree, what was going on with the plant, you couldn’t achieve productive output. And it’s a very unforgiving environment if you don’t have your arms around the numbers. And then marketers kind of embrace data.

Probably with the start of marketing automation, so maybe 15 years or so ago, and we got this vast amount more marketing data, salespeople always to me seem like the last bastion, right? Sales somehow still feels like this process that people think of as being extremely personal judgmental rather than a process that can be guided by data. So, it feels like there’s different levels of adoption across the enterprise today.

Erik Brynjolfsson 
Yes, I think that’s exactly right. And a lot of it comes with when the data became available. Like even within manufacturing, we’ve seen big differences. We’ve analyzed hundreds of thousands of firms and the ones that have continuous flow operations where there’s a lot of data often in real time, they’ve become data driven much more quickly than the ones that work in batches. They don’t have the frequency of data, but even there, we all know about the Six Sigma movement. And as you said, in marketing, it’s become, there’s a revolution there.

A lot of my best students go into marketing and advertising because there’s such good data there and it just allows you to exquisitely do a lot of especially A -B tests. We should hopefully talk a little bit about the whole experimental cycle as well. And then you write sales. A lot of people think that there’s a lot of EQ, emotional intelligence involved in sales and there certainly is, but it’s also something that if you do it right, you can use data to make better decisions.

Richard Owen 
Yeah, you know, I think it was, I’m trying to remember exactly who said this, I think it might be one of your colleagues from MIT, Andrew Lowe, started talking about data as an asset on the balance sheet at companies, right? And the mentality that says companies need to think about data as an asset. And interestingly enough, you see this, the emergence of these chief data officers. And I was a little skeptical at first whether companies really understood what that meant, but I think… a lot of these guys have quite impressive mandates and thought processes around building data as an asset within the company.

Erik Brynjolfsson 
Yeah, I think we are in the early stages of a revolution in how economists think about the economy. It used to be capital and labor were the two big inputs and even before that land to some extent. And now we’re seeing data on a par or, for many kinds of companies, more important. I mean, think about what are the most valuable companies on the planet? Apple, Amazon, Alphabet, Microsoft, Facebook, they’re all intensely digital companies. And for all of them…

Data is really the lifeblood of their success.

Richard Owen 
You know, the argument against that often comes from companies that aren’t technology companies intrinsically, right? And so, they’ve always said, well, that’s all well and good. We’re always folks like myself, we’re always going around saying, well, the best examples are companies like this through the digital natives or tech companies. But ultimately, doesn’t this only work when mainstream businesses, maybe 100-year-old companies, sort of change their management processes as well?

Erik Brynjolfsson
Yeah, well, I’m glad you brought that up. I mean, it’s easy for me and I went to those go-to examples, but the reality is, is every company is becoming more and more digital and every process is becoming more digital. There’s the pure digital companies and it’s really easy to see there, but even like, you know, you take retailers, I did a careful study of CVS a while back and they were able to take the data from all their customers and then learn what works and what doesn’t work and replicate it across lots of their different operations. And you saw a lot of the same kinds of patterns that you saw in the pure digital companies. And it’s very hard for me to think of an industry that isn’t becoming more and more data driven. There’s always a mix, but the data is often what separates the winners from the losers.

I mean, to give you some quantitative evidence on this, we’re just finishing up a paper, we call it digital assets. And what we found was that every company has some degree of digital assets. These are the data and digital tools that complement their computer hardware and software. But it’s very concentrated. If you take out those pure digital companies that I mentioned earlier, you look within industries, you look within manufacturing, finance, retailing. Even there, it’s a very skewed distribution. The top 10 % of firms within each of those industries, tends to use their digital data much more effectively than the other 90%. And so, we’re seeing a, I would have expected a conversion. We’re actually seeing a separation where the digital companies are kind of pulling away. Sorry, let me be clear about that. The companies within each industry that are using digital assets effectively are pulling away from their competitors.

Richard Owen 
Yeah, that doesn’t surprise me as much given that there’s a sort of cumulative effect here. I mean, one of the advantages of being early to investing in digital asset development or data asset development is that you get a cumulative effect over time. If you’re capturing the entire data exhaust from your customers, then you build that data asset over multiple years. Somebody late to the party is always disadvantaged because they don’t have this historical depth of data to draw from.

So, it makes it very hard for them to make smarter decisions. And I’d add, let’s face it, building data assets is a laborious and often challenging process that does take years to get right.

Richard Owen 
Yeah, it’s interesting that in some ways the developed world has a productivity challenge, right? If we’re going to keep pace with the developing world, which has relatively low-cost labor, our solution is to increase productivity in developed countries. And right? And, you know, we’re not necessarily seeing population growth, we’re seeing aging populations at the same time.

Erik Brynjolfsson
You have to.

Richard Owen 
So, you put the two and two together and it becomes an imperative to find ways, if we’re going to keep the standard of living or ideally improve our standard of living, we have to improve human productivity across the board in these developed economies. So, the idea of augmentation fits into that very elegantly. And frankly, if it’s not going to be augmentation through data, what are the tricks do we have up our sleeve? I mean, it’s probably, we’ve probably gone through the sort of manufacturing productivity curves at this point. Supply chain productivity curves. This seems like the most promising approach now to create much more productive populations and enable us to keep our societies moving when we may only have 2 % population growth, but we want 5 % to 6 % GDP growth. This is the missing part.

Erik Brynjolfsson 
Right. No, we’re going to have to boost productivity. And I have to confess, I’ve been pretty disappointed by the productivity numbers. In the late 90s and early 2000s, productivity grew in the United States by about 2 .4, 2 .5 % per year. Since then, it’s only grown by about 1 .3%. Less than half as fast. So that’s been pretty disappointing. And it’s not just the United States. You look at Europe, Japan, just about all the OECD countries have had a slowdown.

And to me, that was a puzzle because I think we have some eye-popping amazing technologies rolling out. But when we look more closely, it gets to part of what you’ve been saying, that we’re not making the organizational changes. We’re not rethinking the business to take advantage of these technologies. If you simply paste the technology on top of what you’re doing or try to swap out one person with one machine, that rarely leads to significant benefits. You need to step back and reconceptualize the process.

That takes longer and it requires more creativity by managers. But if you do that, that’s where the bigger gains come from.

Richard Owen
So, what are the big opportunities for companies now to sort of augment human judgment with data?

Erik Brynjolfsson 
Well, I think that it’s something that is there’s a lot of small specific areas, but the basic template you’re looking for is a place where there’s a lot of digital data as inputs and a lot of digital data as outputs. So, if you can measure things reasonably well, and we were talking earlier at the beginning of the discussion about how much more data is available. Anytime you kind of mapping a set of Xs into a set of Ys and you’ve got digital data on both sides, there’s an opportunity.

for machine learning to really change what you’re doing. I think we’re very far from AGI, the Terminator, or the Androids that can kind of do everything, but we do have, I think, not just human level, but superhuman level ability to start mapping Xs into Ys. The real trick is making sure you have the data. And most organizations, if they look around, they’ll find that they have just a flood of digital data that can be organized and processed.

One of the biggest barriers actually is just the data management issues of getting into a form where you can use it effectively. Because in most cases, it wasn’t collected for the purpose of running a machine learning system. It was collected for some other purpose. And now you have to reorganize it. And that’s where the successes lie.

Richard Owen 
Yeah, and I think a lot of companies are very nervous about that problem. You know, the most common thing I hear in conversation is always, you won’t believe how bad our data is. By the way, that’s a universal statement. So, everyone thinks they’re terrible at it, but the reality is they’re probably close to average. And it seems like it becomes almost paralyzing. Either we’re going to wait for the silver bullet to arrive, which is some sort of grand unification of all our data assets and in this beautiful data warehouse in the sky where all of these things become immediately abundant and accessible, or we’re just going to freeze. We can’t deal with the reality of where we are. And it seems like the general guidance, well, I would suggest is you’re going to have to start, and you’re going to have to recognize this is going to take a while.

Erik Brynjolfsson 
Yeah, no, that’s exactly right. You want to start, and the data wasn’t collected for this purpose. So, you’re going to have to be willing to kind of manage it and deal with the noise and the errors and have a robust enough system and go through an iteration cycle where you learn from it. You know, start relatively small and then iterate and learn from that. That’s been the, that’s what, especially out here in Silicon Valley, you see that they’re very willing to test. You know, put a beta version out there, learn from it and keep iteratively improving. It’s a formula that eventually gets you to a lot of success.

Richard Owen
When you talk about the government, I wonder to what degree also do we have social pressures at some point, you know, the Luddites throwing out Spinning Jenny because they didn’t want their jobs done away. I mean, we have a modern equivalent to that, right? The population would, I think, rightly be very concerned. There were some stats I saw that said that the biggest single employer of high school educated Americans was grocery checkout.

Erik Brynjolfsson
Yeah. Yeah.

Richard Owen
Clocks. I’m not sure whether I got that stat right, but it sounds pretty feasible. So, this notion…

Erik Brynjolfsson
Mm-hmm.

Yeah, I think it depends on how you measure it. Truckers is also a big category, but depending on how you slice it, either of those are sort of the leaders.

Richard Owen
Yeah, I mean, you’d have autonomous trucks and self -checkout grocery stores. And, you know, we could potentially eliminate the most challenging segment of our population to create success out of. Hardly desirable.

Erik Brynjolfsson
Yeah. Yeah. Well, you know, we’ve always been destroying jobs. We’ve always been creating jobs. There’s this constant churn and America and no country has ever succeeded by trying to freeze in place the existing jobs or the existing industries. It’s always been a churn. And I don’t, when I look around, I see no shortage of work that humans can do better than machines. And I think it’s going to be like that for decades to come. Now there may be different sets of skills.

I mean, one of the things that for now, I think humans have an advantage is in some of the, we mentioned earlier, the emotional intelligence and connecting. And so it may be that as you combine, say, like you described it, your experience at Dell, maybe the machine does some of the configuration, but the human is kind of understanding better what the customer’s needs are in a more nuanced way. So, there will be a continuing kind of turnover as some jobs disappear and new jobs emerge.

Now, I could imagine going like decades into the future where we have a different issue where machines are better at that as well. But I don’t think that’s the challenge for the 2020s or even the 2030s.

Richard Owen
I mean, right now, I mean, the way certainly our perspective, if we can get better data, better sort of decision support information in front of people who are asking to make choices in extreme ambiguity in our universe, that’s how do you manage a customer? And today you literally have people guessing. It’s entirely judgmental. If we can give them data that essentially simply informs their choice and gives them a reason to have a different type of conversation or gives them pause for thought. the productivity increase associated with that could be quite profound. They could start making better judgments because guesswork is the worst of all possible. We don’t call it guesswork often enough, but reality is guesswork, what most people are doing when it comes to these things. So, the potential is certainly there.

I did want to ask perhaps a closing question here in terms of where you see the future going. Where are the areas for breakthrough that you’re seeing on the horizon?

Erik Brynjolfsson
Well, I’ve just been blown away by some of the things that are happening with these very large language models. So, I mean, the past decade, there’s been a big set of breakthroughs around supervised learning systems, deep neural nets, and it’s what I described earlier, mapping set of Xs into a set of Ys, and there’s still so much low -hanging fruit on that. And that’s really what most companies are focusing on right now, as they should be. But looking forward, these large language models like GPT-3 or their relatives that are doing it for images like DALL-E it’s breathtaking. When I talk to my very smart AI colleagues, even they are surprised. They don’t even fully understand why it’s working so well. And when you come to like generating content, I’m working with Percy Lang and we’re doing a set of experiments to have these systems work with humans to generate ads, work with them to generate stories, work with them to generate social media posts, and they are really good. You look at them, I would say they’re better than not the average college student, but probably like the 90th percentile college student in many cases. And that’s pretty good. They’re writing software, they’re writing code. Microsoft has done some studies where when they introduce it, 30, 40 % of the code that the people that are working with these tools end up being generated by these tools.

In each case, there’s always a human in the loop. These systems, they don’t quite understand the truth very well. So, they’ll generate content that sounds really, really good. And you have to check it because they just kind of spring together words. And they’re really well written. But sometimes you have to be a little careful that it’s kind of not exactly right. But by keeping a human in a loop and kind of steering it and testing different ways,

I see a huge enhancement in the ability of a lot of marketing, customer service, creative folks to do much, much better jobs. I think it’s a big frontier. Most of our work today in the United States and in advanced countries is information work, knowledge work. So, this is going to be a huge boost to tens of millions of people.

Richard Owen
Yeah, our head of content, Maurice, turned me on to DALL-E. I have to say it’s remarkable when you look at it. It’s mind blowing how this works. And again…

Erik Brynjolfsson
And it’s only a couple of years. It’s getting so much better. I’ve seen some of the new ones. They’re doing some motion stuff, 3D. It’s like, wow.

Richard Owen
Yeah. But I think as you said yourself, and that’s a good note to close on for a lot of businesses, the low hanging fruit is application of what is now very mainstream approaches in machine learning to particular problems. And, you know, yeah, there are going to be challenges around data wrangling, there’s going to be challenges around organization, but this is the frontier for companies. To get themselves organized and really there’s no alternative. I mean, you said yourself, the winners are separate themselves from the pack and if you don’t get in front of it, you’ve really no choice.

Erik Brynjolfsson
No, that’s exactly right. I mean, you don’t have to go to the bleeding edge of what we just described to get value. I think that most of the value is just in these basic supervised learning systems. Standard machine, we call it standard machine learning. It’s only like a decade old at most. And there’s, most companies have barely scratched the surface of where they can do that. Or even data-driven decision -making, that earlier wave, which is, mostly worked its way through, but a lot of companies just from basic data-driven decision-making.

That’s where there’s a huge amount of benefit. It’s not going to automate anybody’s job entirely, but people who use these tools are gonna replace people who aren’t using the tools. And so that’s where the real transition is, is there’s gonna be a set of companies that have figured out how to use them. Right now, it’s mainly the top 10 % and they are just crowding out the people who aren’t using the tools.

Richard Owen
That’s a great point to end on. Eric, thanks very much. Absolutely brilliant talking to you on this topic. Could talk for hours. It’s fascinating. Congratulations on the move to Stanford. Sounds like it’s a lot of fun for you. And I hope we get the chance to talk again soon.

Erik Brynjolfsson 
Absolutely, it’s always such a pleasure talking to you.