TEG Speaker Series: Dr. Kelly Stephani

>> Kelly A. Stephani: Thank you so much, John, and thanks to everyone for joining today. It really is a privilege and an honor for me to be a visiting fellow at Hoover. I started just late last spring with Dr Siegert, and it has honestly been one of the highlights of my career.

The ideas and the rapid implementation of those ideas that Hoover brings to our community it's just so critical and so important, especially when we're in the times that we are now. Technology is becoming so important, so critical of everything that we do. And it's also bringing folks together who might not have otherwise had conversations about how technology can and could be informed policy, and vice versa.

So, thank you again, and I'll be sure to convey my gratitude to Dr Siegert as well, I hope the chance to touch base with her while I'm here. So, for those who don't know me, my background is in engineering. So, you might ask yourself, well, what is an engineer doing at Hoover?

But there was an excellent opportunity to engage with Doctor Siegert and other fellows at Hoover on this challenge, which addresses, again, US technological readiness and competition. So, my background in engineering is in hyper-sonics specifically, and I've been working in the field of hyper-sonics for my entire career, which is longer than I'd like to admit at this point.

But I started working on challenges associated with high-speed vehicles, engineering challenges surrounding that. But it was around the time of 2019 or so, that I really started to be pushed, in a sense, in the direction of policy and how science and technology inform policy. My very first experience was back in 2019, I had a visit to Washington DC.

This was after an award ceremony, I had a chance to engage with some folks at the Pentagon, and of course, I went there approaching them with, well, I've got these great ideas. I'm an engineer, I wanna work with the DoD, and I want to make a huge impact in the area of high-speed flight.

And they said, that's great, but are you aware of what's going on overseas right now? I was a new professor, I geopolitical situational awareness was not as good as it is now, but I was quickly brought into that in that space. Essentially everyone who is working on NASA-type problems, hyper-sonics is known to be cyclic.

Everyone who's working on those kinds of problems was suddenly shifted towards the DoD side. And that's really where we started to learn about the challenges, about the competition surrounding technology. And how we implement technology in a way that gives the US competitive advantage. So, that's my background, and that's why I'm here at Hoover, because we have a lot of challenges to address.

And with my background in hyper-sonics, especially, I get to kind of see I get to work in the trenches. But then also lift those concepts, those ideas, those innovations, into the space where we can interface with our policy and policy fellows. So, the basis for discussion today extends far beyond hypersonics obviously.

My own background and in some cases, my own examples will stem from my experience with hypersonics. But obviously, there are a number of emerging technologies that are critical to our national defense strategy. And these are outlined annually, these are published annually, and so we do get briefed on these.

But the emerging technologies that are in the forefront, especially here at Stanford, I would argue, are related to artificial intelligence, biotechnology, quantum technology, and of course, several others. In my field, hypersonic weapons is an important component of the work that I consider as is directed energy weapons. And as I mentioned, biotech, quantum, and also lethal autonomous weapon systems.

So, these technologies, if we are to understand how these technologies emerge and how to leverage those capabilities. One of the first things that we need to understand is what the state of those technologies are. And there's a formal process by which we basically assess technologies, and we consider the technology manufacturing and integration readiness levels, we call them readiness levels, the RLs.

And there are more than just the TRL, MRL, and IRL, there's also a systems readiness level, so I'll focus on just these three today. And this is, again, thanks to Ash for this recommendation of starting with this introduction on readiness levels, just to kind of level set the community here.

If you have questions, please feel free to interrupt. So, the readiness levels, what are they, and what do they really indicate to us? Are they useful and how do we use them? So, of course, I have to give homage to NASA, which basically founded the readiness levels back in the 1960s.

They had used these readiness levels to basically define state of a technology and understand where in the technology maturation mission process are we. This is, of course, a bit of a play on the readiness levels, because, of course, we're discussing technology as a Unicorn, right? The DoD Unicorn, if you will, so just an outline, here we have our first TRL1, technology readiness level one is really tied to the idea of basic research, fundamental research.

So, it's exploring ideas of what, within the realm of possible, so within the laws of physics, from the periodic table of the elements. What kind of technologies could we possibly envision developing and then expanding in the future to enable new capabilities for our missions, right? So, the first technology readiness level addresses, well, what if there were Unicorns?

Second technology readiness level asks, well, if there were Unicorns, can they actually, are they feasible? Is it something that we can actually realize? And, of course, that has to be grounded in physics, we can't violate laws of physics in order to bring a technology to bear. Once we finally have theorized and even demonstrated existence, I changed this from drawing a Unicorn, to actually demonstrated existence of the Unicorn.

And we have our third TRL, which finally says, okay, this is what we envision it to be. This is what it looks like, this is how we design it. And if you want to move forward with an actual implementation, we then need to take that drawing and make it realizable.

So, TRL 4, we basically say, well, we don't have a Unicorn, but if we had a unicorn, what might it start with? So, let's take a horse, and if we have a horse, let's go ahead and place a horn on that horse so it at least emulates the Unicorn, it's not a Unicorn yet.

We don't get a unicorn until we get out to TRL 5 and TRL 6. So, we take our horse outside and the TRL 6, now we just start calling this horse with a horn a Unicorn, okay? So finally, once with TRL 7, we're asking, well, what can we do with this Unicorn?

If we can design it, if we can realize it, what can we actually do with it? So, TRL 7 says, well, the application might be launching this unicorn in space, can we do it? We're not really sure, but we can at least try it, so we're pretty sure we can make it.

TRL 8, O-M-G, it survived, so, that's demonstration, of course, that this unicorn can actually function in space. And then finally, TRL 9, our reference design incorporates high heritage space unicorns. This is obviously very playful example, and this is, of course, taken from a good friend and a VC firm, from a LinkedIn post.

But in reality, these are,

>> John: I don't know, so would like that character.

>> Kelly A. Stephani: Well, they had a very rigorous set of readiness.

>> John: Yeah,

>> Kelly A. Stephani: This is much more in line.

>> John: They did get us to the moon at times.

>> Kelly A. Stephani: Yes, they did, probably not through a unicorn prototype.

Yes, in fact, NASA, the work that they did in defining these readiness levels has just been instrumental. And if anything can be said about a success story, that is one of our success stories. That readiness levels, they require inputs, they require care and crafting, but they do quantify how ready a technology is and what we can do with that technology.

So, as far as the TRLs go, I have a few tables here, this just shows technology readiness levels. Their levels are from 1 to 9, the definition and then the description. Again, TRL1, that's really founded in this idea of basic research and basic principles. So, this is stuff that we see every day in our university labs, in our DoD labs.

This is the kind of work that's basically connecting physics with what we know to be possible and seeing how much further can we push that with engineering design? So, this is our lowest level of technology readiness. And once we get to TRL 2 and TRL 3, we're starting to think about, well, what is this technology concept and what are we actually hoping to do with it?

So, at that point, we see that invention begins and what they refer to as active R and D. So, the TRL 1 is really basic research and development. TRL 2, TRL 3, those start to get to the point we call active R and D, meaning that we're finding uses, applications, and posts, making connections to end users, but we're still years off from an actual fielded capability or a deployed capability.

In TRL 4, we basically work our way up to what we describe as a component. So, rather than just having a technology operate in a vacuum or an isolation, we want to see how this component or how this technology plugs into a compound and how it can actually be used in a setting that's representative, right?

So initially, these technologies are developed in pristine laboratory environments, but ultimately, if we want to be able to do things in our everyday lives, we have to be able to expand beyond a pristine laboratory environment, and actually demonstrate how these technologies function in relevant environments. So, once we get to the relevant environment in TRL 5, we see that the fidelity of technology workflow increases significantly.

So, it's not just a matter of onesies, twosies, or one-off technologies. We're actually looking at something that interfaces, it has inputs from a representative environment. Once we get up to TRL 6, we see systems and subsystem models, and we get to the prototype phase. So, we actually have a demonstration of what a working technology or working tool could look like.

And once we get up to that point, we're really operating what we call an S and T environment. So, the DoD likes to refer to the sort of technology maturation as either an R and D question or an S and T question. And S and T is getting to the point where you have confidence in the environment for testing, you have confidence in the operation of that component in that environment.

And it gets you to the point that you're now starting to talk about how if you were to take this to the next level, to the TNE level, how would you actually be interfacing with now the full system, or with members of the services that actually have to operate this technology, and how would they use it?

So, that ties into the conversation about DOTMLPF, which I'll get to in just a little bit. TRL 8 and 9, here we're approaching systems that have been tested, qualified through test and demonstration, and then finally for operational tests and evaluation, that we've actually demonstrated the system through successful mission operations, okay?

So, in order for us to take these steps through technology maturation and work our way from TRL 1 up to TRL 9, there is a huge amount of coordination required. Foresight is required, and years, and investment is also required. So, generally speaking, when we're operating at TRLs 1 and 2, we're looking at kind of a three to five-year investment.

And working with TRL 9 requires a sustained investment that goes across the basic research programs, but also into the transitional programs, and then finally into acquisition programs. To accompany the TRL, the technology readiness level descriptions, we also need to consider our manufacturing base. So, the manufacturing readiness levels basically give us a lexicon and a structure or a plan to take a technology that's identified and provide the support, if you will, the infrastructure to actually bring that technology to a place where you can produce at scale.

So, I guess the key word here is production at scale or manufacturing at scale. So again, the MRLs in this case go from 1 to 10, but generally, they keep pace with the TRLs that I outlined on the previous slide. So, I'm not gonna go through all the details here.

You can take a look at the descriptions and kind of read through them yourself. But the driver here is that you really need to have a manufacturing base, which means that we have to have raw supplies or raw materials. We have to have trade persons that can actually provide support for manufacturing.

At scale, of course, means that you need to have that full supply chain established, and then all of these pieces and components have to be integrated. So, it's not just about building the parts, also about building the entire component once we've established its utility, excuse me.

>> John: Does that happen concurrently with the technology development or after the technology has been developed?

 

>> Kelly A. Stephani: That's a good question, and it does depend on where they see the need being driven. So, if you were to look at a technology requirement, so I'm gonna go back to hypersonics because that's one that I've seen firsthand. When we're talking about developing a certain component on a hypersonic platform, we know that we can develop a prototype.

Prototypes have been developed, they've been flown, they've been proven. When it comes to actually deploying them in sort of DoD mil PF framework and making them effective in a battle space, then we have to think about, well, how many are we going to need? How quickly do we need to be able to launch?

If we are depleting our stock, how rapidly can we replenish that stock? And so, those questions are ultimately gonna drive when the manufacturing entity or when that piece comes online, and how it supports the technology readiness. So, if we are sprinting towards a program of record, for example, or if we want to have a certain technology fielded by a certain time, then you do need to have leadership sort of anticipate when those need to be brought online.

And just as an example, there have been executive orders that are essentially enabling manufacturing at scale in anticipation of technology readiness to reach level nine by a certain fiscal year, for example. So, that's a very long answer, but hopefully that helps to at least provide the pacing. Okay, so, we've gotten through TRLs, MRLs, and the final piece is the integration, readiness, I'm sorry, I passed that one.

This one is really addressing an important aspect of technology insertion. So, if we're fielding an entirely new capability and there's a need to deploy that capability with an existing vehicle or with an existing platform, then integration readiness is absolutely imperative. One example that we might point to again, this is back to hypersonic, is if you consider the navy programs, and the army programs, they are trying to integrate hypersonic platforms on existing platforms.

So, the navy has the Zumwalt, and the army is using one of their long-range hypersonic weapon platforms. So, this is a new capability, and this technology is inserted, if you will, in an existing platform, and there needs to be an assessment of integration readiness. So, just to give an idea, if they both use the same all up round, each launcher then is responsible for being able to interface, to take information, to receive information, and to communicate effectively across these different all up rounds.

So, having this integration readiness level characterized, basically provides the means and sort of the path forward to guarantee that when these are delivered, that all of this ties together, and everything is fully operational for the given mission.

>> John: Can you give an example of 0.1 interface identified? And if 5 issue after control the interface, could you give specifics for that?

 

>> Kelly A. Stephani: Absolutely, so, as far as the interface between technologies being identified, if you consider again that the navy and the army working together on this all-up round, one is the conventional prompt strike, one is the long-range hypersonic weapon. Both of them have the same stack, but they need to be able to integrate into an existing platform.

So, the Zumwalt destroyer is the navy platform of choice, and the army has their long-range hypersonic weapon platform. And so, the interface in that case is literally, how do I plug this all around into this platform? So, that interface is not only it's hardware and software, but it's also considerations left of launch, if you will.

So, if I need to understand what it takes to receive the authorization to use this component and then use it on the battle space, there's an entire process left of launch before you even make that decision. You have to be able to pass that information along, and then make the steps to actually use that capability in a fielded scenario.

Yeah, go ahead, please.

>> John: How does control fit in this?

>> Kelly A. Stephani: So, control, this is control established between technologies to initiate, manage, and terminate integration. That just basically tells us, if I'm given the authority to use this capability, do I have the means to provide the inputs? Basically, turn it on, turn it off when authorized, and that kind of control, again with the inputs, is what gets you to that integration readiness.

One can demonstrate, but in a sense, its providing autonomy, it's providing sort of on demand control that brings you to that readiness level.

>> John: Where do people fit into this whole structure?

>> Kelly A. Stephani: I would argue throughout. So obviously, when you integrate the human in the loop, there is a level of control or authority that human has, earlier on in the integration readiness, they'll have less and less authority.

So, everything will be sort of prescribed, controlled, but as you get higher up in integration readiness, especially as you're moving towards more autonomous systems, then the human in the loop is well defined, but it has a much tighter, how do I put this? More interface and more interface opportunities than if you're at the lower integration readiness.

 

>> John: Okay, for now.

>> Kelly A. Stephani: Okay, so after establishing these readiness levels, one of the most important elements that we need to consider with developing these technologies is, first of all, the technology risk. And I don't wanna call these just technology risks, I think these are better classified as risks and opportunities.

So, we have technology risks that have identified here, which are mapped out in terms of financial investment, but also as technology risk, where we have the most risk on the left-hand side. Early on in our technology readiness phases, where we had TRL 1, TRL 3, and then as we work our way through technology maturation, so as we work our way to the right, towards a high TRL, high MRL and high IRL, then the technology risk reduces at least.

And the reason for that is obviously as we learn more about the technology, we develop the technology further, we gain confidence in the technology and we understand how to use it, what its limitations are, what its possibilities are. When we consider the technology risks, one of the elements as well, if we have an idea, and we want to use this idea for a specific application, it may not work.

So, the laws of physics may work against us, and it may just be an ill post problem to begin with. The other risk is more programmatic, and that is, as money is invested into a given technology, or developing a certain capability, there is a possibility that we can mature the technology to a certain point, to a certain TRL, say TRL 2, TRL 3.

But then the question is, what happens to that technology? What happens to all of that work that was put in to develop a certain capability, if there is no support to bring that technology to higher TRL levels. So, when I say support, I mean an actual program that champions that technology, or a certain program that provides the capital or the funding for that technology, then there's a very strong likelihood that technology it'll simply die or disappear.

And we call that the valley of death, these are valleys of death that are identified. You can see them on the chart, we have one valley of death located here. I'm gonna try to use my mouse to highlight this, at the proof of concept and prototype phase. And basically, what happens here is that you develop a capability, technology, government invests, maybe you find industry to invest, or they have their own IRAD activities, and so they invest their own funds to develop these new technologies.

But eventually, over time, someone has to pick up the torch. Someone has to say, yes, this is worth investing in, and I'm going to map this technology readiness level onto a program that will pick it up and carry it forward. If that does not happen, then we end up in this valley of death here.

And if we can carry it across the valley of death, then there's possibility for a venture capitalist, or for some other investors to basically pick it up and carry it across the finish line. The challenge, and I'll call this a risk and an opportunity, is that when trying to bridge the valley of death, there are many different opportunities, many different entities that one can approach to try to carry this technology forward.

If you're lucky enough to be in an area of research that is supported by government, for example, like hyper-sonics is, we have programs in place now that their sole purpose is to try to bridge that valley of death. And so, we have very purposeful S and T roadmaps, very purpose technologies tied to those roadmaps that are brought across TRL 3 and 4 and are brought into prototype phase and into eventually fielded capabilities.

So, that's a good news story, not every technology fits into that category. And when it does, you're then left looking for other options. Some of those options might be US friendly, either US government or allies, or some of those options that individuals may seek out for investment may not be a friendly ally.

And when that happens, we have seen evidence of technologies basically being invested by foreign entities that basically leverage these technologies and capabilities, and ultimately end up working against the US and our allies. So, I call that a risk, but I also call that an opportunity. When we see that demand signal, when we see that pull coming from our adversaries, and they're trying to basically recruit either through workforce, or they're trying to recruit or adopt technology that others have developed.

When we see that demand signal, that's an opportunity for us to identify a gap that our adversaries have, they're trying to fill. And we can also use that as basically a signal to try to mitigate those risks of technology loss. So, the valleys of death, or the TRLs, I would argue, are both risks and opportunities.

And I think depending on how we see those signals, there are certainly ways for us to fold at least elements of that discovery through the TRL maturation process into our own policy.

>> John: That's a great thing, inflation this is true, this picture, too, the replacement of the government by venture capital.

 

>> Kelly A. Stephani: Sorry, can you repeat?

>> John: You have the government being replaced by venture capital, it doesn't happen automatically.

>> Kelly A. Stephani: No, not at all.

>> John: And do we want that to happen with hyper-sonics, in commercial phase?

>> Kelly A. Stephani: That's an excellent question, so to the first point, John, this chart, this is an example.

This is taken from an energy sector, so I completely agree with you that government, especially in hyper-sonics, that's present throughout the entire maturation cycle, do we want venture capital to supersede, if you will, government investment? That's a very good question, and I don't know that I have a great answer for that.

I think that's a perfect question to pose to our economists and policy experts. And the reason I take that position is that government is known to be methodical, and in some cases, slow. If there's a way to accelerate technology maturation and accelerate innovation and adoption of new technologies, then I think that can help to accelerate our technological readiness.

But at the same point, when government oversees and sort of manages technology maturation, they basically can control those levers and determine to what extent technologies are open, to what extent we protect that information. And so, that's an important component of advantage, right? If we go to the venture capitalist, if we go to the private sector, again, IRADs are one example, their innovation cycle can actually be considerably shorter than what we see on the government side.

But there's limited ways to protect that information and to ensure that it doesn't get into the wrong hands, so to speak. So, I think there's a bit of a tension there, and the question is, which one is the right way to go?

>> John: I recently saw the movie Openheimer.

 

>> Kelly A. Stephani: Yeah.

>> John: Now, that didn't seem to be a case where the government was slow, because, what, in about three years, we went from a physics paper-

>> Kelly A. Stephani: That's right, yes.

>> John: To production at scale, so it's a matter of will and resources.

>> Kelly A. Stephani: Exactly, and that's an excellent example, so there are a number of considerations, I think, that have to go into that equation, right?

When there is a demand signal, when there is an absolute need and an urgent need, then yes, there are mechanisms like OTAs, for example, are the most, I guess, modern mechanism that allow us to accelerate when needed. So, the question then becomes, well, in hyper-sonics, for example, is it a matter of need?

Is it a matter of speed, where are we right now? And I would argue that there are, as Honorable Hsu has stated many times, there are many different tools in the toolbox, right? So, is hyper-sonics the be all, end all? It's not the silver bullet, right? But is it something that we need to accelerate?

And that's always a question then of well, how would these be fielded? How would they be used? And so, those are, I think, questions that discussion as well. But yes, very good point. Any other questions before heading into national security risk opportunities?

>> John: Do you think there are any best practices for those purposeful activities to get through the valley of death that you've seen.

 

>> Kelly A. Stephani: Can you,

>> John: So, the infrastructure, I need wind tunnels, I need infrastructure to do the testing, I need access to operators to be able to understand how to use it, I need things like that.

>> Kelly A. Stephani: Okay, sure, so what have we done well, I think that's one way to kind of pose it.

So, what we did really poorly, let me start there, what we did really poorly in the last 30, 40 years perhaps, is to basically outsource important elements or important aspects of our manufacturing base, industrial base. We've certainly ramped down in the last 30, 40 years on test infrastructure, and then the other piece is the workforce development.

Those in my mind are becoming good news stories now, workforce development is, has received a huge push from DoD, specifically through very purposeful, very targeted workforce opportunities that basically bring talent through university system. And then they're committed to working with the DoD so that we can replenish an aging workforce.

As far as the test infrastructure is concerned, that's another.

>> John: How long is that commitment to working in DoD?

>> Kelly A. Stephani: It depends on the program.

>> John: I know of people they go to the government, they get to a particular agency where they get training in big data analysis.

But of course, the commercial people pay a lot more than the federal government. So basically, the government just basically trains somebody who then goes on and works in the commercial sector. So, this development of workforce, yes, it's developing a workforce in the US, but meeting defense needs is, I think, not so clear, because people can't make people a slave, and the thing is that the government will not pay them what they can make elsewhere.

And so, I don't know what the return is in that investment.

>> Kelly A. Stephani: Yeah, so as far as the workforce, that's an excellent point. We've seen that as well, that there's a demand signal, and students also see what the opportunities are, so they make that choice. Do I want to say work for the defense industry as far as commitments?

Again, those depend on which program you're involved, I think the smart fellowships, those are common after graduation. There are also other activities that are, well, let me just say that in the defense industry, private sector does very, very well with recruiting students, especially in hyper-sonics. We've seen clear evidence of that, there is a demand signal there.

If you're talking about other areas where you don't have the same level of support, you can take any of the other examples that we have in national defense strategy, lethal autonomous systems, for example. Do they see that same pull, possibly not, again, just given that they don't have a program that's targeting specifically workforce development in that space.

So yes, there's plenty that still can and should be done policy wise to guarantee that workforce and commitment.

>> John: I wonder if you've thought about where this framework applies or what classes of technologies it applies to, and where maybe you might have more skipping across steps or where this kind of breaks down, I can see the, I think it's very useful letting the themes of this working group.

Areas where, I'd say, markets are constrained, you have perhaps fewer sellers, fewer suppliers, and fewer, even just one buyer at the other end, you can kind of sketch a restructured path, between those two fronts, put in financing as needed, plasma thruster, hypersonic, how does it work for other sorts of innovation?

I come from an energy background, this uses similar sorts of things, but other classes of technologies where you see jumping across TRLs. SpaceX talks about sort of their inner process as being like a spiral technology development process, which I think is sort of trying to get away from the sort of linear conception of TRL progression over time.

Yeah, are there alternate models, or is it helpful? How does it adapt to different technologies and models?

>> Kelly A. Stephani: That's an excellent question, and I think one way to think about it is maybe the opportunity is short circuit, if you will, to leap across TRLs, that you don't have to walk your way up and progress through years and years of development.

What that example might look like, if it's a combination of capabilities, you can maybe take a technology that's already been demonstrated at TRL 3 and then combine it with another technology at TRL 3, and then bump that straight up near to a flight ready capability.

>> John: What's an example of that?

 

>> Speaker 3: So, I can give one, the Air Force was trying to do the same thing when he was the senior acquisition executive with regard to early proof of concepts for hypersonic, which was, let's not reinvent the wheel. But let's take what we already know, ballistic missiles, some of those other technologies, and can we piece these things together in order to develop like an initial kind of rapid capability?

Right, that was kind of some of the things that strategic capabilities office was kind of tasked to do was to look across and say, how can we Frankenstein a new capability together so that you don't have to start a TRL 1 and work your way through? You already have a couple other iterations that you can work through.

 

>> John: How does that turn out?

>> Speaker 3: I can't speak to that unfortunately.

>> John: Your phrase Frankenstein gave me some prediction as to how it turned out, but you can't talk about. Okay, well, I have one that I can talk about, by the way, I'm Austin Street, I'm an Air Force fellow as well from CSAC.

The C-17 was paired with long-range rockets and missiles, and they put those two capabilities together to now basically turn the C-17 into a bomb truck that can fly in formations with.

>> Speaker 3: I think kind of the question, I want to go back, I think it kind of piggybacks off of some of this discussion is where you have seen strictly the Department of Defense find some level of success.

Because if I look, I like this chart, even though it's from Maryland, from a different kind of industry. But this idea that our system is built so that after that research and development, like that's where the program of record effectively has to come into play, and those don't get killed off very easily.

So, I'm more concerned, more curious your perspective on how this has been successful maybe across the hypersonic range as you look at, right, you're trying to obviously spread risk across. Hopefully, multiple different technologies to do so. But also understanding that these programs of record are very hard to be able to steer between technologies very rapidly whenever there's failure kinda later in production.

 

>> Kelly A. Stephani: Yeah, so that question about program of record, we don't technically have one yet, so it's almost impossible to even project. But given that the combination, if you will, or the, I suppose you can use the Army and Navy as a Frankenstein example as well with all around.

That commitment, I'll say that commitment, that demonstration of commitment of the two services to take a common all around and use existing platforms to field. I think that also, you still have to work your way through the integration readiness, right? But that idea of taking existing platforms and making one comments that we have one common manufacturing base to support production, I think that's a success story as well.

Will it accelerate what otherwise would have been a longer program of record establishment? I can't say, simply because we can only really compare how fast were we able to do it versus how fast our adversaries been able to do it. The two examples obviously come to mind are Russia and China, there are arguments over how effective Russia's platforms are, China, I think, has demonstrated.

So, if you look back in that history and see how long it has taken them versus how long it is taking us, I think we'll be able to look back and say that we didn't manage to accelerate. Because of these steps that were taken to have common features, if you will.

 

>> John: One of my colleagues here when I came to Hoover was a guy named Ed Teller, and he didn't believe in waiting to see what the enemy was up to. He wanted an H-bomb to be the first to do it, and that disappoints me when I saw that they had hyper-sonics and we had nothing, and nothing coming, that bothered me.

This is like another Sputnik, Teller, very controversial character, very young, interesting guy. But he said, let's get one now and be ahead, be the first, and that seems to be missing in our policies.

>> Kelly A. Stephani: Right, and I think to some extent, that was a question of, do we or don't we weaponize, right?

We've known how to do hypersonics for a long time, It's just having that.

>> John: And there was serious debate with the H-bomb to, it was a serious debate, but Teller pushed a point, we need to be first.

>> Kelly A. Stephani: Right, right, I think we're at that point now. I think, once the program record is established, then we can look back, but we're definitely at that point now where it's going to happen, just a question of when we finally have something fielded.

 

>> Speaker 3: So, one thing about this chart, government phases out, why is that?

>> Kelly A. Stephani: In fact, it should not, and it does not.

>> Kelly A. Stephani: I'll pull a better chart next time, but yes, government, depending on, we need to have military, we need to support our military. And so, government in that case, in this example, it's definitely present throughout the entire technology risk.

 

>> John: Also, if you compare Russia and the United States, that's going to be a big difference, right?

>> Kelly A. Stephani: Yeah, absolutely, right, right. And I mean, China is of course evolving, as they acquire more and more of their private sector, and basically fold that into government or fold that into the military sector.

You're going to be seeing a much larger swath of involvement depending on how the US pursues either DoD related technologies, or others, that will determine that balance, of course. And so, a few good examples of leveraging private sector. Of course, DIU has been very successful in leveraging the private sector technologies and bringing those to bear for defense.

And so, that might also change the ratio as well, only time will tell.

>> Speaker 4: Kelly, really quick, just to keep pulling the thread on this one, so as you look at more and more of these dual use technologies and the government venture capitalists, in some cases they're now switched.

We have the VCs actually doing most of the initial thing, does that create more opportunities or what does that do there? I'm thinking things like space, cyber, where you mentioned the Space Act, that wasn't government investing flight or automated flight safety, that was completely industry.

>> Kelly A. Stephani: So, I think that's exactly how we're going to innovate faster and identify emerging technologies and bring them to bear sooner.

Again, that sort of tugs at that tension of, how do we then ensure that investments, if government does invest, how do we ensure that those technologies are protected? And that we are not essentially shooting ourselves in the foot S and T wise. So, that is going to be, I think, a very important question to address.

I'm not a policy person, but I'm happy to have those conversations with the very smart policy folks to help understand what that balance needs to look like. So again, DIU, they're very much in the stage for what that conversation needs to be. And again, they've just done such a good job of identifying and then pulling in those technologies from the private sector.

So, I think that care has to be taken, but just the fact that we can innovate so much more rapidly now because we bring in that element. I think we can't go back to a fully government driven enterprise that sort of drives technology insertion points, if that makes sense.

 

>> John: So, as the iterative timeline for the generation of technological development has shortened, and particularly in emerging technologies, things are developing rapidly. The kind of cycle you've laid out here, it really is as a government, we're letting everybody kind of compete and then we're selecting the windows for a long-term champion.

It doesn't seem like that program is gonna work going forward. If we have this iterative technology cycle, how do we adjust, or how do we kinda capitalize on our technological innovation. If we can't use the same model while we're back in the winters and we gotta produce at scale, but we gotta do it at a much shorter iterative timeline.

 

>> Kelly A. Stephani: That's a great question, so you're ultimately going to be limited by, in the case of manufacturing readiness, how quickly can I actually turn out components? If you are not supported by that base, then that's going to limit, can I produce at scale? The question though about, basically picking, if you will, one performer or a couple of performers that sort of make it through the sort of TRL bottleneck, they come out as the winning bid, if you will, on a certain program.

I guess I wouldn't consider that like a failed model, simply because you do have to have competition and there are limited resources. Should one go back and possibly revisit and try to work out how technology is inserted to improve. One case that we could use as an example is the CPS, that program of record by design, is going to be purposefully and intentionally, deliberately inserting technology as those years go on, right.

Conventional prompt, strike, apologies, that's the hypersonics platform for DVD. They're starting off with a capability that doesn't have all the bells and whistles. Those were added on as tech insertions down the road. And so, that does provide an opportunity as new technology is developed. Again, it has to be something that's feasible, has to obey the laws of physics.

But if you see an opportunity and you can skip past the, somehow combine effects, skip past the TRL, that would at least allow for new tech insertion, and I think that's part of the design. Can it be done better, absolutely but at the very least, that does allow for new innovations to be inserted down the road.

 

>> John: Plan to Frankenstein.

>> Kelly A. Stephani: Plan to Frankenstein. Yeah, okay just a couple more charts, if I may, and this is one that I wanted to address specifically because of the very important role that Stanford faculty play in technology innovations, technology maturation. Both through the work we do in the labs with government funding, basic research, but also in the private sector.

So, many spin offs, so many startups that come through from Stanford, just tremendous innovations that have changed the way that we live. And the question is really given that this is all incubating in an academic environment, what is our situational awareness as faculty and with students working in our labs.

How do we understand what the opportunities are, but also the risks are as far as national security. So, what I did is I just kind of overlaid this red map highlighting a few national security risks and opportunities, a few identified here. Academic espionage, IP theft, and also recruitment, especially through thousand talents, is probably one of the more well-known programs.

That faculty have to at least be aware of and have options to participate in. And so, the discussion, I think around this is that we are academics, we're conducting research in an open environment. How do we foster this open environment and enable research at the level that it needs to be performed and have those conversations about discovery while still protecting our national investments?

And when I say national investments, again, it's easy to go back to DoD and say, well, if it's DoD investment. Obviously, we need to have good understanding and situational awareness about what this risk environment looks like. But on the other hand, we also do work with the National Science foundation.

We do great basic research that should be open. So, how do we move past that risk and opportunity debate? So just a few examples, many of you have probably seen these before. These are stories that have been in the news over the last ten years or so. But there are very real cases in which DoD invested in certain technologies and those technologies ultimately ended up in the hands of an adversary.

Are they being weaponized right now? Maybe, maybe not, are they part of our adversary's portfolio now? Are they using what they've learned from these interactions now build on other capabilities? That's what the academic research enterprise is intended to do. So, I guess the main point here is that if there is a need to protect intellectual property or developments that are at a sufficiently high TRL that the US is investing significant money in.

And they want to keep this as a protected piece of intellectual property or information. There needs to be a conversation about how to implement that, how to implement it effectively without harming the research enterprise that we require in academia. To that note, obviously there are new policies in place, how they're being implemented, I think it's a little bit too early to tell.

The first that I wanted to highlight was the new policy that came out from the DoD. This was rolled up by honorable HSU back in June of 2023, which is basically providing guidance. And in some cases, stipulations about how to strengthen efforts to counter the unwanted foreign influence on DoD funded research.

So, I won't go through all the details there, but ultimately the recommendation is that DoD funded activities cannot be tied to certain entities that are known to develop technology for military purposes. And so, there are certain universities identified, and those are then folded into sort of a policy recommendation from DoD, does that need to be stronger?

Perhaps, like I said, we've just gone into the new fiscal year, so I think this is the first time that we actually see how the policy is implemented. And it'll be very interesting to see how that actually shapes out as far as research enterprise. And of course, the last piece from last week, right, or two weeks ago.

The Five Eyes principles were rolled out during the visit with the Hoover institution, and those are listed here to know your threats, your business environment, etc. The one piece that I do want to emphasize, when it comes to knowing your threats, it's not clear. It's never perfectly clear what a threat looks like, it can come in the form of basically an email spam.

It can come in the form of actual academic human espionage and so it's not for academics especially. It's not the first thing on our mind, maybe it should be, but it's not. If we see something that looks suspicious, of course we say something. But oftentimes, this espionage or IP theft occurs, and we kind of look back on it and say well, wow, I didn't really see that coming.

So, I think it's not enough to just say, you have to know your threats. We have to actually understand examples, specific examples of what those threats are and what to do about them before it's really too late. And we've lost technology to adversaries. Okay, so just a final note, in order to make these leaps across the valleys of death, I think we have three key items that we need to underscore.

The first is understanding our technological need, so it's not enough just to develop technology capabilities. We have to understand how they're actually going to be adopted, that will at least provide connective tissue across that valley of death workforce pipeline obviously is important. And then we need to have technology transition advocates or at the very least be informed of what the DOTMLPF requirements are so that we can make more targeted technological innovations.

And then finally, for protecting technology and innovation, again, I think there's a need to strengthen the requirements for the DoD-funded activities. All of that is tied back to honorable shoes policy from June 2023. And if we decide to really push on this path forward with the private sector, we need to incentivize those sectors to secure innovations.

How that is done, obviously that needs to be another conversation, but I think it's an important piece. So, I'll end it there, thanks everyone very much for the discussion and questions, and happy to continue discussions, we've got a few minutes left.

>> John: So, many of us heard the bye byes not too long ago.

And one of the questions what do the civilian agencies do? Civilian agencies do? Do you have some civilian agencies on the table? FBI director, so do they fit into this? You didn't mention this too much, where does that fit in? What do we do that are not currently anymore?

 

>> Kelly A. Stephani: So, the civilian agencies, I wanna think about this one. So, many of my interactions are with government. So there needs to be guideline in place for what protection means, from government is relatively straightforward. So, we have classifications on the research that we conduct. If it's with private sector, most of the conversation is around non-disclosure agreements, intellectual property.

So, there's protection in that sense, but it's not protection in the sense of.

>> Speaker 3: Just on that issue I mean, it's earlier securing the sort of deity fund. So, much security and stuff that's almost diminishing issue. One of the responsibilities elsewhere, I just reflect on one issue that's live today, and I don't have the right take on this is, is a question of outbound investment controls in private sector.

Institutionally, treasury has basically said, we don't want this, we don't want anything to do with this, it would be falling under their authority. And they, for better or worse, they see themselves as encouraging markets and exchange across borders as well. Everyone's telling them, well now you gotta care about what American VCs are doing in China.

And they say we don't want anything to do with this, and so, in this delay back and forth, you get sort of interagency wrangling with different people pushing on each other, congress, that stuff. Nothing happens, that's the right result, but it's almost like the conversation is not even really big.

It had just because I think success means different things for different agencies. And treasury sees the costs of doing that. And maybe some of the security folks see the cost of not doing it and they just kinda talk past each other I don't know how, how do we resolve this sort of thing going forward?

 

>> Kelly A. Stephani: I don't have an answer, but I can tell you that we're looking at it. There are a number of signals that can be leveraged if that information is provided. Can't speak to the specifics, of course, but that is an element of this sort of situational awareness that we have.

So again, do we try to block innovation, no, right? Or do we try to gather information from it, characterize it, right? And so, that is one element that can, and I think should be used to understand, for situational awareness what are we dealing with, where are the gaps, what are we seeing for demand?

And how do we now pivot, how do we counter that strategically. So, I think that's a really good and important point.

>> Speaker 3: This discussion with export controls and commerce says, well, you don't stop the export of technology. You dramatically expand the number of things that need licensing, notification for export.

So, at least, you know you have a better sense of where these things are being sold and how technologies are flowing. There's overhead to that, it's not costless without actually controlling the flow of those things.

>> John: Yeah, do we acquire technologies from our adversaries? So, for example, conversation about how we're an open ecosystem and we have all this great stuff and protect some of it, but it's hard because of the trade-offs with the oPEx, I get all that.

What about the other guys, do they have this problem to? Or do they not have technology that we need to acquire, or are we unable to acquire their technology? Is this specific only to the United States? In other words, the presentation we made, or could we have the same presentation in Chinese language, Russian language?

 

>> Speaker 3: You have six eyes.

>> Kelly A. Stephani: Yeah, excellent point, you're very correct, I am speaking from the blue side, the same conversation can we had from Brad, yeah.

>> John: And so, maybe there are some lessons from that other side, as it were for us, both positive and negative. Maybe we can't talk about that because there's a veil of secrecy over some of the key issues, which is fully understandable given what's at stake.

But I just wonder sometimes we seem to think that we uniquely have these problems, and why doesn't the other guy have the problems? Or how come we only have these problems, I wonder if that would help to open that up a little bit.

>> Kelly A. Stephani: So, that's an excellent point, and to that point, that is directly tied to activities that I pursue, basically, to understand, again, we don't wanna block innovation.

We want to understand so that we can strategize and leverage, right? If we can leverage that information that we have so everyone spies on each other, right, let's just take that as an assumption. If we can leverage that information in a smart way, strategically, that can and should, I would argue, give us an advantage.

So, we should use that information for sure.

>> John: I've gotta stop, though thank you so much for your wisdom, words.