stanford-university-alumni

"In this commentary, I advocate for more detailed incorporation of in , to improve our projections of . Current Earth system models display relatively low predictability of stocks, which limit our ability to estimate future climate conditions. A more explicit incorporation of microbial mechanisms can increase the accuracy of ecosystem-scale models that inform the larger-scale Earth system models. Of the numerous microbial groups that can influence soil C dynamics, AM fungi are particularly tractable for integration in models. Arbuscular mycorrhizal fungi are globally abundant and perform critical roles in , such as augmentation of net and soil C storage. Moreover, AM communities exhibit relatively low diversity within ecosystems, compared to other microbial groups. In addition, global datasets of AM ecology are available for use in model development. Thus, AM communities can be readily simulated in next-generation trait-based models that link microbial diversity to ecosystem function. Altogether, we are well-poised to incorporate the dynamics of individual AM taxa in ecosystem models, which can then be coupled to Earth system models. Hopefully, these efforts would advance our ability to predict and plan for future climate change."

"Nitrogen (N) enrichment is an element of that could influence the growth and abundance of many s. In this , I synthesized responses of microbial to N additions in 82 published field studies. I hypothesized that the biomass of , bacteria or the microbial community as a whole would be altered under N additions. I also predicted that changes in biomass would parallel changes in soil CO2 emissions. Microbial biomass declined 15% on average under , but fungi and bacteria were not significantly altered in studies that examined each group separately. Moreover, declines in abundance of microbes and fungi were more evident in studies of longer durations and with higher total amounts of N added. In addition, responses of microbial biomass to N fertilization were significantly correlated with responses of soil CO2 emissions. There were no significant effects of biomes, fertilizer types, ambient N deposition rates or methods of measuring biomass. Altogether, these results suggest that N enrichment could reduce microbial biomass in many ecosystems, with corresponding declines in soil CO2 emissions."

"I asked whether —applies to microbes. I conducted a synthesis of empirical studies that tested relationships among microbial traits presumed to define the competitive, stress tolerance and ruderal, and other ecological strategies. There was broad support for Grime's triangle. However, the ecological strategies were inconsistently linked to shifts in under environmental changes like nitrogen and phosphorus addition, warming, , etc. We may be missing important ecological strategies that more closely influence microbial community composition under shifting environmental conditions. We may need to start by documenting changes in microbial communities in response to environmental conditions at fine spatiotemporal scales relevant for microbes. We can then develop empirically based ecological strategies, rather than modifying those based on . Synthesis. Microbes appear to sort into similar ecological strategies as plants. However, these microbial ecological strategies do not consistently predict how community composition will shift under environmental change. By starting ‘from the ground up’, we may be able to delineate ecological strategies more relevant for microbes."

"When trees build wood, is incorporated. It takes a long time to decompose ... It’s a smaller scale in . Some of the material they make can be almost woody. The is tough to decompose. The cell walls stay in the soil, microscopically. ... It adds up to a lot ... twice as much carbon in the soil as in the , and much of that is in [fungi]."

"... I would give every single person in the department an entire month anything they wanted. For a month. It’s kind of insane because you’re talking thousands of people for a month, millions of dollars of salary spent for a month for people to do whatever they wanted and they would work so hard that month coming up with incredible ideas. In fact, one I saw was this idea of a 10-foot user interface, and we turned it into the Apple TV. Apple TV was invented because someone was encouraged to do whatever they wanted for a month. You can have that kind of environment to support creativity."

"... That guy was talking about how Microsoft had solved ... tablet computer and they're gonna do it with pens and he just, like, shoved it in Steve's face the way that they were going to, like, rule the world with their new tablets with their pens. Steve ... show them how it's really done ... At that time, touch screen was resistive touch ... He said, "we need to do capacitive-touch and has to be multi-touch." The moment you saw that, you knew this was the way to go."

"Siri is your humble intelligent personal assistant that goes everywhere with you and can do things for you, just by you asking."

"Beyond individual intelligence, nature has also cultivated intelligence through swarms. For example, bees, birds and fish act in a more intelligent way when acting together as a swarm, flock or school"

"The reason that fish form schools, birds form flocks, and bees form swarms is that they are smarter together than they would be apart. They don't take a vote; they don't take a poll: they form a system. They are all interactive and make a decision together in real time."

"How does nature amplify the intelligence of groups? It forms swarms."

"Taking a vote or poll is a great simple way to take a decision, but it doesn't help a group find consensus. It actually polarizes people and highlights the differences between them. People end up getting entrenched in their views."

"Forcing polarized groups into a swarm allows them to find the answer that most people are satisfied with"

"A poll will give you the most popular answer but not the answer that optimizes the preference of a group."

"We take the sense of touch for granted. Think about it: Without it, you're missing one of the basic senses that enables you to interact with the world."

"We focus on a unique form of artificial intelligence called artificial swarm intelligence"

"You have the sense of touch because you need it."

"And so any approach that works effectively in a domain like this is going to have to handle very large spaces, very long horizons, and really lots of uncertainty. So we have kind of a standard structural decomposition to this problem. We call this belief space hierarchical planning in the now. I'll decode what that means a little bit."

"Fundamentally, the way we think about it is that we decompose the computation that's in the robot's head now into two parts. The first part is in charge of taking the sequence, the history of actions and observations, and trying to synthesize them into some representation of a belief for a probability distribution about the way the world might be and then another module that takes that belief and decides how to behave."

"So the dimensionality of the space is kind of unthinkably high. It's also not exactly clear what constitutes an object here. If you were going to behave in this world, it would be a very long sequence of primitive actions that you would take in order to clean this kitchen. And also there's just a fundamental amount of uncertainty in this problem, right? So you don't know what's in the blue bowl or what will happen if you try to pull out a certain thing. You don't know when the people are coming home or what they want for dinner all sorts of stuff you don't know."

"The lesson to be learned from this is that it is often undesirable to go for the right thing first. It is better to get half of the right thing available so that it spreads like a virus. Once people are hooked on it, take the time to improve it to 90% of the right thing"

"OK, so what's my research goal? I come from the machines end of this world, roughly. And what I really want to do is figure out how it is that we can make intelligent robots. And I do this mostly because I'm interested in intelligence more than I'm interested actually in robots. But I think that trying to make a physical agent who goes out and interacts in the world is a really good test bed for understanding what kinds of reasoning and perception and control we need in order to make it an intelligent system."

"reuse is properly a process issue, and individual organizations need to decide whether they believe in its long-term benefits."

"For instance, everyone's used to the idea of convolution now in image space. But if you think about it. And I've had people tell me who work on convolutional neural networks that they don't build any structure into their system. It's just a neural network. But of course, as soon as you build the compositional structure into a neural network, you are taking a position on some regularities that are in the input signal and so on. And you're taking advantage of that so that you don't have to learn a whole fully connected network, but you just learn some compositional kernels."

"So one set of possible kind of general ideas includes convolution in space also and time. Maybe understanding the kinematics of the system that it's connected together in joints and segments, a notion of planning to move through space, being able to do causal reasoning-- if I were to do this, what would happen? -- Abstracting over individual objects, various kinds of state and temporal abstraction and so on. So our view-- I don't want to commit to a particular list-- but is that there's a list of structural principles that are pretty generic and very broadly useful and we should build them in."

"And just again, to kind of give you the motivation. I really want a robot. This isn't my kitchen, by the way, just in case you were worried, my kitchen. But imagine that you had to clean this kitchen or make breakfast in it or something. It would be very hard. And imagine programming a robot to do it. It's extremely hard. And so one thing that's useful to do is to think about what makes this problem hard. So one of the things that makes it hard is that there are lots of objects."

"So just as convolution gives us a great leverage when you apply it to the right part of the problem, then the intuition as well, hopefully there there's a few mechanisms, hopefully not like 100 mechanisms, but maybe 10. And then if we figure out how to use those mechanisms to bias learning and to structure behavior that we could learn robust ways of behaving that are efficient and so on."

"So I'm still there, OK? So I'm still in I'm still trying to figure out how we can design an architecture that can learn efficiently. And so the research strategy that I have really adopted, I work closely with a colleague, Tomás Lozano-Pérez. Our strategy has been the following, which is to try to think of some very generic representation and inference mechanisms and build those in and then figure out how to learn the rest of the stuff. And we're all used to I think by now the idea of some representation that inference mechanisms that we would want to build in."

"The problem with traditional approaches to abstraction and encapsulation is that they aim at complete information hiding. This characteristic anticipates being able to eliminate programming from parts of the software development process, those parts contained within module boundaries. As we've seen, though, the need to program is never eliminated because customization, modification, and maintenance are always required-that is, piecemeal growth."

"It is only when we forget the ideas behind building something wonderful that we can actually do the building that makes things wonderful.”"

"Programmers are not mathematicians, no matter how much we wish and wish for it.”"

"OK, I'll keep going. I'll surely be able to offend some people soon. And I'll work harder at that. OK. so if we kind of accept this idea that we're going to build in some structure, then what? And the thing that my colleague and I have done recently. Well, now, maybe not super recently, but recent. In order to test out the idea that there's a set of mechanisms that would work well, what we did is we hand built the rest of the system. So we hand-built some transition models, inference rules, ways of doing search control and so on and connected them up to these general mechanisms and made a system."

"So I imagine that there's some distribution over possible environments that the robot could find itself in when it actually goes out into the world, right? So maybe it's going to go to houses and the houses are all somewhat different. And once I put that program in the house, maybe it's going to do some estimation or learning. It's going to adapt to the circumstances it's in. My job is to find a program that does a good job of adapting in all the environments that might find itself in."

"So imagine that you have some kind of probability distribution over the worlds that the robot could actually end up operating in. I want to find a program that's going to behave well, let's say get a lot of reward in expectation on average over all the environments that it could possibly find itself in. So that's, I would say, kind of a reasonable formal objective for a robot. And one thing that's good about this as an objective is that we don't have to argue about it, right? It doesn't say whether there should be learning in there or what kind of learning or should it be a genetic algorithm or should it have planning. In some sense, you could say, "I just want to make the program that's going to be the best that can be on average over these environments.""

"And so what happened was I programmed the robot. And it would run into the wall. And I would bring it back. And I would fix the programming. And it would run into the wall again, hopefully for a slightly different reason. And over the course of a couple of weeks, I managed to write a program that would use these funny sonar sensors on the robot and make it drive down the hall without crashing into the walls. And so that was good. And I was happy, in a way, at the end of that, that I had gotten it to work. But I reflected on that a bit more. And what I decided was that I had learned how to navigate down the hallway using the sonar sensors. And what I thought was that-- and it had taken a long time. And I was kind of a hassle. And really, the system should have been doing the learning, not me. So my view was that I should figure out a way to get out of the loop to build systems that could learn on their own to do stuff. And then I could just wait for them to do that. And that would be better. So that was flaky."

"reuse is easiest within a project instead of between them. A manager's success depends on performance on a given project and not on performance over several projects. And preparing code for reuse requires additional work, not only by the reuse expert but also by developers. Therefore, preparing for reuse has a cost for any given project."

"Then I sort of reinvented reinforcement learning in a not very good way, really. But it was kind of entertaining. And I this is a slide by the way for those young people in the audience. You might know, but back in the day, we used to write with colored depends on pieces of clear plastic. And that's what we used to give talks. So I had this kind of pseudo reinforcement learning thing. And by 1990, I actually had this little robot called Spanky that did actual reinforcement learning during my actual defense. So it didn't learn anything too complicated. But it did do it in real time. So that was kind of fun."

"But the problem is now I've written down an objective function. I've said, "Oh, if you could tell me a distribution over possible worlds that you'd like this program to work well in, then I know in a certain mathematical sense with the best program is." But now my problem as the engineer, as the person who is in the robot factory, which is again, the kind of maybe analogous to the problem of nature, is I have to figure out how do I how do I find this program that's going to be good and all these situations?"

"In some sense, this is a thing that people have thought about for a long time, right? This would be like running some kind of evolutionary algorithm or some search or simulation inside the factory. And it's very attractive, but I think generally speaking, hard to make work well. So the question is what should I do, right? I could maybe I can set up this whole evolutionary setup somehow. And then I could just snooze for a really long time while some very complicated program tries to figure out the best robot program to put in the head of the robot. But I don't know. I am simultaneously too impatient for that."

"And so then the question is can I somehow take pieces and parts of all these ideas, some human programming, some robot learning in the wild, some kind of search or evolution offline, some inspiration from humans. Can I take all those things and put them together and see if I can find a way to engineer intelligent robots? So that's basically what I'm up to."

"So if humans have to engineer a very complicated system, then it has to be the engineering process has to have some modularity to it because humans are really bad at understanding one big messy system. They're good at understanding pieces and parts that work together. So it may be that we have to take a modular design approach in our engineering efforts for intelligence, not because the intelligence needs to have that architecture, but because we, the human engineers, need those tools for actually building a system."

"I'm going to-- well, no. OK, let me say something about this. So then the one way to view the research agenda is to say that first of all, I'd like to be inspired by what we know about humans. And in particular, I'm very interested in this bulky core knowledge type stuff because that tells me something about what evolution, in some sense, saw fit to engineer into natural intelligences. And if I understand that natural systems seem to be born with a bias or some built in structure to think in terms of other agents, to understand that they move through 3D space, to talk about, think about objects as clumps of matter that cohere, that's a very helpful engineering bias for building a system."

"Habitability is the characteristic of source code that enables programmers, coders, bug-fixers, and people coming to the code later in its life to understand its construction and intentions and to change it comfortably and confidently."

"So the way I want to think about this, the job of this program. So I'm going to make a robot. I'm going to put a program in the head of the robot. So let's say, I'm not going to worry about hardware. I'm just going to read about the software. And so the program that I'm going to put in the head of my robot, it has to do this job that's written in the formula up here. And what this is just shorthand for saying is that it has to represent some kind of mapping from observation and actions that it's had in the past. So o, a star means the whole history of observations and actions that it's ever had. Based on that, it has to pick the next action. So that's not really saying much of anything at all. That's just a description of every single robot control program basically that's been written. You have to take your history of actions and observations, compute the next action."

"The reason I want to start by backing all the way up to this like very basic control theory picture is that right now there's an enormous amount of argument about how one should make robots. Should they do planning and reasoning? Should they do reinforcement learning? How should we do it? So there's a huge kind of crisis almost in the field about what the best methods are. And what I want to start out this talk by doing is actually thinking about how we can answer that question in a way that's not political or religious, but technical."

"And so what we want to do is think about first of all, what's the best-- what would be the best pi to put inside the robot. How can we think about that? And then we have to think about the problem of how is it that we, in my case, as me, as an engineer, I'm going to find that pi that I should put in my robot."

"So one way to think about the whole problem set up then is that I, as the robotics engineer, have to do for my robots the job that nature did for you. That is to say, I have to think about I'm a robot factory. I'm going to make these robots. And the robots are going to go out in the world. Maybe they're going to go and work in people's Kitchens or something. And every kitchen is going to be different. So there's going to be a lot that I don't know about the world. But somehow, I have to figure out the best program, what program, to put in the head of all my robots, so that when they go out in the world to behave, they can do a good job so that's the way I think about the problem that I face. And in order to think about what would be the best program, I kind of think about it this way."

"So if I enumerate my options and they all don't look very good, I don't know what to do. So one thing to think about, though, is this last thing. So the kind of evolution idea. So let's just pursue this a little bit more. So imagine that we want to try to find a program that works well in expectation over all environments. One way to think about that is that inside the factory, we kind of simulate a bunch of environments. We try a bunch of robot programs. And we try to find one that works well in all those environments. And that's like a really interesting strategy. We would have to think of a space of possible programs for the robot, some objective function. We figure out, well, what are we trying to optimize, a distribution over problems to test."

"So all these constraints need to somehow come together into a way of building intelligence systems. Actually, I would stop here for a minute just because it's a convenient spot and see if there are questions. I see some red Q&A button. So maybe someone can ask."

"So the way I think about the problems-- this is kind of a definitely a computer scientist way to think about the problem-- is to think about the robot as a transducer, as some kind of a system that's connected up to the world. And it makes observations of the world. And it takes actions that change the state of the world. And presumably, there's some objective, right? We want to take actions that change the state of the world in some way that we think will be good."

"Yeah, actually. For years there has been. So a more typical formalization would be in terms of predictive models and planning or reasoning. So reinforcement learning. Also, it depends. The phrase unfortunately, the phrase, "reinforcement learning," grows and stretches too. And sometimes for many people and in many discourses, it's come to mean all of intelligent behavior, in which case, I would say, well, no it's all reinforcement learning. But that's vacuous. Other formulations involve reasoning about objects and their relationships and thinking about the long term consequences of taking actions in the world and so on. So there's really different ways of framing and formalizing the problem. And they give you very different computational profiles and different learning strategies. OK, good."