Andrew is the Vice President of Infrastructure at Dropbox. In this role he oversees all infrastructure engineering and operations efforts which are responsible for scaling Dropbox’s infrastructure stack in order to support hundreds of millions of users worldwide. Prior to Dropbox he was at YouTube, Google and AOL in various infrastructure capacities.
"If we can operationalize this, the micro decision making on the ground becomes much more powerful and it doesn't force us into a command and control environment."
- Andrew Fong
Nominations for the 2020 Inspiring Leadership Award are now open!
We created this award to recognize role models of engineering leadership, for the work they do every day to make a difference in their teams and organizations.
Share their story with us and submit a nomination HERE
Patrick Gallagher: Andrew, welcome to the show.
Andrew Fong: Thank you for having me here today, Patrick and Jerry.
What it means to operationalize value and why it matters
Patrick Gallagher: the prevailing wisdom seems to be that values mission-driven leadership is paramount to success. And I think we have some great examples of this in the Bay area, Dropbox being one of them, but actually applying this as a leader and operationalizing values at scale is extremely challenging. what does it mean to operationalize values and why does it matter? And do you have story to introduce us to this topic?
Andrew Fong: Yeah, I think that's actually a pretty good place to start.
So at Dropbox, I head up the infrastructure organization. So it's an org of roughly about 200 people, 220 people. And one of the things that we actually believe pretty strongly in is making sure we have, a fairly strong concrete set of values and mission within an infrastructure.
And, we went through the normal process of, of coming up with those. And we can talk more about that later potentially, but we have, these three values about being, reliable, efficient, inclusive, and these are meant to be behaviors as well as how we drive system design, as well as how we just operate ourselves on a day to day basis.
And late last year we were going through a design review of a major data center migration. And the team had come to the review and said, you know, I want to have a bunch of engineers. Like most of engineering have to do a bunch of these migrations. And we started just. Starting from the values. And we said, is this, going to be efficient. And everyone was like, no, not, not really. We should probably figure out a different way.
We said, do you think we're just going to be reliable and coming on time, if we have to involve a couple of hundred people and it was like, probably not, And then we started talking, you know, a little bit, like, did we talk to anybody else before we did this?
And the team was like, Oh yeah, we, we kind of skipped that step actually. So we built this back in the napkin plan. We have this plan, we're ready to go, but we haven't like actually lived up to our values. and at that point it was pretty easy to say, okay, let's just pause. let's spend a week, maybe two weeks. Do a little research, see if there's a way to do this. That sort of thing was up to our values, lives up to our purpose, which is, maximize product velocity sustainably. so this is probably not gonna maximize product velocity and probably not going to be super sustainable. so it checks like sort of all of our boxes to say like, okay, yeah, like kind of a pretty Frank conversation without anybody feeling like defensive about it, because it gave us a framework to sort of apply. Our day to day, sort of just a design review through, through that framework.
so that that's like probably the simplest example. It comes to my recent memory of sort of like where we applied them. we do them on a bunch of other places as well. OKR's, our monthly business reviews, all of that is driven also from the values and from purpose.
How to operationalize values in OKRs
When I think about sort of our OKR planning process. As thematically, we want to have things that ladder up to those values. So the actual OKR may not be, "be efficient."
but when we think about how do we think about cost savings or how do we think about developer velocity? All of those things go towards either maximizing, product development sustainably. And then they also have to be reliable and efficient. And so we look for those attributes as we look at the OKRs. And so if we have OKR is that don't match those attributes. Or if we feel ladder up to the purpose, then it's pretty easy.
"Okay. Why are we doing this? Are we the right team to do this? This is our charter, still the right charter." You know, it can lead to a second set of discussions. It doesn't mean we shouldn't do it. It just means that given the framing that we have, it holds us, honest and accountable to ourselves to make sure we're actually going after the things that we think are the most valuable.
Jerry Li: Yeah, as an OKR, I guess that happens on, multiple levels, the team level, org level of company level. everyone's sort of using the same set of values to, make a decision or asking the hard questions.
Andrew Fong: I should give a little bit of a, caveat. These are the infrastructure values. The company sets of values... ours are not orthogonal to the company values, but they're more specific to how we operate in our organization.
So, you know, be worthy of trust, make work human. All of those things are much more at the company level, but they're not in opposition to the sets of things that we care about inside of infrastructure.
Jerry Li: Got it, but essentially it's a set of criteria or things that people can use to make collective decision making it scalable.
Andrew Fong: Yep. that, that was the entirely the intent. and I have done this in smaller size org before, and we had said like, okay, if we can make this work at scale, pretty much all the research also shows that if you make it work at scale, it can work at scale.
it was really about if we can operationalize this, the micro decision making on the ground becomes much more powerful and it doesn't force us into a command and control environment where everything that gets sent to the most senior person, the most senior person okays it, you know, go down the chain. Because there's an implied, almost explicit actually, you know, if you're using this framework, right. And you have the best intent, it's very hard for... you can't get in quote, unquote trouble, right? we have to have like a question about okay, like what happened here? where's the failure, it's a failure of the mission, not a failure of the, person making a decision.
Jerry Li: I think it's very powerful, empowering to be able to leverage that set of value and anyone in the company regardless of their position, right ? Their, title level... Everyone's sort of operating with set of values. That means, one person is as, knowledgeable, as any other person to make a decision. This is gives a lot of power to the company.
Andrew Fong: Yep. That was the thesis. when we go through it, we actually have some sort of outcomes that we're looking for.
And one of the outcomes' hypothesis we had is that this should increase, we call company pulse survey score, every company runs some right. The survey about like, how, how engaged are you? How you feeling about the company?
One of the ones we have is around decision making and we had a hypothesis that by rolling this out inside of infrastructure, at least inside of infrastructure, we would see those scores that have scores increase. And we actually have. We've had sort of this operationalized probably for a little over a year plus now. and quarter over quarter, we definitely see a rise, across all metrics, that relate to empowerment. and you know, decision making is a derivative of that.
How to identify values in your team or organization
Jerry Li: how do you find or identify those values for your organization in the first place? And, do you make changes over time?
Andrew Fong: The short of it is, is it's not that hard, but it, once you've done it once it's sort of, we just did sticky notes on a board. The prompt question that we ask, we start from belief and then we go to purpose. And then we go to that values.
Where the belief is, what would we believe to be true about infrastructure, regardless of where we worked, regardless of what industry we were in. and so for us, the belief is "infrastructure is a force multiplier."
Now, if I worked in construction and I built bridges, I'd still believe that, right? Because I'd be able to get people from Marin into San Francisco at a much higher rate than if they had to, you know, take a ferry right. Like it's a force multiplier.
Then we've wrote a bunch of them about like what we thought about infrastructure and we spent about an hour just like going through those and coming up with like very short and succinct...
one failure mode that I typically see, but I see missions and I see, values is that they're wordy they're long and you can't remember them.
ours is a belief is infrastructure is a force multiplier. We did the same thing with purpose, which is maximize product velocity sustainably. and then our values are reliable, efficient and inclusive. And so it's, they're meant to be short and sweet memorable, in a way where you don't have to go and like, look it up on the Wiki, a the company and say like, okay, why, why do we exist? like what's the reason here?
And so we did this with, My reports, which include five or six engineering directors each with like org sizes, probably 40 to 50. and then some principal engineers, and we spent that cohort did it. We sent it out for review and a little bit more general form, but it wasn't, it wasn't contentious actually. It converged within maybe three hours? Like the first two values of reliable and efficient there was no controversy.
The last one around inclusive, we started with customer focus. you know, everyone starts like that. And we, I actually sort of said, the prompt question we sort of asked ourselves is like, why did we join? Are we here? and everyone really felt that they. Through the interview process and through talking to the managers that they had worked with, you know, as they came in, that it was really about that. They felt that they had a voice and they could actually get the things done.They wanted to do both personally, professionally and with their teams.
And so that sort of expanded to inclusive, where we felt like, okay, we have such a wide range of disciplines. One of the things that's makes us sort of unique in a, is an engineering org is we have everything from supply chain, network engineering, data center, software engineering, some front end, like that runs the gambit in infrastructure.
So you really have to have something cohesive to pull these people together. Otherwise, you know, you know, it's not just a single product you're building, you're shipping something from sheet metal all the way up to, you know, a data center. and then the software that runs on it. So we had to have a set of things that pulled everyone together. And so that really led to inclusive. because we just said like, we can't do this. You know, you have to include the right set of people when we go through this process.
So that's sort of the Genesis of what we've done at least in the infrastructure side. I've done it with a couple other words in slightly similar fashion, though.
Patrick Gallagher: To get clarity on the prompt question. So the belief was what would we believe about infrastructure regardless of the company? And then for purpose? , what was the generative question there?
Andrew Fong: Why do we exist here?
Patrick Gallagher: Why do we exist?
Andrew Fong: Why do we exist at Dropbox
Patrick Gallagher: Why do we exist at Dropbox? Ok. And then for the values that was, why did we join? Why are we here?
Andrew Fong: the general prompt was like, what do we value? and reliability and efficiency, were just like, so crystal clear. If you're, if you're a force multiplier, these are the two biggest levers you have for the company. These are, the metrics are sort of goaled on that. everything else is like, That was just not controversial. Then it became, what other behaviors do we want to incentivize? What are the things that we want to see.
And to be clear, like the original spin on reliable and efficient people thought about these as metrics.
And then we started to dig into it. And we said like, okay, if we think about trust, you know, trust has a big component about being reliable. Well, what makes us really not trust people? And infrastructure team was like, well, we don't trust people when they don't deliver. So it was like, okay, then reliability HAS to be a value because that's actually something that we care about for ourselves. So every time we send an email, we expect a response. So that's about being reliable. Right?
So efficient. We don't want to have meetings that run six hours that go in circles. So you want to make an efficient meeting. You want to make sure it was on time. So you want to be reliable and you show up. all of those were attributes that the org felt like they had as behavioral elements that we wanted to keep further.
So we'd actually, don't spend as much time talking about these as metrics or things like that from like an optimization function. But it's more about how we behave. Because we believe if we behave that way... systems will be reliable. Systems will be efficient because that's just the, that's the environment you're put into.
and so this is, you know, we talked a little bit of operationalizing it. Like that environment, how you behave, creates an envelope for how people will write code.
Jerry Li: when you have identified those set of values and send it out for the entire team, do they have the opportunity to, generate ideas and come back sort of a combination of top-down plus bottom up...
Andrew Fong: infrastructure. We did not do it as much top down, bottom up. In smaller orgs we have, we sent out to like the second layer of the org if I remember correctly, but we didn't go all the way down. I have definitely seen this work where you can go all the way down.
We wanted to balance sort of the time to deliver versus how far do we think the gap was? And we've had similar frameworks in the past within infrastructure. And so, trustworthy, efficient, useful was like one incantation of this.
So we knew that there was like a pretty good nucleus seed on like, what was there? And I think the company had rallied so much about increasing shipping velocity and things like that, that the purpose really resonated. Like it wasn't controversial, from that perspective.
Jerry Li: Yeah. Yeah. That makes sense. And also it's a large org and, the process that identifying and deciding the value is reflective of the value itself, like being efficient.
Are there times that, you feel there's a need to revisit ?
Andrew Fong: So one of the things, and covid's thrown a little bit of wrench in, has a little bit harder right now.
But we actually, every time we onboard a new leader, we definitely revisit cause we want to make sure they actually understand that. It's something that I think about on a daily basis. Like, are these still serving us? Have we outgrown them? Is there anything we should tweak?
The plan was actually right around this time. This is about 18 months, was going to be, to sort of do another offsite onsite to go through it. We need to figure out how to operationalize that in the COVID world in a way that's efficient, because it's, definitely a little bit harder just to get on Zoom and throw a bunch of sticky notes on a board. It's not the same.
I did it with a group the other day though with like, with an online work tool and it works reasonably well, but the goal is to definitely revisit by the end of the year, just to make sure that the values still serve us.
How to reinforce values in your organization in meetings, all-hands, and personally
Jerry Li: Now you have a set of values everyone's sort of endorsed that.
How do you encourage people to, internalize that and, turn that into a habit?
Andrew Fong: there's a bunch of different tools. my mindset is like, it needs to be everywhere all the time with everything we do. So there doesn't go a day in my week where I don't think about them or I don't apply that lens.
The easy one is, OKRs. We mentioned that we use that through the OKR process. We use it through headcount planning because headcount planning should tie directly to OKRs. So, OKR process goes and headcount planning process goes, and those should be the same. Monthly business reviews, any key decisions... we have a column that says like, how did, how do we arrive at this conclusion? Where that conclusion is framed around our values or our mission.
Metrics reviews. Like then there's the things that are not overt in your face, like directly saying we're reliable or efficient. Things like making sure we have metrics review every month. That ties directly to being reliable because our number one of our metrics we go on is reliability. So that's like part of that.
My smaller support team, just like my EA and my chief of staff. Like the two of them know that we never will cancel a monthly business review. It'll just shift because we just firmly believe that there's a set of things that if I'm only asking of 10 things of you a month or eight things of you a month, then we have to be reliable with them and we can't ever let them move. so one on ones are the same. I try not to let them move one on ones. So there's like a way I operate that hopefully starts to cascade a little bit.
So that's sort of like my own personal operating mode, cause I can definitely lead myself the best, in that sense. So I try to put a structure around how I am operating that goes by those. Then we have to like address the broader organization.
So all hands are always structured around this way. We start every all hands with a recap of, belief, purpose, and values. That's pretty simple and easy to do. and then we frame a lot of the content in that way. And when I say frame the content, we look for things that spike in those areas, other than those values around the way they around how people are using them.
but I think the one that I have found the most useful, outside of performance reviews obviously, is we run a weekly podcast that's 15 minutes. And on that podcast, I spend the time to talk to an engineer or a manager or someone in the organization. Sometimes we have guests... that exemplify one of the values. Or have shipped a project or shipped a piece of code that goes directly to our purpose, or has enabled and shows impact in that dimension.
and so that gives us a way to highlight on a regular basis, the value system in a medium, other than email, or other than quarterly, all hands or monthly, all hands, but that's consistent too, right? Where we're being in, like gives you a really easy way to look at this. And has a nice side effect of creating a library where any new hire that starts can start from the first podcast, which is where we rolled them out. Or one of the places we rolled them out. When, so they can listen from there up to where we are today. So they can also see the evolution of how the org has thought about this, how people have reflected upon it. and so they're not meant to be, you know, two hour podcast or anything like that. It's like you could listen to him in the car when people are coming to work or, you know, you getting ready in the morning type thing.
And we find, you know, an org of 200, we probably have 30 to 50 listeners a week easily. it's hard to track. Like we don't have like a Dropbox folder, so, and kind of just like do a poll on it. But we definitely see the adjacencies are at least two teams away or three teams away from the person that was interviewed. So that's typically if whatever the cohort of people, they know, they all listen. That's one of the main mechanisms.
I think the other thing that's very important when you start going through this process is that you always have people that don't believe. And what we did is we made sure that we put people on the podcast, within the organizations of the leaders that didn't believe on day one. And so we operationalized it from a very strategic point of view of, okay, how do we make sure we're showing true recognition and reward the people that may not be the deep believers of this so that they find value.
And actually one of the biggest skeptics in the room when we went through this process is the one that holds me most accountable to them now. I probably don't go through a week without getting an email from him where he's like, I saw you did this. I don't think you lived up to this value that we have. What do you think about that? and it gave him agency, right? So like actually one of my directors is giving me, give him agency to like, have that conversation with me, in a way that he didn't feel, I'm guessing he didn't feel that he was empowered to have previously.
How to operationalize values in recruiting
Jerry Li: that's a fascinating story. you mentioned, different applications of the values and one of those examples you named is recruiting.
How do you, apply those, set of values to recruiting for your organization?
Andrew Fong: There's two ways. there's the internal way that, how we behave just towards a candidate. Right. And then there's how do we reflect upon like a candidate as well?
so very easy, right? For the managers inside of infrastructure, if they don't respond to candidates, if they're not on top of that, right? Like that, just like we asking you to be reliable. So if a candidate has a question, make sure you get back to them. Make sure you're including them.
I take the point of views that you're going to hire this person. This person is going to have a one on one with, with you, at some point, you know, in the next end number of weeks, right from that point. So why are you not building that relationship before as well? It's crazy to me that hiring managers don't want to engage with the candidate pool up until the point they come in and they're full time employee because... what makes that person any less special than then, like having a offer letter? Like it makes no sense to me.
So I take that point of view. And so I try to use the values to make sure the management team is like, you know, it's not this transactional process of recruiting. Like you have to include them. You have to make them feel like they're part of this. Right. You have to actually be responsive to them. So that's on the internal side. That's the part that's just about us.
I would say that what we do is we use a company values more so in the hiring process and a lot of our hiring feedback is geared around that.
the tool we use actually has prompts around that, around the values, like, How did this candidate demonstrate the worthy of trust? How did this candidate, did they have a story about that? Did this candidate demonstrates an ability to, make work human, right?
we use the company values cause we have a standard set of interview questions that are sort of more company specific, that actually go towards the values of how the company operates.
Then once they're inside typically, cause they're not orthogonal values... that they tend to, um, Are to a little bit more specific of versus the company values. That tends to work out just fine.
How operationalized values impact projects: Dropbox's data center migration story
Jerry Li: You mentioned earlier, that there's a very big project, related to data migration. Can you walk us through, the lens of operationalized values in that project?
Andrew Fong: So the project we were talking about, we're doing a data center migration, from one of our original data centers at Dropbox to another geography and it involves tens of thousands of servers probably over an exabyte of data to be migrated, touches virtually every team inside of engineering. It's like the horizon is like Probably North of three and a half to four years. so large capital investment as well.
so as we look at that project, you know, the first cut of the project was, the team had just sort of gone in isolation and said, like, we have a plan, we've got a plan, let's go do it.
and you know, At some scale, it probably would have been fine. and what we did is, you know, went through the initial design review and when we looked at it, you know, in fact to the first story we were talking about, we said, do we think we can do this on this horizon? And we'll be reliable. We're going to commit to finance that we're going to do this. This is actually going to happen.
And everyone was kind of looking at it like saying like, we're going to involve a thousand people. I don't know. This is like, like that horizon, that law, like we have to, it's probably not going to go. Like it wasn't gonna be efficient. Wasn't gonna be reliable, it didn't include the right set of people on day one.
So the team went back to the drawing board and they put everything through that lens. And what they came back with was a strategy that was going to be multi-phased where the bulk of the work would be involved with just infrastructure.
They would involve and make sure key stakeholders were much more informed such as the finance team. the, well, I guess the substance side of the financing, I was gonna say the, the real estate team, but all of the various teams were like part of the planning process from that point on. and they established sort of a roadmap and checkpoints of where key decisions would have to be made and what criteria would need to be made at those decision points.
and so it was a very different approach from like a TPM perspective of like, okay, we have a thousand things... I'll go find a thousand people and we'll ping them every week to make sure it happens. Which is just a very different approach.
And so it wasn't that there was a fundamental set of, constraints that changed or anything like that. It just was the mindset of the team shifted. and all I did was send them like a quick email after the first meeting saying like, we should ship this.
And so when we went through that I mean, we're six, seven months in... I can say that the results that we've seen is we had to move. I think it was roughly around 800 petabyte it's of data. And we had to do it you know, over like a six month horizon. They did it in about four and a half months. The type of work I saw from the team, they were tracking. the migration on a daily basis, they're sending a weekly summary out, you know, they knew exactly how many... I'm gonna say, but they're not files. How many files had to be moved for the completion.
And we're talking about billions of objects, right? We're not talking about like something that just like easily fits in a spreadsheet. You can like, kind of go down. So they built software that tracks all this, right. And they were able to like, actually monitor this real time and track all of this in a very different way than you know, if you had just said like, okay, let's put this in a, you know, a checklist and have a TPM try to drive this.
It became a process that was much more repeatable. Cause we know we'll have to do this again someday. It became a process that was including, you know, everyone from finance to the customer teams who also would work sort of in the loop at some level about like what's happening here. just, it changed the entire team's mindset of like how to approach it.
Now, we had rolled the values out maybe three months before us. I don't fault the team at all for, you know, not finding a way to operationalize on the first time, but it was a good learning experience for the team that goes through and see this.
And what I've seen is that team whenever they look at things now it's always through that lens. And everything they do, it becomes a mindset more than like a set of tactics that they're doing. They took the number of people down from like a thousand to like something like less than a hundred. And they were able to like get started in a way that moved the bulk of itfront-loading it upfront and bought everybody like months of time as well within the project.
It was just a, such a different output than because they clearly weren't rowing in the same direction. The first time everyone was like, kind of rolling in their own direction, optimizing for their own thing. But by putting this optimization function effectively on it, they were able to find a much better solution.
And this wasn't like we said, Oh, you have half the money. This was just like, well if you apply this lens, like, what do you come back with is different.
Lessons learned from Dropbox's "Magic Pocket" project
Jerry Li: And this is not a first time , Dropbox had done a massive data migration there was a, project codename, "Magic Pocket."
Can you share a little more on that and how that was, executed through the lens of values
Andrew Fong: So magic pocket is a project we did about six years ago? My memory is shaky on the timelines right now. Six years ago, maybe six and a half years ago, where we were taking the bulk of our data out of S3, probably 650 petabytes 700 petabytes of data at the time, had to be flawless execution, giving the timelines. And so the financial constraints,
That one... I think everyone understood the mission and. What we learned there is. And what I took away from that is that as the project evolved, we had values that were put into place, but we weren't explicit about them upfront. We were much more startupy at the time. I think engineering was maybe 250 people. So like, it was, you know, you had this collective wisdom that was there.
And, you know, for that project, what came out of it is like durability is paramount. Trustworthiness is paramount. You can not lose a single bite of data or a single bit of data. and so the systems were built that way and the teams ethos became that and they still have that to the day.
Like I would say if you had reliability and go down a level, if you go into the storage teams durability is their value, right. Like, and I would say that's how they operate personally, how they operate from a, from a systems perspective. Durability was a value that came out of that. and what they did was build fantastic validation systems to make sure that everything could be found.
There's a, I think there's a talk online about this at some point. We did a. We had the SRE team site reliability. one of the things they were challenged with, if you corrupt one single bit of data in this, like transfer, can you detect it and less than a one week period?
And so without ever using the code base, that's migrating the data. So build your own code base that can independently validate all this. And so they built these like incredible frameworks around validation for this, all of that still exists today.
And I think that that's actually, you know, they have that value system now. So anytime they have any. If they think anything ever goes wrong in the storage systems? It's like all hands on deck and there's no questions. Like the teams. Don't like, there's no finger points, no blame. It's just so entrenched in the value system that they have, that you must be reliable about durability. You can't lose data, you have to have a mindset that this is the most important thing that we have. Systems that record systems at rest can't ever have a byte of data or a bit of data in any sort of danger mode.
So I think what we learned there is that, you know, as you go through that process, we learned that values are very important. But we didn't think about them on day one and say like, this is how we're going to operate ourselves.
What we learned as like, as we built this thing, we have to have these. And so those have been now baked into the system as, the team's evolved.
I would definitely, if we started to bootstrap it today, I would definitely do it differently because I think it took time for the team to realize that took time for the team to converge on sort of how this think about it .Definitely caused friction because, you know, you had different people thinking about, is one bit of data the most important, or like or is getting off because the dollar value is more important.
I mean, that trade off becomes hard then because there wasn't an aligned on set of principles. And then it becomes very much, you know, at the time VP of engineering has to make that call. Versus the team understanding. Okay. Yeah, no, data's more important than cost on this. We need to flag it, but like, we should do the right.
Like if we continue down this path, that's okay. How to make values endure beyond people in projects with long time horizons: be explicit with your decision making process
Patrick Gallagher: If you have a six year project, you probably have people that then leave, but then the values and the principles that guide that decision making have to persist to continue the reliability and the durability of the project.
Do you have any insights about how you can help values outlast the people?
Andrew Fong: I think the most important thing there is making sure that as you hire and as you bring in new talent to the teams that the previous generation that's moving on or going on to new projects, Spends the time to teach and educate on that. And that you're explicit about the decision making process.
I think way too often, we're much more implicit about how things work. I see this in mentorship, I'd say like, it's a difference between active mentorship versus like passive mentorship. just making sure the why is understood, which is why, you know, from my directs having like monthly business reviews where they explicitly framing things around values is important.
So design reviews, similarly, if you're a senior engineer using them to talk through why you may. made this decision, and being explicit about the verbiage, right? You have a vernacular, so use that vernacular. Because that actually helps people discuss it and write as they talk about it. That's the only way I find that they internalize it.
Like just reading a paper, internally, we have a tool called it's on paper and reading a paper document, or reading a Wiki. You have to live and breathe it, right. You have to get indoctrinated into it. So I think it's a big part of it is like, you can't let the team go to zero and then try to reboot it. Right? Like if you want to value system persist, you have to transition it. And then you have to be willing to see it evolve. I think that's the other, the biggest part.
The reason this is struggling a little bit to answer this question is that. The leadership like the senior leadership and infrastructure Dropbox has been fairly stable over eight years. There's basically been three senior leaders in infrastructure and myself being one of them. the other two have moved onto different roles inside of Dropbox. So there's been a continuity of values there. So that's why I'm a little bit struggling. Cause I probably say that I was the one that persisted that, but I think the leadership team now actually has that. And like I spent very little time on worrying about reliability, durability of systems. I think they just, they get it and they spend their time there.
Patrick Gallagher: I mean, I think even just diving into being explicit with how decisions are made and thinking through some of the other examples that you've shared where that exists. Like in terms of being able to replicate that decision making process or to. You know, make that disciplined throughout the whole organization, I think is really powerful.
Are there other, other inputs to help make decision making process explicit within the operations and things that you do?
Andrew Fong: I think that the biggest is creating an environment where... somebody doesn't know, that's like my general assumption is like, somebody doesn't understand like, why that occurred and like making it safe for somebody to ask that question.
Because, If I think about sort of the turnover rate and the hire rates and lately Silicon Valley, like there's always someone new in the room, right? Like it's never the same set of people that were there last week, even in some sense. so like when I say they don't know, they genuinely don't know because they weren't here for the, like the last three discussions.
So making sure you take the time to go through that, walk through the thought process, even if it means spending extra time offline with them, you have to do it otherwise. Like. it takes too long to ramp up. And a lot of the technology stacks that exist today are a lot of the companies that exist.
the two of you probably are familiar with this, but like, if the average tenure is like somewhere around two, two and a half years, like that is not a lot of time to like pick up a value system, ship some code and like understand what you're doing.
So I think a lot of it is like, I want to have a mindset that there's a longterm horizon of people staying, because these systems have to be durable. So then how do you create an environments that incentivize that? How do you create an environment where like you want to invest in people and they want to stay here for the long term? Because they're seeing the value in that. They're understanding why something is happening more so than just like. Yeah. Yeah. I shipped that thing and I got my bonus and I'm out of here now. that's not the culture. Like I want the culture that's like much more about, okay, I understood what's happening here. And this is about me growing as an engineer or as a person like through the organization.
Patrick Gallagher: Total meta, comment, Andrew, I'm having so much fun diving into this. Like, I just think seeing all of the different ways That you've been able to engineer and apply how values impact behavior at even at such a granular level, but then also at some of the big decisions and big projects is ... is really powerful because I think. you can have these big values, but what does that look like every day? How does that actually impact your decisions and behavior? And that's the big challenge.
Andrew Fong: Oh, I totally agree. I mean, literally my, my next to do is to talk, to talk to my chief of staff about like, okay, what's the podcast for this week? what are we shipping? What's the email I'm going to go send. That's literally my next, the next thing I'm doing, right after I get off with you guys.
. I think it's about the discipline to be repetitive on it.
And I actually think the word repetition is a bad word in some sense for it. Right. It's like, I think when I used to hear that, you know, when I was a more junior manager and people would say like, Oh, just keep repeating yourself. Right? It's like, there's this undertone of like, why am I repeating myself? But I don't, I think that's the wrong mindset to have around it.
I tend to think of it as like, my mindset should be these things. And so it's not repeating myself. I'm just living the set of things that I believe. And so it's not that I'm repeating it cause I want other people to believe it.
It's like, I believe this. So therefore I'm following the set of things and I'm just being explicit. I'm not repeating myself. I'm just making my thought process explicit.
Jerry Li: Yeah. That's a very useful perspective to have.
How to operationalize values in small teams and start ups
so you talked about , the value and , how it applies at a larger, like 200 people organization. Do you have examples where it applies to a smaller team?
Andrew Fong: So I had the, it was a great opportunity too, and I know this is a challenge for a lot of organizations right now, which is, development environments.
I spent nine months, that's my only foray out of infrastructure with Dropbox. Spent nine months in the platform organization, helping fix up some of the development environments. And we had built this environment called EC, Enchanted Container, that, Simply wasn't scaling...
the best way to put it. Is that engineering managers had interns in their offices being like, Oh, my god I can't get my job done. I can't like, am I going to get a return offer? I'm gonna get an offer. Like, I can't ship my code because this development environment is just like is blocking me. Engineers will be like, we're not going to update our environment. Cause if we update it, we don't know if it's going to work next week. I mean, this, this is probably. Four years ago, three and half years ago. And it was just, not great.
and I've definitely heard horror stories from other places it's similar, you know, grow really fast. You don't think about like the tooling and the tool change you need to actually become, you know, productive, efficient, and effective through the organization.
and so I moved over from infrastructure and I started working with his team was about probably 17 to 22 people somewhere around that size at the time. and I had the luxury of actually working with like, An engineer that's just absolutely phenomenal. And he and I sat down and said like, okay, what are the things we need to change?
Well, we need to build some behaviors in the system that where the team, which was a little more junior, understood where to go directionally as well as we got an output that gave like senior engineering agency to say like, no, where it wasn't like a philosophical, sort of Holy war type debate, but it became like, no, there's some principles and like you'll adhere to these principles.
and so it wasn't as crisp as the infrastructure sort of way of thinking, but we did come up with a mission statement, it was a little more wordy, that was just more around...
"Engineering is fundamentally a creative task. And our job is to remove the friction between your brain and the keyboard as much as possible"
Effectively I think that was pretty close to what it was it's three years ago.
And then we came up with some engineering principles and there are sort of the values. Like we want things to be modular. We want things to be composable. we want things to be reliable. we want things to operate on the principle of least surprise.
and so, yeah that was the lens, because this was a much more tactical problem at the time. And it wasn't about building a durable organization. It was literally about like, can we solve this problem? So we used the similar process. Like we went through this with the whole team, all 17 people, like, and we came up with this framework.
And we use that to basically rebuild the development environment for all of Dropbox at the time, probably 600 to 800 people we had to ship it to. we built a new development environment that allowed people to get onboarded within 30 minutes. Write code within 45. Was basically unimpeachable from like, it would not break because the development environment would also be validated through CI. Like it was basically impossible for if the code shipped to production, the development environment, therefore must work. Like it was impossible for one not to be true.
And so built that, shipped it out to all of engineering in like less than nine months, the team had been working or like we'd been hacking at it on, you know, for probably. Better half of two years before that, in sort of an iterative way, we just did a forklift upgrade in like less than nine months to get it out the door.
Like from idea to fully all, not like MVP, but like fully shipped, everyone moved and migrated onto it. which was a pretty big deal at the time. It took sort of the.
Perception of engineering from like, you know, very low in terms of like how well people get the job done. Like almost tripled the score. I think at the time, instead of the surveys we ran.
Sort of the same principles, like we just use it every week in a design review. We always go through the principals. We'd always talk about it in that way. We were just like, always look, I think there's a tagline at the time that we would basically put on everything , I think we changed the team's mindset should be like, your dev is our fraud. because like, I think there's a mindset that , okay, developed environments, development environments, who cares if they break?
Like, I think it's fairly true from what I've seen across Silicon Valley. Like I think it's like, you know, it's a bunch of people all hacking out at the same time. But coming from infrastructure I wanted them to take the mindset that like you're shipping a product. This thing has to work. if you have a say thousand engineers and you have a 1% error rate... that's a lot of emails a day, right? That's a full time person's job to respond to those emails. Ao like, if you want this to be scalable, right. And you want this to operate well . You have to like, in our principles, like sort of outlined all of this... you have to get down to like, you know, three, four nines of reliability on like development environment. which if you put that constraint in shifts how people think about doing development, building the environment itself.
so that's like on a smaller scale sort of a project I think the total size of the project, the number of people that worked on it was probably like North of 25, around 25. But it was also distributed team. It wasn't all one organization. So we had people that joined from other places that had similar that we had to indoctrinate her on the, on the engineering principles around it.
Jerry Li: Yeah, the part I feel really impressive is that, it's not, so the team that's put together temporarily for this project and also you start by first identifying the principles, the values and use that to guide a lot of decision making, , along the way, I guess that itself. save a lot of time in terms of making the tough decision to go a lot of conversations,
Andrew Fong: I will say it also, I mean the side effect, right. Is like people self select out. Right. , it becomes very, I mean, to get a little spicy, right? Like, it becomes pretty easy for people that don't adhere to that to say like, look, this is not the organization I want to be in.
And it's not bad. Right. Like, it's not like okay, you're a performance problem.
Right. It's just like, I don't, this value system doesn't work for me. It's like, okay, let's have that discussion. that's a personal judgment, right? , it's not an efficacy of like, how good are you at your job? it's just a question about like, do you adhere, do you believe in this? If not it's okay. Right? Like you should work on something you believe in, you should come into work every day believing... truly believing the thing you're going to do is going to be maximally impactful. If it's not, then we'll have that discussion.
Jerry Li: Yeah. And this is I think can be generalized by, this is probably one approach can take, Towards any large or medium sized or, you know, smaller sized project. Either explicitly or , implicit.
so that's for, running organization use values and the principles.
How Andrew operationalizes his personal values
How does those reflected in your personal growth? Do you have a habit of doing that outside of work as well, or...? What kind of routines or things you do?
Andrew Fong: I have a very similar structure. So like, work and personal is like, use the same framework and like a lot of ways in the same way. I think that my work personal North star is probably similar to my personal lifeline. Like it's about maximizing people's ability to get the thing they want to do done. That's sort of the idea behind everything.
so I try to like, it's similar framework. I definitely maintain personal OKRs. I definitely have a framework for them where it's personal growth, family, like, those are like the. Themes. Right.
My wife and I had a discussion around values the other day because we have a toddler and we were talking to like, okay, how do we want to raise our son? like, what's the value system we want him to have? It's been implicit between the two of us. Right?
she's a big believer in like, growth. So she's like, okay, I want a kid with a growth mindset. Right. I want that to be like a value that's there. So we definitely had that conversation in a car ride out to her parents' house before COVID, we need to be a little more explicit, but he's only 21 months. So I don't think he understands right now. So we have a little bit of time to like roll out one. but yeah, no, I definitely have that same sort of mindset in my personal life. I'm a big, I'm a pretty disciplined person , from that perspective, both personally and professionally tends to be sort of, I won't say I'll say rigid, but not inflexible. And some of my processes and like what I do.
Jerry Li: Yeah. And do you see like, the application in personal life and work reinforcing eachother?
Andrew Fong: I definitely see they reinforce each other. I mean, just things I do habitually just for my personal for myself. Like, probably the three things.
I'm a huge believer in meditation and journaling. So I do both of those. This is the third one. I'll throw you guys for a little bit of a loop. my personal sort of sport of choice is powerlifting. if you know anything about the sport of powerlifting, the theory is it can always sort of like get stronger in some way. So it's like a growth mindset sort of like, like it fit naturally for me in that sense. and there was like a process in what you follow to get better at this thing. and so it felt very natural from like the engineering side of me.
So like, those are the three sort of outlets I have, which I know it's like, journaling, powerlifting and meditation, not the three things I thought you've probably heard from me, but, it's sort of like what I do, to like maintain that structure. Those are really the three pillars, sort of my personal side.
Patrick Gallagher: the manifestation of your personal values.
Andrew Fong: Yeah. Yeah, no, it's definitely. Yep.
Patrick Gallagher: Do you have a question that you ask yourself to realign? Like if your behavior is out of integrity with your values, like, do you have a question or a process in how you realign within your values?
Andrew Fong: journaling kind of gets at that. I like to the reflection there, I'm also lucky enough to have like a fantastic coach who like holds me pretty accountable is a lot of this stuff. so like that's probably the two mechanisms.
I think you have to be brutally honest with yourself. you're always going to tell a story. You're always have some lens. Right?
I think the other is. I definitely want a team around me that's willing to call me on it. and I think it's like super important to have peers that will do that as well. I have, you know, former manager who will do this on a regular basis. Like I can ping him at any time and be like, Hey, can I talk to you? he'll spend two hours, talking through something, and like not let me kind of get away with anything.
I have two also good friends and peers, one of which I've known for probably almost 20 years now, 18 years, worked together in a couple of places and he's, he's definitely, would hold me to that and be like, no, no, no, no. Like this is not, you know, I don't know what you're doing here. Like yeah, I think about that differently.
so that's sort of like how I do it. Cause I think it's very hard to always be super self-reflective on it because everyone has their own self perception and their own story they're telling themselves. And getting those external perspectives. Right. And making sure that they're not going to be people that's like reinforce your perspective. I think that's very important. It's really easy. And I see this a lot to find an echo chamber. I think that's the part that is very important to me is like, just making sure that echo chamber doesn't exist.
Patrick Gallagher: I think that the mechanisms that you highlighted to realign you with your personal values, I think what's so special. It's like we can talk about the peers. Are you talking about your coach and the people you surround yourself with? Like that's such a powerful...
Andrew Fong: I should also not be averse to this... my wife also does.
She definitely gives me. She definitely won't call me out on it. She called me out this morning on something. So I would say that's probably the that's a big support structure.
Patrick Gallagher: I mean, especially if you have any explicit conversations about the values of the system and what you want to raise your children, like that's a really special conversation to have too.
but just how powerful those mechanisms are, to surround yourself with those types of people who want the best for you and you living in integrity with your values, being the best expression of who you are.
Andrew, we're getting close on our time We wanted to end with one more question for you. was wondering if you could leave us with a story of a difficult moment or a decision that you had to make, where having operationalized values really helped you navigate that situation and made a big difference in the outcome.
And this can be within Dropbox or. Another organization or other involvements or personally, but would love to hear like when the stakes are high, how did those decisions really lead to a big impact?
Andrew Fong: is there always hard questions? Probably the biggest ones for me are like, anytime it's like a feedback conversation, right. It's like really easy to like go into the realm of like, okay, what's the easiest thing for me to do to feel safe in this. So this other person also doesn't feel like this is an awkward conversation.
And I think that I tend to go straight to values in that, in those conversations, like from my own head about okay. If I don't, express this, to this person, I will not giving them like what they need to hear. I'm not being like, I'm failing them in some way. Right. , Probably personal value is big part is around integrity. And so like having that conversation internally, it's , like if I'm not actually having this conversation with this person, like, am I holding that value? to be true?
that one's, that's the one that goes through my head, like consistently, because I think it's very easy to explain it away and then you just don't have that hard conversation you have to have, I can think of a couple of hard conversations. Like they're all like sort of people think people related things recently. So I don't want to go into much detail with them,
but like, those are definitely the ones where I sit there and I'm like, okay, I have to have this conversation. It's gonna be hard, but like, otherwise I just am not living the values that like we want to have.
Patrick Gallagher: it's really cool to be able to look into the mental conversation that you're having with yourself in that moment.
Andrew Fong: Yeah. It's yeah. It's always a fun conversation internally. It's like, okay, what's the easiest thing to do. Well, it's definitely not this. What is the right? But is it the right thing to do?
Actually the better story for this is I have a good friend. and I've known her for like my whole life. She said, when. We were probably 14 or 15 years old. She submitted a comment that, you know, she's a doctor at Stanford now that you have a duty. If you have a set of skills that you have to use them and maximize them to the fullest extent.
and like, I remember it was like the conversation centered around duty. And I, that tends to be the way I think about it. If I'm in this job and I'm in this role or I'm doing these sort of things, you have this duty to do that. and I think that's probably the way I get through it. And integrity was probably the wrong word. duty is the word that comes to my mind now that I reflect a little bit on it.
Patrick Gallagher: Andrew, thank you so much. This was just been an incredible conversation about values, about decision making and about ultimately living, the best expression of who you are.
Andrew Fong: I try, but it's definitely hard.