The Happy Philosopher – Semiretirement and Finding Happiness

Mad Fientist: Hey! Welcome everybody to the Financial Independence Podcast, the podcast where I get inside the brains of some of the best and brightest in the personal finance space to find out how they achieved financial independence.

On today’s show, Jeff from the Happy Philosopher is joining me. I met Jeff back at the first Camp Mustache I attended. We had some great chats there and it was the first time I actually realized how unique the pursuit of FI is for doctors and lawyers and anyone else who’s invested a large amount of money and time into their careers. I didn’t really realize how much identity is tied up in those careers. And obviously, when you make such a huge investment like that, it’s hard to just step away after less than a decade of working.

So, I was excited to get Jeff on because he’s a practicing radiologist. And he’s come up with the unique solution to this. And it’s sort of like a semi-retirement which seems to be working really well for him.

So, I’m looking forward to diving into some of these unique challenges and some solutions to them. So without further delay, Jeff, thanks for being here.

Jeff: Brandon, thank you. I’m delighted to be here, man.

Mad Fientist: Yeah, I know! It’s been a while since we last chatted actually. We met back at Camp Mustache which I believe was in May of 2015, is that right?

Jeff: Was it 2015? I think it was 2016.

Mad Fientist: Wasn’t it? Was it last year?

Jeff: Yeah!

Mad Fientist: Oh, right! Geez, I don’t know how long I’ve been there anymore.

Jeff: When you retire, you get senile, and the years just kind of float by.

Mad Fientist: That’s right, man. I know I never know what time it is, what day it is. It’s great.

So yeah, we had a great weekend in the Pacific Northwest. We had some good chat over coffee one morning. I got to learn all about your story, so I knew I had to get you on eventually. I’m finally glad we made this happen. I appreciate you coming on.

Jeff: Yeah, absolutely.

Mad Fientist: So, before we dive into all the stuff I know I want to talk about, maybe just introduce yourself and tell a little bit about your back story and the Happy Philosopher.

Jeff: My story is pretty typical I think for—I’m a physician, a radiologist. I grew up in a typical middle-class world. Nothing particularly crazy about my childhood. I did well in school, got good grades. And it was just sort of assumed that I would go to college and get some sort of fancy job that required an advanced degree. And I was interested in science and medicine. It seemed like a pretty good fit for me. And I went for it.

I went to medical school which was an awesome experience. I just met a bunch of brilliant people while learning a skill that’s quite unique and quite awesome so.

So, I went through medical school. I had a little bit of a crisis in the latter part of med school. And I realized that I just didn’t know what kind of doc I wanted to become. Eventually, I settled on radiology. And my wife and I—who we got married in med school, met in college—went to go do our residency which is a training program after medical school. And then, at the tender age of 31, I was done with all my training and went into private practice.

And private practice was challenging. It was exciting. I was just living a typical life.

On paper, everything in my life was perfect. But after a few years of work, I just slowly got less and less satisfied with my job. I was noticing that I was becoming more anxious. I couldn’t really recharge on my weeks off. All of the exciting things about medicine, after a few years, just no longer were exciting for me anymore. So, all of the positive things, all the joy I got out of work, really went away. And all that was left was stress and anxiety and just sort of wanting to get out. I felt trapped.

And this was 35 or 36.

Mad Fientist: And how old are you now just to give the audience an idea of timescales?

Jeff: Forty-four.

Mad Fientist: Forty-four.

Jeff: Forty-four.

So, I came out around 2005 (I think I started my job). And so right around 2010-ish is sort of when I was going through this crisis.

And the economic situation around that time—so my entire investing career was sort of centered from 2000 to 2010. If you pull up a graph of the SNP500, for instance, between those years, it’s quite depressing. I wasn’t seeing any progress in my financial situation over that decade.

Mad Fientist: Did you have a lot of medical school loans to pay off? Or did you come out of med school pretty unscathed?

Jeff: By the end of residency, it was about $200,000 between the both of us. I know that sounds like a lot probably to a lot of your listeners. But really, that’s almost nothing for docs. And we could touch on this a little bit later in the interview, but some docs now are coming out with half a million dollars in student loans which is…

Mad Fientist: Crazy!

Jeff: Pretty crippling, yeah.

But we were always fairly frugal and good savers. And so when I came out of residency, when we started work at again around age 31, my net worth was still negative, but it was almost zero—which was a victory coming out of that.

Mad Fientist: Yeah.

Jeff: So anyway, here I am kind of going through burnout, and I really didn’t know what to do. And the only thing I knew that was certain was that I didn’t want to work forever. I didn’t want to work into my 60’s.

So, I went online and I typed and started researching early retirement. And of course, the stuff that comes up back then when you type in “early retirement” are the big brokerages, saying, “Hey, you can retire at 55” and that’s early retirement. I was like, “What?! I mean, that’s 20 years away. I can’t do that. That’s just not feasible.”

So, I typed in—I think I typed in “extremely early retirement.” You can imagine what popped up in my web browser, right? So, Early Retirement Extreme is the first hit.

I went on Jacob’s site. And I think the first article I read was how to retire in five years. And after I sort of picked up pieces of my brain that had flown all over the room and put them back in my head, I just started reading everything on the site. And after a few days, I was very impressed and thought it was awesome and also completely unfeasible for my life.

Mad Fientist: So, at this time, are you a pretty big spender? Obviously, you’re out of med school year radiologist, you’re making pretty big bucks, I would imagine, right off the bat as soon as you qualified, I would imagine. Is that correct?

Jeff: Yeah. The financial life of a physician is really interesting. It creates an interesting dynamic. So you go from poverty, which is medical school, where you’re going into debt (you’re borrowing all the money that you can and barely scraping by and still going into debt), and then you go into residency where you make a typical middle-class average salary.

I think now the average salary for residents is about $50,000 or $55,000 (which I think is about average in the United States for earning).

And then, you live that for a while. And then you go into practice and it’s multiples of that. So it’s sizable. It’s a lot of money.

And to be honest, I don’t know how much I spent those first five years. I don’t think it was crazy. I mean, we bought a nice house. We had to furnish the house. So there were some expenses there. But again, we were pretty naturally frugal, and just didn’t have the inclination to spend a lot of money. But we probably spent a lot more than we’re spending now. I still kept the car that I was driving in residency. I didn’t go out and go crazy or anything.

Mad Fientist: So, it wasn’t you’re spending then that you didn’t think that early retirement in five years wasn’t possible. And it wasn’t your income obviously because you’re making more than enough to make that a reality.

So, was it just the psychological aspect of pulling away from work after just such a short career or is that the sunk costs of you’ve put so much money and time into this career to then walk away? I imagine there’s so many different challenges for doctors (which is why I was really excited to get you on) that maybe some normal careers don’t have.

So, can you talk a little bit about why your initial reaction to Early Retirement Extreme was like, “I couldn’t do that in five years?”

Jeff: Yeah. I mean those are great points that you make. And I definitely want to dig into those.

For me, it was that Early Retirement Extreme was so extreme, and I couldn’t really visualize taking those actionable steps and principles and applying them to my life. For me, I was earning plenty of money, and I was not spending a lot. For me, it was lack of knowledge. I didn’t really know that I could retire in five years because I didn’t know how much I needed.

I remember one conversation I had with a spouse of a physician that I know. He said, “Yeah, you probably need like $10 million to retire.” I was like, “Holy!” I’m doing the math in my head. “$10 million? That’s a lot of money. I’m going to have to work awhile for that.”

But I just sort of accepted that. I didn’t realize, I didn’t know really about the 4% rule and all the things that most people seeking FI know about.

And so, I was reading Early Retirement Extreme, and I read a guest post by this guy. You may have heard of him, Mr. Money Mustache. I’m reading this, and I’m like, “Mr. Money Mustache? Who are these people? What’s going on here?” But that’s sort of the post that got me over to Pete’s site.

And this was back in 2011 or 2010 or whenever it was. He had just started.

Mad Fientist: Yeah, it was probably the same guest post that I found out about him from I think as well.

Jeff: Yeah, yeah. So, there wasn’t a lot on his site, but he was very prolific back then, and he was posting new material like every few days. So every few days, I would read these earth-shattering things and ideas from him.

And it didn’t take me long to realize that the math part was really easy for a physician as long as you can get your spending under control. It’s not an earning problem. The math part was really easy. But you’re right, the psychological part is a lot bigger deal for physicians in particular and I think a lot of other high income or sunk cost professions like you mentioned.

So, becoming a physician is a different job than any other that I’ve had. My identity sort of merged with being a doctor.

For instance, when I was a busboy or waited tables or even did some door-to-door sales for a while, as soon as I stopped doing that for the day, I no longer identified with it. I went back to being Jeff. But becoming a physicians a little bit different—and I imagine a lot of other professions—where you never really detach from that identity when you go home from work.

So, the idea of giving that up early in life is like giving up a part of your identity. And ego has a really tough time with this.

It’s interesting. Physicians, if you look at the statistics, they retire later than the average population (a few years later than people on average). And that’s kind of ridiculous when you think about how much money they make. But I think a lot of it has to do with just they can’t detach. It’s psychologically painful.

And if you look also look at statistics like job satisfaction, there’s a lot of really dissatisfied docs out there. I’d like to believe that it’s a psychological problem, not a money problem or a job satisfaction thing. I don’t know that for certain. But that’s my guess. Talking to a lot of docs, I think that’s true. I think that’s true. It’s hard to untangle from that. That is probably the biggest challenge because the math is easy for a doc to retire early.

Mad Fientist: And there are other things to consider. It’s pretty much impossible to go back after stepping away. That’s something I’ve heard you say in person and write about. So can you maybe talk about that and how that’s definitely different for a lot of professions? I have no doubt that I could go back to software development in a few years and say that I was just working on my own stuff in the meantime. Maybe I would spend a few months catching up on the latest technologies, but I could go back. But that’s not really the case with your profession.

Jeff: Yeah, and I think that colors a lot of my views with respect to retirement, how much money I need. You’re absolutely right. It’s not impossible to go back into medicine after a long hiatus. But for all practical purposes, it’s very, very difficult. You have to use these skills to sort of stay on top of things. The technology is constantly changing. And to be honest, you have to keep all the licensing and all the continuing education.

And who’s going to want to hire a physician that’s been out of practice for 10 or 15 years when you can just hire somebody fresh out of training who is eager.

There are just a lot of questions. And I think that it would be very hard to get back into the field after being out for so long.

That, and it’s just so darn hard. I mean it’s really intellectually demanding. I think it’s hard to keep that edge after being out for a prolonged period of time. I know there are some people out there that have done it, but it’s hard. I think it’s easier to go back in if you love it, if you’ve always loved it, and then you go out for some reason and you go back in. It’s easy. But if you burn out, then you’re out 10 years, the thought of going back is—psychologically, I don’t think that I would be able to do that.

So, I feel like this is my chance. I’m reimbursed very high amounts of money for my time. And this is my chance to make that trade, the most advantageous rate that I can. And then, when I’m ready to step away. I don’t want to have to worry about that.

I think talking to a lot of physicians, sort of people that I’ve interacted with who’ve come at me through my blog, a lot of them feel the same way. They just have this underlying unease about leaving medicine too early and fear of having to scramble later in life. And I think it’s justified.

I mean, a lot of people in the early retirement community, the FI community don’t have those thoughts because it’s so easy for them to just slip back into work.

Mad Fientist: Okay. So you stumbled upon Early Retirement Extreme. You realized you weren’t happy. You wanted to make a change. So how did you get to the point where you actually were comfortable with the idea of walking away at some point much earlier than you probably anticipated before?

Jeff: So, that was actually kind of a compressed crisis in my life. I was going through all of the reading, learning. And it didn’t really do anything to improve my situation at work. Just after a particularly bad stretch of work and call, I came home one day, I had rough math in my mind, and kind of knew what I could do, and I just told my wife like, “I can’t do this long-term. I’m committing to five more years, and then I’m walking away. I think we’ll have enough money. But that’s all I can mentally and emotionally commit to right now.”

Mad Fientist: How did she take it?

Jeff: Surprisingly well! I’ve made a lot of mistakes in my life, but the big decisions have been really good. My decision of who to choose as a spouse was perhaps one of my best decisions. She was very supportive. She kind of understood what I was going through. And I think we both really want each other to be happy. And this was my path to happiness. It really is what I thought.

Mad Fientist: Now, she’s a doctor too. But she did step away to become a full-time mom, is that right?

Jeff: That’s right.

Mad Fientist: So, had she done that at this point or is she still working as a full-time doctor?

Jeff: Yeah, she was a stay-at-home mom at this point. That was the irony of this. She probably would have been the better one to keep working. But I would have made a really bad stay-at-home dad. She’s way better being a stay-at-home mom than I would be a stay-at-home dad.

But yeah, she was out of medicine for a few years at that point. And I mean that’s a whole other podcast, that transition…

Mad Fientist: Yeah! How was that transition in a nutshell? Did she handle that okay or was she going through the same things that you were going through thinking about stepping away?

Jeff: She was going through a lot of the same things that I was going through. But she struggled with walking away, potentially not going back to it. But she really wanted to be a stay-at-home mom too.

And when we were both working, we just could not find the balance in our lives. Our lives were not fun—trying to arrange the childcare, trying to arrange just living life around two call schedules and the kids.

So ultimately, that was a really good decision for all of us. And I don’t think that she’s regretted it.

Mad Fientist: So, you tell her that you’ve got five years max left. What happened after that?

Jeff: I realized that although I shortened my prison sentence, so to speak, I didn’t really make my cell any more comfortable. I still had five years to go, and I was still burned out, and I still wasn’t happy. So I had to figure something else out.

I had to start focusing on being happy now with whatever situation I was in rather than saying, “Okay, I’m just going to go through this and wait five years, and then I’ll be happy.”

So, to me, just that decision is really what pushed me in a different direction, focusing more on being happy, rather than delaying it, rather than just being miserable now, and then at some point in the future, sort of guessing that I’ll be happy because I’m financially dependent.

Mad Fientist: So, what steps did you take to work on that during the five years?

Jeff: Mainly, I did a lot of self-reflection, self-experimentation, reading. I went down the Mr. Money Mustache philosophy of life. You read his blog. He’s got a great philosophy on doing things for joy, having gratitude, using stoicism and negative visualization, meditation, all these sort of tools.

I started doing a lot of meditation, experimenting with that, experimenting with diet, and just doing all kinds of things in my life to try to make myself the best version of me regardless of what my situation was.

Mad Fientist: And did it work?

Jeff: It did actually. I mean, it was amazing. It was a complete transformation. And a big part of it was just eliminating things from my life that were negatives. I know you’ve written about this, Brandon—maybe one of my favorite articles that you wrote was happiness through subtraction.

Mad Fientist: Yeah, yeah, it was.

Jeff: Yeah, absolutely brilliant article.

And so, I started just getting rid of unnecessary things in my life, both physical and mental obligations. I got rid of news and most of television. And it’s just amazing.

Mad Fientist: I know that one will make you happy. That one’s so huge!

Jeff: People don’t believe me when I tell them this, “Just get rid of news.” They look at me like I told them to chop off their arm. I’m like, “No, just try it 30 days. Nothing will change in your life except for the better.” There’s no actionable information. It’s just designed to flick your adrenal glands and get cortisol into your blood and make you anxious.

Mad Fientist: Right, exactly! It’s all outside your control, so don’t even stress out about it.

So, that was a big effective one for you. Was the meditation good? That’s not something I’ve ever tried.

Jeff: Meditation is amazing. I wrote a big post on it on my experiences with it.

Mad Fientist: What’s the name of the post? I’ll link to it on the show notes.

Jeff: I think it’s something really stupid, like 1 Simple Trick to Increase Your Awesomeness.

Mad Fientist: Okay, cool. I’ll link to it.

Jeff: It was, by far, my most ridiculous title ever on the blog. But I’ll send that to you.

But the great thing about meditation is you don’t have to be good at it. You don’t have to do it consistently. It has lasting effects. I mean, it’s amazing! If it were a drug, it would be illegal because it’s so good. There’s little to no downside. It’s such a small time commitment.

And it improves sort of the day-to-day mindfulness without you even really realizing it. I’m terrible at it. I’m not a great meditator at all.

Mad Fientist: So, what does it look like for you because I know there are lots of different types?

Jeff: I do more of a mindfulness meditation. I break it down into two categories—the mindfulness or just sort of paying moment-to-moment attention, and then the more concentrative, focused concentration. I don’t do that. I just do the mindfulness where I sit quietly. And I usually do guided meditation for 10 minutes, kind of just pay attention my breath. And then, every time I get distracted by a thought, I just observe it, and then gently go back to the breath.

It’s just this repetitive practice of not getting caught up in your thoughts and just letting them sort of float around like clouds and not get too attached to them.

Mad Fientist: Yeah. I’ve heard lots of really intelligent, successful, smart, happy people talk about it. When enough people say something, then you should probably take notice and try it out for yourself.

Jeff: Yeah, I got into it, really, I was listening to Tim Ferris and his podcast (which is pretty great). And he made this observation that all of these high achievers that he interviewed, a ridiculously high percentage (something like 70% of them) had some sort of meditative or mindfulness practice. So he realized there was something to it. And that was my gateway into it.

Mad Fientist: Right! So, you’re doing all these things, and you’re becoming happier. Are you happier work or is it just you’re happier in general so you can deal with not enjoying work as much, you can deal with that better?

Jeff: Yeah, I think I did become happier at work in spite of nothing changing. I mean I still had the same job. It was still just as stressful. But the simple act of not worrying about things that I couldn’t control—There are so many things that are out of our control. And once I stopped worrying about those, I noticed a big improvement. I could just deal with the day-to-day stresses of the job a lot better.

It still wasn’t optimal. I still didn’t want to do it full-time, and I was still looking for ways to either shorten it or change it, but at least I was in a place where I wasn’t spiraling down. I wasn’t getting worse. I was getting better.

Mad Fientist: That’s great! So, can you talk a little bit about where you’re at now and how you got there?

Jeff: Yeah, absolutely. So, what you’re referring to is I now work half time. I work part-time. And this was a job share that I engineered. And it was about three years after my initial episode of burnout.

So, I actually worked full-time two to three years after that initial five year declaration of independence. And at that point, I finally engineered this job share and convinced my group that that would be a positive addition. So, me and my job share partner are…

Mad Fientist: One full-time employee?

Jeff: Yeah. So, when we’re plugged into the schedule, one of us is working on that day. And we just decide which days we’re going to work and which days the other person is going to work. So it’s great. There’s a lot of flexibility.

We are dependent on one another. So, we have to coordinate which days are working. And we have to both sort of be on the same page there. But it’s fantastic! It’s definitely saved my career. If I had been full-time and not been able to do this, I’m convinced I’d probably not be working right now.

Mad Fientist: Right! So yeah, because the five years is pretty much up, and you’re still going. So, what’s your plan from here on out?

Jeff: That’s a great question. It’s something that I’m still trying to figure out. As I get closer to achieving this bulletproof FI number—and as you know, there’s a lot of deliberation and debate about when do you actually become financially independent. I like how some of the people—like JD Roth I think has written about this, The Degrees of Financial Freedom and Independence.

Mad Fientist: Yeah, that’s a fantastic post, and I will link to that in the shownotes. That’s one everyone should read I think.

Jeff: Absolutely! One of his best, I like it a lot.

But as I get to this place where work is becoming more and more optional, I don’t know. I don’t know what I’m going to do. A few more years… tomorrow…? I don’t know.

Mad Fientist: It’s really that up in the air.

Jeff: I’ve matured my thinking to a point now where I’m working for money and I’m going to do that as long as I’m satisfied and doing good work in my job. And at some point that that exchange will not make sense—you know, the marginal utility of additional work is not going to make sense just because I don’t need any more money—

Mad Fientist: Yeah, you have a great post on that, which I’ll link to as well, where you talk about how money gets less valuable as you get older and time gets a lot more valuable as you get older because you have less time left. So each second is more valuable. So I will link to that because that’s really good as well.

Jeff: And stacked upon that, each additional dollar spending you stack on the top of your topline spending, you’re going to get less marginal return on the happiness. You have three things working against you as you keep working later into your life.

Mad Fientist: So, I end all my interviews with asking, “What’s one piece of advice you would give to somebody on the path to financial independence?” But I’m going to ask if you could maybe give two pieces of a advice—one for someone in a job similar to yourself that has unique struggles that we touched upon on this interview, but then also, you can give one general piece of advice to everyone else out there.

Jeff: Yeah. So I guess my one piece of advice to a physician or somebody in my situation is to think about separating your identity as a physician from who you are as a person. Do your best to keep boundaries between those two things. You can really become unhappy fast in medicine and dissatisfied. And if your identity is that of a physician, and it’s so intertwined, when you fail at medicine, or you fail to be happy, you become the failure. And it can be really emotionally destructive.

Physicians have a pretty high rate of depression and suicide compared to the average population. So it’s just important to be aware of these things and to get help if you need it.

Mad Fientist: So, separating your identity from your job, that seems like it’s easier said than done. Is there any sort of specifics that you could give to help someone do that? I’m not exactly sure where I would start with that if I was working all the time and just totally consumed by the job.

Jeff: Trying to create boundaries like when you come home, try to create some sort of ritual maybe to leave your work at work and focus on things at home that are important to you. Learn to say no to obligations that suck all your energy—because medicine will. It will take everything from you—and more—if you let it. It’s never satisfied. There’s always more to be done.

But I think just knowing yourself and deeply reflecting on these things and being aware of them is a step in the right direction.

Mad Fientist: Excellent! So now, your general piece of advice to anyone out there that’s pursuing financial dependence?

Jeff: Yeah, I think my one piece of advice is to not focus on financial independence, but instead to focus on happiness. When I started down this path, I used to think that freedom, or more specifically, financial independence, leads to happiness. And I think it can. But I think it’s reversed. I think happiness leads to freedom.

And the reason I know this is because I know a lot of people that are financially independent that are 100% miserable. They’re lonely, they’re anxious, they’re fearful… but they’re free. Financial independence is not the end goal; happiness is the end goal. So we need to always keep that in mind. We just need to internalize that message. Happiness leads to freedom.

And to be honest, it’s not that expensive to become happy. When you can cultivate happiness in your life with really simple things that I write about like gratitude, getting rid of negatives from your life, decluttering all the physical things you don’t need, the mental baggage and the obligation—like I said, stop watching the news, just walk and meditate—those things don’t cost anything, but they’ll make you happy.

Happiness is freedom. You have to practice. It’s a skill. Happiness is definitely a skill. It’s not something that we just blunder into for the most part.

Mad Fientist: I completely agree. And that’s a fantastic advice and a great way to end the interview.

So Jeff, I really appreciate you taking the time to talk with me today. And if anybody’s interested, head over to theHappyPhilosopher.com. You can get in touch with him there.

So Jeff, I appreciate it, man!

Jeff: Yeah. Hey, keep doing great work, man.

Mad Fientist: Thanks very much. Aright! I’ll talk to you soon.

Jeff: Alrightee!

Mad Fientist: Bye.