Reading a math paper is a bit like having dinner at a nice restaurant. The entrée might taste delicious, but it doesn’t tell the full story of how it was made. Clever recipes that end up tasting funky don’t make the menu; undercooked dishes are (usually) not served to customers. But missteps in both conception and execution are important parts of the process. Danny Calegari, a topologist at the University of Chicago, wants mathematicians to be more transparent about what goes on in the kitchen.

Calegari grew up in Melbourne in a mathematical household. His father taught calculus and his mother statistics at a local technical college. Their bookshelves were filled with textbooks that he and his little brother Frank would browse through for fun.

But Calegari didn’t always plan to be a mathematician. When he enrolled as an undergraduate at the University of Melbourne, he considered becoming a writer, or perhaps a cognitive scientist. He decided to take a topology class to better understand some models of how the brain worked. “It was just electrifying,” he said. “In this class, every sentence was somehow interesting. … It was like the world’s greatest Lego set or something, where it all just connected up in this brilliant, unexpected, powerful way.”

He was hooked. He decided to major in math, and later got his doctorate from the University of California, Berkeley, ultimately sticking with topology. (Frank would follow him to Berkeley and then join him on the math faculty at Chicago, specializing in number theory.)

Calegari remains an avid reader — of Joyce, Dickens, Nabokov, Austen, Vonnegut, Lorrie Moore, Raymond Queneau, Anne Carson. He also writes short stories of his own from time to time. But mostly, he’s drawn to math, to problems in low-dimensional topology and geometric group theory that keep him awake at night. “Once I had this encounter with topology, which was a little bit like getting run over by a train, everything changed,” he said.

In a recent issue of the Notices of the American Mathematical Society, Calegari published an essay about the importance of failure in mathematics. Disappointment, he wrote, “is both a crisis and an opportunity.”

Quanta spoke with Calegari about what failure in mathematics looks like, and why it’s important. The interview has been condensed and edited for clarity.

Why did you decide to write about failure?

It’s something I know about — and something that people don’t talk about often enough. So I felt like I could at least try to be honest about what that experience is like, and the ways in which it can sometimes be useful.

Math is so competitive. Most people are struggling to get access to very limited resources, so no one wants to portray any kind of weakness. There’s huge sociological pressure not to admit you’re struggling.

Disappointment hurts every time, but it’s not always a bad thing.

Your essay mentions your failure to make the International Math Olympiad team in high school, and how you later saw that as an opportunity to learn math more deeply. How have you experienced failure since?

It’s always embarrassing to admit that you really wanted something that you didn’t get. A job, an invitation to a conference, acknowledgment, recognition.

My main disappointment had to do with work I did in a field called “stable commutator length,” about the relationships between elements in so-called free groups. I felt like I proved some really unexpected theorems and uncovered a lot of fundamental structure and connections to other areas of math, like dynamics. I even wrote a book about it. I thought it would be incredibly interesting to people working in low-dimensional topology, and that it would inspire them to start working immediately on related questions.

But no one cared. There were maybe one or two people in the world who built on my work, but it was basically just crickets.

So I was very disappointed. I put five or six years of my life into that. And I stopped working on it, not because I’d exhausted the subject, or because I didn’t think there were interesting things to work on, but just because it was lonely to keep doing it by myself and get no reaction.

But there was a good lesson to come out of it, which is, I ended up caring a little bit less what other people think, and am driven much more by curiosity alone. If you want to go into mathematics, doing the mathematics itself has to be the thing that’s the reward, because no one cares, and what’s considered important doesn’t always make sense.

In the spirit of that lesson, did you ever return to this work?

Just in the last few months, actually. I’m hoping to formulate more precise conjectures in this area.

Is this a common experience?

Yeah. I think 100% of mathematicians think that no one cares, that no one even knows anything about their best work. Even when someone proves some famous conjecture and gets a lot of attention, the number of people who actually read the details of the proof and fully appreciate the thing that was done — it’s microscopic. Which makes sense: It’s very hard to learn a thing that’s not in an area that you already know well, and there’s very little reward if you do it. So most people don’t do it, most of the time, which is perfectly reasonable. Why would they?

That is very lonely, though.

It is. I think this is one reason why people collaborate much more commonly these days — something that’s only happened in the last couple of decades. Because at least your co-author cares about what’s in the paper and is interested in what you’re doing.

It’s an incredibly positive development, actually. It wasn’t common for people to write multiple-author papers when I was a grad student; now it’s rare for a paper to have fewer than three authors. I think it’s a really, really good thing in math. Collaboration makes for papers with fewer errors, for a start.

So that’s another way that failure has led to something productive.

Yeah. And through collaboration, especially large-scale public collaborations like the Polymath project, failed proof attempts have become part of the public record. By reading through those failures, mathematicians can better understand the current cutting edge. People tried things, and some of them didn’t work, and there’s a record of that. The failed experiments, these hard-earned lessons, are out there and visible.

Usually, people who have been working in a field for a long time learn from experience that a certain kind of approach has certain shortcomings, for instance. And young people entering the field have to learn those things by hanging out with other mathematicians and going to conferences. But collaborative efforts are providing a new way to get those things out into the community.

What’s an example where mathematicians have learned from a failed proof?

Bill Thurston was a mathematician who turned the subject of three-dimensional topology upside down. Part of that was his geometrization conjecture, in which he gave a model for how geometric structures can be associated with three-dimensional spaces.

While starting to formulate an early version of the conjecture, he thought many times that he had a proof that certain structures were impossible. But another mathematician showed that these structures could exist. So he was wrong.

Thurston said that all his wrong attempts allowed him to develop a deeper insight into the geometry and topology of three-dimensional spaces. This was the scaffolding that the eventual geometrization conjecture rested on. Analyzing the points of failure in his program, and why his proofs had foundered, directed him to make this powerful, important conjecture.

It’s now one of the biggest results in the field, right?

Right. The hard part is that you have to shift gears at some point and say, “Well, maybe what I’m trying to prove isn’t true.” I think that happens a lot in math, but that people are very reluctant to change gears.

G.H. Hardy was once asked what distinguished Ramanujan as a mathematician. One of the things he said was that Ramanujan had a remarkable capacity for coming up with hypotheses very quickly, but that he was also very quick to revise his hypotheses. He was nimble: If something didn’t work, he was able to pivot and revise his way of thinking. Mathematicians don’t always appreciate the power of that.

Most of the time, at least in my experience, they are a little bit too quick to just accept the conventional wisdom that a conjecture is true or false, to go along with whatever everyone else is saying.

This also taps into the idea that it’s important to make conjectures in the first place, even when they’re wrong.

Absolutely true. There’s a related issue, which is that sometimes there are deep errors in papers that can’t be fixed. And if people really want to imagine that something is true and that they’ve proved something, it can be very hard to admit mistakes.

That dovetails unfortunately with how people tend to work on the conventional-wisdom version of conjectures. Because if everyone has agreed that some conjecture is probably true, and you come up with a proof of it, of course people will be critical to some extent, but they’re going to be less careful about it if it agrees with their expectations. It can be a long time before errors are found.

One of Hilbert’s problems, about families of curves defined by differential equations, was considered solved. Henri Dulac, who gave a proof of it in 1923, went to his death believing he had solved it. And then Yulij Ilyashenko found a counterexample 60 years later. These days, work in this area is much more active and lively as a result.

This sort of stuff can happen in mathematics. There can even be very famous problems where people claim a proof that everyone accepts, and a long time goes past before one discovers that it’s just not true. And the same kind of psychological pressures that make it difficult for people to deal with failure are at work in making it hard for people to carefully and critically evaluate arguments that they have a huge personal investment in being correct.

You spoke about how collaborative efforts have made failure more visible. Are there other ways to achieve that?

One other way is through teaching. There’s a tendency sometimes when you’re teaching to just present the material as: Here is the path, let’s go steadily along it. But I feel like it’s useful to introduce other directions, subtle aspects of the topic one might find confusing. Sometimes you need to interrupt the flow, in the sense that at first everything looks like it’s coming out naturally, and you want to say, let’s stop, this is not actually as easy as it maybe appears to be, what about this, what about that?

How does this relate to your interest in reading and writing?

It’s always hard to know what’s going on inside other people’s heads. It’s not easy to communicate the special intimate experience of one’s internal world.

And this is important in mathematics, because the way in which we think about mathematical objects is very different from what appears on the pages of a paper, or on the blackboard or something like that. Even if you have what you think of as a very clear and straightforward idea, you still have to communicate it to someone who has a very different way of thinking about things.

One of the great, tremendously useful and valuable functions of good writing is that it gives you a way of seeing what it’s like to be other people. You can see inside people’s heads. This is one of the great gifts of literature: You get to see that everyone else is weird, too, not just you.