The recipe is a simple cold infusion of Earl Grey (15g leaves for 300g liquid).
I've cooked it three times, the third time without infusing the liquid and the texture was perfect. It's a small sample size, but my hunch is that there's something about the tea that messes up the custard somehow. To be clear, the only variable I changed was removing the infusion step and that made it turn out great.
I've attached a picture. As you can see the color isn't great either and I'm thinking about scrapping the dish altogether if I can't get the texture to turn out smooth. Really would appreciate some insight into why it turned out so poor.