My recipe actually specifies ripe, yellow, key limes -- and I wrote it for this thread that way. Unfortunately there were server problems last night, which combined with my usual obsessive editing, caused part of the recipe to drop out. Perhaps it's more accurate to say the server problems confused me, and when it comes to confusion there isn't much head room. My bad.
Here in Southern California, we still grow lots of key limes commercially, and also import an enormous amount of key limes from Mexico, mostly because key lime plays such an important role in Mexican cuisine. These limes are usually harvested, sold and used while small and green. Small and green as opposed to the limes on my tree which are large and yellow -- when truly ripe. These green limes, like the immature green limes from my tree, are more limey compared to the ripe limes, which are more generically citrus.
We obviously disagree on lime zest. This may or may not have something to do with how it's taken -- I take mine with a micro plane which avoids pith. It's the pith which is mostly responsible for unpleasant bitterness. The volatile oils in the skin add intensity and (to me) a pleasantly bitter aftertaste, adding one more contrast to sweet/sour/custard-creamy.
FWIW, the recipe I posted for the key lime filling tracks Joe's Stone Crab's recipe very closely, including the use of zest. Joe's Key Lime Pie
The differences between my recipe and Joe's are that, while allowing for variation in the flavor and intensity of the limes, I tend to use less lime and less zest than Joe's. On the other hand, I use considerably more lime juice than do you. We have our own lime tree, and use all our own fruit. Even so, or perhaps because, the quality of the fruit varies tremendously. If you disagree on the use of any lime zest at all, you should take it up with Joe's. Boar D. Laze only pawn in game of life. I also use a slightly different crust, but that wasn't at issue here so I didn't include it.
As you may have guessed, my real concerns on this forum are far more about teaching strong cooking techniques than disseminating any recipe or particular set of recipes. Considering that you and I often run into one another on baking threads it may surprise you that I don't consider baking to be one of my strengths -- still more true with pastry. I'm adequate, and no more (K-A on proteins and sauces, if I say so myself).
Nevertheless, my experience is that most home bakers follow recipes too closely to the extent they rely too heavily on repeating accurate measurements without much regard to their materials. Especially with fruit, the products vary too much for a tsp of this and a cup of that to be reliable ratios which will produce a particular flavor. What's important are tastes and textures. To the extent that it's possible, that means tasting. Whether you can taste or not, you always need to use your senses of taste and smell -- not to mention the good sense the good Lord gave you. Good cooking requires active involvement. That's why I posted the variable measurements and the tasting advice. IMO, that way of thinking about food is more important than the question of zest or no zest.