A better question would be do you have the physical and mental stamina to go along with your passion? :D
Rather than pinning the decision on age, I'd look instead to how you cope in highly crowded and stressful environments, ability to take criticism and do as the chef asks, and what kind of physical shape you're in. Also consider if you can live on 9 - 14$/hr (the going rate in Seattle for cooks, will be less in most other parts of the US) for the next few years, and then 60 hour work weeks if you go the salary route. Not to mention lack of health insurance, and you probably won't see much of your family during holidays, since it's our job to provide leisure for other people.
I am a career changer as well, now past the three year mark in experience. But chefness and restaurants also run in my family, and this combined with personal hobbies and interests which line up with those of guys in their 20's means I get along fairly well with younger cooks. That is a very serious matter in a job which plays more like a team sport! Still, kitchens are all different, and some will suit better than others.
If my own experience so far is any indication, other things I would want to keep in mind would be making a plan as soon as possible and sticking to it, in terms of future goals. Provided it speaks to your soul's content, of course. Time in the long run is more critical for us than for the kids, IMO. Ex: I'm figuring on ten years myself. I know what kind of place I want to be chef-owner of at the end of that time, what kinds of places I want to work in first, and how I'd like to advance in the ranks to get both the cooking and business skills I need, but if I can't pull it off, I'll become a full time personal chef instead.
If following orders and doing the best possible job is important to you, accept that this might work against you because chefs sometimes have that artist/crafter eye which translates into contradictory instructions. Basically, the way you do something one day might be perfect, so you keep doing it that way, until suddenly out of the blue the chef starts screaming at you how it's wrong, even though they had approved the original way in the first place. I would say this and the sheer multitasking involved were/are my two biggest challenges; if you can find a way to be ok with that and not take the illogical personally, all the better. Same for coping with the frequency of needing to make do in conditions always not ideal.
If you're not already doing this, consider regular exercise which gets or keeps your back and legs in good shape. Same for slimming down, if necessary. Less pounds to carry makes for a less tiring day, and many kitchens are also cramped spaces.
Last comment about age: starting young isn't a guarantee of success either. Already in the time I've been doing it, I've seen plenty of line cooks who hit their 30's and are complete burn-outs looking to get out of kitchens for good. But for myself, there's no place I'd rather be than in a kitchen, and I like thinking about food when I'm not working. Definitely the right choice for me.