It sounds to me that in addition to researching all the regulatory issues, you need to also research from a marketing perspective. We sell our products in many farmers markets around the NY Metro area and see lots of people such as you come and go.
First, farmers markets are really about farmers and promoting sustainable, local agriculture. You must keep this in mind when deciding what kinds of products you want to offer. That will also determine how you approach the farmers market board with your idea. If you just want to go to your grocery store and whip up some baked goods and soups to offer, I doubt you'll get very far with them. You need to be able to succinctly communicate how your product line supports the overall mission of that particular market.
Second, you need to come up with a product idea that is unique and not widely available in your region's grocery stores, restaurants and markets.
Third, limit your offerings so you can create a brand identity for yourself. We sell at markets where one vendor constantly gripes about how she never really makes much money after her costs, but one week she shows up with pickles, baked goods & salad dressings, then the next week it'f fresh mozzarella, french bread and flavored olive oil, then the following week it's pickles and marinara sauce and who knows what else. It's all about developing a following. If no one can predict what you're going to have, no one will bother to stop by your booth.
Fourth, you need to invest some money into presentation of your product. Your signage needs to look interesting and eye-catching in addition to being clear and understandable. Your packaging should also reflect your message and identity. Think carefully about how you will package your product. Promoting sustainability through your packaging is also a real plus with market boards.
There are many more things you should think about before you venture into the farmers market world as a plan to earn a living, but one that is most important is whether you really like going to farmers markets and having lots of face time with people who shop there and other vendors. If talking to lots of people who are fussy about their diets, ingredients, organics and vegetarianism gets on your nerves, look for something else to do. If you are uncomfortable talking to strangers and initiating conversations with them, you will not do well in the fm arena. You must project an open, engaging, approachable and friendly demeanor. Never sit in a chair behind your booth waiting for customers to come to you. Stand there, give samples and be able to answer lots of questions about ingredients, calorie counts and nutrition.
My best advice is to visit lots of markets and familiarize yourself with the products offered there. Some markets are better than others. Find a way to fill an empty niche at the markets to choose to join. Talk to the vendors and ask what niche they think should be filled.
Lastly, it's unlikely you'll make much money at one market alone. We do ten per week, plus our shop during the busy season and only in this way can we truly support ourselves. Given the right approach, you can do well here but you must be ready and willing to become part of a FM community and culture.