Salt kills yeast, agreed.
Sugar -- or honey, syrup, beet reduction, whatever -- is simply a purer form of food for the yeast than the sugars present in flour.
Remember, yeast is a small organism that basically eats sugar and farts. When it's eaten enough, it generally either goes dormant or dies. So if you have a huge amount of pure sugar of any kind in your dough, the yeast its it all up, farts massively, and stops. So you get a rapid rise that ends quickly. If you have no pure sugar at all, i.e. your dough is all flour like in a French baguette, the yeast takes a good while to produce much farting action.
Let's try this differently. What are you trying to bake?
Edit: Clarification. Yeast is a small organism that eats sugar and THEN farts. It doesn't eat your farts. I'd not recommend that as a baking technique, at all.