First cook your starch fully!
I read some where about waxy maize flour or rice flour being a better starch for frozen applications. You might also want to try a modified cornstartch for stability.
I also know that other thickeners are used in commercial icecream's for the same purpose. For example: Carregenean, Agar-Agar, geletin.
You may have to increase the amount of suger, to compensate for the lack of fat and emulsifiers.
You could also try using almond, oat, or rice milk as your base. My favorite is almond milk
The ingredients you are eliminating promote a fine ice crystal size, richness, and smoothness. So depending on your formula, you are probalbly going to end up with a grainy sorbet type texture versus a smooth rich ice creamy texture.