How to get rid of bugs on plants?

[eveglass]

Sometime in the last few weeks, I think someone spilled some sugary drink near my plants. It got on my windowsill and also the plant containers and (here's the important bit) on some of the leaves. I didn't notice until today when I went to go water them. It looks like I now have tiny white flying bugs living on the leaves of my poinsettia and my african violet. I've cleaned up the flat surfaces and the pots as best I can, but how do I fix the leaves? Do I just have to cut off the ones that are sticky? Will the bugs leave if there's no sticky leaves left, or do I need to get rid of the whole plant?

Please help a panicking plant mommy!