Personally, I'd ditch the woods totally and stick with the irons, especially with pitch and putt experience. I'm pretty crap, but I've finally realised that straight is more important than distance. A decent 5 iron (or even 7 iron) is better than a sliced or hooked 200 yard plus drive*. If you do end up playing semi regularly, get a forgiving set of irons designed for high handicappers, and a rescue club or two.
*giving this advice makes me feel ancient.