It was a port town with ships from all over the world arriving. They decided they'd base their colours on the flag of the first ship that arrived, which came from Sweden.
I'm a bit of Boca fan (thanks to Maradona obviously) and trust me, no Irishman was involved with them at the start.