I think it's Eddard Stark. The reason why littlefinger thinks he took Catelyn's maidenhood is because he regarded Lysa as Catelyn when he is drank. This is from what Lysa said, so it is not for sure, either.
By the way, is "Tansy" mentioned in Hoster's uncouncious really Lysa, as Catelyn suspected?