'Most of all' or 'least of all'

Which is correct? I also want to know whether semicolon is the right punctuation after “…like that”.
I would never tolerate anyone speaking to me like that;most of all, my wife.
I would never tolerate anyone speaking to me like that;least of all, my wife.

An independent clause must follow a semicolon unless it is part of a list of items with internal commas.

Thank you, Mister Micawber.
And,are both the sentences correct?If so, what is the difference in the meaning of the sentences?

When I read them to myself, both seem possible. The first (‘most of all’) refers to ‘anyone’ and the second (‘least of all’) refers to ‘tolerate’. The second is probably the expected.

Thank you once agian.