'each of which' or 'each of that'?

English as a Second Language English Vocabulary, Grammar and Idioms
1

The United States consists of fifty states, ______ has its own government.
A. each of which D. each of that

The answer is A. But as I’ve learned before, ‘that’ and ‘which’ is interchangeable. So D is okay, isn’t it?
Let give me an useful advice. Thanks in advance.

2

‘Each of which’ is the correct one because ‘each’ refers to the 50 states. ‘That’ can only refer to one state.