With-in the end.

Seems like its only me you are having issues with .
Seems like its only me with whom you are having issues .
Seems like its only me whom you are having issues with .

-Which is more appropriate ?

Thanks in advance.

If you are speaking naturally, then the first is most appropriate.

I agree, the first seems most natural.