Hi pivosz
“It’s (high) time we left/went/etc.” is a very standard expression and I’d say this form of the verb (left/went/etc.) is an example of the subjunctive. It basically means “we should go” / “we should leave”/ etc.
It only looks like the past tense, but it actually refers to the future.
It’s similar to what you see in some “IF-sentences”:
“If we left at 5 a.m. tomorrow morning, we would miss most of the rush hour traffic.” The word “left” in this sentence also refers to the future.
Amy