Are they interchangeable, "it's not the time to watch the match" and "the match hasn't started yet"?

They're obviously not fully interchangeable - it's possible that it's not the time to watch the match because the match has already finished.

Overall, you could use "it's not the time to..." in a literal fashion to mean that something cannot be done at this time, but to me it wouldn't sound natural. Usually, this phrase means that you shouldn't be doing something, rather than that you aren't able to - because it's inappropriate, for example, or because it'll make things worse.

For instance, if someone has just died and your friend wants to watch a match, you could tell him that "it's not the time to watch a match" - meaning that it's inappropriate to do so when you're grieving.