Some comments from a physicist:

The universe is a closed system by definition.

The big bang theory states a bit more than the existence of an infinite dense and hot universe at some point in time. It states as well that there was no point in time before that. Clocks started ticking at the big bang and in these sense one can say the universe “began to exist”. Although everything for which there is a conservation law in physics was already there. General relativity gives us that a universe in the big bang state should be frozen in time, i.e. time does not advance there.

As for evidence on something that starts to exist at some point in time we have two: the universe is generating space through cosmic expansion and it is generating time, every second that just has passed wasn’t there a second ago.