Because you could use your time to work and make money. Labor is kind of selling your time to other entities and doing their stuff. And even an enterpreneur can use time to grow their business. Or capital alone regularly grows with interest or by investing it.

You could also use your time to make waffles. That’d be oddly specific, but even then you could sell those and you’d end up with money again… And exposing waffles to time just results in them growing mold.