“Inevitable” to delay Tokyo Olympics: PM Abe