My Ex had surgery, when kids visit him he sends them to his?

gf's sisters house instead as he is unable to visit and respond to them. Should'nt he be allowing me the Mother and custodial parent this time?