Why you shouldn’t glitch in 旅かえる (Tabi Kaeru)

I fast forwarded the time in my phone so my frog will be home early. However, once I changed the time back to normal, the game won’t continue until the new time is arrived. I have let my frog down.