Generally, the rule of thumb is not to leave perishable foods like meat, dairy, or casseroles out for more than two hours. After that, bacteria can start to multiply, and it might not be safe to eat. So, if your tater tot casserole has been chillin' on the counter for longer than that, it's probably best to toss it. Better safe than sorry, right?