So I did the usual "apt update && apt upgrade" on two of my Debian 10 boxes and did a reboot. The upgrade process went just fine. I had no issues with one of the boxes as it came up pretty quick after reboot and I can access it through SSH as usual.

For the other box though, I was waiting for it to come back up by pinging it continuously and it never did until I needed to go to the actual server itself and powered it off using the power button (without holding it, just one press, so I assume this is not a force shutdown) and then it came back up as normal and I can access it through SSH again. Which log file should I be checking to see what happened why I was not able to access the server after that first reboot after the upgrade?

For something like this, I'd start with /var/log/syslog, /var/log/messages, and /var/log/kern.log.

You may find this article helpful: https://vitux.com/how-to-watch-or-mo...-in-debian-10/

And the journal, assuming vanilla Debian, and persistent logging is on (it should, on a server):
for the last boot, or
for EVERYTHING.

In any case, you need to find the relevant timeframe.

Got it, thanks!

How is this an answer to a request for troubleshooting information?
If your problem is SOLVED, please mark it so, and share the solution.

Wh can't you just tell me to mark the thread as solved without being condescending? Really, what is wrong with you?