3 identical Raspberry Pis, same disk image, yet PHP one one (different location) takes forever to load. Ideas on how to identify the issue?
 

This is a weird situation.

I have 3 Raspberry Pis. All 3 are identical hardware, and all run the same disk image. Each are in different physical locations (different LANs and WANs). These Pis serve as web servers for security cameras, and run php to serve up the files.

In two of the locations, everything works perfectly. In one of the locations, everything works except that the PHP script to display images on my browser takes >1 minute to load (not a trickle, but as in, delay to load).

I've compared the output of

<?php
phpinfo();
?>

from all three Pis, and they are the exact same. In the location where the PHP takes forever to load, I've changed routers, too (general upgrade) and this hasn't worked. Finally, the logs don't seem to indicate any problems.

It feels like some waiting / timeout issue, but I can't think of what it could be. I've tried local (192.168) and remote access test, and same result. Since posting this, I've tried other (basic) PHP scripts and they work without delay. So it's a problem with the code, perhaps trying to read something? Any ideas on good first steps to troubleshoot the issue?

Thanks a ton!!

Print the current time at useful locations in the code.

Is your php code accessing information from other network devices or just from itself? Is running the phpinfo script also slow?

Is the misbehaving pi connected by wifi or ethernet? Is it a bandwidth problem? Have you checked lan bandwidth speed? If accessing other devices are you using DNS or IP address.

YES!!!!!!!!!!!!!!!!! It was calling an old Yahoo weather API (doesn't work anymore), and for whatever reason, the script seems to have hung on that. Remove it and presto, immediate load. Thanks for that idea.

Now I need to figure out why one of the cameras isn't uploading anything to the Pi and there are no errors at all in the vsftp log (even with verbose logging)...

Thanks!

So you are saying there actually are differences in the scripts. It might be worth deeper investigation as to how the camera is configured as well as the script on the pi.

Interestingly the scripts are the exact same, since I have an SD card image and write that to any new Pi SD card I use. You have a great point that I should dig into this further. I decided to query the URL in question, https://query.yahooapis.com, trying a simple "wget https://query.yahooapis.com". From the Pis where I haven't had any loading issues, I get an immediate response,

But from the bad Pi, I get:

As mentioned, these are the same exact SD card images, so what do you think could be the reason for this difference?

The good Pis
The bad Pi
Looks like a DNS problem. Are you using the ISPs DNS servers via DHCP?