I have an issue with running python scripts on linode cloud server. I create a scrapping script to scrape data by check in and check out so that i run the script using screen method. It run over one day very smoothly but today it hang. No response. When i call getRequest() method then i print the proxy number and send request with proxy and timeout=10.
Code:
def getRequest(self,url):
self.proxieName = random.choice(self.proxyDict.keys())
print getYellow("Proxy: ")+getCyan(self.proxieName)
self.proxie = self.proxyDict[self.proxieName]
try:
self.getRqst=requests.get(url,headers=self.nrmlHeaders,proxies=self.proxie, timeout=10)
except Exception, e:
print e
time.sleep(10)
return self.getRequest(url)
else:
if self.getRqst.status_code==200:
self.rqstErr = 0
return self.getRqst
else:
self.rqstErr+=1
print getYellow("Hi Dear, Don't worry. May be Server down. Exception in Url: ")+url
print getRed("Error!!! ")+getBlue("I get Status Code: ")+str(self.getRqst.status_code)
print getGreen("I am sleeping now for 10 second")
time.sleep(10)
return self.getRequest(url)
Here is the image snapshot link
https://www.dropbox.com/s/yh2bs7z1mx...error.png?dl=0
after getting the content i parse them and save information into a CSV file and send another request.
And i see also that after 01:40 of 6th July the csv file alos not updated. So it's hanging.
