LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 11-29-2017, 05:31 PM   #1
dmchess
Member
 
Registered: Jan 2005
Posts: 108

Rep: Reputation: 20
Google Finance problem


I have been checking my stocks with a script daily. First Yahoo cut me off and now I am having trouble pulling it from google:

wget -q -O - http://www.google.com/finance?q=$1 | \
awk 'BEGIN{first=1;vol=0}
/ref_/ { if(first) { gsub( /<[^>]*>/, "");print;first=0}}
/data-snapfield="vol_and_avg"/ { vol=1}
/class="val"/ && vol == 1 {sub( ".*>","");sub( "/.*","");print;exit}'


I tried just running the wget command and am getting nothing.
Is something going on with Google?

Terry H.
 
Old 11-29-2017, 08:37 PM   #2
michaelk
Moderator
 
Registered: Aug 2002
Posts: 16,823

Rep: Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072
I would guess it is a user agent problem which also might be the problem with Yahoo.

Code:
 
wget -U firefox -q -O - https://www.google.com/finance?q=$1 | \
awk 'BEGIN{first=1;vol=0}
/ref_/ { if(first) { gsub( /<[^>]*>/, "");print;first=0}}
/data-snapfield="vol_and_avg"/ { vol=1}
/class="val"/ && vol == 1 {sub( ".*>","");sub( "/.*","");print;exit}'

https://www.linuxquestions.org/quest...d-this-606755/
 
Old 11-29-2017, 10:24 PM   #3
dmchess
Member
 
Registered: Jan 2005
Posts: 108

Original Poster
Rep: Reputation: 20
Well, is there something I can do about this?
 
Old 11-29-2017, 10:42 PM   #4
JJJCR
Senior Member
 
Registered: Apr 2010
Posts: 1,442

Rep: Reputation: 253Reputation: 253Reputation: 253
Check out this link, it might provide some info:

https://stackoverflow.com/questions/...le-finance-api
 
Old 11-30-2017, 01:26 AM   #5
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 7,932
Blog Entries: 5

Rep: Reputation: 1898Reputation: 1898Reputation: 1898Reputation: 1898Reputation: 1898Reputation: 1898Reputation: 1898Reputation: 1898Reputation: 1898Reputation: 1898Reputation: 1898
back story:
https://www.linuxquestions.org/quest...le-4175617060/
andd why did you say it was working again, if it doesn't now?
 
Old 11-30-2017, 04:34 AM   #6
michaelk
Moderator
 
Registered: Aug 2002
Posts: 16,823

Rep: Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072
Quote:
Well, is there something I can do about this?
Did you read the thread I posted? Did you look at or try the command I posted?

I added -U firefox to your command but if that does not work there were other suggestions in the thread.
 
Old 11-30-2017, 12:25 PM   #7
dmchess
Member
 
Registered: Jan 2005
Posts: 108

Original Poster
Rep: Reputation: 20
I tried the command it's not working correctly. I tried it with F for Ford and got nothing. When I run the wget command it pulls some data out, but after piping it into the rest of the command I get nothing.

I know a little awk, but not enough to know what is going on with the script.

Terry

ps aas for Ondoho, it did work for a while, but failed yesterday.
 
Old 11-30-2017, 12:28 PM   #8
dmchess
Member
 
Registered: Jan 2005
Posts: 108

Original Poster
Rep: Reputation: 20
I should mention the block of data coming out of the wget command is big and complex and I don't know what I am looking for.

Terry
 
Old 11-30-2017, 02:24 PM   #9
michaelk
Moderator
 
Registered: Aug 2002
Posts: 16,823

Rep: Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072
Neither do we...

The user agent option i.e -U firefox fixed the 403 forbidden error so that you are now downloading the web page but no idea what data you are trying to pull from it.
 
Old 11-30-2017, 11:58 PM   #10
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 7,932
Blog Entries: 5

Rep: Reputation: 1898Reputation: 1898Reputation: 1898Reputation: 1898Reputation: 1898Reputation: 1898Reputation: 1898Reputation: 1898Reputation: 1898Reputation: 1898Reputation: 1898
Quote:
Originally Posted by dmchess View Post
as for Ondoho, it did work for a while, but failed yesterday.
if it is the exact same script, that would suggest something with the site has changed, no?

fwiw, i just tried it and got no output.
i then removed everything after & including the pipe |, and got a web page.
so it suggests you need to change how you're pulling data from it.

unfortunately, this awk regex monster is totally cryptic to me.
i prefer a commented shell script any day.
but for this task, i'd consider using an XML parser like xmllint.
 
Old 12-31-2017, 05:02 PM   #11
dmchess
Member
 
Registered: Jan 2005
Posts: 108

Original Poster
Rep: Reputation: 20
I am now piping the html out of nasdaq.com for each stock into a pascal program which prints the price and volume. This is called by another pascal program which loads that data into a mysql database. After running the program for a few days, Nasdaq changed their web site. I then altered my pascal program that filters the html. I have now been watching this for a few weeks and it seems to be working pretty good. Why so much pascal, because I am comfortable with it and it is actully relatively fast.

So I am closing this ticket,

Terry H.

Last edited by dmchess; 12-31-2017 at 05:06 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Why is Google Finance Stock Screener not showing any shares? grumpyskeptic General 3 04-16-2018 06:28 AM
Google map and Google earth problem? nec207 General 1 02-28-2016 06:22 AM
personal finance (how about now?) jbeiter Linux - Software 5 07-13-2011 06:18 PM
UK Finance Directory,Online Finance Directory,UK Finance,Finance Directory in UK seekuk General 1 12-30-2005 04:04 AM
Finance Software cajunaggie Linux - Software 4 05-26-2005 12:10 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 06:54 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration