LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 04-26-2022, 10:27 AM   #1
Bartonsen
Member
 
Registered: Oct 2016
Posts: 41

Rep: Reputation: Disabled
Download assistance


I have a raspberry pi, and I'm trying to make a small script to download the betting program from this page: https://www.norsk-tipping.no/sport/oddsen

The purpose is just to dl the football program to the pi, but I'm struggling in how to dl the matches and odds. I've tried curl and wget, but not getting anything...
Any suggestions? Is it doable?
 
Old 04-26-2022, 10:36 AM   #2
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,882
Blog Entries: 13

Rep: Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930
Anything you can type which works on the command line, you can put into a shell script.

So you need to find a command line way to fetch the data you want first. Wget usually works though.
 
Old 04-26-2022, 10:59 AM   #3
Bartonsen
Member
 
Registered: Oct 2016
Posts: 41

Original Poster
Rep: Reputation: Disabled
Thanks for input.

My challenge is that the file retrieved by wget "https://www.norsk-tipping.no/sport/oddsen"
does not contain any matches. It's just a html file but not of interest.

I guess they are using a script program or something to "hide" the content...?
 
Old 04-26-2022, 12:36 PM   #4
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,882
Blog Entries: 13

Rep: Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930
Quote:
Originally Posted by Bartonsen View Post
Thanks for input.

My challenge is that the file retrieved by wget "https://www.norsk-tipping.no/sport/oddsen"
does not contain any matches. It's just a html file but not of interest.

I guess they are using a script program or something to "hide" the content...?
Can't say I actually didn't visit the link.

But ... from your first post I sounded as if you wanted to dl their program? Or do you mean a game program that is data covering the details of a specific game versus their actual code? I'd doubt they'd publish their code or image to be accessible on a site.
 
Old 04-26-2022, 01:39 PM   #5
Bartonsen
Member
 
Registered: Oct 2016
Posts: 41

Original Poster
Rep: Reputation: Disabled
No, I'm not interested in their code. I'm interested in the matches and the odds, e.g.
Manchester City Real Madrid 1.45 4.50 6.20
Liverpool Villarreal CF 1.28 5.50 9.50
etc.
But the file I download does not contain any matches, so I'm curious where they've "hidden" the matches and odds...
 
Old 04-26-2022, 04:07 PM   #6
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,882
Blog Entries: 13

Rep: Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930Reputation: 4930
Well under the browser if you save website full, does that contain the data? If so you'll have to figure out how the browser saves the whole page and then filter out what you want.

Or .... it's been a long time, some browsers had or have like an attribute like view, I forget the name. I use the keyboard a lot versus mouse and so I use CTRL-L to allow me to type into the browser location bar for my website choice. At times I would "fat finger" it and hit CTRL-I and whatever they call the view, it would bring up like expert page information into a sub-window.
 
Old 04-26-2022, 06:28 PM   #7
dugan
LQ Guru
 
Registered: Nov 2003
Location: Canada
Distribution: distro hopper
Posts: 11,226

Rep: Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320Reputation: 5320
If you can’t find a JSON API, then use PhantomJS to scrape the site.
 
Old 04-26-2022, 11:06 PM   #8
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 19,872
Blog Entries: 12

Rep: Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053
^ This is actually the right answer.
Yes, the site does not work without javascript and wget doesn't do javascript.
There's no actual content on that page's HTML, it's all provided by javascript.
Good luck picking that apart.
 
Old 04-28-2022, 10:34 PM   #9
JJJCR
Senior Member
 
Registered: Apr 2010
Posts: 2,150

Rep: Reputation: 449Reputation: 449Reputation: 449Reputation: 449Reputation: 449
Yes, indeed scrape it.

Maybe the OP is hoping like "save as" and all data is downloaded. Save as used to work on the old days (table era, but now with bootstrap and other tools; scraping with python or other language could be your best option), but newer websites doesn't let you save some data easily.

https://realpython.com/beautiful-sou...craper-python/

Last edited by JJJCR; 05-01-2022 at 10:58 PM. Reason: edit
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
sudo apt-get install --download-only *Assistance* DatSik Linux - General 3 12-14-2012 06:52 PM
Assistance needed to investigate unauthorised download by unknown service nic2 Linux - General 2 11-18-2008 07:26 PM
Assistance Needed! Yellow Dog 5.0.1 Download kgaouette Linux - Newbie 8 05-22-2007 07:44 AM
assistance with an algorithm mandrake_linux Programming 3 07-27-2001 04:03 AM
Need assistance on setting up a server wa03 Linux - Networking 1 09-04-2000 03:28 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 11:00 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration