LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 08-29-2011, 04:12 AM   #1
Aquarius_Girl
Senior Member
 
Registered: Dec 2008
Posts: 4,731
Blog Entries: 29

Rep: Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940
wget -c website's certificate verification


Code:
anisha@linux-dopx:~/Desktop> wget -c --no-check-certificate https://www.smart-developer.com/lnmshop/download/SmartDeveloperUS/203/SD_02.pdf
--2011-08-29 14:47:49--  https://www.smart-developer.com/lnmshop/download/SmartDeveloperUS/203/SD_02.pdf
Resolving www.smart-developer.com... 80.237.227.161
Connecting to www.smart-developer.com|80.237.227.161|:443... connected.
WARNING: cannot verify www.smart-developer.com's certificate, issued by `/C=US/O=Thawte, Inc./OU=Domain Validated SSL/CN=Thawte DV SSL CA':
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `SD_02.pdf'

100%[========================================================================>] 19,286      77.1K/s   in 0.2s    

2011-08-29 14:47:52 (77.1 KB/s) - `SD_02.pdf' saved [19286]
The book that gets downloaded is in KB,
and shows error on opening too, where as
on downloading through Firefox directly,
I have no issues.
 
Old 08-29-2011, 04:26 AM   #2
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
I tried to download the same and it results in a file of size 19286 bytes (the same as yours) which is a HTML document. From Firefox it redirects me to a login page. Maybe you have simply to provide username and password by means of --user and --password options of wget. Just a guess.
 
Old 08-29-2011, 04:30 AM   #3
Aquarius_Girl
Senior Member
 
Registered: Dec 2008
Posts: 4,731

Original Poster
Blog Entries: 29

Rep: Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940
Thanks for bothering, but that didn't help:
Code:
anisha@linux-dopx:~/Desktop> wget -c --no-check-certificate --user abc --password xyz https://www.smart-developer.com/lnmshop/download/SmartDeveloperUS/203/SD_02.pdf
--2011-08-29 15:11:53--  https://www.smart-developer.com/lnmshop/download/SmartDeveloperUS/203/SD_02.pdf
Resolving www.smart-developer.com... 80.237.227.161
Connecting to www.smart-developer.com|80.237.227.161|:443... connected.
WARNING: cannot verify www.smart-developer.com's certificate, issued by `/C=US/O=Thawte, Inc./OU=Domain Validated SSL/CN=Thawte DV SSL CA':
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `SD_02.pdf'
    [   <=>                                                                   ] 19,286      10.4K/s   in 1.8s    
2011-08-29 15:12:00 (10.4 KB/s) - `SD_02.pdf' saved [19286]

Last edited by Aquarius_Girl; 08-29-2011 at 04:34 AM.
 
Old 08-29-2011, 04:39 AM   #4
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
That's only a WARNING. Instead if you don't use the --no-check-certificate option you get an ERROR and the file is not downloaded:
Code:
┌─[colucix@ocean-4]-[~]
└─• wget -c  https://www.smart-developer.com/lnmshop/download/SmartDeveloperUS/203/SD_02.pdf
--2011-08-29 11:36:21--  https://www.smart-developer.com/lnmshop/download/SmartDeveloperUS/203/SD_02.pdf
Resolving www.smart-developer.com... 80.237.227.161
Connecting to www.smart-developer.com|80.237.227.161|:443... connected.
ERROR: cannot verify www.smart-developer.com’s certificate, issued by “/C=US/O=Thawte, Inc./OU=Domain Validated SSL/CN=Thawte DV SSL CA”:
  Unable to locally verify the issuer’s authority.
To connect to www.smart-developer.com insecurely, use ‘--no-check-certificate’.
It looks like it checks the certificate in any case, but using the --no-check-certificate it ignores it afterwards. This is clearly stated in the wget manual:
Quote:
This option forces an “insecure” mode of operation that turns the certificate verification errors into warnings and allows you to proceed.

Last edited by colucix; 08-29-2011 at 04:42 AM.
 
1 members found this post helpful.
Old 08-29-2011, 04:42 AM   #5
Aquarius_Girl
Senior Member
 
Registered: Dec 2008
Posts: 4,731

Original Poster
Blog Entries: 29

Rep: Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940
So, what to do now Mr. Moderator?
 
Old 08-29-2011, 04:44 AM   #6
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
You can try to provide username and password, Mrs. Senior Member! Maybe when downloading from Firefox you were already logged in.
 
Old 08-29-2011, 04:48 AM   #7
Aquarius_Girl
Senior Member
 
Registered: Dec 2008
Posts: 4,731

Original Poster
Blog Entries: 29

Rep: Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940
Perhaps you didn't notice that in my second post I have
provided username and password but to no avail, Alessa.. spellings are too difficult!

Last edited by Aquarius_Girl; 08-29-2011 at 04:50 AM.
 
Old 08-29-2011, 04:50 AM   #8
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
Indeed, I didn't noticed that. Hmmm... need for further investigation.
 
Old 08-29-2011, 05:06 AM   #9
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
Two things coming in my mind: does the password contain some special characters expanded by the shell? Have you tried to enclose it in single quotes? An alternative to send credentials is to embed them in the URL:
Code:
wget -c --no-check-certificate https://abc:xyz@www.smart-developer.com/lnmshop/download/SmartDeveloperUS/203/SD_02.pdf
 
1 members found this post helpful.
Old 08-29-2011, 05:19 AM   #10
Aquarius_Girl
Senior Member
 
Registered: Dec 2008
Posts: 4,731

Original Poster
Blog Entries: 29

Rep: Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940
Seems the shell can't tolerate @ in the username!
See the placements of quotes too.
Scroll right.
Code:
anisha@linux-dopx:~> wget -c https://x.y.z@gmail.com:abc@www.smart-developer.com/lnmshop/download/SmartDeveloperUS/203/SD_02.pdf
https://x.y.z@gmail.com:abc@www.smart-developer.com/lnmshop/download/SmartDeveloperUS/203/SD_02.pdf: Bad port number.

anisha@linux-dopx:~> wget -c https://'x.y.z@gmail.com':abc@www.smart-developer.com/lnmshop/download/SmartDeveloperUS/203/SD_02.pdf
https://x.y.z@gmail.com:abc@www.smart-developer.com/lnmshop/download/SmartDeveloperUS/203/SD_02.pdf: Bad port number.

anisha@linux-dopx:~> wget -c https://'x.y.z@gmail.com:abc'@www.smart-developer.com/lnmshop/download/SmartDeveloperUS/203/SD_02.pdf
https://x.y.z@gmail.com:abc@www.smart-developer.com/lnmshop/download/SmartDeveloperUS/203/SD_02.pdf: Bad port number.

anisha@linux-dopx:~> wget -c https://x.y.z:abc@www.smart-developer.com/lnmshop/download/SmartDeveloperUS/203/SD_02.pdf
--2011-08-29 16:01:38--  https://x.y.z:*password*@www.smart-developer.com/lnmshop/download/SmartDeveloperUS/203/SD_02.pdf
Resolving www.smart-developer.com... 80.237.227.161
Connecting to www.smart-developer.com|80.237.227.161|:443... connected.
ERROR: cannot verify www.smart-developer.com's certificate, issued by `/C=US/O=Thawte, Inc./OU=Domain Validated SSL/CN=Thawte DV SSL CA':
  Unable to locally verify the issuer's authority.
To connect to www.smart-developer.com insecurely, use `--no-check-certificate'.
Unable to establish SSL connection.
What to do?

Last edited by Aquarius_Girl; 08-29-2011 at 05:27 AM.
 
Old 08-29-2011, 06:00 AM   #11
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
What if you try --user and embed the user name in single quotes? Otherwise you can try to use URL escapes, for example:
Code:
wget -c --no-check-certificate https://x.y.z%40gmail.com:abc@www.smart-developer.com/lnmshop/download/SmartDeveloperUS/203/SD_02.pdf
 
1 members found this post helpful.
Old 08-29-2011, 06:45 AM   #12
Aquarius_Girl
Senior Member
 
Registered: Dec 2008
Posts: 4,731

Original Poster
Blog Entries: 29

Rep: Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940
Quote:
Originally Posted by colucix View Post
What if you try --user and embed the user name in single quotes?
Look at the second output in the above post, please.

Quote:
Originally Posted by colucix View Post
Otherwise you can try to use URL escapes, for example:
Code:
wget -c --no-check-certificate https://x.y.z%40gmail.com:abc@www.smart-developer.com/lnmshop/download/SmartDeveloperUS/203/SD_02.pdf
Tried this, it didn't show the bad port error though,
but didn't result in anything meaningful too. The size
of the pdf (after downloading) is still 186 KB, whereas
the actual file is of 20 MB.
 
Old 08-29-2011, 07:05 AM   #13
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
Quote:
Originally Posted by Anisha Kaul View Post
Look at the second output in the above post, please.
I noticed it but I meant to use the --user option instead of embedding the user/pass in the URL. Anyway, since the bad port error disappeared, I think the @ is interpreted correctly. Unfortunately, I am running out of suggestions, now!
 
Old 08-29-2011, 07:10 AM   #14
Aquarius_Girl
Senior Member
 
Registered: Dec 2008
Posts: 4,731

Original Poster
Blog Entries: 29

Rep: Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940Reputation: 940
Quote:
Originally Posted by colucix View Post
Unfortunately, I am running out of suggestions, now!
Nevermind, and thanks for hanging around patiently. Also,
I have put in the question here: http://unix.stackexchange.com/questi...e-verification

Let's us if they are more witter than LQ'ers.
 
Old 08-29-2011, 08:15 AM   #15
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
I found an interesting example in the wget man page. It assumes the remote server uses cookies to manage the authenticated session:
Code:
This example shows how to log to a server using POST and then proceed to download the desired
pages, presumably only accessible to authorized users:

        # Log in to the server.  This can be done only once.
        wget --save-cookies cookies.txt \
             --post-data 'user=foo&password=bar' \
             http://server.com/auth.php

        # Now grab the page or pages we care about.
        wget --load-cookies cookies.txt \
             -p http://server.com/interesting/article.php

If the server is using session cookies to track user authentication, the above will not work
because --save-cookies will not save them (and neither will browsers) and the cookies.txt file
will be empty.  In that case use --keep-session-cookies along with --save-cookies to force saving
of session cookies.
Maybe it's worth a try.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
wget to login to a website uks Linux - General 6 11-01-2012 04:44 AM
fetchmail socket error and certificate verification lmcilwain Linux - Software 8 09-24-2010 05:45 AM
wget through an authenticated website davee Linux - Networking 2 02-11-2009 04:57 AM
Using wget to test if a website is up ericcarlson Linux - Software 3 10-14-2008 08:36 AM
Installing Certificate to an existing website. drendz Linux - Security 2 07-01-2004 07:38 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 10:05 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration