LinuxQuestions.org - Need help to use sock proxy while sending http request using AnyEvent in perl

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - Need help to use sock proxy while sending http request using AnyEvent in perl (https://www.linuxquestions.org/questions/programming-9/need-help-to-use-sock-proxy-while-sending-http-request-using-anyevent-in-perl-4175629484/)

Need help to use sock proxy while sending http request using AnyEvent in perl

I am trying to send multiple http get requests using perl. I need to send those request using sock proxy.
If i use following code, I am able to send request with sock proxy

Code:

#!/usr/bin/perl

use strict;

use LWP::UserAgent;

my $ua = LWP::UserAgent->new(

  agent => q{Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; YPC 3.2.0; .NET 

CLR 1.1.4322)},

);

$ua->proxy([qw/ http https /] => 'socks://localhost:9050'); # Tor proxy

#$ua->cookie_jar({});

$a = 10;

while( $a < 20 ) {

my $rsp = $ua->get('http://domain.com/type?parameter=1&parameter=2');

print $rsp->content;

$a = $a + 1;

}

It works successfully but I need to use AnyEvent to send multiple get requests parallely.

Code:

#!/usr/bin/perl

use strict;

use AnyEvent;

use AnyEvent::HTTP;

use Time::HiRes qw(time);

use LWP::Protocol::socks;

use AnyEvent::Socket;

my $cv = AnyEvent->condvar( cb => sub {

    warn "done";

});

my $urls = ["http://url-1-withallparameters",

"http://url-2-withallparameters",

"http://url-3-withallparameters",

];

my $start = time;

my $result;

$cv->begin(sub { shift->send($result) });

for my $url (@$urls) {

    $cv->begin;

    my $now = time;

    my $request;  

    $request = http_request(

      GET => $url, 

      timeout => 2, # seconds

      sub {

        my ($body, $hdr) = @_;

        if ($hdr->{Status} =~ /^2/) {

          push (@$result, join("\t", ($url,

                                      " has length ",

                                      $hdr->{'content-length'}, 

                                      " and loaded in ",

                                      time - $now,

                                      "ms"))

              );

        } else {

          push (@$result, join("",

                              "Error for ",

                              $url,

                              ": (", 

                              $hdr->{Status}, 

                              ") ", 

                              $hdr->{Reason})

                );

        }

        undef $request;

        $cv->end;

      }

  );

  }

$cv->end;

warn "End of loop\n";

my $foo =  $cv->recv;

print join("\n", @$foo), "\n" if defined $foo;

print "Total elapsed time: ", time-$start, "ms\n";

This is working fine but I am not able to send these requests using sock proxy.
Even in terminal, I export proxy, commands like curl wget work fine with sock proxy but when I use perl command to execute this script as a sock proxy, it does not work. I could integrate it with LWP agent, but it is not working with Any Event.

I have used
proxy => [$host, $port],
below
GET => $url,
but it works for http and https proxy only, not for sock proxy. I am not sure what I am doing wrong. If you suggest any solution, it will be great help from your side.

Thanks
Sam

I'm not sure what problem is caused with AnyEvent, but this may be useful, i like to use IO::Socket::Socks for any socks proxy connections;

You'd use it like this for HTTP requests:

Code:

#!/usr/bin/env perl

use strict;

use warnings;

use IO::Socket::Socks;

$|++;



# set the socks proxy

my ($socks_addr, $socks_port) = ("127.0.0.1", 9050);



my $sock = new IO::Socket::Socks(

        ProxyAddr        => $socks_addr,

        ProxyPort        => $socks_port,

        ConnectAddr        => 'checkip.dyndns.org',

        ConnectPort        => '80',

        SocksDebug        => 0,

        Timeout        => 4,

        SocksVersion        => 5

) or die "[-] $SOCKS_ERROR";



print $sock "GET / HTTP 1.1\r\n\r\n";

print join ('',<$sock>)

Of course, you'd have to send raw http requests for things like "GET /path/to/page" etc.

I think you could use 'runsocks', 'tsocks' or 'redsocks' or similar program to redict traffic to SOCKS-proxy.

Maybe you could use threads

Code:

#!/usr/bin/perl



use strict;

use warnings;

use threads;

use Thread::Queue;



my $queue;



# worker thread

sub getUrl {

    require LWP::UserAgent;



    my $ua = LWP::UserAgent->new(

        agent => q{Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; YPC 3.2.0; .NETCLR 1.1.4322)},

    );

    $ua->proxy([qw/ http https /] => 'socks://localhost:9050'); # Tor proxy

    

    while (my $url = $queue->dequeue) {

        my $rsp = $ua->get($url);

        print $rsp->content;

    }

}



$queue = Thread::Queue->new;

my @workers;



# run 4 threads

push @workers, threads->new(\&getUrl) for (1..4);



$queue->enqueue(

    "http://url-1-withallparameters",

    "http://url-2-withallparameters",

    "http://url-3-withallparameters",

    "http://url-4-withallparameters",

    "http://url-5-withallparameters",

    "http://url-6-withallparameters"

);



$queue->end;

$_->join() foreach @workers;