[SOLVED] redirecting from a subdirectory via apache

Nominal Animal · 11-19-2011, 09:49 PM

Quote:

Originally Posted by compused

Thanks again Animal, your persistence is greatly appreciated. This information doesnt seem to be available anywhere on the net. As a shortcut, I tried adding the Header always edit Location: rules but that didnt make any difference.

I only realized after you tested it, that I forgot to verify the Header directives work. Well, they did not, so feel free to blame me for yet another brainfart

Quote:

Originally Posted by compused

running: wget --post-data 'username=username&secretkey=password&just_logged_in=1' --http-user=myusername --http-password=whatever -O - http://www.myserver.com.au:2014/office/src/redirect.php produces (I will put it all in):

Ah ha! Look at the Location: line; it redirects to /mindex.php, when it should redirect to /office/mindex.php or http://www.myserver.com.au:2014/office/mindex.php . Remember, only the /office subtree is obtained from my2ndserver; we need to edit all links and redirects and other URL references to point to the /office subtree.

It seems the only problem left is the Location: headers not being fixed. (I think cookies should work okay, but obviously you'll need to test as much as you can to make sure; these edits are not foolproof.)

I fixed (and this time lightly tested, too) the Header directives in my post above.

I think I forgot the QSA flag from the original rewrite, too. It retains the GET query parameters (that follow a question mark in the URL). Also, the ProxyPassReverse should probably have /office as the first parameter, too. I think we can also drop the host part from any URLs we edit in the content; /office/... works just as well as http://www.myserver.com.au:2014/office/... (I think). And I forgot about URLs in CSS files; they need Substitute rules too.

Here is my suggested configuration example thus far, with all the changes incorporated:

Code:

<VirtualHost *:2014>
    ServerName                         http://www.myserver.com.au:2014/

    UseCanonicalName                   Off
    UseCanonicalPhysicalPort           Off

    # We do not need these proxy capabilities, so disable them.
    ProxyVia                           Off
    ProxyRequests                      Off

    # Followsymlinks option is needed for RewriteEngine to work.
    Options                            FollowSymlinks

    # Append a slash. This lets http://.../office URL work correctly.
    DirectorySlash                     On

    # The published web tree for this server.
    DocumentRoot                       /var/www
    <Directory /var/www/>
        AllowOverride                  None
        Options                        FollowSymlinks
        Order                          Allow,Deny
        Allow                          From All
    </Directory>

    # The subtree that is reverse-proxied from my2ndserver.
    <Directory /var/www/office/>
        RewriteEngine                  On

        # /office subtree is obtained from http://www.my2ndserver.com.au:103/,
        RewriteBase                    /office
        RewriteRule                    ^(.*)$   http://www.my2ndserver.com.au:103/$1 [QSA,P]

        # and Apache should fix headers accordingly.
        ProxyPassReverse               /office   http://www.my2ndserver.com.au:103/
        ProxyPassReverseCookieDomain   www.my2ndserver.com.au   www.myserver.com.au
        ProxyPassReverseCookiePath     /   /office
 
        # mod_substitute is used to fix the links in the content.
        # Limit link fixing to content that we know may contain links.
        AddOutputFilterByType          SUBSTITUTE   text/html text/css text/plain

        # Fix absolute paths, from / to /office.
        Substitute "s|(=[\"']?)/+|$1/office/|q"

        # Fix full absolute URLs that contain a path part (at least /).
        Substitute "s|(=[\"']?)http://www\.my2ndserver\.com\.au:103/+|$1/office/|q"

        # Fix URLs that do not contain a path part; i.e. they end after the port number.
        Substitute "s|(=[\"']?)http://www\.my2ndserver\.com\.au:103([^0-9])|$1/office/$2|q"

        # CSS files may also contain URLs, as "url(...)".
        Substitute "s|(url\([\"']?)http://www\.my2ndserver\.com\.au:103/+|$1/office/|q"
        Substitute "s|(url\([\"']?)/+|$1/office/|q"

        # Location: header is used for redirection; they need editing too.
        # Fix full URLs that have no path part (end with the port number):
        Header edit Location:  ^http://www\.my2ndserver\.com\.au:103$   http://www.myserver.com.au:2014/office/

        # Fix full URLs that have a path part:
        Header edit Location:  ^http://www\.my2ndserver\.com\.au:103/+(.*)$  http://www.myserver.com.au:2014/office/$1

        # Change path-only URLs to a full URL. This makes further rules simpler.
        Header edit Location:  ^/+office$         http://www.myserver.com.au:2014/office/
        Header edit Location:  ^/+office/+(.*)$   http://www.myserver.com.au:2014/office/$1

        # All paths left need to be moved to under /office.
        Header edit Location:  ^/+(.*)$           http://www.myserver.com.au:2014/office/$1
    </Directory>
</VirtualHost>

I added some comments in the hopes that it clarifies things a bit -- and because I seem to keep making mistakes, it might help catch my mistakes.

It is getting to be pretty complicated at this point, but I do think it is fairly complete now, if it works for you.

Could you please test this configuration?

You can obviously adapt it to your host names and IP addresses and so on, there is no need to use this exact one, as long as the /office Directory directive looks more or less like the above. Note that the order of directives of the same kind (Substitute, Header, RewriteRule) is important; do not change their order unless you are aware of how that affects Apache processing.

compused · 11-20-2011, 07:37 AM

Animal, congratulations, the masked redirection from a subdirectory is now working flawlessly as requested, on myserver.com.au:2014 ...and thanks again for your persistence! Your last code is a lot more elegant than initially, too.

However, there is just one...um...small problem.

That is, I have been unable to successfully move this block of code back to port 80, on the main server, which is a drupal installation. I get the same problem when trying to access http://www.myserver.com.au/office....i.e. page not found. I think it might be something to do with the DocumentRoot setting being different to /var/www which we have been working with, but I am unsure what to do.

Basically, all thats in there originally is:

Code:

<VirtualHost *:80>
DocumentRoot "/home/html2/drupal"
#the DocumentRoot is the same in another drupal installation on my server; only the ServerName changes
ServerName myserver.com.au
<Directory "/home/html2/drupal">
allow from all
Options +Indexes
AllowOverride All
#I notice we are using AllowOverride None in /var/www
</Directory>
#Then I have added the new stuff immediately below it:
    Options                            FollowSymlinks
    DirectorySlash                     On

    <Directory /var/www/>
      AllowOverride                  None
      Options                        FollowSymlinks
      Order                          Allow,Deny
      Allow                          From All
</Directory>
<Directory /var/www/office/>
   #
        RewriteEngine                  On
        RewriteBase                    /office
        RewriteRule       .....
etc

I tried enclosing some of it in <Proxy *> tags, aliasing...what further info is useful here?
Thanks!
Yours truly,
Compfused

Nominal Animal · 11-20-2011, 03:51 PM

You need to create the /office directory under the DocumentRoot, i.e. /home/html2/drupal/office in your case, and point the office Directory to that one. The directory must exist and be accessible, but it can be empty.

I adapted my suggested configuration to your configuration above. The publicly-visible host name is in red, the hidden office host in gold, and the subtree path in blue, to make it easier to modify for you. If a host uses some other port than 80 (or 443 if using https scheme), it must be included in the colored host name part. In regular expressions, . means "any character", so it has to be written as \. so it will only match an actual dot.

I have not tested this complete configuration, so there may be typos lurking in there; please carefully test and adapt it to your needs. If you find any bugs in it, let me know.

Code:

<VirtualHost *:80>

    DocumentRoot        /home/html2/drupal/
    ServerName          myserver.com.au

    DirectorySlash      On
    ProxyVia            Off
    ProxyRequests       Off

    # Make sure RewriteEngine can be enabled.
    Options             FollowSymlinks

    # Drupal tree, /.
    <Directory /home/html2/drupal/>
        AllowOverride   All
        Options         FollowSymlinks Indexes
        Order           Allow,Deny
        Allow           From all
    </Directory>

    # Reverse-proxied tree. This directory must exist and be accessible, but it can be empty.
    Alias /office /var/www/office
    <Directory /var/www/office/>
        AllowOverride   None
        Options         FollowSymlinks
        Order           Allow,Deny
        Allow           From all
        RewriteEngine   On

        # /office subtree is obtained from http://www.my2ndserver.com.au:103/,
        RewriteBase     /office
        RewriteRule     ^(.*)$   http://www.my2ndserver.com.au:103/$1 [QSA,P]

        # and Apache should fix headers accordingly.
        ProxyPassReverse               /office   http://www.my2ndserver.com.au:103/
        ProxyPassReverseCookieDomain   www.my2ndserver.com.au   myserver.com.au
        ProxyPassReverseCookiePath     /   /office/
 
        # mod_substitute is used to fix the links in the content.
        # Limit link fixing to content that we know may contain links.
        AddOutputFilterByType          SUBSTITUTE   text/html text/css text/plain

        # Fix absolute paths, from / to /office.
        Substitute      "s|(=[\"']?)/+|$1/office/|q"

        # Fix full absolute URLs that contain a path part (at least /).
        Substitute      "s|(=[\"']?)http://www\.my2ndserver\.com\.au:103/+|$1/office/|q"

        # Fix URLs that do not contain a path part; i.e. they end after the port number.
        Substitute      "s|(=[\"']?)http://www\.my2ndserver\.com\.au:103([^0-9])|$1/office/$2|q"

        # CSS files may also contain URLs, as "url(...)".
        Substitute      "s|(url\([\"']?)http://www\.my2ndserver\.com\.au:103/+|$1/office/|q"
        Substitute      "s|(url\([\"']?)/+|$1/office/|q"

        # Location: header is used for redirection; they need editing too.
        # Fix full URLs that have no path part (end with the port number):
        Header edit Location:  ^http://www\.my2ndserver\.com\.au:103$        http://myhost.com.au/office/

        # Fix full URLs that have a path part:
        Header edit Location:  ^http://www\.my2ndserver\.com\.au:103/+(.*)$  http://myhost.com.au/office/$1

        # Change path-only URLs to a full URL. This makes further rules simpler.
        Header edit Location:  ^/+office$         http://myhost.com.au/office/
        Header edit Location:  ^/+office/+(.*)$   http://myhost.com.au/office/$1

        # All paths left need to be moved to under /office.
        Header edit Location:  ^/+(.*)$           http://myhost.com.au/office/$1
    </Directory>
</VirtualHost>

Does this work for you?

compused · 11-21-2011, 06:46 AM

Thanks again Animal
Thanks for tidying that all up...I replicated the code above, created the ./drupal/office directory (permissions 755) but I get a 403 error msg:

Quote:

Forbidden
You don't have permission to access /office/ on this server.

I tried a range of owners for /office, chmod'd to 777 and also moved 'office' into other subdirectories without success. There are no clues in the apace logs

Also, using the following now draws a 404 blank:
wget --http-user=myusername --http-password=whatever -O - http://www.myserver.com.au/office/src/login.php

I think it must be a Drupal thing?

So close....

Compfused

Update: putting a .htaccess file in /office with the following content:
DirectoryIndex index.php index.html index.htm
Options +Indexes
produces a web page displaying this content:
Index of /office

Nominal Animal · 11-21-2011, 11:57 AM

Quote:

Originally Posted by compused

Thanks again Animal
Thanks for tidying that all up...I replicated the code above, created the ./drupal/office directory (permissions 755) but I get a 403 error msg:

Permissions 755 should be ok. I think I forgot 'Order' and 'Allow' directives from the directory. You might also need the Options FollowSymlinks in the VirtualHost body to allow RewriteEngine to work. I changed those in the post. Could you retest, please?

compused · 11-21-2011, 02:58 PM

Hi Animal
The Order, Deny options were in place already. You had not forgotten them. I have attached the .htaccess in the drupal root directory. Maybe there are some clues there

Or would it work to modify and put all the code above, into the .htaccess in /office?

Compfused.

Nominal Animal · 11-21-2011, 05:54 PM

Quote:

Originally Posted by compused

The Order, Deny options were in place already. You had not forgotten them. I have attached the .htaccess in the drupal root directory. Maybe there are some clues there

You, sir, are spot on. Seeing the settings in that .htaccess file clarified it for me too.

The issue is that since /office is now a subdirectory of Drupal, the Drupal security settings affect it. We do not want that; we need a clear office directory for the reverse proxying stuff.

The solution is to use an Alias /office /path/to/somewhere/else/office, with corresponding <Directory /path/to/somewhere/else/office/> . This moves the /office subtree out from under Drupal, so the security settings are no longer inherited. You can pick any directory (drwxr-xr-x = 0755 mode, owner preferably root to keep it empty), as long as it exists and is empty, and is not under Drupal or a directory with similar settings.

I edited my last config again, using /var/www/office -- but just because /var/www already exists; it was the first choice that sprang into my mind. Feel free to use something else. Could you please re-test once again? I'm sorry for the seesawing; it's not intentional.. but this is, more or less, the way these complex configurations must be developed and adjusted and adapted to fit one's needs.

compused · 11-22-2011, 07:52 AM

...sadly, its not working Animal. Thanks again for the detail and explanation above.

I have attached the relevant part of my httpd.conf

There is some discussion in the Drupal forums about accessing non-drupal subfolders...I seem to have done whats advised but its still not working, eg
I tried adding a .htaccess so I have /var/www/office/.htaccess with contents:
DirectoryIndex index.php index.html index.htm
Options +Indexes

I also meant to say that apache wont start unless I have:
ProxyPassReverse http://www.my2ndserver.com.au:103
NOT
ProxyPassReverse /office http://www.my2ndserver.com.au:103

Compused

Nominal Animal · 11-22-2011, 11:14 PM

I sat down for another testing session, and found out a few details.

ProxyPassReverse really does not like to be set in a Directory directive.
The Location: headers it produces in that case use the filesystem path.
The Alias directive (and preferably also the Directory directive) for the reverse-proxied section must be before the Drupal stuff, for the Drupal stuff to not override them.
Accept-Encoding: request header must be unset, before reverse proxying. This makes sure the response is uncompressed. If the response were to be e.g. gzip-compressed, the content substitutions would not work at all.

After a bit of tinkering, here is a configuration that works for me. Just so that it works for anyone wishing to test it, I again switched the hostnames and paths back to being directly testable:

Green indicates the hidden server host name and port. Port 80 can be omitted for HTTP, and 443 for HTTPS.
Gold indicates the user-visible host name and port. Port 80 can be omitted for HTTP, and 443 for HTTPS.
Red indicates the user-visible URL prefix used for reverse-proxying the hidden server.
Magenta indicates the directory (filesystem path) used for the reverse-proxying. This must exist and be world-accessible, but it should be empty. It is just an anchor for defining the configuration, nothing more.
It should be outside the normally published web tree. In particular, it must not be under the Drupal tree, if using Drupal.
Blue indicates the document root (filesystem path) for this virtual host.

For Compused, the replacements should be

my2ndserver.com.au:103
myserver.com.au
/office
/var/www-office
/home/html2/drupal

Note that certain values have a slash suffix, which is not part of the string to be replaced. Some of the values need to end with a slash, some need to omit a trailing slash. See the Apache documentation to decide for yourself on a directive-by-directive basis. Indentation is obviously not important; I just like to try to make it easier to take it all in at a glance.

Code:

<VirtualHost 127.0.0.1:80>

    DocumentRoot        /var/www/
    ServerName          127.0.0.1
    DirectorySlash      On
    ProxyVia            Off
    ProxyRequests       Off
    UseCanonicalName    On

    # Make sure RewriteEngine can be enabled.
    Options             FollowSymlinks
    RewriteEngine       On

    # Make sure the reverse-proxied tree is Aliased first.
    Alias               /office         /var/office
    ProxyPassReverse    /office/        http://www.w3.org/

    # Disable access to the tree other than explicitly allowed parts.
    # (This is usually included already in the root configuration.)
    <Directory />
        AllowOverride   None
        Options         FollowSymlinks
        Order           Allow,Deny
        Deny            From all
    </Directory>

    # Reverse proxied tree.
    <Directory /var/office/>
        AllowOverride  None
        Options        FollowSymlinks
        Order          Allow,Deny
        Allow          From all

        # Use mod_rewrite to reverse-proxy these requests.
        RewriteEngine  On
        RewriteBase    /office
        RewriteRule    ^(.*)$           http://www.w3.org/$1 [QSA,P]

        # Do not allow special encodings; they would disable the content substitution rules.
        RequestHeader  unset Accept-Encoding:

        # Reverse proxying rules for cookies.
        ProxyPassReverseCookieDomain    http://www.w3.org   127.0.0.1
        ProxyPassReverseCookiePath      /                   /office/

        # Content substitution is limited to types that contain links.
        AddOutputFilterByType           SUBSTITUTE          text/html text/css text/plain application/xhtml+xml application/xml

        # Content substitution rules for HTML, XHTML and XML.
        Substitute      s|(=["']?)/+|$1/office/|q
        Substitute      s|(=["']?)http://www\.w3\.org/+|$1/office/|q
        Substitute      s|(=["']?)http://www\.w3\.org([^0-9])|$1/office/$2|q

        # Content substitution rules for CSS.
        Substitute      s|(url\(["']?)/+|$1/office/|q
        Substitute      s|(url\(["']?)http://www\.w3\.org/+|$1/office/|q
        Substitute      s|(url\(["']?)http://www\.w3\.org([^0-9])|/office/$2|q

        # These fix links that are split onto a separate line. Rare, but seen at w3.org.      
        Substitute      s|^"http://www\.w3\.org/+([^"]*)"|"/office/$1"|q
        Substitute      s|^'http://www\.w3\.org/+([^']*)'|'/office/$1'|q

        # Edit Location: header to point to the reverse proxied tree.
        Header edit     Location:       ^/+office/+(.*)$    http://127.0.0.1/office/$1
        Header edit     Location:       ^/+(.*)$            http://127.0.0.1/office/$1
    </Directory>

    # Main web tree.
    <Directory /var/www/>
        AllowOverride   All
        Options         FollowSymlinks Indexes
        Order           Allow,Deny
        Allow           From all
    </Directory>
</VirtualHost>

Could you please test this one?

compused · 11-23-2011, 06:59 AM

Hi Animal
Thanks for making that so easy to follow.

Unfortunately its not working

There are various references to accessing a non-drupal folder ... eg:
http://drupal.org/node/47092

Maybe a .htaccess file is needed?
Compfused

Nominal Animal · 11-23-2011, 08:23 AM

Could you please make sure the office directory is not under Drupal? In fact, that it is outside the web tree entirely? In your case, not under /home/html2/drupal, and not under /var/www?

I modified my suggestion to you above, to /var/www-office.

The location of the directory is important, so that Apache does not walk through the Drupal directory containing the .htaccess files. When the office directory is well outside any of those, the reverse proxy works for me, no matter what I put into the document root (or Drupal) .htaccess or Directory element.

compused · 11-24-2011, 07:30 AM

Have some very good news, Animal. ITS WORKING PERFECTLY!

I think the problem might have been another <VirtualHost *:80> entry *above* the entry we were working on. This other entry pointed to another Drupal site, viz:

Code:

<VirtualHost *:80>
DocumentRoot /home/html2/drupal
ServerName AnotherDrupalWebsite.com.au
#
<Directory "/home/html2/drupal">
allow from all
Options +Indexes
AllowOverride All
</Directory>
</VirtualHost>

So based on your methodology above, this <VirtualHost> entry was moved *down* to below the <VirtualHost> entry for MyServer.com.au

I then found all was now well with Internet Explorer, but not for Firefox. Suspecting a Firefox cache problem, I hit Ctrl and F5...all was then well with Firefox! I've changed an entry via about:config FF to ignore cache, for the moment

Well, thanks again Animal...May I call you 'Nominal'?...you have been an absolutely fantastic help with this problem! There is no way I could have solved it myself and no where else on the net with a ready solution! I hope your ability to persist with a problem serves you well in the future! Bye for now!