Welcome to the most active Linux Forum on the web.
Go Back > Forums > Linux Forums > Linux - Server
User Name
Linux - Server This forum is for the discussion of Linux Software used in a server related context.


  Search this Thread
Old 12-16-2015, 06:37 AM   #1
Registered: Jun 2015
Posts: 88

Rep: Reputation: Disabled
unable to crawl into angularJS website for sitemap generation

i developed a website based on angularjs which will redirect to!/home resulting in home page of my website.

when i generate sitemap for i get only home page. but there are few more pages. google is crawling into website but it is stopping "#!".

i need to write a rewrite rule in apache so that i can redirect to and google will be able to crawl and generate sitemap.

i've edited my apache conf file as follows but no luck

RewriteCond %{REQUEST_URI} ^/$
RewriteCond %{QUERY_STRING} ^_escaped_fragment_=/?(.*)$
RewriteRule ^(.*)$ /snapshots/%1? [NC,L]

how can i achieve this?
Old 12-16-2015, 08:42 AM   #2
Registered: May 2005
Distribution: LFS, CentOS, Slack, Ubuntu
Posts: 112

Rep: Reputation: 8
Check this out

Looks like "!#" will prevent the search engine from crawling.

Don't use HashBangs #!

Hashbang urls were an ugly stopgap requiring the developer to provide a pre-rendered version of the site at a special location. They still work, but you don't need to use them.

Hashbang URLs look like this:!path/to/resource

This would be paired with a metatag like this:

<meta name="fragment" content="!">

Google will not index them in this form, but will instead pull a static version of the site from the _escaped_fragments_ URL and index that.

Pushstate URLs look like any ordinary URL:

The difference is that Angular handles them for you by intercepting the change to document.location dealing with it in JavaScript.

If you want to use PushState URLs (and you probably do) take out all the old hash style URLs and metatags and simply enable HTML5 mode in your config block.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Crawl local website lilleskut Linux - Software 9 06-24-2012 07:46 PM
help with google sitemap index gaga General 2 05-26-2011 07:41 PM
LXer: Three sitemap generators for WordPress LXer Syndicated Linux News 0 05-29-2008 04:00 AM
Wrong link in the SiteMap? exit3219 LQ Suggestions & Feedback 1 01-06-2007 04:41 PM
LQ sitemap feedback titanium_geek LQ Suggestions & Feedback 0 09-01-2006 09:00 AM > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 06:27 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration