Why My Website Still Appear in Google While Already Disallow In Robots.txt File
robots.txt is a valuable record which sits in your site's root and controls how web crawlers list your pages. A standout amongst the most valuable statements is "Disallow" — it stops web indexes by Google search engine to private or insignificant segments of your site or web pages, e.g.
Disallow: /junk/
Disallow: /temp/
Disallow: /category1/myprivatepage.html
You can even piece web crawlers ordering each page on your space:
User-agent: *Disallow: /
User-agent: *Disallow: /
I don't know why anybody would do this, but rather somebody, some place won't need their site to show up in web search tool comes about.
Be that as it may, blocked pages can even now show up in Google. Before you venture on your soapbox to rage about Google's infringement of robots.txt and the organization's injurious control of the web, take a short time to see how and why it happens.
Expect you have a page at http://www.mywebsite.com/privatepage.html containing private data about your organization's new Foozle venture. You might need to impart that page to accomplices, yet don't need the data to be open information at this time. Consequently, you hinder the page utilizing an affirmation in http://www.mywebsite.com/robots.txt:
User-agent: *Disallow: /privatepage.html
User-agent: *Disallow: /privatepage.html
After fourteen days, you're looking for "Foozle" in Google and the accompanying section shows up:
mywebsite.com/privatepage.html
How could this happen? The main thing to note is that Google lives with your robots.txt directions — it doesn't record the private page's content. In any case, the URL is still shown in light of the fact that Google found a connection somewhere else, e.g.
Google in this way relates "Foozle" with your private page. Your URL may show up at a high position in the indexed lists on the grounds that Foozle is an once in a while utilized term and your page is the sole wellspring of data.
Moreover, Google can demonstrate a page portrayal below the URL. Once more, this isn't an infringement of robots.txt rules — it shows up on the grounds that Google found a passage for your private page in a recognize resources, for example, any Open Directory Project. The depiction originates from that site as opposed to your page content.
Can Website Or Webpages Be Blocked From Google?
There are a few arrangements that will stop your private pages showing up in Google indexed lists.1. Set a “noIndex” meta tag
Google will never demonstrate your private page or take after its connections on the off chance that you add the accompanying code to your HTML <head>:<meta name="robots" content="noindex, nofollow" />
2. Use Google’s URL removal tool
Google offer a Free URL removal tool inside their Webmaster Tools.3. Add authentication
Apache, IIS, and most other web servers offer fundamental validation offices. The guest must enter a client ID and private word before the page can be seen. This may not stop Google demonstrating the page URL in comes about, however it will stop unauthorized visitors perusing the website or web page contents.4. Review your publishing policies
On the off chance that you have top-private content, maybe you shouldn't distribute those reports on an publicly available to the internet!
If you like my blog and if it is helpful to your business or blog's then don’t forgot to share on your social media so that others get help from your sharing.
If you still stuck in the problems or not a technical knowledge then you can contact me any time for the any website and SEO Help and Support.
My direct Contact: rakshithirapara@gmail.com, Skype: rakshithirapara, Mobile: +91 9898141247
Thanks for Reading,
Have a Good Day :)
No comments:
Post a Comment