PHP Classes

Discussion forum of package Robots_txt

Recommend this page to a friend!
      Robots_txt  >  All threads (1)  >  Post a new forum message  >  (Un) Subscribe forum alerts  
RSS 1.0 feed RSS 2.0 feed

Class:Robots_txt
Description:Test if a URL may be crawled looking at robots.txt

Subject Last update Replies
Says not allowed also if it is: http://www.
Package rating comment		 2011-12-11 12:26 0