1. SpotPlex Closed Down
    SpotPlex, a Digg-like site based on page views instead of voting, has shutdown their site and their hyperbot.
  2. Blocking Bad Bots and Scrapers with .htaccess
    Titan ... toCrawl/UrlDispatcher. True_Robot ... turingos. TurnitinBot. Twisted PageGetter. UCmore. UdmSearch. UMBC. UniversalFeedParser. URL..
  3. Twisted PageGetter is… SpotPlex?
    So, if you currently have Twisted PageGetter blocked, and you’re signed up to SpotPlex, you may want to unblock the eager little bot.



    The Charming Alyssa Milano
    Ingrid Martz - Model and actress
    WWE Diva Maria Kanellis

    (Search for more on: ‘Twisted PageGetter’)

    Twisted PageGetter - block or not

    I found a Bot identifying itself as Twisted PageGetter in the access log of my site.
    After I googled for it I found on Matt’s Blog that he suggests to block this Bot with an htaccess entry.
    This is working quite well, I tried it.
    But: I am using Spotplex as a counter and wondered why they were [...]

    read all » Twisted PageGetter - block or not