is there any web crawler can get access to users region for download?

-4

Actually I am using Httrack as a web crawler, can it use my credentials to access members area and download the zip files because they are restricted from public access.

Thank you in advance.

Update:

After all the problem was with my IPV6
it must be disabled; with (Windows 8 x64), I created a registry Dword32 "DisabledComponents" with the value of 0xffffffff
restart the system
httrack works with capture url, because with ipv6 it gives me a proxy adress like xxxx::xxxx:xxxx:xxxx%19 and this is not useful with proxy settings in windows

Thank you for the answers

web-crawler
asked on Server Fault Nov 17, 2013 by M. Abdelhafid • edited Nov 17, 2013 by M. Abdelhafid

1 Answer

0

It's a little difficult to fully understand what you are actually asking due to the way you have worded your question. I'm guessing you are trying to access a restricted part of a website and download files from that section?

If so, yes it can be done but its not easy to achieve and it will fail if the website provides a logout link which undoubtedly it will. As the crawler follows the logout link you will no longer have a valid session/cookie with the website and the crawl will come to a halt.

You can view everything you need to know here: http://forum.httrack.com/readmsg/288/284/index.html

answered on Server Fault Nov 17, 2013 by twigg

User contributions licensed under CC BY-SA 3.0