Finding Domains that Don't Want to be Found

I am curious whether anyone knows or is working on an alogirthm or implemenation to search the internet for domains. I am posting this now before I have given it very much thought to find out if anything is being done in this area (maybe something for r&d if not). Regular internet searching and even "deep" digging reveal very little. If capable, please advise.

93

Ummm,...do you mean domain names? Expired domain names? Available domain names? or Pre-existing domain names?

Searching is never a problem. The problem is going to be the WLS via NetSol.

I wasn't thinking of NetSol. More along the lines of nslookup() or dig() against the root name servers. However, its quite possible that hidden domains may reside on a completely separate network outside of the "www" universe we're all(most) trapped in.

Smirkley, I am speaking of domains, ip address ranges, etc., used by sources wishing anonimity. I am not proposing infilitrating anything US gov't related. I am thinking more off the radar than that.

OK. I understand.

All IP #'s exist whether assigned or not. Phantom IP's are commonly used by spammer's.

I would not reccommend delving into the neighborhood you suggest, as you may attract the attention of organizations that you may wish you have not attracted.

one thing you may try looking for is robot.txt files... robot.txt tells search engines to ignore certain sites, such as the robot.txt from the gov pages:

www.whitehouse.gov...
(notice the gov uses robots.txt instead of robot.txt... it does the same thing, and sometimes comes up with a search query of robot.txt)

Here's a quickie on robot.txt from Dreamweaver:

The robot.txt File at Dreamweaver FAQ.com

These should help you get to some of the pages you may be looking for, but not all. Another thing is that not all IP's you find will be web pages, but rather FTP servers, or just BBS txt type pages... I recommend using CameraShy when you are doing your searches. First, it doesn't leave a trace of where you've been, and secondly it finds the info sometimes put into GIF images. The only problem I have with CameraShy is that it doesn't allow pop-ups... basically every page will need to be viewed in the original browser window, but it's well worth it (and is a single exe file... no extra components are installed!).

There are multitudes of h4xT0yZ all over the internet that could help you accomplish whatever you are trying to do. Do searches for hacking toyz on www.altavista.com...

Also, definitly check out CDC's site (Cult of the Dead Cow). This is a good hacking group to check out, even though you aren't gonna be 'hacking' anything, per se. Basically it is probably the best place to start finding what you need in tips and open-source toolz.

Hope that helps out a bit! Happy hunting!

(be vewy, vewy quiet... I'm hunting web pages...)

There's also something called "The invisible web" that's mostly database pages. Here's an article on that:
searchenginewatch.com...

There exist ways to reach and search them... searchenginewatch.com is always a good source for info.

thank you Earthscum and Byrd, most helpful.

62.227.245.245 193.159.2.151 149.225.49.243 193.159.3.129 217.4.80.88 193.159.1.115 62.158.7.37 217.2.232.165 213.21.43.147 212.98.131.68 212.184.153.40 213.7.48.206 193.159.117.121 193.159.77.180 193.159.3.137 193.159.3.137 212.64.80.213 172.177.148.123 195.186.192.204 128.250.1.21 62.96.171.5 193.159.0.62 217.4.81.252 193.159.117.41 202.92.95.199 62.158.226.204 62.155.248.182 62.158.226.204 217.1.72.199 62.158.225.214 62.158.228.246 62.158.226.208 216.220.31.216 217.50.26.105 193.159.3.130 62.54.157.222 62.224.87.11 217.0.159.118 62.158.227.99 64.41.46.104 62.155.177.72 195.57.121.217 62.155.177.72 217.0.1.241 193.159.117.30 193.159.24.96 193.159.2.72 212.4.75.201 193.159.2.72 62.180.220.43 195.39.147.77 212.144.253.126 145.253.141.230 213.7.11.239 213.6.101.188 193.159.39.163 62.158.10.37 62.158.22.213 213.186.145.92 193.159.117.133 212.185.252.73 62.224.244.151 62.224.8.3.11 194.25.134.91 193.158.183.189 62.224.81.197 62.224.20.171 62.155.233.129 193.159.89.215 213.47.142.223 193.159.3.97 213.47.142.223 62.224.49.80 217.12.7.41 193.159.3.144 212.234.31.129 62.158.225.184 62.224.94.231 217.2.88.161 193.159.2.6 62.180.24.252 62.180.24.252 62.180.26.40 217.2.88.161 193.159.2.6 62.157.0.215 213.21.32.109 217.80.198.117 193.159.0.141 172.163.40.223 193.159.0.141 212.10.8.236 193.159.0.141 213.6.110.194 193.159.116.153 62.54.184.16 62.158.226.112 24.4.254.203 62.158.224.194 217.4.97.82 193.159.3.176 217.4.81.35 193.159.3.176 193.159.139.242 193.159.2.126 195.102.200.56 193.159.1.68 ---- 62.155.147.211 ---- 193.159.116.17 62.158.45.208 62.158.227.200 149.225.71.149 193.159.3.146 193.158.163.173 62.54.135.41 62.54.9.57 62.156.8.11 62.224.81.61 62.224.5.136 62.224.81.95 193.159.2.6 216.104.228.104 216.104.228.102 62.158.224.62

I am not responsible for any misuse legal or otherwise of the above IP Adresses.