Ok now I found the info in my computer I will explain how I did this:
1. cpy+pst:
"robots.txt" "disallow:" filetype:txt into the google search
You will get some names organizations you have heard of like BBC etc. If you click for eg the BBC link link you will get weird lists of text pages eg:
Disallow: /cgi-bin
Now the trick is while your looking at these lists they are path ways that can be copy and pasted then added to the url in your browser to kinda jump
to the location you chose. Sorry this is difficult for me to explain but its really simple.....
2 choose something that takes your fancy then copy it over the end of what is already in your browser
For example while on the bbc bit your browser will have
"http://www.bbc.co.uk/robots.txt" as the page your on.
So lets take the first one on the list which is "Disallow: /cgi-bin" , now copy it except for the "Disallow: " part.
So we have copied "/cgi-bin" and we paste it on to the example above so it looks like this:
www.bbc.co.uk..."
And when you press enter you will jump to a location which informs you, you dont have access.
Its hit and miss but thats how I found it in Google(Above example was by clicking BBC, I found my thing by clicking Google at step one) I havnt looked
at hardly any other site as I was very happy with Govt search thing I found.
Dam im no good at explaining things with text
[edit on 8-9-2009 by draildatrain]