Massive UFO disclosure in USA : A challenge for ATS

Section 1 : Introduction

I have a challenge for members of ATS, outlined below (particularly in Section 7).


A massive amount of UFO documentation has been made available online.


No, I’m not talking about the relatively small volume of FBI material that got so much press and blog coverage in 2011…

No, I’m not talking about the documents made available online by Canada, Australia, Britain, France or other European countries in the last few years…


I am talking about the online publication of over 129,000 pages of UFO reports, US Air Force documentation, including sighting reports, internal memoranda, statistical analyses, project reports, press releases and commentaries. That material includes a huge amount of documentation on some projects which are (in outline at least) fairly well known (including Project Sign, Project Grudge, Project Bluebook) in addition to material relating to various less well-known projects (including Project Twinkle and Project Stork).


You may be surprised to learn that this mass of documentation was made available FREE online in 2007/2008 The relevant website (Footnote.com) has hardly been mentioned on ATS and other UFO websites since it that mass of UFO documentation became available. (Two discussion threads on ATS went precisely nowhere when these documents were raised. I hope this thread does not suffer the same fate…).


This is possibly because examining that mass of material on the relevant website is, well, a pain in the backside and very time consuming (hence the challenge in Section 7) below…).


The relevant UFO documentation can be found on the Footnote.com website at the link below:

www.footnote.com...


Since that mass of UFO documentation became available on the Footnote.com website, that resource has generally been ignored by members of ATS (with a few exceptions). Indeed, quite a few people have made comments on ATS, on other UFO websites and in some recently published UFO books stating or implying that the USA has not disclosed any UFO documents or that other countries have disclosed more documents than the USA. One recent article in a national newspaper in England stated that "The US Air Force had a secret unit that probed UFO sightings for 22 years", as if the relevant project (Project Bluebook) was not well known to the public at the time. I’ve become sufficiently irritated to write this thread (see Section 11 : Postscript …).


Most of this US Air Force material was available off-line for decades – just like the FBI material which was mistakenly reported in 2011 to have been “just released”. However, the number of researchers that looked at this mass of US Air Force material when it was off-line was relatively limited. Rather than look at the primary material, the authors of many UFO books and websites have been content to repeat summaries of that material written by a relatively small number of researchers. This has involved the repetition of errors and omissions made by that relatively small number of researchers. Also, a few more recent releases (including unredacted versions of some of the documents) mean that more investigation can be done of relevant issues.


Now that this material is available online (for free), there really isn’t much excuse for researchers continuing to fail to go back to the source material and see what has been missed.


Well, there is one excuse for failing to go back to this source material : the sheer mass of material – combined with a fairly slow and unhelpful means of accessing it – means that it is a quite painful to look at these documents on the Footnote.com website.


Basically, it is too much like hard work to examine this material on the Footnote.com website. After all, virtually all UFO researchers are indulging in ufological research as a hobby in (often limited) spare time. Work and family do not leave a lot of time or energy for the more boring detailed work that could be done. Brief comments on the latest Youtube video are much, much easier and generally as much fun…


So, can the mass of UFO material on Footnote.com be efficiently downloaded for research off-line? Once that is done, can some aids be developed for finding relevant material?

Is ATS (and the wider UFO community) up to this challenge?


Can the relevant technical issues be addressed here?

Over the years, I’ve been impressed by the technical abilities and computer savvy of members of ATS. This is an issue where ATS could get ahead of the wider UFO community.

We’ll see…



This post is split into the following sections:

Section 1 : This introduction

Section 2 : The USA has now made available online more UFO documents than other countries

Section 3 : FARmore UFO documentation is available online from the US Air Force than any other agency in the USA (including the FBI, CIA and NSA)

Section 4 : The witness reports on Footnote.com are not just “letters from farmers”

Section 5: The UFO documentation on Footnote.com is not limited to witness reports

Section 6 : There are significant gaps in Air Force FOIA documentation available on its website and via other official websites

Section 7 : The Challenge : Efficient Batch Downloading?

Section 8 : Guides to the documentation?

Section 9 : Conclusion

Section 10 : References

Section 11 : Postscript


The lack of discussion of this massive collection of UFO documentation from the USA does rather raise the question : Why do so many people call for documents relating to UFOs to be released but then not bother looking at them when they are released? Are many people just waiting for a document signed by a president which says : "UFO reports are caused by people seeing alien spaceships visiting Earth" and they aren't very interested in any other documents?

Section 2 : The USA has now made available online more UFO documents than other countries

In the last few years, quite a few countries have been making available online their archives of UFO reports. Some of those archives have included fairly substantial volumes of material (e.g. in France and Britain).

Those releases are outside the scope of this thread, but I think it is worth referring to a thread started by Internos (“The state-of-art in UFO disclosure worldwide”) which highlights a paper by V-J Ballester Olmos which sought to present a concise chronological picture of the international history of releases of UFO archives. Similarly, another (less detailed) thread has also sought to cover the same ground – see List of Countries who have Disclosed/are Disclosing UFO Files.


In many of the discussions of the releases during the last few years, various members of ATS have commented that it is time that the USA made a similar release of its archives of UFO reports. For example, in one of the relevant discussions (www.abovetopsecret.com...), one experienced researcher (“Free_Spirit”) argued in posts in that thread that the title was misleading when it referred to Denmark following the USA and that the OP was “lying”, since “the US government has never opened their UFO files” Similarly, in the thread ”Sweden to release UFO files could USA be next?”, franspeakfree commented “We are now running out of countries that are releasing their UFO documents,is the US going to be next”.


I have found those comments (and the lack of challenge to such comments) to be a bit irritating given that the volume of material made available in the USA (particularly the Project Bluebook material on the Footnote.com website) exceeds the volume of material made available in other countries – yet the material in the USA has largely been ignored.


For ease of reference, links to the items (including some relevant discussion threads) about the material released by countries other than the USA are briefly included below.

I’ll first list the English language countries that have been reported to have released UFO documents in the last few years, i.e. Australia, Britain, Canada, and New Zealand.



English language : Australia

Australia has made a limited volume of UFO documents available on the website of the National Archives of Australia (enter searches for “flying saucers” and “ufos”):
recordsearch.naa.gov.au...

One website I found useful in relation to Australian UFO documents is at the link below:
disclosureaustralia.freewebpages.org...

Australian documents are briefly discussed in the ATS Thread entitled www.abovetopsecret.com...




English language : Britain

Releases in Britain in the last few years have gained a large amount of international publicity, so are relatively well known. Relevant threads on ATS include the following:
Now Britain is to release UFO files
Britain to release 160 UFO files
New release of UFO files in UK - March 2009
More UFO files released - UK February 2010 (including VIDEO overview showing images)
Britain's latest batch of UFO files - March 2011


Britain’s UFO files include (in addition to numerous UFO sighting reports) several reports analysing the phenomenon, including the Project Condign report. That report is discussed in my thread Project Condign - A valuable official report.
[atsimg]http://files.abovetopsecret.com/images/member/8d8124faa528.jpg[/atsimg]

Paragraph 1 of the Executive Summary of the Condign Report includes the following:

… That UAP exist is indisputable… and [they] clearly can exhibit aerodynamic characteristics well beyond those of any known aircraft or missile – either manned or unmanned.

English language : Canada

Canadian UFO files can be found on the government website at the link below

www.collectionscanada.gc.ca...


The full list of files can be obtained by leaving the search form at that link blank and then performing a search.
Relevant threads on ATS include:
Canadian UFO files
More Canadian UFO govt files
Official canadian investigation concludes UFOs"probably extraterrestrial"




English language : New Zealand

For discussion of the release of UFO documents in New Zealand, see the ATS thread at entitled ”New Zealand to disclose UFO files - within a week”

Other countries – Foreign languages (in alphabetical order)


For threads on ATS relating to the release of UFO documents in Brazil, see:
Brazil UFO Disclosure Reports Now Available To View
Breaking News: Brazil Declassifies New Set UFO Secrets Documents


For threads on ATS relating to the release of UFO documents in Denmark, see:
www.abovetopsecret.com...
Strange Photograph Released In Danish Department Of Defense Report!


For threads on ATS relating to the release of UFO documents in Ecuador, see:
Ecuador has released all of its UFO files
2 UFOs Chased by Military Jets Ecuador, English translation


For threads on ATS relating to the release of UFO documents in France see France releases UFO files

Those French files they can be found online at the link below:
www.cnes-geipan.fr...

See in particular the section entitled "Recherche" at:
www.cnes-geipan.fr...



For threads on ATS relating to the (inaccurate?) reports of the release of UFO documents in Russia see Russia Discloses/Releases UFO Files and Russian Navy UFO Records Declassified


For threads on ATS relating to the release of UFO documents in Sweden, see: ”Sweden to release UFO files could USA be next?”


For threads on ATS relating to the release of UFO documents in Uruguay, see Uruguay ~ Declassified UFO Files

Section 3 : FARmore UFO documentation is available online from the US Air Force than any other agency in the USA (including the FBI, CIA and NSA)

Documentation produced by the FBI, CIA and NSA appears to be regarded as more exciting and sexier than documentation produced by the US Air Force. A few documents from those agencies have received much more attention in the last few years than the Air Force documentation.


In fact, as can be seen from looking at the UFO documentation released by the FBI, CIA and NSA (and other websites containing documentation from agencies in the USA other than the US Air Force), the volume of US Air Force material on the Footnote.com FAR outweighs the volume of material released by other agencies in the USA.

For ease of reference, links to the material released by those other agencies (and links to relevant discussion threads) are briefly included below.


Many non-governmental websites purport to contain UFO documents generated by US governmental agencies, but the contents are often not verified. Most such websites fail to distinguish between those documents which have questionable origins (e.g. the various MJ-12 documents) and those documents obtained as a result of Freedom Of Information Act requests. An honourable exception is the CUFON website, which includes a section on verified documents, with helpful brief comments.

Another under-appreciated non-governmental website is the unofficial Bluebook Archive website, which made quite a few Project Bluebook documents available online. It has been dwarfed by the Footnote.com release, but is still relevant for a few reasons (not least is its inclusion of various unredacted documents which appear with redactions on the Footnote.com website).

Probably the best known non-governmental website containing UFO documentation is, of course, the Black Vault website. It contains a large section on UFO documents from various US government agencies, but is somewhat difficult to navigate and is very incomplete.


Given the volume of UFO documentation online of doubtful provenance, it is highly desirable to focus on official websites.


The websites of quite a few US Government agencies include sections on UFOs. These usually give a good indication of the response that will be received if a blanket request is made in the USA under the Freedom Of Information Act (“FOIA”) to that agency for documents relating to UFOs.


Every year or two, these documents are re-discovered by some newspaper or researcher, leading to reports of documents having been “just released” (e.g. as with the FBI documents in 2011 and 2007).


The FBI’s “Vault” website contains a total of about 1,600 pages of short (often almost illegible) memoranda and letters relating to UFOs. My personal favourite sequence of documents within the FBI files relates to the memo dated 24 September 1947 from FBI Assistant Director D M Ladd to FBI Director J Edgar Hoover summarises, and attaches, a memo dated 23 September 1947 from Colonel R H Smith (Assistant Chief of Staff Intelligence) revealing that the FBI had been asked to assist the Air Force in UFO investigations “to relieve the numbered Air Forces of the task of tracking down all the many instances which turned out to be ash can covers, toilet seats and whatnot”. However, another FBI document has received much more attention over the years, i.e. a memo to the Director of the FBI (Hoover) by Special Agent Guy Hottel (SAC Washington) dated 22 March 1950 (see image koi_t19500322_1 below), which I discussed in my thread Debunked! The FBI alien bodies memo – A case study in the reinvention of the wheel and is on the FBI’s own website at the link below:
vault.fbi.gov...:

[atsimg]http://files.abovetopsecret.com/images/member/8fad24cf2b51.jpg[/atsimg]


The CIA’s webpage includes a search function which can be used to search for UFO document. The resulting list of files is somewhat difficult to navigate. An article entitled “A Die-Hard Issue : CIA's Role in the Study of UFOs, 1947-90” by Gerald K Haines (a National Reconnaissance Office historian) is a useful, if somewhat controversial, guide to the CIA’s UFO documents. (For some balance, see also Bruce Maccabee’s comments on the most controversial part of the Gerald Haines’ article). The most commonly mentioned CIA documents are probably those relating to the Robertson Panel (see Karl12’s thread The Robertson Panel: UFOs and Ridicule, which includes this video of Terry Hansen discussing the Robertson Panel report:



The memo below was prepared following the meeting of the Robertson Panel and reflected its conclusions and recommendations. The most controversial sections recommended a program with the aim of “debunking” UFOs and the monitoring of civilian UFO groups.

www.foia.cia.gov...

A few relevant pages of that memo are embedded below:
[atsimg]http://files.abovetopsecret.com/images/member/02ad3607849a.jpg[/atsimg]
[atsimg]http://files.abovetopsecret.com/images/member/6f8b0340749e.jpg[/atsimg]



The NSA’s website includes a fairly limited number of UFO documents. Several of these are, however, quite interesting. They include the infamous “UFO Hypothesis and Survival Questions” document. As indicated by the threads below, various NSA documents have been discussed repeatedly on ATS: “Disclosure NSA Style”
“NSA releases UFO documents online. We are not alone!”
“NSA releases 29 messages from space?”

Those documents include several relating to the Tehran incident, discussed in the video below from Paul Kimball’s “Best Evidence” documentary:



The NSA documents relating to this incident at the link above include the following:
[atsimg]http://files.abovetopsecret.com/images/member/144d3924be6d.jpg[/atsimg]
[atsimg]http://files.abovetopsecret.com/images/member/cd9917edf802.jpg[/atsimg]
[atsimg]http://files.abovetopsecret.com/images/member/c55b72dc7a18.jpg[/atsimg]



The Department of Defense’s webpage on UFOs includes a concise summary of the USA Government’s position in relation to UFOs and links to a selection of relevant Government documents. These include links to rather large PDF files of the full text of the US Air Force’s “Fact vs. Fiction in the New Mexico Desert” report on Roswell (1995) and its subsequent “Case Closed” report on Roswell (1997).


The Navy’s FOIA website includes a UFO “fact sheet” at:
foia.navy.mil...

Section 4 : The witness reports on Footnote.com are not just “letters from farmers”

In one recent thread about UFO documents in which I referred to documents released by the USA and was told such documents would only comprise letters from farmers.

In fact, the witness reports aren’t just the ones “from farmers”. The reports released include sightings by military personnel, pilots, policemen, astronomers and many others.

While many of the incidents have only a handful of documents (indicating the superficial nature of most of the investigations), a smaller number of incidents have a surprisingly large number of documents. If you have seen one or two documents online in association with some of the classic cases, you may be surprised by the number (and content) of relevant documents online.

For example, the Project Bluebook documents include quite a few relevant photographs and documents relating to the classic Socorro sighting in 1964.

Many of you will know that on 24 April 1964, there was a report of a landed UFO with nearby crew by Deputy Lonnie Zamora sighting near Socorro, New Mexico. This incident came first in a survey in 1965 by Jacques Vallee of the opinion of various UFO groups as to the most significant UFO sightings. J Allen Hynek has said this incident was “one of the soundest, best substantiated reports as far as it goes”. It was also included in a list of twenty UFO cases produced by Bruce Maccabee in 1999 in response to a challenge to produce “the best evidence for an extraterrestrial origin for the UFO phenomenon”. For anyone not familiar with this incident, a relevant video is embedded below:



I have made available a list of references to discussion of this incident in about 180 UFO books in a table can be sorted by author, date and length of discussion by clicking on the relevant word in the top line of the table on an early draft of my UFO website at:

www.isaackoi.com...

Despite being a very well known sighting, the Project Bluebook documentation in relation to that Socorro sighting does not seem to be looked at by many people. For example, in 2009 ufologist Anthony Bragalia posted an article which claims that "the truth is now revealed", alleging that the sighting "was a hoax" : see THE SOCORRO UFO HOAX EXPOSED! (Famous 1964 sighting was a college prank). A follow-up article by Anthony Bragalia claimed that the physical evidence points to a prank, stating that a “single buried sentence” in the Project Bluebook documents which refer to “charred cardboard” found at the scence “speaks volumes” and makes the following comments:

The "charred cardboard" found at the site by AF investigators is an extremely important detail that does not seem to have ever been brought up by "civilian" UFO investigators who support Soccoro as an ET or secret aerocraft event. And of course the reason for this is obvious: such mundane material should not be there if it were ET or if it was an experimental vehicle. Instead, this "find" is indicative of something very terrestrial. This is because "charred cardboard" makes complete sense when considering the event as a student-created hoax:

Pyrotechnics could very well account for the found material. Such cardboard tubes or "casings" are used in shell inserts, bottle rockets and fireworks. When ignited, such spent explosives leave a a distinct charred cardboard appearance upon cooling. Burned cardboard and cardboard powder char are left in their wake.

…

Or perhaps the charred cardboard came from the "craft" itself. One NM Tech Professor speculated that the "craft" seen by Lonnie was a large white balloon. In fact, Lonnie's immediate reaction was to characterize it as a balloon. He even radioed to his partner: "It looks like a balloon." The Professor believes that this balloon may have been "over-fitted" with white coated craftboard (or light cardboard) to create the "landing struts" and other features. Such cardboard or craftboard material may well have ignited and charred at the bottom- potentially leaving such cardboard residue as was observed by AF investigators.

I am not aware of any of the numerous responses to those articles actually pointing to another relevant document in the Project Bluebook files which rather undermines Anthony Bragalia’s argument. That other document a letter from Hynek to Menzel dated 29 April 1965 in which Hynek refers to having personally picked up some cardboard "at the landing site". Hynek’s letter discussed the possibility of a hoax in quite a bit of detail. There are numerous points made in that letter which undermine the recent allegations of a hoax. For example, I think it is important to note that Hynek (i.e. the person that collected the cardboard) noted at the time that it was charred but also had "plainly been weathered quite some time and is hardly the kind that would have been used to fake a model of a spaceship".

www.footnote.com...|8695713

[atsimg]http://files.abovetopsecret.com/images/member/8d83b17ba4f8.jpg[/atsimg]

[atsimg]http://files.abovetopsecret.com/images/member/3816fc9d8d4c.jpg[/atsimg]

[atsimg]http://files.abovetopsecret.com/images/member/ce3c8ae04a9b.jpg[/atsimg]

[atsimg]http://files.abovetopsecret.com/images/member/fd6d995607ab.jpg[/atsimg]



During 2009 I found it surprising that nobody else seems to have pointed to that document on any of the numerous websites which discussed Anthony Bragalia’s articles claiming that physical evidence indicated that the Socorro sighting was a hoax. That omission rather implies that the content of the Project Bluebook documentation is not as well known as I had assumed, even in relation to the classic cases.



The Project Bluebook documents also include various documents relating to other classic cases which could supplement the previous discussions on ATS and elsewhere.


A surprising amount of material can be found in relation to analysis of photographs and film footage of UFOs.


For example, jkrog08’s thread: The 1952 Tremonton, Utah UFO Fleet discusses the Tremonton, Utah incident which occurred on 2 July 1952, involving film taken during a sighting by US Navy Warrant Officer Delbert C Newhouse. The Tremonton film was one of two motion pictures of UFO sightings considered by the Robertson Panel, organized by the CIA, in January 1953. References to 64 books containing discussion of this incident are included in the relevant table of references on an early draft of my website. The relevant footage is shown below:



The memo below from the Project Bluebook files refers to the movie as “possibly the best documentary evidence yet obtained on unidentified flying objects”. This memo is forms a small part of a fairly extensive set of documents relating to this movie:

www.footnote.com...

[atsimg]http://files.abovetopsecret.com/images/member/ff24c79e6297.jpg[/atsimg]



The well known sighting on 7 January 1948, during which Captain Thomas F Mantell Jr (Air National Guard F-51 Mustang fighter pilot) died during an attempted intercept near Godman Air Force Base (near Fort Knox), Kentucky has been discussed in jkrog08’s thread: The Mantell UFO Incident. References to about 179 books containing discussion of this incident are included in the relevant table of references on an early draft of my website. The Project Bluebook file on Footnote.com relating to this incident starts at the link below:
www.footnote.com...|9669879
and includes numerous documents and photos, such as the one below:
[atsimg]http://files.abovetopsecret.com/images/member/345549941bc9.jpg[/atsimg]


I could go on and on, but the point here is simply to highlight the Footnote.com resource (and the challenge in Section 7) below) rather than attempt to outline the entire content of the files.

Section 5: The UFO documentation on Footnote.com is not limited to witness reports

While the bulk of the Project Bluebook documentation on Footnote.com is dominated by material dealing with individual sightings (including sighting reports and case files relating to those sightings), that documentation also includes material evaluating and the UFO phenomenon as a whole, statistical analyses, project reports, press releases and numerous other documents.

The material even extends to a review of a movie about UFOs and the script for one infamous documentary – the Armstrong Circle Theatre show in which Keyhoe was cut off because he departed from that script.
[atsimg]http://files.abovetopsecret.com/images/member/75acfbaa002e.jpg[/atsimg]


A small sample of those other documents are indicated below.
The Project Sign Report from February 1949 begins at the link below:
www.footnote.com...

The introduction is shown below:
[atsimg]http://files.abovetopsecret.com/images/member/63788cc70566.jpg[/atsimg]



The considerably longer Project Grudge Report from August 1949 begins at the link below:
www.footnote.com...|11885542

A couple of pages of the Project Grudge report are shown below:
[atsimg]http://files.abovetopsecret.com/images/member/141c9d30f0d1.jpg[/atsimg]
[atsimg]http://files.abovetopsecret.com/images/member/d12ad6979fad.jpg[/atsimg]



Any mathematicians and statisticians are likely to be particularly challenged by Project Bluebook Special Report No. 14 (dated 5 May 1955), entitled “Analysis of Reports of Unidentified Aerial Objects” and based on studies by the Battelle Memorial Institute. That report is frequently discussed by Stan Friedman (e.g. in the video below).


Project Bluebook Special Report No. 14 begins at the link below:
www.footnote.com...|11884638

A few pages of the Project Bluebook Special Report No. 14 are shown below:
[atsimg]http://files.abovetopsecret.com/images/member/425ac5f1eb5f.jpg[/atsimg]
[atsimg]http://files.abovetopsecret.com/images/member/572fbc4372fa.jpg[/atsimg]
[atsimg]http://files.abovetopsecret.com/images/member/847e4e8ff97b.jpg[/atsimg]



An interesting 57 page booklet entitled “Aids to Identification of Flying Object” (1957) begins at the link below:
www.footnote.com...|11884055

A few pages of the “Aids to Identification of Flying Object” booklet are shown below:
[atsimg]http://files.abovetopsecret.com/images/member/bc33aec9a022.jpg[/atsimg]
[atsimg]http://files.abovetopsecret.com/images/member/1c20114c4fce.jpg[/atsimg]



The Project Bluebook files also include various documents relating to numerous explanations that have been offered for UFO reports, including ball lightening and unusual aircraft. For example, the files include a small file entitled “AVRO Car”. That file begins at the link below:

www.footnote.com...

A couple of pages from the AVRO Car folder are shown below:
[atsimg]http://files.abovetopsecret.com/images/member/6225574db776.jpg[/atsimg]
[atsimg]http://files.abovetopsecret.com/images/member/c4288089bf1b.jpg[/atsimg]



Such documents may not contain any massive revelation, but some of us find them interesting since they contain:
(a) reports of incidents that have not been seen by the public before,
(b) some additional information on known sightings,
(c) an insight into how government works (or doesn't work) when dealing with unusual problems.

Each of these items, particularly Item (c), may have nothing whatsoever to do with aliens...

Section 6 : There are significant gaps in Air Force FOIA documentation available on its website and via other official websites

One of the reasons for wanting to make it easier to review the documentation which has been released online relating to Project Bluebook and other US Air Force projects is to make it easier to identify material which HAS NOT been released online.

A fairly preliminary consideration of the material which is online on the Air Force website, the National Archives website and Footnote.com indicates some of those gaps.


This is not the place to discuss those gaps in any detail (particularly since identifying gaps could be done much more easily once the challenge in Section 7) below is met), but obvious ones include:

(1) Documents released by the Air Force in other FOIA releases;

(2) Documents mentioned within the documents which are available online;

(3) Documents which must exist (or have existed) in relation to the creation of the documents available online;

(4) Documents mentioned by those involved in the relevant projects.



This topic may be worth revisiting in more detail once the challenge in Section 7) below is met.

Section 7 : The Footnote.com Challenge : Efficient Downloading?

Looking at documents on the Footnote.com website can be frustrating experience.

The speed at which documents appear on the screen, and (even more so) can be downloaded to read off-line, varies greatly from document to document and day to day. On one or two occasions, I have given up looking into issues due to pages repeatedly appearing very slowly or with patches of text missing.

Due to these problems (and the risk that Footnote.com may decide to stop supplying these documents at some point), I think it is highly desirable to download the ENTIRE relevant collection.

I have not been able to identify an efficient way of achieving that goal.

Calling up each image and downloading it in turn would not be practicable.


So, the challenge to members of ATS is : Can you find an efficient way to download the entire free collection of over 125,000 documents?


I am aware of some download management tools (such as Flashget) available for Firefox and other browsers which can assist in downloading all the files of a certain type (e.g. all the jpeg images) on a webpage, but have had difficulty in using those tools to make any real inroads on downloading the entire collection of Project Bluebook documents.

Some of the problems that I’ve encountered are that:

(1) it seems to be much easier to find ways just to download jpeg thumbnails which are available for each image, with URL beginning “img.footnote.com”, such as the link below:
img.footnote.com...

However, these thumbnails are too small to be very useful for research. It would be much better to be able to efficiently download the full images, without having to select the download option on each individual page.


(2) While it may be possible to use a script or tool to download a list of URLs and then use a script or tool to generate a list of URLs with consecutive numbers, the URLs for consecutive images on Footnote.com are not necessarily numbered consecutively. For example, image /#9668013 is followed by /#9668016 which is in turn /#9668018.


Basically, this sort of issue is beyond my technical abilities.


I know that there are quite a few technically minded members of ATS, so I would be very grateful for any thoughts or links to tools which would make downloading the entire collection efficient…

It would also be highly desirable to download the documents in a way which retained as much of the data associated with the images as possible, e.g. that image #9668033 is page 21 within the Project Bluebook/[Blank]/[Blank]/[Blank] directory. This would be useful when seeking to find an unredacted copy of the document (if available) on the under-appreciated non-governmental Bluebook Archive website mentioned above.


I think there is scope for members of ATS to make a real contribution to the wider ufological community by seeking to address this challenge.

Section 8 : Guides to the documentation?

Sightings can be found within the Footnote.com archive by searching (although the search function is far from perfect, possibly because of the poor quality of some of the documents that have been scanned and the redactions from the documents) or by browsing. The documents can be browsed by selecting a year, then a month, and finally a location, as shown in the sample screenshots below:
[atsimg]http://files.abovetopsecret.com/images/member/ce169ba66a3d.jpg[/atsimg]
[atsimg]http://files.abovetopsecret.com/images/member/c6f0194e33ad.jpg[/atsimg]


Documents which do not relate to any specific sighting tend to have “Blank” in the location column, so the more general reviews and reports appear on Footnote.com either in with a blank year, blank month and blank location or with their year and/or month but blank location.


Due to the mass of documentation available, I thought it worth mentioning some of the guides available to the Project Bluebook files (and invite members to add links to some of that documentation).

As many members will be aware, quite a few UFO books discuss Project Bluebook and some of the documentation which it generated.

Particularly illuminating (although far from uncontroversial) are books by those involved in the Project.
The best known such book is by Ruppelt, the text of which is available free on various websites, including the invaluable NICAP website (run by Francis Ridge). See:
www.nicap.org...


(I’d recommend reading that book to those that have not yet done so. It’s free and has been used over the years as a source by many, many UFO authors (often without them giving credit to Ruppelt). For those that would prefer just to listen to a relevant discussion, Ruppelt is discussed by Colin Bennett in the interview below.)




Although not as well known or influential, another member of Project Bluebook (Hector Quintanilla) also wrote an unpublished manuscript (entitlted “UFOs, An Air Force Dilemma”) which is also available online:

www.ufocasebook.com...


Several other UFO books have focused on the content of the Project Bluebook documents, including a book by Brad Steiger entitled “Project Blue Book” (which largely comprises extracts from the relevant documents) and a more analytical and critical book by Kevin Randle entitled “Project Bluebook Exposed”, which contends that Ruppelt’s account of Project Bluebook was misleading.

A few researchers have concentrated on the sightings within the Bluebook files which were either labelled by the Air Force as unidentified or are considered by the relevant researcher to be unidentified. See, in particular, the guides by Don Berliner and Brad Sparks at the respective links below:

Berliner: www.bibliotecapleyades.net... Part 1

Sparks: www.cufos.org...

The National Archives guide to the Project Bluebook files is available on the Footnote.com website at the link below:
www.footnote.com...

Personally, the documents I have found most interesting have generally been within the “Administrative” Files. I have included below the part of the National Archives guide relating to the Administrative Files, supplemented with URLs I have noted to some of the relevant documents. Most of these documents do not relate to any specific location and, as noted above, tend to be filed either under blank year, blank month and blank location or under the relevant year (and month, if known) with blank location.


It should generally, therefore, be relatively easy to fill in the gaps where I have not currently mentioned a URL. Finding the links that I have included so far did not take a great deal of time, particularly after I noticed the system used as outlined in the paragraph above.


Roll 85 Project (Administrative) Files

Administrative Files: Box 1

Technical Report, “Unidentified Aerial Objects, Project SIGN, “Feb. 1949: 74
pp.
www.footnote.com...


Technical Report, “Unidentified Flying Objects, Project GRUDGE,” Aug.
1949: 408 pp.
www.footnote.com...|11885542


Final Report, Project TWINKLE, 27 No. 1951: 25 pp.

Status Reports and Special Reports, GRUDGE – BLUE BOOK
www.footnote.com...

Report 1: 33 pp.
www.footnote.com...

Report 2: 29 pp.
www.footnote.com...

Report 3: 11 pp.
www.footnote.com...

Report 4: 19 pp.
www.footnote.com...

Report 5: 17 pp.
www.footnote.com...

Report 6: 20 pp.
www.footnote.com...

Report 7: 28 pp.
www.footnote.com...

Report 8: 40 pp.
www.footnote.com...

Report 9: 46 pp.
www.footnote.com...

Report 10: 81 pp.
www.footnote.com...

Report 11: 34 pp.
www.footnote.com...

Report 12: 35 pp.
www.footnote.com...



Roll 86 Administrative Files: Box 2

Staff Study – 1952 (Capt. Ruppelt): 49 pp.

AISS folder (Miscellaneous dates); 58 pp.

AFCIN Plan on UFO’s (May 1957 – Feb. 1958): 11 pp.

Staff Study, Dec. 1958: 36 pp.

FTD UFO Panel (1959-1960):25 pp.

Review of Motion Picture “Unidentified Flying Objects” (1960): 17 pp. plus
newspaper clippings

Staff Study, 28 Sept. 1959: 58 pp.

Proposed Transfer to SAFOI (Dec. 1959 – April 1960): 20 pp.

Proposed Transfer to ARDC (Feb. 1960 – April 1962): 15 pp.

Deputy Director File, Jan. 1961: 39 pp.

Untitled folder containing copies of Air Force Regulation 80 -17 with Revisions
(1966-1968): 11 pp.

Summary to Project BLUE BOOK (Jan. 1956): 3 pp.

First Status Report: Project STORK (Preliminary Work for Special Report #14)
April 25, 1953: 102 pp.

Special Report No. 14 (May 5, 1955): 315 pp.
www.footnote.com...|11884638


Correspondence and miscellaneous file relating to Special Report No. 14: 54
pp.
www.footnote.com...



Administrative Files: Box 3

Dr. Fitts’ Reports (April 1949): 17 pp.

USAFE “Aids to Identification of Flying Object” (1957): 57 pp.
www.footnote.com...|11884055

Untitled folder of photographs, containing about 50-60 photos, mostly of USAF
personnel, with many duplicates

Advisory Panel on the Scientific Use of Balloons Meeting, 15 No. 1965: 78 pp.

Sub-Committee Hearing, March 1966 (USAF SAT): 40 pp.

SAT and Recommendations, 1966: 31 pp.

USAF Test Group: UFO Material, 19 Sept. 19

Foreign Technology Division: “Soviet Effort to Contact Extraterrestrial Life,”
3 Feb. 1967: 67 pp.

87 Project BLUE BOOK (USAF SAT, 3 Feb. 1966): 137 pp.


Administrative Files: Box 4
ANTIC UFO Briefing 1 April 1952: 44 pp.

Press Conference – Gen. Sanford 1952: 42 pp.

Capt. Ruppelt ADC Briefing 24 Jan. 1953: 29 pp.

Standard BLUE BOOK Briefing 9 Feb. 1953: 18 pp.

Indoctrination D/T Representatives, 18-19 Feb. 1957: 2 pp.

McClellan Sub-Committee Hearings, Feb.–Mar. 1958: 4 pp.

20 June 1958 Briefing, Representative Henderson: 16 pp.

8 Aug. 1958 McCormack Sub-Committee Briefing: 30 pp.

Proposed Hearing 1961 (Including Mr. Hine’s Visit): 17 pp.
11-15 July 1960: Congressional Briefings (Mr. R. Smart): 21 pp.

Eglin AFB Briefing by Dr. Hynek (27 April 1960): 39 pp.

Briefing AFIT by Lt. Col. Friend, 9 Feb. 1961: 6 pp.

UFO Briefing – Troy, OH, 14 March 1961: 6 pp.

UFO Briefing, 29 Oct. 1962 – 30 Jan. 1963: 40 pp.

FTD Correspondence Folder, 10 July – 1 Aug. 1962: 27 pp.
The UFO Program to the American Society of Newspaper Editors, 22 April
1967: 21 pp.

Briefing 7 July 1967 to Gen. Garland: 24 pp.

August 1952 Air Intelligence Digest Article (Ruppelt): 34 pp.

AVRO Car Folder: 8 pp.
www.footnote.com...

UFO Policy Meeting, Feb. 1959: 27 pp. (Note: All materials this folder
marked “For Official Use Only”).

ATI School Training Lecture, “The UFO Program” (1957): 54 pp.

Armstrong Circle Theater, “UFO: Enigma of the Skies.” Broadcast of 22 Jan.
1958: 78 pp.
www.footnote.com...

Col. Tacker Radio Interview, Dec. 1960, on “Washington Viewpoint”: 13 pp.

Committee on Science and Astronautics (1961): 150 pp. publications @ 2 pp.
per reproduction; 6 pp. other material

Library of Congress Legislative Reference Service: “Facts About UFO’s” 5
May 1966: 48 pp.

D.O.D. News Release and Fact Sheet (1952-1966): 185 pp.



Roll 88 Inspector General, USAF Office of Special Investigations (OSI)
www.footnote.com...


Roll 92
Overseas Commands, 1949-68
Alaskan Air Command
www.footnote.com...

Caribbean Command
www.footnote.com...

Far East Air Forces
www.footnote.com...

USAF in Europe
www.footnote.com...

Military Air Transport Service
www.footnote.com...

Newfoundland Base Command
www.footnote.com...

Section 9 : Conclusion

Many of you will have noticed that a lot of nonsense is written about UFOs.

Some of that nonsense is repeated time after time after time.

Once an inaccuracy (or complete fabrication) has entered into the UFO literature – particularly online – it keeps circulating for years.

Many current UFO websites simply cut and paste material that someone else has written. Indeed, in the UFO and SETI books I’ve read (which number over 1,000) much of the material can be identified as coming originating with a fairly small number of researchers. So, if one of those books gave a wrong or incomplete account of a UFO sighting then the incompleteness and errors can get reinforced by repetition in numerous subsequent books.

Given the increasing ease of access of primary material (i.e. online copies of original UFO reports, online copies of original investigative case files, online copies of the full text of relevant press articles etc etc), it is not unreasonable to expect the quality of UFO literature to have improved in recent years. Unfortunately, in my experience that does not seem to have been the case.

Making significant sources of primary material even easier to access should help improve the situation.
I have outlined in Section 7 a challenge which may help address this situation.

So, is ATS (and the UFO community) up to the challenge?

We’ll see…

Section 10 : References

After reading approximately 1,000 UFO books, I have gathered fairly long list of references to discussion of some of the documents and incidents mentioned above.

Rather than include long list of references here, I will simply include a few links below to pages on an early draft of my website which have tables containing those references for those interested in finding discussions in UFO books of some of the material mentioned above. Some tables have hundreds of references. Those tables can be sorted by sorted by author, date and length of discussion by clicking on the relevant word in the top line of the table, making them more useful than merely cutting and pasting the relevant text here.



Incidents:
Socorro incident:
www.isaackoi.com...

Tremonton film:
www.isaackoi.com...

Mantell incident:
www.isaackoi.com...



Documents

CIA : Robertson Panel report:
www.isaackoi.com...

Project Sign Report:
www.isaackoi.com...

Project Grudge Report:
www.isaackoi.com...

Project Bluebook Special Report Number 14:
www.isaackoi.com...



Personalities

Edward Ruppelt:
www.isaackoi.com...

Hector Quintanilla:
www.isaackoi.com...

Section 11 : Postscript

[atsimg]http://files.abovetopsecret.com/images/member/5db6c492921e.jpg[/atsimg]

!

Sand flags to you very informative and thanks this must be what they mean by the elephant in the room they have disclosed from a breif over scan of your work

In the world of "text walls" this is the great wall of China!

Thank you for putting this all together. (I was good and did not add a reply before you were done!) It will take some time to go through.

An initial thought: what an idea, that perhaps disclosure would come in the form of an impenetrable website that everyone ignores? What if?

Good job on compiling all this data. This topic should be tagged because it is VERY important to people who would like to research and reference material on there own.

Wow cool thread your obviously not the type to mess around with something.
I haven't read it all but i have a idea which i will try soon if someone doesn't beat me to it. But have you tried a program which will literally download a website?
EDIT: im going to try it now but sadly i currently only have my netbook so it could take a while...

reply to post by IsaacKoi


The following WP entry should help those of good will

Website-Mirroring Software

reply to post by IsaacKoi


Good job, you put a lot of work into this thread.

Section 7 : The Challenge : Efficient Batch Downloading?

I can recommend HTTrack for a first attempt to download the entire site. Windows and Linux version available, free open source.

Once the data is on your computer, you may try to index it with Google Desktop Search (free, but quite heavyweight) or Archivarius 3000 (commercial).
I do not have experience with Google Desktop Search, but it is very popular. I am using Archivarius and love it, well worth the money and does semantic search.

Originally posted by SacredLore
reply to post by IsaacKoi

 

Good job, you put a lot of work into this thread.

Section 7 : The Challenge : Efficient Batch Downloading?

I can recommend HTTrack for a first attempt to download the entire site. Windows and Linux version available, free open source.

Once the data is on your computer, you may try to index it with Google Desktop Search (free, but quite heavyweight) or Archivarius 3000 (commercial).
I do not have experience with Google Desktop Search, but it is very popular. I am using Archivarius and love it, well worth the money and does semantic search.

Yea thats what im trying right now its taking a while . i just hope it works...