The 154 GB NARA Blue Book Archive

A new FootnoteReap release (v0.0.1.5.1) is available to download.

code.google.com...

Build v0.0.1.5 was broken. So I updated the code to make it compatible with the new Fold3 URLs. You should only have to grab the updated software if you have found a new case that isn't already in the 154 GB archive.

The software is still alpha and is probably still a bit confusing. So if you haven't located a new case and just want to download the Blue Book archive. You don't have to do anything different. =) Just continue to use BT Sync.

twitter.com...

originally posted by: sn0rch
a reply to: Spacespider

Me too, I tried the torrentsync and it just say there not moving, no peers Im guessing, no initial seed.

There should be at least six seeders. You may want to click on the peer list to check to make sure your machine doesn't have too big of a time difference with the rest of the peers on the network.

I see at least two people connected who need to update their system time:



You can find your BT Sync device name here (scroll right):



To update the system time on Windows:

I've been a lurker on here for years, and this is the first time i've actually made an account.
This is fantastic work, thank you. I'll be running the sync as long as I can on 100/100.

a reply to: Xtraeme

130,000 pages?! This is an insane amount of information. To the guy who put this together: if you are ever in Ontario, the beer is on me!

a reply to: TheMalefactor

I never turn down a free beer. =) Too bad ATS doesn't have a tip bot.
Hopefully the library helps in a small way to unravel a couple mysteries and helps to make better sense of everything.

Just wondered, has anyone managed to sync the whole lot yet? I'm at 118Gb so far.

a reply to: simplesquig

At least seven other people have fully synced and downloaded all the updates. There are two other machines online at the moment, besides my own, that have a full copy. A big thank you to John Greenewald, Jr. and everyone else that is seeding for helping out.





Be on the lookout for a couple updates in three or so weeks.

originally posted by: simplesquig
Just wondered, has anyone managed to sync the whole lot yet? I'm at 118Gb so far.

I've fallen behind a bit!
I'm at 60GB now but will have the machine on syncing more.

I'm still good to do some transcribing, do we know what formats (and formatting) we want this done in?

Thank You Xtraeme for sharing this ! in the past the blue book archives rolls cover only 1947 to 1954 partim and 1968 69, but the fold3 archive was more complete but very slow to consult, I have scan and extract and analyse 1947 to 1957 cases one by one (yes a lot of job ! ) , but with this new archive finish the job would be more easy for me so thank you !

My opinion is that blue books have many non closed cases , many many more than what usaf said at the time, blue book is an important archive .

for the scan problem in my opinion the better first step is to scan the "synthesis file", first image of each container , because some texts doesn't pass the ocr test

Thank you again

Now that the shouting is ver...I'd like to ask a few questions...

1. How many of y'all actually downloaded this file?
2. How long did it take?
3. Did you find the data usable
3. Did you find the data helpful?

Thanks...

originally posted by: tanka418
1. How many of y'all actually downloaded this file?

At the peak there were 110 people simultaneously connected. There are 38 connected now. So over the last couple weeks probably a couple hundred people have downloaded bits and pieces.

2. How long did it take?

The pipe is a 50 Mbps FIOS connection. So on average it took people with a 10-15 Mbps connection about two to three days to download all the material.

If you are trying to figure out how long it will take to download for yourself. It is a pretty simple calculation: Total size / megabits per second (Mbps).

So in other words: 154 GB (or 154,000 MB) / (your connection speed). To convert megabits to megabytes simply divide by 8. Then just divide the result by 60 seconds / min and 60 mins / hour to calculate total time in hours. Here is a quicker way to do it:

www.wolframalpha.com...

3. Did you find the data usable
3. Did you find the data helpful?

I think that has already been answered:
www.abovetopsecret.com...

Though I'd love to hear from other people.

Why not download it yourself?

I am curious what happened to this project?

Well, I didn't hear back from anyone, so I thought I would post an update.

I have succeeded in writing a script to convert this entire collection to PDFs. This has been completed. I am now running a mass OCR conversion on all of them, which will probably take a few days.

I will upload the collection, complete with a search engine soon... just waiting for the OCR to complete.

I'm looking forward to getting this online.

a reply to: blackvault

Howdy John,

Yep, there were a couple of pages still outstanding. I got a form letter style response from Fold3 about some of the missing pages on their end, but I don't expect they'll add the pages any time soon. I also wrote a little script to help sort out the cases in 9667997 (8198 pages), 9669100 (1415 pages), and 9669191 (3710 pages). If you'd like, I would be happy to give you write access to SVN so you can add your scripts to the project. We can divide up the task of running the OCR leg. I have a couple of machines that are just idling. So it would probably speed things along.

Thanks for helping out again. =) Happy New Year's John!
-Xt

Glad to know you are still pursuing it. FULL credit to you for compiling the data. I emailed you directly, asking what credit you'd like on the site... but as I mentioned above... figured out a few things, and built a searchable database.

First, they are entirely converted to PDFs, with the exception of a few that contained THOUSANDS of jpegs... it was just too large and not feasible (for now)... re-aligned and adjusted the images to then run it through the best OCR process I could. I then uploaded the entire collection, and then built a server site search engine to catalog them all (ie: not using Google which may only search 4% of them!) and it is now available at:

projectbluebook.theblackvault.com...

Let me know if I can give you a credit, and if you want contact information on it...

a reply to: blackvault
Outstanding! Thank you!

This is amazing! They have the Kenneth Arnold sighting concluding the event to be a mirage. If you look through he archives, you will find some good ones! It would be nice if the unexplained ones were archived separately since those account for about 5%. S&F

a reply to: blackvault

proofofexistence.com...
blockexplorer.com...

I don't understand why anyone would want a copy of the prices of used cars.

It seems to stop downloading/syncing at 126.784 files, I seem to be missing 468 files from Titan BitTorrent Sync Server.
Anyone know why ?