2 Million+ New Pages – Searchable transcripts of UFO podcasts, videos etc : Leveraging AI

Section A : Introduction

This thread is about the latest tools I’ve developed to deal more efficiently and effectively with audio/video material (such as podcasts, documentaries etc) relevant to UFO / UAP / Fortean research – a collection of searchable transcripts and related spreadsheets.

I don’t think anyone (at least within ufology, and possibly even outside it) has tried something like this before, so I hope you find this attempt interesting/useful…

I don’t have a lot of spare time for ufology (or anything else…) so am always keen to find more efficient ways to do effective and comprehensive UFO research (particularly by leveraging Artificial Intelligence software and by creating a searchable archive).

During the last six months or so, I’ve taken the occasional short break from my ongoing behind-the-scenes research on a, well, pretty large item on AATIP / AAWSAP / Bigelow / metamaterials / SWR to prepare this relatively small thread. I’d like to share (where I’ve obtained permission to do so) some of the searchable transcripts I’ve made of audio/video material regarding UFOs and the paranormal, plus some wider sceptical / scientific material.

The searchable spreadsheets summarise information on each podcast/video, help find podcasts or documentaries featuring particular individuals/guests. Given the volume of material, I’ve found the searchable spreadsheets a useful tool in my own research and am happy to share them.

Over the last couple of decades, I’ve generally focused on written UFO material (including books, magazines/newsletters, UFO databases and – of course – official documents). I like to have information in writing, rather than in audio/video recordings. I can read pretty quickly whereas I find listening to podcasts or watching UFO documentaries to be relatively time consuming. So, I’ve wanted to create such transcripts for a few years so that I can read the material to at least get the gist of the matters discussed and points made. Of course, rendering the material searchable is an added bonus.

I hope this little side project has created some useful additional tools for the UFO community. As is often the case with ufology (at least for me…), the little project I originally envisaged grew a bit, due to an element of mission creep.



TL;DR - I’ve previously quietly help coordinate efforts by numerous UFO groups/researchers to get a mass of digitised written material about UFOs online (after obtaining relevant permissions), including numerous defunct UFO newsletters/magazines plus official documents, PhD dissertations, UFO databases/catalogues and other material (see Section B below). I’ve now created over 2 million pages of searchable automated transcripts of over 50,000 audio/video recordings (see Section D and Section E). Most of the transcripts so far are of UFO podcasts (see Section F below), with smaller numbers of transcripts of other UFO audio-visual material (see Section K and Section L below). I’ve obtained permission to upload most of those transcripts as PDFs (and as text files) to the links below. Due to the volume of transcripts, I have also produced some guides to the relevant transcripts and recordings, by creating and uploading about 20,000 pages of spreadsheets providing an outline of the recordings covered by the transcripts – summarised briefly below. I have various ideas for further potential leveraging AI and a bit of crowdsourcing (_not_ crowdfunding – I don’t want your money…) to help make a lot more searchable transcripts freely available online (see Section H, Section I, Section N, Section O, Section P, Section Q, Section R, Section S, Section T and Section U below), not to mention further official UFO documents and other UFO data from around the world if there is any interest in speeding up the pace of UFO research (see Section V below). Increased speed may, of course, simply result in some UFO researchers going around in circles faster, but I’m happy to continue my efforts to attempt to improve the current state of affairs.

Link to the UFO transcripts, and related spreadsheets, that I have uploaded so far



I’d suggest glancing at a sample summary spreadsheet. A summary spreadsheet appears at the top of each of the directories of transcripts (both as a searchable PDF version and a Microsoft Excel editable version), starting with “0 – “ and ending with “Info Koi”. To make it easier to quickly download a bigger sample of the summary spreadsheets and glance through a few of them, I’ve also uploaded the current collection the PDF versions of the “Info Koi” summary spreadsheets (over 20,000 pages) to THIS temporary Wetransfer link.

(I’ve also uploaded Excel versions of each Info Koi summary spreadsheet, in case anyone wants to organise them differently, annotate them or combine them with other UFO spreadsheets / databases).

The transcripts help find material containing discussions of a particular individual or topic. The imperfect searchable transcripts are intended as a tool to help locate relevant audio/visual material, not to replace that material. The spreadsheets, and the transcripts, contain links to the relevant recording so that no-one is reliant upon the imperfect automated transcripts.

I’ve split this post into the following Sections:

Section A : This introduction

Section B : Background - Project Phoenix

Section C : Background - Rapid searching of numerous PDFs

Section D : Background - Artificial Intelligence and modern automated transcripts

Section E : Using Youtube’s AI and playlists (plus crowdsourcing) for batch production of transcripts

Section F : Podcast Transcripts – UFO Podcasts on Youtube (introduction)

Section F1 : Podcast transcripts - "Arcane Radio" (permission given to me by Lon Strickler)


Section F2 : Podcast transcripts - "Fade to Black" (permission given to me by Jimmy Church)


Section F3 : Podcast transcripts - "Grimerica" (permission given to me by Graham Dunlop and Darren Grimes)
Section F4 : Podcast transcripts - "Inception Radio Network" (permission given to me by Mike Lucas)


Section F5 : Podcast transcripts - “Inside the Outer Limits” (permission given to me by Chris Evers and Philip Mantle)


Section F6 : Podcast transcripts - "Mr UFO" (permission given to me by Tim Beckley)


Section F7 : Podcast transcripts - "OBDM" (permission given to me by Mike and Joe)


Section F8 : Podcast transcripts - "Open Minds Radio" (permission given to me by Alejandro Rojas)


Section F9 : Podcast transcripts - "Paranormal Dimensions" (permission given to me by David Young)


Section F10 : Podcast transcripts - "Paranormal Mysteries" (permission given to me by Sacha Christie)


Section F11 : Podcast transcripts - "Podcast UFO" (permission given to me by Martin Willis)


Section F12 : Podcast transcripts - "Rogue Planet" (permission given to me by Jason McClellan)


Section F13 : Podcast transcripts - “Shattered Reality” (permission given to me by Kate Valentine and Fajrusha)



Section F14 : Podcast transcripts - “Skeptic Zone” (permission given to me by Richard Saunders)



Section F15 : Podcast transcripts - "Somewhere in the Skies" (permission given to me by Ryan Sprague)



Section F16 : Podcast transcripts - "Spaced Out Radio" (permission given to me by Dave Scott)


Section F17 : Podcast transcripts - “Strange Planet” (permission given to me by Richard Syrett)


Section F18 : Podcast transcripts - “The Higherside Chats” (permission given to me by Greg Carlwood)



Section F19 : Podcast transcripts - "The Mad Scientist" (permission given to me by Chris Cogswell)


Section F20 : Podcast transcripts - “The Midnight Hour” (permission given to me by Steve Cambrian)



Section F21 : Podcast transcripts - “UFO Buster Radio” (permission given to me by Emmett and Manny Moonraker)



Section F22 : Podcast transcripts - “UFO Chronicles” (permission given to me by Nik Hunter)



Section F23 : Podcast transcripts - “UFO Garage” (permission given to me by Ben and Joe)



Section F24 : Podcast transcripts - "UFO News Network" (permission given to me by Frank Stalter and Chant Hannah)


Section F25 : Podcast transcripts -"X Zone Radio" (permission given to me by Rob McConnell)



Section G : Videos Transcripts : UFO Documentaries on Youtube – Introduction



Section H : Videos Transcripts : UFO Documentaries on Youtube – Documentary series



Section I : Videos Transcripts : UFO Documentaries on Youtube – Single documentaries



Section J : Videos Transcripts : Government videos (e.g. NASA)



Section K : Videos Transcripts : UFO vodcasts on Youtube

Section K1 : Vodcast Transcripts - “Alien Protocols” (permission given to me by “Buddy”)

Section K2 : Vodcast Transcripts - "Den of Lore" (permission given to me by Chris George Zuger)

Section K3 : Vodcast Transcripts - "Hidden Truth" (permission given to me by Jim Breslo)

Section K4 : Vodcast Transcripts - "Metabunk" (permission given to me by Mick West)

Section K5 : Vodcast Transcripts - "Rich Planet TV" (permission given to me by Richard Hall)

Section K6 : Vodcast transcripts - “Skeptiko” (permission given to me by Alex Tsakiris)

Section K7 : Vodcast Transcripts - "Thinking Allowed" (permission given to me by Jeffrey Mishlove)

Section K8 : Vodcast transcripts - “UFO Bros” (permission given to me by Emmett and Joe Hayes)

Section K9 : Vodcast Transcripts - "UFO Hub" (permission given to me by Adnan Ademovic)



Section L : Videos Transcripts : UFO lectures on Youtube

Section L1 : Lectures - “Society for Scientific Exploration (SSE)” (permission from Annalisa Ventola)

Section L2 : Lectures - “Swansea UFO Network” (permission granted by Emlyn Williams for me to share my spreadsheet and transcripts)

Section L3 : Lectures - “UFO Congress (IUFOC)” (permission granted by Alejandro Rojas for me to share my spreadsheet and transcripts)





Section M : Foreign language transcripts

Section M1 : Introduction regarding foreign language transcripts

Section M2 : French language transcripts

Section M3 : German language transcripts

Section M4 : Italian language transcripts

Section M5 : Spanish language transcripts



Section N : Further transcripts? More of the above

Section O : Further transcripts? UFO podcasts already on Youtube but without subtitles

Section P : Further transcripts? Digitised UFO podcasts not currently on Youtube

Section Q : Further transcripts? Other digitised UFO audio material not currently on Youtube

Section R : Further transcripts? Other digitised UFO video material not currently on Youtube

Section S : Further transcripts? Digitising other UFO audio recordings

Section T : Further transcripts? Digitising other UFO video recordings

Section U : Further transcripts? Translation of foreign subtitles (and other foreign PDFs)

Section V : Conclusion

Section W : Postscript


Some of you know my view that those who fail to learn from ufological history are doomed to repeat it. The history of research into reports of “UFOs” is littered with examples of reinvention of the wheel, inefficiency and ineffectiveness. I'd like to make it easier to learn about the history of this subject in the hope that this _may_ help improve the quality of current UFO research. That’s one of the reasons for my working over the years to obtain permission from numerous UFO researchers and groups for me to share their work freely online in a searchable format.

I’ve worked to help make more source materials relating to UFOs freely available online in a searchable format in the hope this will help everyone to research facts without paying for the privilege and also help them to establish which people in the UFO community are hoaxers, charlatans or just gullible. The vast majority of people in ufology (ufologists and skeptics) have appreciated my efforts to do this. A very wide spectrum of UFO groups, ufologists and skeptics work with me behind the scenes, since most people in ufology across this wide spectrum want to find answers and identify charlatans. A very, _very_ small handful don’t like these efforts.

A lot of UFO documentation has been preserved and circulated over the years solely within various old-boys networks. Despite the fact that I’ve been invited to join (and have joined…) many such networks over the years, I personally think that the sharing of data only within such networks has been a significant problem throughout the history of ufology. The practice of very limited sharing of data with other “responsible” researchers (all too often actually meaning researchers with similar views…) lends itself to an echo-chamber effect. Views can be formed and reinforced within a small group that fall apart fairly quickly once the relevant data is actually shared widely and examined from different perspectives.

Preserving the history of the UFO field isn’t enough to improve the quality of research. We need to make it freely available in formats that make it as easy as to possible to locate relevant data and work. Sloppy research within ufology is becoming less and less excusable in the light of the ease with which relevant information can now be obtained.

I’m hardly alone in many of these thoughts. I’ve worked with the AFU in Sweden and over 100 other UFO researchers/groups to share a mass of written UFO material online, as part of the digitising of UFO magazines/newsletters, official documents and other material. I’ve found written material to be easier to search, skim and share than video and audio material.

However, I don’t think audio/visual UFO material (such as UFO podcasts, documentaries, videos of lectures, audio recordings of witnesses/researchers etc etc) should be completely ignored. They sometimes contain information that supplements the data available in written material. It would just be nice if it didn’t take so long to extract the more useful information from such material – hence this project…


Podcasts, for example, can

(1) contain information supplementing that available in writing,

(2) contain a UFO witness telling his story in different ways over the years, for innocent or nefarious reasons,

(3) indicate a researcher or researchers that have looked into a particular event/issue, leading to possible avenues of enquiry,

(4) let you hear the voice of a witness when giving an account, which can possibly give a bit of insight into the person/event.



I’ve leveraged AI software to produce a substantial volume of automated transcripts on my own so far (over 2 million pages) and have some thoughts on using crowdsourcing to work with others to generate a lot more such transcripts – particularly of further documentaries (see Section H and Section I below) and podcasts not currently on Youtube (see Section P below)– without any single person having to do a lot more work.

Section B : Background - Project Phoenix

For the many newer members that weren’t around on ATS when I posted here far more regularly up until an unpleasant event at the end of 2016, I’ll post a brief bit of background to put this latest set of uploads into context.

I am a barrister in England afflicted with an interest in issues relating to "UFOs". I've now assisted with making over 200 sets of UFO/Fortean periodicals and newsletters freely available online, plus official documentation and other material. Some of my previous threads have outlined a few of the searchable PDF archives I've helped make available for anyone to freely download, including on UFO documents from the FBI (see FBI: “discs recovered”, Air Force “greatly concerned”, “at a complete loss” + more memos"), from the CIA relating to remote viewing (see Remote viewing & UFOs : Stargate, Galactic Federation + the Aviary (CIA index + 92,010 PDF pages), from the US Air Force (see Massive UFO disclosure in USA : A challenge for ATS, from Canada with the permission of the Canadian government (see Canadian disclosure: “UFO Found” and other documents/photos and Canadian PDFs – “At no time should it be made available to the public” + more official memos), Australia, with the permission of the Australian government (see, for example, Oz X-Files : “authenticated sighting”, “appear to be inexplicable”, USA “very interested" and New Zealand (see New Zealand X-Files : Official “fib”, superiors “mystified”, + Kaikoura “whitewash”), plus various out-of-print UFO publications (see, for example, Phil Klass: "Skeptics UFO Newsletter" now online (large collection) and about 30 other UFO magazine/journal collections).

I have helped build up a free online archive of searchable digitised UFO material in conjunction with the Archives For the Unexplained (AFU) in Sweden and over 100 other UFO groups/researchers involved in the UFO digitising work I've been helping to coordinate. I’m biased in favour of material in English (simply because it is the only language in which I am fluent…), so my focus has mainly been on material from the United States, the United Kingdom, Australia, New Zealand and Canada (but more recently I’ve also dealt with the uploading of dozens of the main UFO magazines/newsletters from Spain, Germany and France after working with local researchers to get relevant permissions) - e.g.:
United States
United Kingdom
Canada


The number of UFO groups / researchers that have kindly provided scans for me to share on the AFU's website exceeded 100 during 2019. Some, of course, have done a lot more scanning than others. The AFU itself has done by _far_ the largest single portion of the scanning. Some others have been extremely useful in filling in rare gaps, e.g. Barry Greenwood in the USA. Some have been focused on individual countries (e.g. Association SCEAU/Archives OVNI, Patrice Seray, El Peter EL Baze in France). But all the contributions of scans have helped make an online resource that has, I think, reached a critical mass that means it is now much easier to obtain further permissions for the release of material on the AFU's public website.

I haven’t posted very much on UFO discussion forums, including here on ATS, since December 2016 due to some unpleasantness with (literally) one or two people regarding my use of a pseudonym when sharing material about UFOs. I have always made clear that I use a pseudonym when posting about UFOs, simply to avoid any potential effects upon my career as a barrister. (I’m a member of a rather old-fashioned profession…). My professional reputation is more important to me than ufology, not least because my professional reputation helps pay the bills for my family whereas ufology… – well – frankly trying to find some answers relating to “UFOs” has just been a considerable drain on my time and resources. So, I took a break from ufology for about a year after that unpleasantness but then suffered a relapse in late 2017 after I learnt about AATIP and AAWSAP shortly before the former became public knowledge.

Since late 2017, mainly to avoid any drama, I’ve largely worked behind the scenes rather than in public. I’ve simply got on with putting material online from around the world for the UFO community, and worked on my private research. I launched “Project Phoenix” in 2018 to help speed-up the digitising of UFO material and getting that material freely shared online (after getting relevant permissions).

Basically, there is a massive amount of further UFO material (from official documents to UFO newsletters etc) lying around in numerous official and private archives, just waiting for someone to digitise that material and share it.

In the period since late 2017, I’ve helped coordinate work on Project Phoenix to make much more UFO material freely available online than I’d ever managed before that – but I haven’t been posting about the uploads on ATS or other discussion forums very often. I’ve been adopting an even lower profile on public forums to seek to avoid anyone publishing my personal details online. The further digitised material can be found on the AFU’s website at the link below (particularly in the sections relating to UFO magazines) and anyone doing a Google search will have been able to find that material:

files.afu.se...



My relative absence of posts on public forums has had several advantages. After I largely stopped posting on public UFO forums in late 2016, I was approached by quite a few interesting people and invited to join various further private networks. It seems that some people were more comfortable talking to me, or exchanging material, after I reduced the amount of comment I posted in public.

However, I think that much more could be done (with relatively little effort) to make UFO research more efficient and effective with just a spoonful of coordination of wider public efforts so am dipping my toes back into the waters of more public participation on ATS and a few other UFO forums – including with this new thread.

Quite a bit of my private research since 2017 has been in relation to Bigelow/AATIP and related issues. I have obtained information (and, in some cases, documents) from various people connected with each stage of Bigelow’s involvement in ufology – not just in relation to his higher profile participation in NIDS and AATIP/AAWSAP but also in relation to less well known Bigelow UFO projects such as the Bigelow Foundation, the Triad Research Conference Foundation, Bigelow’s involvement with UFORC / CUFOS / FUFOR / MUFON and some others. Participating in public discussions in relation to such controversial areas is, however, likely to result in more drama for me and speculation about why I prefer to post under a pseudonym. Also, I’ve encountered some blocking tactics from one or two people in the UFO community when trying to put together a definitive account of certain aspects and I strongly dislike posting until I can be comprehensive. Besides, part of the story behind work by Bigelow/AATIP would make at least some of you cry (literally) and I don’t like upsetting people. So, I’ll probably continue to be relatively restrained about voicing opinions in relation to those matters. Heck, I was relatively restrained about posting opinions and speculation regarding UFO research even before the unpleasantness in 2016 since I’ve thought for a long time that there is _plenty_ of speculation being pumped out by other UFO researchers. I think it is more productive for me to focus on finding/sharing material and data rather than focusing on merely expressing personal views.

Since 2017, I've also worked to try to improve the amount of input from experts outside the rather polarised ufo community. The following individuals kindly agreed to my requests to endorse the expert input statement below that I drafted to use when requesting such input : Dr Kit Green (formerly of the CIA), Dr Hal Puthoff (of TTSA), Dr Eric Davis, Dr Bruce Maccabee (formerly of the US Navy), Dr Danny Ammon (of Germany's GEP), Rev Dr Ray Boeche, Dr Irena Scott, Dr Chris Cogswell, Jim Semivan (formerly of the CIA), Jenny Randles, Colonel Charles Halt (formerly of the US Air Force), Lieutenant Colonel Kevin D Randle (formerly of the US Army), Nick Pope (formerly of the UK Ministry of Defence), Professor Richard Noll (of DeSales University), Dr David Clarke, Dr Chris French, Dr Gilles Fernandez, James Oberg (formerly of NASA), Robert Sheaffer, Tim Printy, Lance Moody, Curt Collins, Wendy M. Grossman (founder of "The Skeptic"), Jan Aldrich, Barry Greenwood, Edoardo Russo (of Italy's CISU), Frank Warren, Chris Rutkowski, Christopher O'Brien, Richard Doty (formerly of the US Air Force OSI), Dr Mark Rodeghier (of CUFOS), John Schuessler (of MUFON), Tony Eccles (of BUFORA), Mark Allin (co-owner of the AboveTopSecret website), Rick Hilberg, Paul Dean, Keith Basterfield, Jacques Scornaux (of France's SCEAU-Archives OVNI), Mikhail Gershtein (of Russia) and Robbie Williams (English pop star).

(BEGIN QUOTE OF EXPERT INPUT STATEMENT)
"We consider that obtaining input from disinterested experts on specific points regarding reports of 'UFOs' is likely to contribute to the study of relevant physical, historical, psychological and sociological issues. If you are able to help provide such input, we would appreciate you doing so".
(END QUOTE OF EXPERT INPUT STATEMENT)

Another thing has changed. I used to post quite a lot of material exposing UFO hoaxes and flawed UFO research. I learnt from the unpleasantness in 2016 that sharing material that exposes weaknesses in claims by others results in considerable retaliation from a few people within ufology. On the one hand, my sharing UFO source materials (documents, newsletters etc) online has helped me gain access to numerous useful contacts and a growing network of individuals willing to work together to achieve common goals. On the other hand, my sharing information about UFO hoaxes (such as various viral UFO videos) has, I think, mainly lost me connections and access to information/material from a few true believers that seem to view any criticism of any UFO evidence/researcher (however poor) as a betrayal of “the cause”. Perhaps I should have kept my mouth shut rather than shooting myself in the foot. I wonder how many others within ufology were smarter than me and learned to just sit on negative information. There are some people that seem to be relentlessly positive when interviewing people on podcasts etc - leading me to wonder if they have to act that way to keep getting guests and maintain friendly contacts, i.e. excise any critical thought from their minds.


Oh, and I also collect and share online cartoons on UFOs (particularly newspaper cartoons from the 1940s-1960s).

Section C : Background - Rapid searching of numerous PDFs

A large volume of digitised UFO material becomes unmanageable without tools to help search and analyse that material.

Some of the potential search/analytical tools are hard to master (not to mention expensive…), so are outside the scope of this thread about my recent transcript project.

However, one the simplest (and free!) such tools can search numerous PDFs in a single search. I use this software to search some or all of a directory containing scans of most of the UFO books that have been published, hundreds of thousands of pages of official UFO documents from around the world, thousands of UFO magazines, millions of pages of transcripts of UFO podcasts, numerous UFO newspaper clipping collections, various large UFO databases, case files of various UFO groups/researchers etc).



Basically, I think this simple and free tool (which I’ve mentioned a few times over the last decade or so) has the potential to significantly improve the quality of UFO research without worrying about any of the fancier search/analytical tools. I’d, once again, like to encourage more researchers to try it out.

The relevant tool is the free version of PDF-XChange Editor (which does not require a licence) can be downloaded from the link below (free).

PDF Xchange Editor

There are “Plus” (and licenced) versions of the software available on the same website, but the free version without a licence includes the relevant search functions (and “more than 60% of the features” in the paid-for version). I’ve been using the features available in the free version very happily for about a decade without any problems.

I know that Adobe Reader (and other pieces of PDF software) allow searching multiple PDF files – but the speed of this PDF-XChange Editor’s search function made the Adobe Reader’s equivalent search look incredibly slow when I compared the two pieces of software a few years ago.

PDF-XChange Editor can be used to search an entire directory full of PDF documents (or, indeed, containing sub-directories full of different collections of PDF documents). For example, I have a directory entitled “Journals” which contains sub-directories for journals published by NICAP, MUFON, APRO and about 450 other sets of magazines/newsletters in English. I can search one of the sub-directories relating to a specific newsletter or search all of the newsletters using one simple (and amazingly fast) search in PDF-XChange Editor.

The same searches can now be done off-line on a downloaded copy of the UFO transcript detailed in this thread.

After installing the free version of “PDF-XChange Editor” (see the link above), when you run the program you simply:

(1) Click on the search button (i.e. the image of the binoculars);



(2) Type in the word you want to search for (and – if you want - select any options, e.g. making the search term case sensitive);



(3) Click on the drop down box and select “browse” at the bottom. You can then select the directory of journals/transcripts you want to search (e.g. your “Fade to Black” transcript directory or a directory containing ALL the UFO tran.journals you have downloaded)



The relevant (fairly simple) steps are indicated by the numbers in the screenshot below:





Very quickly, a search for, say, “pilots”, generates a list of results such as the one below:







Crucially, you can very quickly click on the various results in turn and they are displayed (with the relevant keyword highlighted) in a separate window next to the list of results.



An example is shown below.







PDF-Xchange Viewer does not create any index - it needs to run through each specified file/folder each time a search is performed. This means it is quicker to set up than indexing tools, but searches obviously take longer than running a query indexing software.



The search results in PDF-XChange Viewer indicate how many times the relevant keyword or phrase appears in any particular document (with a helpful snippet of surrounding words, which often allows you to eliminate many of the results) and allows you to click on each one in turn very quickly, with the relevant page being displayed almost instantly.



Trying to review the results of a search on most indexing software is, relatively speaking, a pain in the backside. There is often a preview window which displays the first relevant occurrence of a keyword/phrase within a document when you highlight that document's filename, but I've found that opening each search result found by indexing software can be much slower than opening search results found by PDF-Xchange Viewer – as I mentioned in a post back in 2012:

ufoupdateslist.com...



I find PDF-Xchange Viewer much more useful on a day-to-day basis than most indexing tools, since the indexing tools can tell you quickly that a certain document or set of documents contains a keyword but doesn’t let view the search terms in its full context as rapidly as PDF-Xchange Viewer. I’d rather wait a bit longer for the results (which can be generated while I’m sleeping, watching a movie or going for a walk with my family) and then zip through the results, rather than getting results immediately but having to spend much longer reviewing the results to find any useful material.

Section D : Background - Artificial Intelligence and modern automated transcripts

After getting a bit frustrated with UFO audio/visual material back in 2011, I asked on ATS how others afflicted with an interest in UFOs get to grips with audio/visual material (in my 2011 thread : "Getting to grips with UFO documentaries" (2011)), particularly online guides to documentaries (as to which, see Section G, Section H and Section I below).

Since that 2011 thread, I’ve occasionally revisited related topics.

For example, in 2012 I posted an item on the UFO Updates email discussion list about experiments I had done on software available at that time to produce searchable PDF transcripts of UFO transcripts to add to my growing collection of digitised UFO material. As indicated in my 2012 post, the results at that time were (to be polite) disappointing. In 2013, I included some notes on the same topic in one of my threads on ATS in the hope of learning about better software. The most promising, but still disappointing, option I found at the time was built into Youtube.

My posts in 2011-2013 noted that the most promising pieces of commercially available transcript software were Dragon Naturally Speaking and Adobe Soundbooth's Speech Search transcription function.

Dragon Naturally Speaking : After doing some reading regarding Dragon Naturally Speaking, it seemed that the software needed to be trained to the voice of a particular individual, by that individual reading out various passages of text. That was, of course, a considerable problem when I was seeking a means to transcribe large archives of podcasts featuring various presenters and guests.

Adobe Soundbooth's "Speech Search" transcription function did not require training and I used it to produce an automated transcript of a UFO podcast. As could be seen from the sample transcript I shared at that time, the results were basically unusable and I was not convinced that seeking to correct that transcript would be quicker than starting from scratch and simply transcribing a podcast manually.

In my posts in 2011-2013, I also noted that one of the more interesting incidental discoveries I made during the above exercises regarding transcribing podcasts was that the closed captions that can be displayed on almost all Youtube videos (by clicking on the "CC" at the bottom right of each video) can be downloaded quickly and easily using various methods. While this method had the advantage of being free and very easy to use, the results were - once again - very disappointing and almost unusable, as could be seen from the Youtube sample transcript I posted at that time.

Since many UFO documentaries were already on Youtube, I thought this could be a great way of quickly building up a library of transcripts of documentaries. Unfortunately, the automated transcripts on Youtube videos were generally _very_ poor and hence not much use.

For example, if you turned on the automated captions on Youtube's copy of de Caro's "Special Assignment" on CNN at the link below in 2011, you got treated to mis-transcriptions such as "that silly UFO" (4:47) and "crap landing" (5:16), plus my favourite - when Airman Greg Battram ("Airman Greg") states on the video that "I think I saw a UFO, some kind of spaceship from someplace not of this Earth" the automated captions for the final bit say "someplace closer" (4:41).



Many of the comments on Youtube's automated closed captions in 2011 included the word "FAIL". In fact, the weakness of Youtube's automated transcriptions was the source of some humour in Youtube videos, including at the link below:



In a small number of cases, the transcripts on UFO documentaries on Youtube were very good because someone had uploaded a human transcription. But a bit of random sampling of UFO videos on Youtube suggested this applied to, oh, less than 1 percent of videos so this only appliesd to a very small number of UFO documentaries.

All-in-all, I found these results very disappointing.

Redoing some experiments with transcription software in 2017 (including using Youtube’s subtitling system), my impression was that the results were much better (although still far from perfect).

Taking the above example from CNN, instead of “crap landing” the relevant transcription now indicates (correctly) a reference to a “craft landing”.

The current transcription of the CNN video by Youtube’s Artificial Intelligence software can be seen by clicking on the link above for the relevant video and then clicking on the “CC” icon on the bottom right of the screen – as illustrated in the screenshot below.

The full transcript for a Youtube video can be seen and – notably – SEARCHED in videos with that “CC” video by (1) clicking on the gear icon below a Youtube video and then (2) selecting the option to open a transcript, which (3) causes a window pane to display a transcript with time codes which can be searched (simply by searching the webpage for a keyword, e.g. by pressing Control-F to bring up a search box) – as illustrated in the screenshot below. You can also highlight text and then cut and paste it into your notes, a post or an email. (Just pause the video if you want to scroll through the transcript and/or copy some text).



Doing considerably more comparisons in 2019, it became clear to me that automated transcriptions had improved CONSIDERABLY (actually, even with capital letters “considerably” is understating it – let’s say MASSIVELY) since 2011 – resulting in my putting more time and effort into producing relevant transcripts.

During some experimental work on uploading material to Youtube to create transcripts (and using existing online material as samples), I found that the Youtube system (which now utilises gradually improving Artificial Intelligence software) and related tools had improved in various other ways as well. In particular, it is now possible to combine various tools to enable the user to batch produce searchable transcripts. Okay, this still involves a bit of tedious work – but the scope of potential projects is, um, sizeable.

Anyone that has used the natural language and voice interfaces used by Smart home devices such as Google Home and Amazon Echo will not be surprised by the improvement in the ability of Artificial Intelligence software to deal with human speech in recent years. Given the sales of such items and the competition in relation to Smart home devices, presumably this evolution of Artificial Intelligence tools will continue for the foreseeable future. Furthermore, a wide range of Artificial Intelligence language tools are becoming more readily available to non-expert developers to tailor to specific tasks (as I indicated by my creation of a UFO sighting chatbot at the end of 2018 – see my thread "Robert 2.0" chatbot : World's first "robot ufologist"?? (Christmas present 2 of 3)).

While hugely better than a few years ago, automated transcripts currently remain far from perfect. They do not indicate who is speaking. They contain some mistakes, particularly in relation to spelling of names (which somewhat reduces the effectiveness of searches). The automated transcripts are currently certainly not yet good enough to completely replace listening to relevant material.

However, the searchable PDF transcripts are – I think – a useful additional tool for finding relevant material (and I think improvements are likely to continue – so this exercise could be repeated with improved transcripts periodically…).

In any event, I have uploaded text file copies of each transcript (in addition to the searchable PDF copy) so that the text file version could be edited by anyone wishing to create and share improved transcripts. However, I assume that the appetite to improve such transcripts will be limited, judging simply from the fact that few such human transcripts have previously been produced – with a few notable exceptions by, for example, Keith Basterfield, Giuliano Marinkovic and Joe Murgia.

The first batch of automated transcripts that I’ve produced totals over 2 million pages (actually currently over 2.25 million pages, but I prefer to round things down a bit to be consistent with my personal preference for understating matters rather than overhyping them…) and involves transcription of over 55,000 recording – averaging, say, 1 hour per recording. So, it would take a significant amount of time for volunteers to cover all this material and improve the automated transcripts. Frankly, I think that any of the potential further work outlined below (in Section H, Section I, Section N, Section O, Section P, Section Q, Section R, Section S, Section T and Section U) would involve a MUCH better cost/benefit ratio than manual improvement of the automated transcripts. Many of the steps in those sections could be taken with relatively little effort and could be spread among UFO researchers doing a bit of research they would be undertaking anyway – so the burden involved in contributing to a larger project would be almost negligible if spread among a few more people.

(I’ll probably carry on doing a substantial amount of the routine work myself anyway, but that routine work has – frankly – become a bit dull now that the more interesting challenges of developing/combining relevant tools have been dealt with…).

SECTION E : Using Youtube’s AI and playlists (plus crowdsourcing) for batch production of transcripts

Given that improved Artificial Intelligence software has massively improved the quality of the automated transcripts that can be generated in the last few years (with considerable grounds for being confident of continuing improvements for the foreseeable future), I wanted to experiment with batch production of transcripts. Creating one transcript at a time would be time consuming if tens of thousands of recordings are to be dealt with.

Once again, Youtube seemed to offer a few relatively simple possibilities and existing tools that could be connected and modified into a cohesive system for batch producing hundreds (or even thousands) of transcripts at a time. I haven’t seen any indication of anyone trying to do this in the past, but it seemed like an interesting challenge to develop such a system.

After a bit of wasted time experimenting with various possibilities, I found that the key to batch production of transcripts was to use Youtube’s playlist function.

For those unfamiliar with playlists, they can be used to automatically play a collection of videos or audio recordings one after another without having to select manually the next recording after one ends. But playlists can be used for far more than that…

Playlists for Youtube video can be created manually fairly quickly, or you can find existing playlists online or (interestingly) if you select an option on Youtube to play all the videos uploaded by a particular Youtube user then Youtube automatically creates a playlist which lists all of those videos.

Various tools already exist in relation to the creation and use of playlists.

For example, it is possible to download all the videos in a Youtube playlist just by entering the relevant playlist’s id code (visible in Youtube’s URL whenever the playlist is playing) for the playlist into various tools. Many of you are probably familiar with free tools for downloading an individual video from Youtube and similar (free and commercial) tools can be used to download all the videos/podcasts in a Youtube playlist – such as these ones:

ddownr.com...

loader.to...



Similarly, tools exist (such as www.williamsportwebdeveloper.com..." target="_blank" class="postlink" rel="nofollow">the “Youtube Favorites Back Up” tool developed by Robert S Robbins) which enable users to download a list of the URLs in a playlist, together with some other details (e.g. the video title and video description entered by the uploader). There is a free online version of that tool, albeit with a couple of significant limitations on its functions compared to the Windows application which Robert S Robbins sells for the price of 5 US dollars. I found that Windows Application incredibly useful in my UFO transcript project and think that the potential uses of that tool are much wider than its name suggests (particularly when that tool is combined with others). This appears to be an unusual case of a software developer modestly understating the usefulness of an application he has developed, rather than over-hyping it or over-charging for it…

Once a list of URLs is extracted from a Youtube playlist (using the “Youtube Favorites Back Up” tool or other software), that list of URLs can be fed into other tools e.g. to download some or all of them or (using the free Google2srt software) download the automated transcripts for each of the videos in that playlist as a SRT transcript file. (The SRT transcript files can be converted to .txt text files just by changing the file type – e.g. using the very useful free Bulk Rename Utility).

For the initial collection of a couple of million pages of transcripts, I have largely used existing playlists (particularly ones for all the videos uploaded by a particular user/podcaster). However, the next step would involve creating more playlists.

Adding one video at a time to a playlist is easy. However, it would take a long time for some of the larger potential future tasks using single additions. I need to look further into batch additions to a playlist and, in particular, methods of collaborating to create more extensive/comprehensive playlists. Initial work has been fairly promising, with some online discussions including:

(1) Tools for creating temporary Youtube playlists from a list of URLs;


(2) Tools for collaborating to create permanent Youtube playlist from a list of URLs – see DISCUSSION HERE and SAMPLE SPREADSHEET HERE.

SECTION F : Podcast Transcripts – UFO Podcasts on Youtube

Getting permission to upload transcripts of audio-visual material has been simplest in relation to podcasts and vodcasts. I’ve obtained permission to upload transcripts (and summary spreadsheets) of many of the most popular UFO podcasts.

For various technical reasons, I’ve concentrated thus far on podcasts which are already on Youtube. However, as detailed in the various sections towards the end of this post, there are numerous possibilities for extending this work.

SECTION F1 : Podcast transcripts - "Arcane Radio" (permission given to me by Lon Strickler)

This podcast can be found on Youtube HERE and on other platforms via its associated Facebook page HERE.

This podcast, hosted by Lon Strickler, describes itself as “exploring the unexplained - definitive paranormal and cryptid talk with your host Lon Strickler”.



Link to relevant folder containing transcripts and, at the top, summary spreadsheet



Direct link to relevant summary spreadsheet, PDF version



Guests on this show have included various researchers of bigfoot and ghosts, in addition to several UFO researchers (including Nick Redfern, Albert S Rosales, Stan Gordon and Ken Pfeifer).

Section F2 : Podcast transcripts - "Fade to Black" (permission given to me by Jimmy Church)

This podcast can be found on Youtube HERE and has an associated website HERE.

This podcast, hosted by Jimmy Church, describes itself as being “Bespoke radio for the masses”.



Link to relevant folder containing transcripts and, at the top, summary spreadsheet



Direct link to relevant summary spreadsheet, PDF version



Guests on this show have included many leading names in UFO research (such as Steven Greer, James Fox, Richard Dolan, John Greenewald, Richard Doty and Stanton Friedman and many others). The summary spreadsheet for this show is currently over 430 pages long.

Section F3 : Podcast transcripts - "Grimerica" (permission given to me by Graham Dunlop and Darren Grimes)

This podcast can be found on Youtube HERE and has an associated website HERE.

This podcast describes itself as being “A loose cannon podcast featuring casual conversations with whomever we deem interesting, often on the fringe, Like Coast to Coast But on Demand!”.



Link to relevant folder containing transcripts and, at the top, summary spreadsheet



Direct link to relevant summary spreadsheet, PDF version



Guests on this show have included researchers of free energy, ancient mysteries, Sasquatch and shamanism in addition to several UFO researchers (including Grant Cameron, Leslie Kean and Stanton Friedman).

Section F4 : Podcast transcripts - "Inception Radio Network" (permission given to me by Mike Lucas)

This podcast can be found on Youtube HERE and on other platforms via its associated Facebook page HERE.

This podcast describes itself as being “Voice of the Fringe Majority", which is “devoted to delivering the best in audio quality and content delivery of all that encompasses the UFO, Paranormal, Conspiracy, and Cryptozoology field through our copious radio talk shows”.



Link to relevant folder containing transcripts and, at the top, summary spreadsheet



Direct link to relevant summary spreadsheet, PDF version



Guests on this show have included researchers of a wide range of alternative topics, including “mothman” sightings, dream interpretation, what happens to our pets when they die, mind control and time machines - in addition to several UFO researchers and discussions about UFO news.

Section F5 : Podcast transcripts - “Inside the Outer Limits” (permission given to me by Chris Evers and Philip Mantle)

This podcast can be found on Youtube HERE and on other platforms via its associated Facebook page HERE.

This podcast describes itself as being about “UFOs, Ghosts and all things Paranormal”.



Link to relevant folder containing transcripts and, at the top, summary spreadsheet



Direct link to relevant summary spreadsheet, PDF version



Guests on this show have included mainly UFO researchers from the United Kingdom or those involved in researching sightings/incidents in Britain (such as Rendlesham). Past guests include Spyros Melaris, Brian Allan and Peter Robbins.

Section F6 : Podcast transcripts - "Mr UFO" (permission given to me by Tim Beckley)

This podcast can be found on Youtube HERE and has an associated website HERE.

This podcast describes itself as “featuring interviews and information from some of the top names in the field”.



Link to relevant folder containing transcripts and, at the top, summary spreadsheet

Direct link to relevant summary spreadsheet, PDF version



Guests on this show have included researchers of remote viewing, the occult, interdimensional beings. ghosts and lost civilizations, in addition to numerous UFO researchers (such as Greg Bishop, Scott Corrales and Kathleen Marden).

Section F7 : Podcast transcripts - "OBDM" (permission given to me by Mike and Joe)

The Our Big Dumb Mouth (“OBDM”) podcast can be found on Youtube HERE and has an associated Facebook page HERE.

This podcast describes itself as being “a live Conspiracy and Paranormal Talk Show. UFOs, Bigfoot, MK ULTRA”. Its host state “We have a sense of humor and hope you do too”.



Link to relevant folder containing transcripts and, at the top, summary spreadsheet



Direct link to relevant summary spreadsheet, PDF version



Mike and Joe have discussed a wide range of UFO topics, including discussion of Tom Delonge, disclosure and AATIP.

Section F8 : Podcast transcripts - "Open Minds Radio" (permission given to me by Alejandro Rojas)

This podcast can be found on Youtube HERE and has an associated website HERE.

This podcast describes itself as part of an organisation which is “focused on taking a serious look at credible information and the social impacts of belief and knowledge in regards to paranormal phenomena, in particular, the UFO phenomenon and the search for extraterrestrial intelligence”.



Link to relevant folder containing transcripts and, at the top, summary spreadsheet



Direct link to relevant summary spreadsheet, PDF version



Guests on this show have included numerous people discussing the USS Nimitz sightings (e.g. Kevin Day, Joe Murgia and David Beaty), AATIP (e.g. Dr Eric Davis, Dr John Alexander, Nick Pope and Danny Silva) and dozens of other UFO topics.

Section F9 : Podcast transcripts - "Paranormal Dimensions" (permission given to me by David Young)

This podcast can be found on Youtube HERE and has an associated Facebook page HERE.

This podcast is part of Paranormal UK Radio Network, which the describes itself as being “the BEST Paranormal Talk Radio in the UK!”.

Link to relevant folder containing transcripts and, at the top, summary spreadsheet



Direct link to relevant summary spreadsheet, PDF version



Guests on this show have mainly been researchers from the United Kingdom (such as Philip Mantle) and/or discussing UFO incidents and events in the United Kingdom (such as Rendlesham), but with some American researchers (e.g. Don Schmitt, Kathleen Marden and Dr Irena Scott).

Section F10 : Podcast transcripts - "Paranormal Mysteries" (permission given to me by Sacha Christie)

This podcast can be found on Youtube HERE and has an associated website HERE.

The host of this podcast, Sacha Christie, describes herself as being a “Radio show host, Investigator, Writer, Par Abnormal Instigator, Agenda Bender”



Link to relevant folder containing transcripts and, at the top, summary spreadsheet



Direct link to relevant summary spreadsheet, PDF version



Guests on this relatively new show have largely focused on Rendlesham and, in particular, Larry Warren’s credibility (or the lack thereof) and alien abduction research.