Extracting and animating data from the "UFO Video Captured By Homeland Security"

originally posted by: WeAre0ne
You would be done by now if you only did every second.

True. In fact, that was done last week.



Sadly, there are various problems with taking snap shots every second (or longer time frames) - as you can see if you go through the data frame by frame.

For example, the various displays update with different frequencies - so if you take any particular frame in isolation you may not understand that you are looking at data being displayed for various different times. So, you can't take all the data as relating to the situation at the exactly the same time.

Anyway, doing the data for every frame isn't really _that_ much more work than doing it for every second. (I can say this from experience of doing both).

a reply to: IsaacKoi

IssacKoi, do you have the most up-to-date Excel frame data available? Also, is there an up-to-date Google Sheet that you can give volunteers edit access to? It's best to use Google Sheet because we can all work together in real time. It also has revision history and rollback feature if someone decides to delete everything.

I ask because I am writing a program in c# that will take all the coordinates and convert them to a Google Earth KML Tour / Fly By. If you have data available for every 30 frames or every second, that would be great.

originally posted by: IsaacKoi
For example, the various displays update with different frequencies - so if you take any particular frame in isolation you may not understand that you are looking at data being displayed for various different times. So, you can't take all the data as relating to the situation at the exactly the same time.

I don't quite understand what you are saying here. The on-screen display data has a date and time. Example: 23AP2013 01:22:07. If you select only frames that have a unique date and time, you will get a snap shot every second.

In stead of:

23AP2013 01:22:07
23AP2013 01:22:07
23AP2013 01:22:07
23AP2013 01:22:08
23AP2013 01:22:08
23AP2013 01:22:08
23AP2013 01:22:09
23AP2013 01:22:09
23AP2013 01:22:09

You get this:

23AP2013 01:22:07
23AP2013 01:22:08
23AP2013 01:22:09

The data between each second is somewhat useless, so we can select all date times that are distinct, and drop all other frames. With that info alone we could interpolate between each second when animating or simulating the flight path and camera target, and the accuracy would be more than satisfactory.

--edit--

Are you trying to say that some values visible on the on-screen display have a delay? If so do you have documentation to back up that assertion? Is it a delay, or just a slow polling time?

Either way, only a video needs 30 frames per second to make it smooth. We don't need data from all 30 frames per second, we only need data every second. The rest is useless really.

originally posted by: WeAre0ne
The data between each second is somewhat useless, so we can select all date times that are distinct, and drop all other frames. With that info alone we could interpolate between each second when animating or simulating the flight path and camera target, and the accuracy would be more than satisfactory.

I've explained the problem with that approach (and why it is not "more than satisfactory") in my earlier reply to you.

If you disagree and want to do things per second (as was done last week), you are of course free to do things that way.

a reply to: IsaacKoi

Your explanation wasn't quite clear. Specifically this part:

"so if you take any particular frame in isolation you may not understand that you are looking at data being displayed for various different times"

I find it hard to believe that the on-screen data would not display accurate information in relation to the time stamp on that frame. Are you saying if I select any single frame, the data on the screen may not be correct? That doesn't make sense.

p.s. check your ATS private messages.

originally posted by: FederWBush
Can someone clarify to me frame 1721 AU number ? (
prrr.isaackoi.com...

In that frame, the value is changing from 24 to 25. Since it in the process of changing, I would put either value in. (I would probably use 25 for that particular frame).

The problem of changing values has been an issue for several people (including me). I posted a bit about this above, which I'll paste here for ease of reference.

originally posted by: IsaacKoi

originally posted by: kkrattiger
Please take a look at Frames 60 61 & 62. 61 being 02. This is seconds in Longitude (correct me if I'm wrong). It goes from 08 to 02 to 07. 7's can look like 2's but that is not the case here... Unless my eyes are playin tricks on me.

When a value changes, there are sometimes a frame or two where parts of BOTH the old and the new value are displayed. This means that digit that changes can be a blurred mess of two different numbers.

So, in the column you are working on (i.e. Column AU) I would insert 7 in frame 60 and 8 in frame 62 onwards. As for frame in which the change occurs (i.e. frame 61) you could almost toss a coin to decide which value to include. I'd probably go for the 7 on this one.

I would not agonise too much over the change-over frames - either value would be fine for the tiny fraction of a second represented by one frame. I've tended to scroll down a few frames (using the PDF version in Abobe Reader it is easy to scroll up and down) to make it clear what the value becomes and then choose whether the blurred number looks a bit closer to the old value or the new value. (The appearance of a 2 in frame 61 is basically almost all the way to the 7 that is being formed).

And from some reason I can't send you P,M IsaacKoi.

New members of ATS are automatically prevented from sending a PM. I think you have to have made a minimum number of posts first (20??).

originally posted by: WeAre0ne
I find it hard to believe that the on-screen data would not display accurate information in relation to the time stamp on that frame.

You are welcome to believe whatever you want.

However, if you scroll through a bit (any bit) of the PDFs I made available a few days ago, you will see within a few seconds of observation that the on-screen displays do change at different intervals - as I said above. Some of them, when they do update, change values in a discontinuous manner (i.e. they jump) - again showing that the values cannot be correct for EACH frame (resulting in the problems that I've explained with take any particular frame as displaying accurate information).

I'll go into this in much more detail if/when I post any threads about conclusions to be drawn from the data. First, I want to get/animate the data as accurately as possible.

originally posted by: WeAre0ne
Are you saying if I select any single frame, the data on the screen may not be correct? That doesn't make sense.

It may be that to you "that doesn't make sense" but it is fairly trivial to demonstrate it to be the actual factual position.

originally posted by: lovebeck
Are there detailed instructions on how to extract the data/add it to the spreadsheet? I do know a lot about photos, I can use spread sheets, I can use a computer and do data entry, etc. I would just like to take a closer look at the process before volunteering to help out. I'm a pretty smart cookie to boot. However, I'm a nurse and so not an IT professional, haha.

Sure. If you want me to clarify anything in the the instructions I'll paste below, just let me know here or by U2U.

The more people that do something constructive, the sooner we will finish without getting too bored.

originally posted by: IsaacKoi
Basically:
(1) Download the current (incomplete) Excel spreadsheet:
prrr.isaackoi.com...

(2) Pick an incomplete column and post here that you will complete that column. The columns that remain to be done or are currently incomplete are I, R, U, V, W, Z, AD, AH, AJ, AK, AL, AM, AN, AU and AY.

(3) Scroll through the frames and fill in the values. Using the PDF versions of the frames in Abobe Reader is probably the easiest way to quickly scroll through the frames. The values do not change in every frame so you can generally cut and paste (or use the Excel "fill" menu and the down option) to cut and paste a relevant value down a column until the frame where the value changes. The PDF versions of the frames are in two files, at:
prrr.isaackoi.com...
prrr.isaackoi.com...

I should warn you, having done quite a few of the columns myself already, that it is pretty boring work.

Challenge 2 should be more fun, but requires some expertise.

Just to update the position :
Columns A to H : COMPLETED FOR ALL 70727 FRAMES
Column I : High priority - Incomplete
Columns J to Q : COMPLETED FOR ALL 70727 FRAMES
Column R : Medium priority - Incomplete
Columns S to T : COMPLETED FOR ALL 70727 FRAMES
Columns U to W : Unnecessary / low priority. (Analogue displays)
Columns X to Y : COMPLETED FOR ALL 70727 FRAMES
Column Z : Unnecessary / low priority. (Analogue displays)
Columns AA to AC : COMPLETED FOR ALL 70727 FRAMES
Column AD : High priority - In progress - Excallibacca
Columns AE to AG : COMPLETED FOR ALL 70727 FRAMES
Column AH : High priority - In progress - Raymundoko
Column AI : COMPLETED FOR ALL 70727 FRAMES
Column AJ to AN : Medium priority - Incomplete
Columns AO to AT : COMPLETED FOR ALL 70727 FRAMES
Column AU : High priority – In progress - FederWBush
Columns AV to AX : COMPLETED FOR ALL 7027 FRAMES
Column AY : High priority - Incomplete
Column AX : COMPLETED FOR ALL 70727 FRAMES


(Incidentally, I _fully_ appreciate that dealing with some other issues is probably more fun than this boring bit, but I think it will provide the most solid foundation possible for the more fun bits and will ultimately save time by avoiding unnecessary errors and uncertainties).

a reply to: IsaacKoi

I don't need to scroll through a PDF, I wrote a program that flips through each frame image and attempts to OCR the data... I've seen the data. I am now writing a program to convert the data ive collected into a KML Google Earth Tour / FlyBy so I can recreate the view of the camera, and triangulate the exact position of the object in relation to the ground objects and landmarks.. I am then going to model this object and scene in 3D Studio Max, to perfect scale, including camera distortion.

This is not my first rodeo actually.

What you said doesn't make sense because I have extensive knowledge of on-screen data displays, I've created hardware and software that integrates with cameras to do so. Usually sensors are not all polling in synchronization, they usually poll asynchronously on their own threads, but they poll so fast (talking milliseconds) that they are always a good representation of what is being sensed at any time. These sensors usually update global variables in memory. When you go to draw the data on-screen for video, it reads all those variables and draws them to the back buffer, every frame, then displays the buffer when it is finished, rinse and repeat. This buffer swap is much slower rate than the sensor data variables are being updated. So at any given time, the information on the screen is accurate.

It seems like you are confusing one or two things. It seems you are implying the data on-screen is displaying values that are outdated, as if the sensors are being polled slower than once per second. Do you have the technical documentation to support this? Or perhaps you believe that because the data being displayed is simply not showing smallest unit of measurement?

Please, explain yourself further. I don't see a problem taking a snap shot every second to paint a good picture for triangulation. Anything else is useless and wont give you any more information. Sure I might miss some micro movements of the camera, but those are unnecessary for what I want to do.

originally posted by: WeAre0ne
It seems like you are confusing one or two things.

That's distinctly possible. I'm a lawyer, not a technical chap.

I look forward to seeing your data.

In the meantime, I'll be finishing the per frame spreadsheet. It really isn't much more work doing the spreadsheet for every frame rather than a snap shot per second.

Presumably this has all been covered by the OCR data collection you mention you've already done. Well done by the way on getting OCR to work on the frames. I'd tried the OCR tool in Adobe Acrobat and others in this thread have tried using more advanced OCR software.

a reply to: IsaacKoi

The trick was to apply filters to the frames to make the white text stand out to get OCR to work.

originally posted by: WeAre0ne
The trick was to apply filters to the frames to make the white text stand out to get OCR to work.

Does that mean that, in effect, you have completed the spreadsheet per frame anyway?

originally posted by: Mastronaut
I made a pastebin (can't upload it on the site)

Thanks. Interesting. KML files don't look that complicated. I think we can create an animation quite simply, just by adding "timestamp" or "timespan" data in the KML file:
developers.google.com...

(Obviously, some of the more technical chaps on ATS would probably have been able to do this in their sleep...).

a reply to: IsaacKoi

AU is about 60% done , I believe I'll finish it the day after tomorrow.
And I might not be able to have a internet connection in the next couple of days.
So you mind if i'll start another column immediately when i'm done with AU ? (AY might be a good place to continue with...)

originally posted by: FederWBush
AU is about 60% done , I believe I'll finish it the day after tomorrow.
And I might not be able to have a internet connection in the next couple of days.
So you mind if i'll start another column immediately when i'm done with AU ? (AY might be a good place to continue with...)

Great. I think we are nearly there.

One of the other PRRR members (on ATS as CardDown) has let me know that he has done quite a bit of Column AY, so I think the only other high priority is Column I - so I'll note him as working on AY and you as moving on to Column I.

I think that will be all the high priority columns done so we can then focus on animations and data extraction (i.e. the more fun stuff...). I've been looking into KML and think that it will be easier to animate the data than I first feared.

Just to update the position :
Columns A to H : COMPLETED FOR ALL 70727 FRAMES
Column I : High priority – In progress - FederWBush
Columns J to Q : COMPLETED FOR ALL 70727 FRAMES
Column R : Medium priority - Incomplete
Columns S to T : COMPLETED FOR ALL 70727 FRAMES
Columns U to W : Unnecessary / low priority. (Analogue displays)
Columns X to Y : COMPLETED FOR ALL 70727 FRAMES
Column Z : Unnecessary / low priority. (Analogue displays)
Columns AA to AC : COMPLETED FOR ALL 70727 FRAMES
Column AD : High priority - In progress - Excallibacca
Columns AE to AG : COMPLETED FOR ALL 70727 FRAMES
Column AH : High priority - In progress - Raymundoko
Column AI : COMPLETED FOR ALL 70727 FRAMES
Column AJ to AN : Medium priority - Incomplete
Columns AO to AT : COMPLETED FOR ALL 70727 FRAMES
Column AU : High priority – In progress - FederWBush
Columns AV to AX : COMPLETED FOR ALL 7027 FRAMES
Column AY : High priority - In progress - CardDown
Column AX : COMPLETED FOR ALL 70727 FRAMES

originally posted by: WeAre0ne

originally posted by: Uggielicious
Yeah, it did go over roads with automobiles and above buildings so I estimate its size to be between 10 and 15 feet in diameter. Nothing can be identified so pursuing information seems to be an empty effort.

How exactly did you arrive at that size estimate? A wild guess? Do you know the laws of perspective can make your thumb look the same size as the moon? Same with this object, you have no idea how far away the object is, so you have no idea the size it is.

With the information in this video, such as the type of camera, the amount of zoom, the GPS position of camera, and the GPS position of the "lookat" point or "target", which is all available. We can actually get a very accurate size of this object, and if it is the size of a balloon, well, there we go.

We can also get a confirmed speed of the object, which can tell us if its average speed of wind at those altitudes pushing a balloon, or we can determine if it is way too fast to be wind.

We can be more certain than just a wild guess.

You don't have an argument to respond to. My size estimate was a wild guess which I stand by. Knowing everything else that you proposed arrives at the same result as Drake's "equation": "The Drake Equation is used to estimate the number of communicating civilizations in the cosmos, or more simply put, the odds of finding intelligent life in the universe. N = The number of civilizations in the Milky Way galaxy whose electromagnetic emissions are detectable."

Drake had to start with zero, or 1 including earthlings, and he came up with a total of zero because you cannot have a result when there is no evidence for anything else than zero.

So we have a video of a fast moving object that doesn't waver that much and it could never be a balloon and it certainly does not resemble any human-made aerial craft. It could be one hell of a sophisticated drone. And coming up with answers to your guesstimates still leaves one without any usable details.

Of more interest to me would be a transcript of the dialogue between involved parties, such as who saw it first, what instructions were given to the cameraperson who jumped on it real fast and did a great job of tracking it, almost keeping it centered the whole time.

A similar video can be found on the net taken from a shuttle, I think, where a hauling UFO is seen above earth and is also tracked for a long time. That more interesting video did not garner the seemingly interest this video has.

Column AY is complete. It won't be long now!
I'm eager to see the ways this database can be used, especially in the 3-D visualization of the video.

originally posted by: Uggielicious
You don't have an argument to respond to. My size estimate was a wild guess which I stand by.

You stand by a wild guess..... Good one.

originally posted by: Uggielicious
So we have a video of a fast moving object that doesn't waver that much and it could never be a balloon and it certainly does not resemble any human-made aerial craft. It could be one hell of a sophisticated drone. And coming up with answers to your guesstimates still leaves one without any usable details.

Everything you said has not been confirmed. A video of a fast moving object? Just from the basic observations and triangulation I've done so far, that is not entirely true. So far, the only fast moving object was the aircraft filming. They seemed to be circling around the object as it floated over the city. Because of the laws of perspective, and the parallax effect, the object simply appeared to be moving fast and not actually moving fast, because the background was moving faster than foreground. It is no faster than wind.

As for the size, we may just find out its 1 foot wide....

Here is the thing about this video... the aircraft was circling it. That means we can pull data from 3 or more different angles, which means we can triangulate its exact position. We can know exactly how far away it is. How slow it was going... And know the size of the object with a very small margin of error.

This can tell us a lot about the object.

I believe life exists outside of Earth... but I'm not going to mistake a balloon in the wind for alien space ships. I'm smarter than that.

a reply to: IsaacKoi

Yeah, its all true.

a reply to: WeAre0ne

Totally... Agree