Of Cosmic Rays, UFO, and SOHO

reply to post by AnthraAndromda

For now I'd just rather watch Phage carve up your posts. Give him some more to work with.

reply to post by DenyObfuscation


You, like me, will have to wait for the software.
Monday wil come soon enough, and we will have numbers Phage can't dispute.

reply to post by AnthraAndromda

Converting this to something that makes a bit more sense; we have one chance in 11,806,375.44 for one of our special vectors.

And yet we see these "special vectors" fairly often. So there would seem to be something wrong with your assumptions about the probabilities of these "special vectors". Unless, as you claim, they are not cosmic rays.

I'd like you to consider this image. Notice that we see a cosmic ray whose trace fades and brightens. Something which you say indicates that it cannot be a cosmic ray.

Now, as I've explained before, these images are highly processed. They are also cropped, removing the outer margins. But (as I've explained before) the raw images are available. Here is a section of the raw image. There is something interesting about it. Notice how the cosmic ray extends beyond the portion of the sensor which is exposed to the optical path of the telescope. Can you explain how that can happen if this anything other than a cosmic ray striking the CCD? If it is a cosmic ray and if these vectors are so "special" (one in 11,806,375.44) why do we see them so often? Could there be a problem with your assumptions about the probabilities?

Here is the link for the raw FITS file.
sharpp.nrl.navy.mil...

Here's another "special vector". This time behind the C2 occulter. How can something outside the imager show up behind something that blocks the light of the Sun? 7/20/12 07:05
If cosmic rays with "special vectors" are so rare (one in 11,806,375.44), why do we see them so often?

sharpp.nrl.navy.mil...
soho.esac.esa.int...

reply to post by Phage


You know as well as I the answers to those (stuid) questions. And, those "special vectrs"? You seem unable to identify the, but you have shown none in you images so far.

Anyway, something a bit more important:

At what level of "brightness" should we ake the distinction between Cosmic Ray and "background" (substrate noise I was thinking at least 10% above "typical" background; What do ya think; enough? More? Less?

reply to post by AnthraAndromda

You know as well as I the answers to those (stuid) questions.

Yes, I know the answers. They are cosmic rays. All of the streaks in all of the images are cosmic ray tracks.

And, those "special vectrs"? You seem unable to identify the, but you have shown none in you images so far.

You're right. Based on your definition I can't identify anything "special" in any of my examples or yours. You calculated (let's leave the validity of your calculation out of it) the probability of any particular vector:

These seem like rather slim odds, but remember, this isn't the probability that we will see a cosmic ray, but rather, that that cosmic ray will have a specific / required vector.

You calculated those odds to be

one chance in 11,806,375.44 for one of our special vectors.

What makes one vector "special"? Is a "special vector" one that has a track adjacent to it? What makes it more special than one with a track that is on the other side of the frame but parallel or perpendicular to it? What makes it more special than one that is crossing it? Are you saying that cosmic ray tracks are extremely unlikely to do any of these things? Or do you get to define that specialness? It's special because it looks like a spaceship. "It's special because I think it's special". That's circular logic. Now, if you had showed me two examples of specialness which were exactly the same you might have something, but you haven't.

From the OP.

The cosmic ray labeled cosmic ray 2 while appearing to be an extension of the longer one likely isn't.

Tell me, was my example from C2 a single cosmic ray which demonstrated something unlikely or was it two very special cosmic rays?

At what level of "brightness" should we ake the distinction between Cosmic Ray and "background" (substrate noise I was thinking at least 10% above "typical" background; What do ya think; enough? More? Less?

The brightness would depend on the energy carried by the particle. Are you claiming it's not a cosmic ray and it's not a "special vector"? Are you claiming it's noise? Noise that just happens to form a straight line. Special noise.

reply to post by Phage


The question about "brightness" was in regard to the aplication I'm creating over the weekend.

So, I take it, 10% is good with you.

The trace you are asking about, while I haven't had a close lok, appears to be two cosmic rays. And, your "fits" data, for the time being, is usless; I have no software that can handle that forat.

reply to post by AnthraAndromda

The trace you are asking about, while I haven't had a close lok, appears to be two cosmic rays.

Two cosmic rays on the same vector. What are the odds?

The question about "brightness" was in regard to the aplication I'm creating over the weekend.

So, I take it, 10% is good with you.

I don't think you can simplify to that extent. The "background" values vary a great deal due to the variations in the brightness of the corona. But if you aren't using the FITS files your results may not mean much if you are relying on brightness values. The FITS files are the raw data as uploaded by the spacecraft. As explained, the "browse" images have been heavily processed to bring out details of the corona . All of the original brightness values have been altered according to the background subtraction mask and the dynamic range constraints of JPG compression.

Originally posted by Phage
reply to post by AnthraAndromda

 

The trace you are asking about, while I haven't had a close lok, appears to be two cosmic rays.

Two cosmic rays on the same vector. What are the odds?

They're NOT on the same vector, nor do they apear to intersect: odds ... about 2 in 11 million.

The question about "brightness" was in regard to the aplication I'm creating over the weekend.

So, I take it, 10% is good with you.

I don't think you can simplify to that extent. The "background" values vary a great deal due to the variations in the brightness of the corona. But if you aren't using the FITS files your results may not mean much if you are relying on brightness values. The FITS files are the raw data as uploaded by the spacecraft. As explained, the "browse" images have been heavily processed to bring out details of the corona . All of the original brightness values have been altered according to the background subtraction mask and the dynamic range constraints of JPG compression.

Yes, I know about that. So, filtered and compressed. As far as "Dynamic range" goes, JPG should do just fine for early work. But then, I get impression that no atter what results I find, you will attept to reject them.

Ya know, I just might an algorithm to find the point at which we call it a"cosmic ray". You do know that the only way to identify a cosmic ray using any file formatis to "find" that "brightness" point where it is presumed to be a cosmic ray. The fact that there are filters, masks, and some unknown degree of compression, while, inconvenient, are not that "big a deal". I'll let you consult Hermes on that.

reply to post by AnthraAndromda

But then, I get impression that no atter what results I find, you will attept to reject them.

That's a little premature. I don't have any idea what it is you think you're going to accomplish. You asked my about brightness values and I told you something about them.

odds ... about 2 in 11 million.

Wow. Pretty long odds. Are these following those same odds? Oh, look and there's an "intersection" if you extent the vectors in the first one. Are these not cosmic rays too?


Extent these vectors and what do you get? Intersections. Are these not cosmic rays?

This one is tricky. What do you think? Five rays on two intersecting tracks? Not cosmic rays?



There is something wrong with your probability calcs. Here's another thing (besides diregarding flux). You are actually not dealing with 3 dimensions at all, you are dealing with a 2 dimensional image. Sure there is third dimension involved but it has no real effect on the vectors seen in the images. All it would really do is affect the length of the track. But since there are other factors that also affect that, it's pointless to consider it.

reply to post by Phage

You are actually not dealing with 3 dimensions at all, you are dealing with a 2 dimensional image.

Actually, we are dealing with 3 dimensions. That third dimensin you want t throw away; determines the third diensin of the path through the device. Depending on this third angle we might see a cosmic ray pass from one pixel to another. It can also give us clues as to "when" penetration through the depleteion region occurs. So, this "third" angle becomes rather important.

Your 5 ray iage; 5 cosmic rays on 5 different vectors, and, while it may appear that way, they probably would not intersect.

How many different kinds / types of cosmic ray trace have you identified? Think you can tell what kind of particle made it? I think there is enugh information in a JPG to determine this, though, sadly, nt that damn third angle.

Originally posted by Phage
reply to post by AnthraAndromda

 

Now, as I've explained before, these images are highly processed. They are also cropped, removing the outer margins. But (as I've explained before) the raw images are available. Here is a section of the raw image. There is something interesting about it.

Notice how the cosmic ray extends beyond the portion of the sensor which is exposed to the optical path of the telescope. Can you explain how that can happen if this anything other than a cosmic ray striking the CCD? If it is a cosmic ray and if these vectors are so "special" (one in 11,806,375.44) why do we see them so often? Could there be a problem with your assumptions about the probabilities?

Here is the link for the raw FITS file.
sharpp.nrl.navy.mil...

I think we have a problem. In the image you referenced above, and provided a link to sees a bit odd.



I had some time so I downlooaded a viewer (gimp) that was suggested on the FITS site. Upon loading the file I noticed; it doesn't look anything like the image you provided. What's up with that Did you spec the wrong file?

Also.
If this is what we can expect from the FITS images, then I think we may be better off using the JPG. It appears that the only processing going on is to improve contrast, and provide some color. It is unfortunate that these FITS files don't seem to contain much useful image data.

Initial tests of my application indicate that during the Proton Storm there were around 85,000 strikes during the exposure period / frame. This works out to a cosmic ray probability of: 0.079. This has the effect of altering the original probability from 1 in 11 million to 1 in 110 million.

I havent looked at "normal" days much, but the cosmic ray count deops from 80,000's to 20,000's - 30,000's. This is rather significant and will further damage the probabilities.

After several hours of studying the FTS iages and files. I find that they are little more than perhaps a glofified GIF. These images are black and white, native resolution of the sensor, with 256 levels of gray.

It seems that with all the contrast enhancement, colorizing that is being done, the JG images may be better for Phage's argument. Many of the "cosmic rays" we see that are so bright, in the raw data / iage, they are barely above ambient, and might not be cosmic rays at all, but rather artifacts caused by an obsolete system.

If we were to attempt to "detect" cosmic rays on the original raw images, we are actually not too likely to find much. I will have to check many more images, but, if what I've seen so far is typical, then we will prolly have to go with the JPG's.

reply to post by Phage

I don't think you can simplify to that extent. The "background" values vary a great deal due to the variations in the brightness of the corona. But if you aren't using the FITS files your results may not mean much if you are relying on brightness values. The FITS files are the raw data as uploaded by the spacecraft. As explained, the "browse" images have been heavily processed to bring out details of the corona . All of the original brightness values have been altered according to the background subtraction mask and the dynamic range constraints of JPG compression.

Simplifing like this, in the way that I have, is the only way I can see that we may be able to extract the data we want. If you have a better way, perhaps we can try that too.

On these "FTS" files. Where exactally did you get them? I've looked at the Navy site for this, and don't seem to be able to duplicate your search.I can get all kind of result, but, so far, no data like those two FTS files you linked. And, even more interesting is the fact that when I open one of these FTS files you linked; I seem to get a different image. So that kind of beggs the question; "How much did you process that image before you posted your thumbnail?" If I can't get known valid images other than the JPGs, then we will have no choice but to use them.

I will post udated probabilities later today. However, please expect the probabilities to become less by orders of magnitude. Just in reliminary results the probability of a "given" cosmic ray, during that Proton Storm, has degraded from 1 chance in 11 million to 1 chance in 110 million.

During the Proton Storm the probability of a given cosmic ray was on the order of 0.079. The average number of cosmic rays was on the order of 83,000 per frame. On quieter days, "normal" days (as oposed to quiet days), cosmic ray "hits" average around 15,000 to about 30,000, this is a fraction of the Storm's "density", theprobabilities will degrade by at least another order of magnitued.

I know you are wanting to throw proability away, and say that ALL these "hits" are cosmic rays. What you fail to understand is that probability is a well proven and quite useful branch of Math. It is used to support a rather large industry that spans the globe. It also lends a bit of support to science.

reply to post by AnthraAndromda

After several hours of studying the FTS iages and files. I find that they are little more than perhaps a glofified GIF. These images are black and white, native resolution of the sensor, with 256 levels of gray.

Once again. The FITS files are the raw data uploaded by the spacecraft. They are uncompressed and unaltered. The images are greyscaled, you can make them red and green if you wish. You think there is some advantage to going beyond the native resolution of the sensor? The dynamic range typically varies from levels of about 500 to over 16,000, that is what makes them so good for analyzing the corona and CMEs.

It seems that with all the contrast enhancement, colorizing that is being done, the JG images may be better for Phage's argument.

As explained, it is not "contrast adjustment" which is performed on the browse images. It is background subtraction. I'll stick with the raw data thank you, even though it is not quite as convenient as the processed images.

if what I've seen so far is typical, then we will prolly have to go with the JPG's.

Perfect. Reject raw data in favor of altered data, the nature of which processing you don't understand.

Ok, I tried to "lead" you to thinking that just maybe there was a problem with your calculations. I tried to get you to apply some science in showing you that what you claim is extremely unusual doesn't seem to be. But you didn't get it so here it is. You screwed up big time.

First, the odds you “calculated” would have been (if it had been done correctly) to exactly duplicate a particular track. But you only specified that the extended azimuths of the tracks should “intersect” in order to be considered "special". This leaves a wide range of possibilities and casts the odds in an entirely different light (especially when they are calculated correctly).

Second, and most important, we are dealing with two planes; the “horizontal” (xy plane), represented by the CCD array, and the “vertical” (xz plane). While it takes three Cartesian coordinates to define a point in space, it only takes two radial values (in degrees) to define a vector. You are using radial values but you include an extraneous factor. There are 360 degrees of direction available in the vertical plane and 360 degrees available in the horizontal. For example, a vector would be described as having an elevation of 30º and an azimuth of 60º. So, by correcting this error we have already greatly improved the odds. The probability of any particular vector (elevation and azimuth) occurring at any given time is actually 0.0000078 (0.0028 x 0.0028), or 1:128,205. But we aren’t looking for just any vector. We are looking for a vector which; a) produces a track on the CCD and b) has a certain “special” relationship to another track on the CCD.

Now, as you point out, in order to create a track on the CCD the particle must strike the CCD obliquely. To calculate that angle we need to know two things; the length of the track and the thickness of the active layer of the CCD. Based on a thickness of 60 microns (typical of front illuminated devices), a track 32 pixels long (at 21 microns each) gives us an elevation of 5º and 55 pixels gives us 3º. So, since we have three discrete options within this range (3, 4, and 5 degrees) the probabilities of such a track (between 32 and 55 pixels in length) being produced is 0.008 (3/360). This corresponds quite well with a statistical analysis of actual data shown in Fig. 5 here:
www.dlr.de...


Now what about those “special” vectors? Perhaps we should call them azimuths since we are working only in the horizontal plane for now. It seems that you are calling intersecting azimuths “special”. This would yet another error. We have two tracks. The azimuth for each can be any of 360 degrees but there is only one radian which will not intersect the “first” track (the same radian as that track). So the odds of an intersection on the xy plane are 179/180 (the original direction of the particle doesn’t seem to matter, does it? We’re just looking at the now static tracks relative to each other.) So now we have a probability of 0.00792 (0.99 for the azimuth x 0.008 for the track existing at all) or 1:126 for two “intersecting” tracks to occur.

But those are the odds if there were only 2 cosmic ray strikes on the CCD in any given frame. So now let’s look at the matter of flux. I know you don’t like it but it has to be considered. We are looking at an exposure time of about 19 seconds for these images (it varies and can be seen in the FITS file for each image). During that period of time there will be multiple cosmic ray strikes on the CCD. How many strikes? Well it seems that cosmic rays in space have a nominal count of about 4/cm2/sec (I gave an erroneous figure previously). www.rssd.esa.int...
Our CCD has an area of 4.6 cm2 so that means we can expect about 350 strikes per image. We are “rolling the dice” 350 times in each image. Since we are using 2 of those strikes we have 175 chances to get a particular pair. We end up with a probability of 1.39. And indeed, it is not at all difficult to find images with at least one pair of “intersecting” tracks.

reply to post by AnthraAndromda

On these "FTS" files. Where exactally did you get them? I've looked at the Navy site for this, and don't seem to be able to duplicate your search.

Perhaps you don't know how to do it right. Not surprising.
sharpp.nrl.navy.mil...

And, even more interesting is the fact that when I open one of these FTS files you linked; I seem to get a different image.

Which one?

So that kind of beggs the question; "How much did you process that image before you posted your thumbnail?

I did nothing to the thumbnail image except convert the FITS file to a TIF so it could be uploaded to ATS.

I know you are wanting to throw proability away, and say that ALL these "hits" are cosmic rays. What you fail to understand is that probability is a well proven and quite useful branch of Math.

Yes it is. When correctly applied.

BTW, it would be nice if you provided the sources of the images you used. It's considered polite.

Originally posted by Phage
reply to post by AnthraAndromda

 

Perhaps you don't know how to do it right. Not surprising.
sharpp.nrl.navy.mil...

Yes, Phage, I have the URL. The issue is that the search parameters are not properly defined, and, I can find no image dats that "looks" anything like what you provided.

And, even more interesting is the fact that when I open one of these FTS files you linked; I seem to get a different image.

Which one?

All of them. Look a few posts above and you will see one of your thumbnails, and the rendering of the actual data in the corresponding FTS. They not anything alike.

Now, I was able to bring up some of the detail in one of your thumbnails, and the NASA JPG. But, that was after removing noise, and perforing a radical contrast enhancement ... several times.

All that I can say about the FTS is that your thumbnail "seems" to contain different data than the original.

I did nothing to the thumbnail image except convert the FITS file to a TIF so it could be uploaded to ATS.

And what application did you use to convert the FTS?

BTW, it would be nice if you provided the sources of the images you used. It's considered polite.

You know as well as everyone; all my images originated at NASA. That is a selfevident fact. And, when it comes to computers; I'm very old school, no more than is required! Basically, I don't care about polite, I care about the data, and its analysis (we'll address that later).

reply to post by Phage

You think there is some advantage to going beyond the native resolution of the sensor? The dynamic range typically varies from levels of about 500 to over 16,000, that is what makes them so good for analyzing the corona and CMEs.

No I did not say anything about going beyond the resolution of the sensor. That actually would require replacement of the device, though at 1 megapixel, perhaps it should be.

Dynamic range of what? The hardware out there will handle 256 levels of gray, that's it.

As explained, it is not "contrast adjustment" which is performed on the browse images. It is background subtraction. I'll stick with the raw data thank you, even though it is not quite as convenient as the processed images.

No it is not just contrast , there is a good deal of data manipulation going on. Your "background subtraction", if it's what I'm thinking of, is an automatic process performed by the CCD to eliminate any substrate noise it may have. Course that isn't really necessary given its operating temp. So what do ya mean, really when you say "background subtraction"?

As for the FTS files, I'd be happy to use the, they would require heavy processing, using tools that aren't available, but, I guess I can create the tools.

The problem is, Phage, after I've created the tools to bring up those seriously weak "signals", I will have done as much to the data as NASA has already done with their JPG's. I also doubt that there will be enough "added" data to make a difference. So, in the end we will only have spent a lot of time producing what we already have.

Perfect. Reject raw data in favor of altered data, the nature of which processing you don't understand.

Oh, but I do understand the nature of that processing. It brings up the impossibly weak "signals" so that we can work with them. You shouldn't argue, using the JPG's will only help your failing position.

But, hey, if you really want to use the "raw" data, we can.

I think I should apologize for confusing you, I sort of take "vector" and "path" as synonymous, I guess for you they are not. So, If I inadvertently say "vector" please know that I mean "path"

Second, and most important, we are dealing with two planes; the “horizontal” (xy plane), represented by the CCD array, and the “vertical” (xz plane). While it takes three Cartesian coordinates to define a point in space, it only takes two radial values (in degrees) to define a vector. You are using radial values but you include an extraneous factor. There are 360 degrees of direction available in the vertical plane and 360 degrees available in the horizontal. For example, a vector would be described as having an elevation of 30º and an azimuth of 60º. So, by correcting this error we have already greatly improved the odds. The probability of any particular vector (elevation and azimuth) occurring at any given time is actually 0.0000078 (0.0028 x 0.0028), or 1:128,205. But we aren’t looking for just any vector. We are looking for a vector which; a) produces a track on the CCD and b) has a certain “special” relationship to another track on the CCD.

Yes, you are correct here. Allow me to adjust the probability: 0.00000392 or 1:255,102.04 You will notice that my calculation is different from yours; cosmic rays must interact with the sensing portion of the device to be recorded, thus those entering from the back aren't "seen" as they do not reach the sensing areas. Thus lower probability.

Now, as you point out, in order to create a track on the CCD the particle must strike the CCD obliquely. To calculate that angle we need to know two things; the length of the track and the thickness of the active layer of the CCD. Based on a thickness of 60 microns (typical of front illuminated devices), a track 32 pixels long (at 21 microns each) gives us an elevation of 5º and 55 pixels gives us 3º. So, since we have three discrete options within this range (3, 4, and 5 degrees) the probabilities of such a track (between 32 and 55 pixels in length) being produced is 0.008 (3/360). This corresponds quite well with a statistical analysis of actual data shown in Fig. 5 here:
www.dlr.de...

60 microns!?! Seriously? Soho isn't that old. A modern device has an active layer more on the order of 0.2 to 0.3 nanometers, much less than you 60 microns (60,000nm). And, yet, the probability of any of these paths is still the same as for all other paths.I suspect your numbers are reasonably accurate, given the huge error. However, your estimation of the probability is wholly inaccurate. The probability of a cosmic ray having the "correct" path remains the same 1:255,102.04, and we haven't factored in the probability of a cosmic ray yet.

continued

reply to post by Phage

Now what about those “special” vectors? Perhaps we should call them azimuths since we are working only in the horizontal plane for now. It seems that you are calling intersecting azimuths “special”. This would yet another error. We have two tracks. The azimuth for each can be any of 360 degrees but there is only one radian which will not intersect the “first” track (the same radian as that track).

Please show mechanics and logic.

Actually "vectors" and "azimuths" are both incorrect, we shall call them "paths". And no we are not working on a single plane.

So yes we have two "tracks" each with the sae probability, which hasn't changed from it's original 1:255,102.04 (your wee song and dance was mostly deleted, as it is wholly wrong), and as I said, we haven't included probability for cosmic rays. You like the term "flus" which is really just anther way of quantizing the probability. In the case of your Proton Storm probabilities were rather high (as they should be); at around 0.079. This value was found by "counting" cosmic ray strikes on 86 SOHO images. Giving us a probability of: 0.00000030968 or 1:3,229,139.757

During more normal times this probability drops dramatically: 0.024. The probability on a normal day becomes: 0.000000098 or 1:10,204,081.632. It shuld be noted; this is for 1 cosmic ray The probability for the second: 104,123,281,965,847.5635.

So, while the absolute values here may have changed it; hasn't helped. Instead of a probability of 1:139 trillion we now hav 1:104 trillion.

We end up with a probability of 1.39. And indeed, it is not at all difficult to find images with at least one pair of “intersecting” tracks.

Now this is rich! A "probability" greater than 1. You are aware that it is not possible to have probabilities greater than 1, right?

The best program i've found to deal with astronomical FITS images is DS9.
It handles FITS natively and it's coded from the start to analyze astronomical data.
I'd recommend DS9 over photoshop anytime when dealing with FITS.
Hope it helps.