The (Raw) State of Space
I was looking at a moon. Sitting in my lounge just outside of Central London, I felt a totally unjustified sense of discovery. You see this wasn’t The Moon. This was another far more alien (albeit still greyly familiar) moon. This was Ariel. And she was spectacular.
Several hours earlier I had become intrigued in the concept of raw historical space imagery, thanks to this fantastic post (and subseqent comments) on Reddit by user mmokkp. In that post, mmokkp had created a seemingly unparalleled high-resolution image of Callisto through a process of downloading, converting, cleaning, and finally stitching multiple raw images — taken as part of the Voyager mission — together into a seamless mosaic. The resultant image is breathtaking, but the process involved fascinated me even more.
The simple fact that this raw data is freely available was something that hadn’t even crossed my mind. On searching further, I quickly realised that just below the surface of the curated space imagery (which has repeatedly and seemingly without-fail fascinated me for as long as I can remember), lies an almost impossibly vast body of raw image, telemetry, and sensor data. Gigabytes upon gigabytes, spanning decades of space exploration, that I’d never once thought to seek out.
A few hastily-written Python scripts later, and a small subset of that data — the Ariel images taken by Voyager 2 nearly three decades earlier — was ready to explore. Advancing from image to image, a small, out-of-focus grey blob danced around my laptop screen amidst the noisy, often-corrupted blackness. Then suddenly it was just there. Downloaded, converted, and clearly definable. I was looking at a moon.
Whilst a step-by-step, code-focussed look at how I obtained the Ariel images will be the subject of an upcoming article, here I want to discuss two things: first, the major difference between the images described above and the space images we are more accustomed to seeing on the web; and second, where exactly you can go to find such raw data.
The Reasons for (and against) Raw Imagery
The last ten years of my life have been spent in the Visual Effects industry where, coincidentally, raw data also plays an exceptionally important role. Broadly speaking, in Visual Effects there are two main reasons for using raw image files over other formats :
Using a raw image gives an artist access to a larger range of visual information, which in turn improves their ability to manipulate that image accurately.
In VFX this often meant ensuring that detail was maintained in the darkest and brightest sections of an image; be it a photo captured on a low-lit set, or a computer-generated image of sunlight gleaming off of a robot’s shoulder. In both cases capturing in a raw format gives the final artist the greatest potential for creative change.
Raw images often contain extra non-visual information that you potentially lose once you convert to a non-raw format.
This extra information (often termed ‘metadata’) can range from details on a camera’s lens and shooting settings, to time and location data — no-doubt familiar to those of you who have viewed or edited EXIF information for your own photos.
Depending on the raw format used, there might be few limitations on what types of metadata the resultant files can contain, or on how that extra information might be used by the final artist.
However, there is also a strong case for not always using raw files in Visual Effects work. Larger file-size and accessibility issues aside, perhaps the biggest reason of all is one of presentation — oftentimes the extra data provided by a raw format is simply unnecessary in a final image, and presenting the raw, unedited state of the data might actually detract from what you are trying to convey.
Perhaps unsurprisingly, the same arguments apply when it comes to space imagery.
Just as the images you see in the cinema have been converted from their raw format and manipulated to elicit the director’s desired emotional response, the space images we commonly see on the internet are indeed also manipulated; oftentimes constructed from multiple separate raw-format images, taken with multiple separate filters, and then recombined, coloured and graded to stunning effect.
Of course, saying that these processed space images are in any way ‘incorrect’ would be the same as complaining that you’re not seeing the greenscreens when you go to see the next Marvel movie. A presentational choice was made, and the resultant images are the reason myself and millions of others are so passionate about the topic of space. But likewise, you wouldn’t use those same images for further analysis.
Simply put, there is a time and a place for both the raw data, and the edited, processed imagery. Although if you intend to manipulate, examine or process that image data in any way, then using a raw format is essential.
Raw Data Sources
On my trip down the rabbit-hole I’ve started to compile a list of sites that provide space imagery in a ‘raw’ form. Whilst it is in no-way designed as an exhaustive collection of every source in existance, I do intend to update the list as and when I find missing/incorrect information (or when said missing/incorrect information is kindly pointed out to me).
Before I present the list however, I feel I should briefly draw attention to the fact that all of this is only possible thanks to the fact that certain organisations (NASA, JPL, ESA, & SETI to name but-a-few) not only achieve the momentous task of capturing these images in the first place, but then choose to make them available to a global audience, in both raw and processed forms — a staggeringly momentous task which I am incredibly grateful for.
- It’s worth mentioning that sometimes just getting the data into a usable format can be a long, technically-challenging process — some users over on the Unmanned Spaceflight forums work on single datasets for months before even beginning to stitch and grade imagery. Whilst this article is intended to serve as just a brief introduction, future articles will hopefully begin to at least scrape the surface of this demanding workflow.
- There are of course a multitude of ‘raw’ image types that I could cover here, however for more in-depth information on the different raw formats available I can highly recommend Bill Green’s fantastic post ‘Finding and Using Space Data’ over at The Planetary Society.
- For further reading on the benefit of using raw archived data over converted formats, Emily Lakdawalla’s ‘Getting to the real science image data: It’s not that hard!” is an excellent, thouroughly convincing article.
- Yes, that ‘construction’ image really is over-simplified. I’ll be attempting to recreate a much more representative version that hopefully does the process a little more justice in an upcoming article.
- Finally please go and visit Planetary Bot. I think I love it a little…