Russell Kirsch helped create square pixels, now he wants to kill them off
By Vlad Savov 
posted June 30th 2010 5:23AM
posted June 30th 2010 5:23AM
According to old Rus, squares were just the logical solution at the time, but now that we can splash bits and bytes around with reckless abandon, he's come up with a new algorithm to smooth images beyond what's possible with simple squares. His new idea inserts 6 x 6 masks where there once was just one pixel, with adaptive calculations making for a more realistic representation of the underlying optical data. The sample above shows what improvements this new technique can deliver, with Russell's son doing the posing once more -- you'll find his decidedly younger visage in the 176 x 176 proto-pic after the break.
Ok...
Only problem is everybody can imagine what pixels are - those other "things" - somehow I can't imagine them, but the difference does look good.
I wonder if that could be used to improve today's JPEG compression?
And on the other hand - how can you actually record an image in any way other than by "dots" - and squares are the easiest to manage.
still looks like a bunch of squares to me, he's just increased the resolution of the image with a nifty upscaling algorithm
@mrqs Exactly, the "pixels" are still square blocks, they're just a lot smaller than before! Maybe we've missed something...
@TimmyRaa
And, the smaller pixels are still arranged in bigger, 6x6 blocks, which are effectively still pixels that can't adapt their boundaries. you can see the jagginess really clearly where his ear canal meets the left edge of his ear lobe. the best application will be in high-compression, low-quality images, like on a cell phone. i probably wouldn't want this on a camera.
@TimmyRaa
They aren't just smaller pixels, I think there is meant to be some element of shape info with each new 'pixel' unit. e.g. the diagonal lines you see as part of the ear will always be diagonal. Yes, at the moment, you could just increase the pixel count by 1:36 to get his result right now. With improved algorithms and screen tech however, his low res pictures could be displayed better in the future without the traditional concept of upscaling.
@in5ane
that sounds to me more like a compression algorithm rather than improving image quality
you can represent an image in many fancy ways in binary, but at the end of the day the existing display technologies know only pixels (unless you wanna bring the crt back)
ie. i don't think this is a software issue
@mrqs
If you want to nit pick, everything is compression until it gets to the screen... :)
Ok, put it this way. Take a normal image, and zoom in until one pixel fills the entire screen. Your screen will be one flat colour, no matter how big the resolution of the original image.
Take a Kirsch image, zoom in to one of his pixels, and there will still be detail even at this level.
@in5ane his algorithm provides you with just more data per "pixel".
or rather, you'd provide it with more data per pixel and then you'd have these sub pixels that you wouldn't have total control over. so the data that you'd have to feed to the monitor wouldn't be so big.
however, thats an imaginary problem, the problem lies more in the display manufacturing side.
[b]
so yeah you could basically emulate this thing by feeding your lcd 640x480 and letting it smooth the picture to 1920x1200. now who in their right mind would do that when they could just feed the monitor the proper 1920x1200 -or more, if the displays could take it.
@mrqs If I understand correctly what he's doing, here is a further description that might help fill in what people are missing:
First, the important point, which hopefully helps emphasize that this really is pretty impressive: Those two images are from the same source data. Or, to look at it another way, the second image is an alternate rendering of the first image.
Basically what he has come up with is a different way of rendering that source data to give less pixelated results. Note that this approach is independent of image compression techniques, and so could be used in addition to that. But it would require a rendering environment that can render dots smaller than the pixels in the image. That's not that uncommon a scenario... images stretched on high resolution displays, or printed (where the DPI of the printer is often higher than the DPI of the image). Or graphics designed for previous iPhones rendered on an iPhone 4 :).
Typical approaches to stretching images use interpolation that generally results in an image that appears blurry. Sharpening techniques can improve that, but generally, ultimately, just end up bringing back some of the original pixelation.
He uses a completely different approach to stretching the image such that "sharpening" is inherent in the stretching, and it follows the natural contrast edges of the picture, which de-emphasizes rather than emphasizes the pixels.
@DingoJunior ah, i see.
tl;dr: it's a better image up-scaling technique, not a new display technology or anything.
@mrqs
This is from the original article:
"Now retired and living in Portland, Ore., Kirsch recently set out to make amends. Inspired by the mosaic builders of antiquity who constructed scenes of stunning detail with bits of tile, Kirsch has written a program that turns the chunky, clunky squares of a digital image into a smoother picture made of variably shaped pixels."
So, from what I understand, he is not using square pixels while generating the image...
@DingoJunior "Basically what he has come up with is a different way of rendering that source data to give less pixelated results."
I don't get this comment board. Your comment is correct and the first comment saying "still looks like a bunch of squares to me" is highest ranked.
Maybe I assume too much from the masses.
It's a way of decoding the same information differently to give a more visually pleasing result and it clearly works. If you increase the resolution during encode then you just use more space so this has to be a decode-only technology.
It's like on a digital TV vs analog TV. When the signal is bad, it's still watchable but when you get digital noise in the form of squares throughout the image then it's harder to watch as your eye is drawn to the noise.
The algorithm could be optimized to work better in certain areas like the jagged edge on the hairline and use a smoothing algorithm to reduce long zig-zags.
It's almost like treating a digital image as a vector image during decode and using the pixels as curve vertices and determining the curve directions based on contrasting areas.
@TinWard "It's almost like treating a digital image as a vector image during decode and using the pixels as curve vertices and determining the curve directions based on contrasting areas." This. It is more like a vector image than a regular raster image, in that it is clumping areas of a picture into something you can quantify color-wise with geometry rather than just summing color information and assigning a color/pixel to the resulting area. You get a higher resolution effect out of a source that is the same size (theoretically anyway).
@DingoJunior
yes, it is very useful and impressive, but at the end of the day it's still just an upscaling technique - he's in no way "getting rid of the pixel" and like glassfin said, you'll always get better results by using the native resolution of your display/printer/phone/whatever instead of upscaling, no matter how advanced the algorithm
of course an image of the native resolution isn't always available and that's where this will come in handy, but it's hardly the second coming of the pixel
don't our eyes see slight ovals or something? perhaps the slight difference to the shape of the square could in the end make for a massive before and after
just look at the retina display, the original was great back in the day but now with a before and after.
Looks blended
But will it blend? Or are you talking about something else?
it sounds better than just cramming the poops out of every pixel, trying to get the highest ppi possible
This guy is 70 to 90 years old and still researching. :D. Some body kidnapp him and drop him in the bahammas
@DefPoet
you were nearly half way there.
....
“Finally,” he says, “at my advanced age of 81, I decided that instead of just complaining about what I did, I ought to do something about it.”
....
It is quite extraordinary that he has still not retired from research.
@DefPoet agree
@DefPoet
years later -- "I've done it! I figured out the perfect live vector tracing algorithm for digital imaging!...uhhh..ooo..." (places hand on heart and drops to the floor)....the world will never know.
@DefPoet Nothing should ever stop people from trying to make the world a better place. If you need to relax, relax-- but by all means, if there's a way you can still make an impact, I could care less how old you are. Go for it. Our culture does our elders the disservice of telling them that after retirement, their lives are best spent wasting away into a comfortable nothing and dying out of sight.
At the end of the day, you're still seeing the image through the series of square dots on the screen. THOSE are the pixels!
Here, it's not pixels, it's just a neat vectorisation of the image.
None-square pixels..? 'tis a silly concept, unless you're about to build a morphing display to go with it.
@Jayenkai
i agree, i can still see the pixels in the right image. and they are still square. they are just interpolated in a new way. So nothing revolutionary here.
Actually the only alternatives to square pixels would be triangular or hexagonal ones, but i doubt they'd make images any better.
@Jayenkai
Agreed the "before" picture shows a single camera/scanner pixel represented by many pixels on your monitor when looking at it. Applying the algorithm then smooths the one camera/scanner pixel over many available on your monitor.
However I think Fujifilm have hexagonal pixels in their camera's which they claimed removed the problem of human-eye sensitivity to diagonal lines as well?
Pixels are the thing that will ultimately hold back digital images. It's the same with video using frame by frame pictures to give the illusion of film. It needs to be superseded by a better technology eventually.
@Pryomancer isn't film just a series of pictures too?
@Pryomancer
Film is a series of pictures anyway; digital or not. Using frame by frame pictures doesn't give the illusion of film - it IS film.
@monkfishbandana Yeah, exactly. Pyromancer, what else could you possibly suggest? Puppets?
@Pryomancer Well, here's a question: Why does it needed to be superseded by a better technology? Just wondering - lots of times better things are invented with no reason for it to be better. What's there already worked fine. Not saying that we should stop progress, but "better" isn't always better if there's no reason.
@Pryomancer Like halovision?
anyone else a little creeped out by the picture of the baby?
@erick390 Wherever you move the eyes follow you. It needs the background soundtrack of echoic baby laugh.
I'll be darned, apparently if you throw more pixels at an image, you can make it sharper. Truely revolutionary.
Nothing to see here, move along.
@Xepol
I think you're missing the point by a couple miles, friend...
@MarcO
Not even remotely. His scheme involves taking each pixel, turning into a 6x6 pixel array (of square pixels I might add), and using digital processing to decide how to extrapolate the data. This is just a swell with a non-linear stretch routine which is absolutely nothing new at all.
Not only is there nothing new here, it doesn't even really address the square pixels - it DEPENDS on square pixels.
He has created a slightly different smoothing routing for his swell - nothing more. If the supplied screen shot is any gauge, it seems to create some very nasty artifacts that many other time tested routines don't.
I got so used to them now...
Soylent Green is made of pixeeelllsss...
Piiiixeeeeeeeelllsss!
Yea, I am not getting it. This seems comparable to someone who increases a polygon count rather than use nurbs.
@raredesign
+1 to you, sir, for knowing what nurbs are.
Russell Kirsch helped create square pixels, now he wants to kill them off - by adding 35 more square pixels.
The answer to the jaggies problem is getting people over the "HD" craze and increasing pixel density. No, 1920 x 1080 is NOT an awesome resolution. Why did I have to pay $100 more to get something more than 1990's display tech? 1366 x 768 (an "HD" screen?!?) is just a wide 1024 x 768 - nothing fancy. We should be at 4k or 8k by now.
So, while he might be on the right track by adding more pixels, its nothing new. Just something that should have happened years ago.
@dcnoren fyi it's not a new display technology, it seems to be more like a better up-scaling algorithm.
@maveric101
Yes, I do believe that is painfully obvious.
so... he's not getting rid of square pixels then. He's just increasing the number of square pixels by 36x. doesn't everyone already know that higher resolution gives a clearer image?
It's not killing off the square pixel. You got that from the person in charge of bogus but provocative headlines in the sick and twisted department.
It's a scaling interpretive. No sense adding anything to an image at the native ppi count of the display, but, as in the examples, if you scale it up, you need to re-render it differently.
It just looks like instead of presenting a pixel as-is he averages its value with the value of the neighboring pixels into 36 smaller pixels.
Ummm. Pretty sure I've seen this done before, more times than I can count, over the last 25 years.
So when robots take over they will have better vision to spot us and..... Do bad things??
Enhance...
Enhance...
Enhance...
Enhance...
this would be the end of pixel art! or....the beginning?