Machine vision: tilting my head
[Audio Version]
Following is another in my series of ad hoc journal entries I've been keeping of my thoughts on machine vision.
I observed something very interesting today. When I look at a fixed position in a relatively static scene and tilt my head slowly left or right - rotate it, essentially - something unexpected happens. The scene seems to "click" into position at a rate of perhaps twice a second. The effect is similar to watching a poster rotate with a strobe light flashing every half-second, minus the blackness. And, funny enough, it feels as though my eyeballs are rotating and clicking with each step.
I thought maybe this had something to do with the fact that I have two eyes, so I closed one eye and repeated the experiment. Same result.
I tried this because I wanted to know how our eyes deal with changes in rotation. I was thinking about how to get software to deal with a change in point of view. When your saccades around a scene, it somehow almost instantly orients itself to the new point of view. It occurred to me that maybe the brain somehow plans the saccade and predicts how much the scene will "shift" by. A computer should be able to do this, too. The hard part is predicting how far a camera's saccade will shift the scene. With a "soft fovea" inside a fixed view, this is easy.
But it seems the tilting-head case throws the brain for a loop. I believe what's happening is that the lower level visual processor doesn't know how to deal with the whole scene rotating and so calls for a "reset" of the image, as though you had blinked and, upon opening your eyes, found yourself in an entirely new scene.
I estimate it takes a little less than half a second to deal with the new orientation. It would be interesting to experiment with the brain's ability to learn to deal continuously with such rotations. I bet it would be like switching from contacts to glasses or vice-versa. At first, the world appears strangely bouncy as I move about. Within a few minutes, I find that bounciness goes away. I assume this is because my brain learns to make predictions about how the scene I see will respond to my movements.
Following is another in my series of ad hoc journal entries I've been keeping of my thoughts on machine vision.
I observed something very interesting today. When I look at a fixed position in a relatively static scene and tilt my head slowly left or right - rotate it, essentially - something unexpected happens. The scene seems to "click" into position at a rate of perhaps twice a second. The effect is similar to watching a poster rotate with a strobe light flashing every half-second, minus the blackness. And, funny enough, it feels as though my eyeballs are rotating and clicking with each step.
I thought maybe this had something to do with the fact that I have two eyes, so I closed one eye and repeated the experiment. Same result.
I tried this because I wanted to know how our eyes deal with changes in rotation. I was thinking about how to get software to deal with a change in point of view. When your saccades around a scene, it somehow almost instantly orients itself to the new point of view. It occurred to me that maybe the brain somehow plans the saccade and predicts how much the scene will "shift" by. A computer should be able to do this, too. The hard part is predicting how far a camera's saccade will shift the scene. With a "soft fovea" inside a fixed view, this is easy.
But it seems the tilting-head case throws the brain for a loop. I believe what's happening is that the lower level visual processor doesn't know how to deal with the whole scene rotating and so calls for a "reset" of the image, as though you had blinked and, upon opening your eyes, found yourself in an entirely new scene.
I estimate it takes a little less than half a second to deal with the new orientation. It would be interesting to experiment with the brain's ability to learn to deal continuously with such rotations. I bet it would be like switching from contacts to glasses or vice-versa. At first, the world appears strangely bouncy as I move about. Within a few minutes, I find that bounciness goes away. I assume this is because my brain learns to make predictions about how the scene I see will respond to my movements.
Comments
Post a Comment