Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Image + Camera = crash

in Bugs Posts: 1,223

I was doing some simple tests of using the camera to capture motion (writing a little program that measures head-bob as a means of tracking running speed). But using the camera to generate images for analysis was causing frequent crashes.

Bring it down to its essence, and it fails within twenty seconds. Looking at the documents, the slowness of this operation is noted, and yes, I'm doing here exactly what the help docs say don't do, so in a sense, the bug is mine.

But it makes little sense to me why this is sure doom.

-- ImageCameraTest

function setup()
    cameraSource(CAMERA_FRONT)
end

function draw()
    background(40, 40, 50)

    img = image(CAMERA)

end

While this is not...

function setup()
    cameraSource(CAMERA_FRONT)
end

function draw()
    background(40, 40, 50)

    sprite(CAMERA)

end

Is there a way to capture camera pixels in a way that lets me query them quickly?

Thanks.

Tagged:

Comments

  • dave1707dave1707 Mod
    Posts: 7,058

    @Mark I don't think img=image(Camera) is meant to do it 60 times per second by having it in draw. Try this, tap the screen to capture an image. You could set a timer to do it automatically.

    function setup()
        cameraSource(CAMERA_BACK)
        print"keep tapping screen to capture image."
    end
    
    function draw()
        background(40, 40, 50)
        if img~=nil then
            sprite(img,WIDTH/2,HEIGHT/2)
        else
            sprite(CAMERA,WIDTH/2,HEIGHT/2)
        end
    end
    
    function touched(t)
        if t.state==BEGAN then
            img=image(CAMERA)
        end
    end
    
  • Posts: 1,223

    No, I'm not trying to record images. I'm trying to spot movement.

    I really do want to check the camera in the draw loop.

  • Posts: 419

    You're creating a full-sized image 60 times a second (or thereabouts). Even though you're reusing the img variable, the image itself won't get removed from memory until the next garbage collection, so the images will stack up in memory and you're reaching saturation.

    Instead of creating a new image each time, try reusing old images. Something like:

    setContext(img)
    sprite(CAMERA)
    setContext()
    
  • Posts: 148

    @Mark, i think garbage collector is just not keeping up with the images you are creating every frame. You should probably only keep a small number of latest frames - just enough for motion analysis, and discard the rest, by forcing garbage collection periodically.

    For example, here, only 10 last captures are kept in the table, and i'm calling collectgarbage() explicitly to recover the memory:

    --# Main
    -- ImageCameraTest
    
    frames = {}
    head = 0
    N = 10
    
    function setup()
        cameraSource(CAMERA_FRONT)
    end
    
    function draw()
        background(40, 40, 50)
    
        -- keep N last frames
        frames[head + 1] = image(CAMERA)
        head = (head + 1) % N
    
        -- periodically force garbage collection
        if head == 0 then
            collectgarbage()
        end
    end
    
  • dave1707dave1707 Mod
    Posts: 7,058

    @Mark I wrote a little program that compares one image to the next and they weren't the same. I have the camera in draw(), so the 2 images are 1/60 second apart. Even if I cover the camera so both images are black, comparing the pixels of the first image and the second image result in a difference. I wonder if there should be a percentage of difference before the images are considered different.

  • IgnatzIgnatz Mod
    Posts: 5,396

    I did this a while back, and took pixel samples, eg every 50th pixel across and down. But that was to detect movements of a uniformly colored object in my hand.

    If you're outside, just about every pixel is going to change colour at every frame, and the complexity of real life scenery is going to make tracking orientation extremely difficult.

    There are plenty of apps that track running, I wonder how they do it.

  • dave1707dave1707 Mod
    edited January 2016 Posts: 7,058

    @Mark If you want to track running speed, have you looked at the location functions. There's location.speed which measures speed in meters per second. I haven't tried them because my speed on the couch is usually 0.

    EDIT: I guess the runner would have to be carrying the device though. So forget this option.

  • Posts: 1,223

    @dave1707 and @Ignatz. I use a combination of location and accelerometer for outside running, I'm looking at the camera for a very specific purpose -- treadmills. I'm thinking the user props the iPad or iPhone somewhere so that they can see the screen, then, using the front camera, I measure points scattered over the screen in a grid. I look for pixels being "hidden" and "revealed" to come up with a steps-per-minute pace and turn that into a (rough) approximation of pace.

    Of course, the runner on the treadmill knows how fast they are going, but that's okay. I'm not trying to provide feedback in mph. I'm trying to move their avatar along a path.

  • SimeonSimeon Admin Mod
    Posts: 4,587

    @Mark I wonder if this might be better handled by a shader? The image analysis could happen in there (where it has no problem analysing the camera stream at full speed). The output of the shader could be a single colour indicating the height of the head, and it would be fairly cheap to read the single colour out for use in analysis.

    Unsure if it's possible, but just a thought.

Sign In or Register to comment.