8000 drivers/genx320: Added raw event reading functionality by jeff-shulkin · Pull Request #2970 · openmv/openmv · GitHub
[go: up one dir, main page]
Skip to content

drivers/genx320: Added raw event reading functionality#2970

Open
jeff-shulkin wants to merge 1 commit intoopenmv:masterfrom
jeff-shulkin:genx320_raw_event_stream
Open

drivers/genx320: Added raw event reading functionality#2970
jeff-shulkin wants to merge 1 commit intoopenmv:masterfrom
jeff-shulkin:genx320_raw_event_stream

Conversation

@jeff-shulkin
Copy link
Contributor
@jeff-shulkin jeff-shulkin commented Jan 24, 2026 •
edited
Loading

Addresses #2945. Adds IOCTL call OMV_CSI_IOCTL_GENX320_READ_EVENTS_RAW, which simply returns the 32-bit words provided by the GENX320 in EVT2.0 format. When streaming, this allows a downstream PC to handle 3x the amount of events.

@github-actions
Copy link
github-actions bot commented Jan 24, 2026 •
edited
Loading

Code Size Report:

Firmware Text Diff Data Diff BSS Diff
OPENMV4P/firmware.elf 🔺0.03% (+544) ➖0.00% (+0) ➖0.00% (+0)
OPENMV_N6/firmware.elf 🔺0.03% (+584) ➖0.00% (+0) ➖0.00% (+0)
OPENMV_RT1060/firmware.elf 🔺0.02% (+560) ➖0.00% (+0) ➖0.00% (+0)

@kwagyeman
Copy link
Member
kwagyeman commented Jan 26, 2026 •
edited
Loading

@jeff-shulkin - Thanks for this PR. To improve performance, you should actually not perform the memcpy at all.

Do this, update the IOCTL to accept a pointer to an image object. The csi_snapshot internally then should be passed that.

case OMV_CSI_IOCTL_GENX320_READ_EVENTS_RAW: {
            if (omv_csi_get_cropped(csi)) {
                return OMV_CSI_ERROR_CAPTURE_FAILED;
            }

            if (csi->transpose) {
                return OMV_CSI_ERROR_CAPTURE_FAILED;
            }

            image_t *img = (image_t *) va_arg(ap, image_t *);
            ret = omv_csi_snapshot(csi, img, OMV_CSI_FLAG_NO_POST);
            break;
        }

Then, update the logic in here to like:

 case OMV_CSI_IOCTL_GENX320_READ_EVENTS_RAW: {
    image_t img;
    error = omv_csi_ioctl(self->csi, request, &img);
    if (error > 0) {
        ret_obj = py_image_from_struct(&img);
    }
    break;
}

This will remove that memcpy. For the python code in the app, you'll also want to remove the fifo buffer implemented in python, that will not be needed anymore as you can use the fifo buffer inside the camera driver when it's in fifo mode. So, basically, you directly pass the result of calling read_events_raw to the protocol channel. Note that you will need to not call snapshot() in the protocol callback, so, in the script you'll want to wait for the first read or size request access of the protocol channel to set a flag to prime the pipe... and then in the main while loop start doing snapshot and return that in the protocol callback.

When in image fifo mode (frame buffers > 3) the first call to snapshot will kick off the dma system and then return the first frame. The next frames behind it are queued up in the fifo. If the FIFO overflows, it's completely reset, which will cause data loss. This is why you want to wait until you get the first request to read the channel or similar from the PC before calling the first snapshot. Calling snapshot in the protocol callback would be the easiest thing to do... but, then things get a bit re-entrant.

@jeff-shulkin jeff-shulkin force-pushed the genx320_raw_event_stream branch from dd97734 to dd5f987 Compare January 29, 2026 06:00
@jeff-shulkin
Copy link
Contributor Author
jeff-shulkin commented Jan 29, 2026

Updated the code to get rid of the memcpy. There's some formatting issues I'm not sure how to best resolve, as fixing them would require either shortening the IOCTL call name or fixing the formatting for every single IOCTL definition in omv_csi_ioctl_t.

In any case, I should also have a PR up in openmv-projects with the raw streaming soon. I'm still working through the kinks of getting this working without any dropped data, so some more guidance when I put up that PR would be greatly appreciated.

@kwagyeman
Co 8000 py link
Member
kwagyeman commented Jan 29, 2026

For the formatting, you just need to add spaces as it shows. Yes, fixing every ioctl spacing.

Thanks for your work on this, Ibrahim, and I discussed in our last meeting, potentially adding RAW event processing and support to the OpenMV IDE. It's not necessarily hard, but I have other, more pressing issues right now. What you are doing currently is the fastest way to get the best data transfer speed for now.

@jeff-shulkin jeff-shulkin force-pushed the genx320_raw_event_stream branch from dd5f987 to f1e704f Compare January 29, 2026 06:27
@jeff-shulkin
Copy link
Contributor Author
jeff-shulkin commented Feb 12, 2026

Sorry for the long lag-time on this PR, I've been busy over the past couple weeks. I just made a PR on openmv-projects (openmv/openmv-projects#7) that shows how to stream out the 32-bit words. I was able to get rid of the Python-side FIFO buffer completely, but I'm still getting weird behavior. To be more specific, the EVS still seems to be dropping data, resulting in skipped frames when visualizing the EVS histograms. I'm not sure if its inefficiency in the altered C code or the MicroPython scripts (though I heavily suspect the latter), so any insight on how to reduce the amount of dropped data would be greatly appreciated.

Furthermore, I haven't pushed this change yet, but I did have to change a couple lines of code to get rid of the FIFO buffer. More specifically, I had to change the suggested lines in GENX320's ioctl from this:

case OMV_CSI_IOCTL_GENX320_READ_EVENTS_RAW: {
            if (omv_csi_get_cropped(csi)) {
                return OMV_CSI_ERROR_CAPTURE_FAILED;
            }

            if (csi->transpose) {
                return OMV_CSI_ERROR_CAPTURE_FAILED;
            }

            image_t *img = (image_t *) va_arg(ap, image_t *);
            ret = omv_csi_snapshot(csi, img, OMV_CSI_FLAG_NO_POST);
            break;
        }

to this:

case OMV_CSI_IOCTL_GENX320_READ_EVENTS_RAW: {
        if (omv_csi_get_cropped(csi)) {
            return OMV_CSI_ERROR_CAPTURE_FAILED;
        }

        if (csi->transpose) {
            return OMV_CSI_ERROR_CAPTURE_FAILED;
        }

        image_t *img = (image_t *) va_arg(ap, image_t *);
        ret = omv_csi_snapshot(csi, img, OMV_CSI_FLAG_NO_POST);
        if (ret == 0) {
            ret = img->w * img->h;
        }
        break;
    }

This is a pretty hacky solution though. Is there a better way of writing this? The reason I reset ret to img->w * img->h is because, if we just return ret as zero, then the branch:

    if (error > 0) {
        ret_obj = py_image_from_struct(&img);
    }
    break;

never triggers.

@kwagyeman
Copy link
Member
kwagyeman commented Feb 12, 2026 •
edited
Loading

@jeff-shulkin - Can you look at this script I did for RAW bayer image tuning on the desktop? https://github.com/openmv/openmv-projects/blob/master/ccm-tuning/ccm_tuning_on_cam.py

It shows how to send a frame buffer to the PC bypassing the stream buffer. Then, you should just need to turn on the csi.framebuffer() fifo by passing in a large frame buffer count, and then finally, you need to make buffer size very large to maximum transfer bandwidth. Doing small 2K event buffers is unlikely to work well. Aim for like 16K to 32K buffer sizes.

As for your question, just change:

 if (error > 0) {
    ret_obj = py_image_from_struct(&img);
}

To:

 if (error == 0) {
    ret_obj = py_image_from_struct(&img);
}

Note that in the example I used... the buffer I am pulling data from doesn't move, you'll have to re-create the memory view after each snapshot return becuase it will be to a new buffer address if you have more than 1 buffer.

@kwagyeman kwagyeman mentioned this pull request Feb 12, 2026
Open
@jeff-shulkin jeff-shulkin force-pushed the genx320_raw_event_stream branch from f1e704f to ed57a27 Compare February 12, 2026 22:16
@jeff-shulkin
Copy link
Contributor Author
jeff-shulkin commented Feb 13, 2026

Just updated the C code to check for a successful return from snapshot as suggested. However, concerning the micropython scripts, I'm running into an issue with inefficiency. After taking a look at the ccm_tuning scripts, I altered my EventChannel class to look like so:

class EventChannel:
    def size(self):
        global primed
        if not primed:
            primed = True

        return event_length

    def shape(self):
        return (event_length, 1)

    def read(self, offset, size):
        global frame_available

        if frame_available:
            end = offset + size
            mv = events_mv[offset:end]
            if end == event_length:
                frame_available = False
            return mv
        return bytes(size) 

    def poll(self):
        global primed
        if not primed:
            primed = True
        
        return frame_available

The problem is that, with the way the script is set up now, its flooding the USB stream with old events, leading to a massive slowdown in speed. If possible, I'd like to do the same optimization as the unpacked ndarray streaming script, where we just send new events. I believe this change would be trivial, as it would just mean that we read from 0 until size, where size is the number of new events instead of the full buffer size. At the same time, with how the ioctl call for GENX320_READ_EVENTS_RAW works, I'm not sure how to get just the number of new events. Do you have any advice in this matter?

@kwagyeman
Copy link
Member
kwagyeman commented Feb 13, 2026

@jeff-shulkin - Let me give the RAW mode a shot. I think I might have some time today.

@jeff-shulkin
Copy link
Contributor Author
jeff-shulkin commented Feb 16, 2026 •
edited
Loading

@kwagyeman Figured out the streaming issue I was running into earlier. The problem was that I forgot to actually use the system_primed variable in the script's superloop, so it was extremely re-entrant. I'll update the script in openmv-projects to be correct, so you should be able to check out RAW mode in the next hour or so.

Problem I'm running into right now is that, if I set CSI_FIFO_DEPTH > 1, the camera freezes on me at 8192 events and above, forcing a resync. If you have any insight here, that would be well appreciated.

@kwagyeman
Copy link
Member
kwagyeman commented Feb 16, 2026

Hi, I'll comment directly on the project's PR. This feature appears fine here in the firmware.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jeff-shulkin @kwagyeman
0