-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
USB Host UVC driver version 2 #84
base: master
Are you sure you want to change the base?
Conversation
7adb8ec
to
e9a9097
Compare
#include <stdint.h> | ||
|
||
// Descriptors from UVC driver version 1 | ||
// TODO check and test this |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Jut FYI, so you don't forget about this, here is TODO, I did not find anything about it in the attached TODO file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I did not add this testing todo to the list. The descriptor parsing has a pretty good coverage and I don't have this cameras so it is not implemented.
It can be removed, I wanted to leave it here for future reference
host/class/uvc/usb_host_uvc_2/host_test/main/parsing/test_parsing_customer_dual.cpp
Outdated
Show resolved
Hide resolved
host/class/uvc/usb_host_uvc_2/host_test/main/parsing/test_parsing_elp_h264.cpp
Show resolved
Hide resolved
host/class/uvc/usb_host_uvc_2/host_test/main/parsing/test_parsing_logitech_c270.cpp
Outdated
Show resolved
Hide resolved
e9a9097
to
4157f1b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@peter-marcisovsky Thank you very much for such a thorough review!
I fixed most of the things, answered other and left a few for later.
I left the changes in separate comment for now, so it is easier to review
#include <stdint.h> | ||
|
||
// Descriptors from UVC driver version 1 | ||
// TODO check and test this |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I did not add this testing todo to the list. The descriptor parsing has a pretty good coverage and I don't have this cameras so it is not implemented.
It can be removed, I wanted to leave it here for future reference
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The need for this file stems from a different thing. In this host_test the freertos
component is not mocked. So the main
function is provided by freertos
component (same as if it was run on esp target) and it in turn requires app_main
function. So linking main
from Catch2 has no effect
# target_link_libraries(${COMPONENT_LIB} PRIVATE Catch2WithMain) # We don't mock FreeRTOS for now |
It also means that we cannot pass any arguments to the resulting .elf file... This can be fixed only in esp-idf later...
I plan to implement another host_test where the freertos component is mocked. In this situation I don't need this file and can use main from Catch2
.advanced.urb_size = 20 * 1024, | ||
}; | ||
|
||
/* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function can be used for initializing SD card and saving the frames on it. It can be useful during testing.
I'll clean the example up before releasing it, thanks!
uvc_stream->bEndpointAddress = ep_desc->bEndpointAddress; | ||
*ep_desc_ret = ep_desc; | ||
|
||
return usb_host_interface_claim(p_uvc_host_driver->usb_client_hdl, uvc_stream->dev_hdl, intf_desc->bInterfaceNumber, intf_desc->bAlternateSetting); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The only functional change is that an error message will be printed if != ESP_OK.
Is that what you meant?
usb_host_interface_claim
also prints an error message internally...
|
||
// 1. Negotiate the frame format | ||
// @see USB UVC specification ver 1.5, figure 4-1 | ||
uvc_vs_ctrl_t vs_result; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yep, this not the cleanest way. I will consider refactoring the negotiation before 1st release
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey Tomas,
I did the first round and it works! Cool!
Regarding the changes, I did check the:
- READMEs and Description
- Folder struct and APIs
Couple of notes I've already put, but to proceed, I think that I might meed some additional information.
Like requirements. I prepared the list of the questions, answers to them will help me understand what should I check and how.
- What are the targeting versions of esp-idf for examples? (I've tried the
basic_uvc_stream
on v5.2 and there isfatal error: sd_pwr_ctrl_by_on_chip_ldo.h: No such file or directory
😐 - What are the requirements for the driver? Should it notify the user while the device is attached to the ESP32 and then user should start streaming? Or it just provides the possibility to open the stream, when the device is already connected and user knows the vid/pid? More like the general idea. Using with hubs, several streams on one device or not, hot detaching (detaching cameras during streaming) and so on
- Single-thread/Multi-thread. Does it support opening several streams on one physical device from several different tasks?
I have checked the docs/
folder, thanks for the png! (also, if we need to provide special additional information, such as "create" above the arrow, maybe we need to think about renaming the name of the API)
I will proceed with the review, feel free to address current comments.
enum uvc_host_dev_event { | ||
UVC_HOST_ERROR, /**< USB transfer error */ | ||
UVC_HOST_DEVICE_DISCONNECTED, /**< Device was suddenly disconnected. The stream is stopped. */ | ||
UVC_HOST_FRAME_BUFFER_OVERFLOW, /**< The received frame was discarded because it exceeded the available frame buffer space. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is that also an error? Or it is possible to increase the frame_size
right after getting this event?
If this is an error, maybe it could be better:
UVC_HOST_FRAME_BUFFER_OVERFLOW, /**< The received frame was discarded because it exceeded the available frame buffer space. | |
UVC_HOST_ERROR_FRAME_BUFFER_OVERFLOW, /**< The received frame was discarded because it exceeded the available frame buffer space. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The frame buffer size cannot be changed of the UVC stream was opened (I can add this into TODO list)
It's not a real error because the stream continues. It just informs the user that a frame was dropped. If this happens too often, the user can closed the stream and open it again with larger frame buffers...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Then maybe?
UVC_HOST_FRAME_BUFFER_OVERFLOW, /**< The received frame was discarded because it exceeded the available frame buffer space. | |
UVC_HOST_DROPPED_FRAMES, /**< The received frame was discarded because it exceeded the available frame buffer space. |
* - ESP_INVALID_ARG: frame or stream_hdl is NULL | ||
* - ESP_FAIL: The frame was not returned to the driver | ||
*/ | ||
esp_err_t uvc_host_frame_return(uvc_host_stream_hdl_t stream_hdl, uvc_host_frame_t *frame); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Based on the logic, described above, maybe it would be cleaner to name it something like:
esp_err_t uvc_host_frame_return(uvc_host_stream_hdl_t stream_hdl, uvc_host_frame_t *frame); | |
esp_err_t uvc_host_frame_process_complete(uvc_host_stream_hdl_t stream_hdl, uvc_host_frame_t *frame); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, is that possible to avoid returning value in the uvc_host_frame_callback_t
and do that via the uvc_host_frame_process_complete()
call?
If the frame is handled right in the callback, it is possible to call it right from there.
If the frame is not handled in the callback, it will be called someplace else.
WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This naming was taken from esp32-camera
. I'll check if it can be made better
case USB_TRANSFER_STATUS_STALL: | ||
// On Bulk errors we stop the stream | ||
//@todo not tested yet | ||
//@todo Stall, error and overflow errors should be propagated to the user |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the STALL is a regular logic?
Like STALLing the unknown control transfers in general? It is not the problem, device just got the unknown request.
Is it the type of an error we need to stop stream after?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have not encoutered STALL during my tests. The BULK IN transfer should never be STALLed, so we might as well just abort
here
430f7d0
to
f2e7547
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, couple of new comments, but nothing serious.
I will try to verify it on a hardware, should I have some special camera or anything I should know before testing?
Also, did you do any measurements to what extend it is better than the implementation, based on libusb?
UPD: Is our current example in esp-idf is compatible with this new class driver?
num_of_transfers, transfer_size, num_isoc_packets, max_packet_size); | ||
|
||
// Allocate array of transfers | ||
uvc_stream->constant.num_of_xfers = num_of_transfers; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here, instead of num_of_xfers = num_of_transfers
we can increase num_of_xfers++
every successful allocation.
Which means, that during the uvc_transfers_free
you will release only allocated xfers.
Currently, you will pass the whole array of xfers, even if there was an error during the allocation of the first one.
|
||
// @todo create a function that will wait for all frames to be returned | ||
if (!uvc_frame_are_all_returned(uvc_stream)) { | ||
vTaskDelay(pdMS_TO_TICKS(70)); // Wait 70ms so the user can return all frames |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems, that this is the right place to add a mutex here....but you already have a TODO here )
xSemaphoreGive((SemaphoreHandle_t)transfer->context); | ||
} | ||
|
||
esp_err_t uvc_host_usb_ctrl(uvc_host_stream_hdl_t stream_hdl, uint8_t bmRequestType, uint8_t bRequest, uint16_t wValue, uint16_t wIndex, uint16_t wLength, uint8_t *data) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we plan to support something else rather then usb?
If no, maybe usb
in not necessary to be in the name.
esp_err_t uvc_host_usb_ctrl(uvc_host_stream_hdl_t stream_hdl, uint8_t bmRequestType, uint8_t bRequest, uint16_t wValue, uint16_t wIndex, uint16_t wLength, uint8_t *data) | |
esp_err_t uvc_host_ctrl(uvc_host_stream_hdl_t stream_hdl, uint8_t bmRequestType, uint8_t bRequest, uint16_t wValue, uint16_t wIndex, uint16_t wLength, uint8_t *data) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here I wanted to emphasize that it is not 'Camera control' but 'USB Control' request.
I'll go through all function names again and revise it
#include "usb/uvc_host.h" | ||
#include "uvc_control.h" | ||
#include "uvc_stream.h" | ||
#include "uvc_types_priv.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need _priv.h
postfix in the filenames?
They are already in the private folder. Or am I missing something?
I mean, I just bumped in that when I tried to find the file, related to "uvc_frame.c" which is usually has the same name, but header. I didn't find it and then I realized that I need to search "_frame_priv.h" )
Just a bit confusing IMHO.
But maybe there is an idea behind such naming.
uvc_host_frame_callback_t frame_cb; /**< Stream's frame callback function */ | ||
void *user_ctx; /**< User's argument that will be passed to the callbacks */ | ||
struct { | ||
uint16_t vid; /**< Device's Vendor ID. Set to 0 for any */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we are providing the option to select any vid/pid, it seems that it is a command like: "I don't know anything about device will be attached, I just want to have a stream".
In this case, maybe it will be a good thing to add the default config as well.
Like literally "open the first device, first stream with the first possible supported settings and give me the handle".
Because I've tried to provide correct config to open the stream and I realized that I have no idea about the VID/PID, stream number and protocols/formats the camera supports.
So either I need to attach camera somewhere in advance to get these parameters, or just ask the driver to use the first supported configuration it'll find.
host/class/uvc/usb_host_uvc_2/examples/basic_uvc_stream/main/basic_uvc_stream.c
Show resolved
Hide resolved
host/class/uvc/usb_host_uvc_2/examples/basic_uvc_stream/main/basic_uvc_stream.c
Show resolved
Hide resolved
// First, check list of already opened UVC devices | ||
ESP_LOGD(TAG, "Checking list of opened USB devices"); | ||
uvc_stream_t *uvc_stream; | ||
SLIST_FOREACH(uvc_stream, &p_uvc_host_driver->uvc_stream_list, list_entry) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe, when the vid/pid == 0
, we don't need to iterate the list, but to return the first member of the list.
Seems, that otherwise the logic is bit complicated with iteration for both cases.
@tore-espressif Will proceed with review shortly... |
@roma-jam that is really weird, Can you share full CFG descriptor? |
USB 2.0 Camera Seems like we have it in host_test, as CANYON_CNE_CWC2. Which probably means that I do smth wrong. UPD: I open the stream on the same device again. But the stream was already opened and interface was claimed, so I don't get any error on that and driver tries to claim the interface one more time. But it is already claimed, that is why the EP already allocated. Good.
|
Yep, I opened the stream, but I keep getting the following error: and I still have no idea why. In process. |
f2e7547
to
104b5c1
Compare
Introduction
This PR introduces version 2 of the USB Host UVC driver, aimed at enabling ESP SoCs to stream frames from connected USB cameras.
The driver provides frames to the user via a callback, offering a flexible interface that can serve as a foundation for more advanced frame processing applications.
Internally, the driver manages a FreeRTOS Queue containing multiple frame buffers, which helps prevent buffer overflows and underflows during streaming.
Detailed feature descriptions, future plans, and usage instructions are available in the README.md file.
Information for Testers
This section provides instructions for testing the driver’s functionality and public API.
Two examples are included for testing:
1.
basic_uvc_stream
2.
camera_display
Information for Reviewers
This section provides guidance for those reviewing the internal code structure of the driver.
The driver includes a
host_test
folder with tests designed to run on Linux. These tests use a mocked version of the esp-idf/usb component, focusing on the following areas, which should need less in-depth review:uvc_descriptor_parsing.c
uvc_isoc.c
anduvc_bulk.c
The following file, focused on human-readable output, also requires only a light review:
uvc_descriptor_printing.c
Areas not yet covered by
host_tests
and requiring more detailed review include:uvc_frame.c
(manages queue of empty frame buffers)uvc_control.c
(handles video format negotiation and other control requests)uvc_host.c
(includes driver installation, device management, stream initiation, etc.)Related Information