8000 [BACKEND] External phone camera management by nicolas-rabault · Pull Request #4 · nicolas-rabault/leLab · GitHub
[go: up one dir, main page]
Skip to content

[BACKEND] External phone camera management #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
nicolas-rabault wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[BACKEND] External phone camera management #4

nicolas-rabault wants to merge 1 commit into from

Conversation

@nicolas-rabault
Copy link
Owner
@nicolas-rabault nicolas-rabault commented Jun 19, 2025 •
edited
Loading

What is this PR about

In this PR we try to use the web interface to link local network devices cameras to feed the dataset.

How do we plan to do it

In the camera configuration dropdown, add a special camera option called "Phone".

This option must:

  • Always be visible
  • Be selectable multiple times

When a "Phone" camera is added:

  1. 🔍 Determine frontend base address:

    • If the user accesses the frontend via localhost or 127.0.0.1, detect the machine's local network IP address (e.g., 192.168.x.x).
    • Use this local network address for any mobile connections (QR code, streaming).
    • If the user is already on a LAN or public address, reuse it as-is.

  2. 🔐 Enable HTTPS locally (required for camera access on phones):

    • If the frontend is running locally (e.g., in development), enable HTTPS using:
      • A self-signed certificate (auto-generated if needed)
      • Or use mkcert to generate a trusted local certificate
    • Make sure the frontend is reachable via https://<local_ip>:<port> over Wi-Fi

  3. 🌐 Setup a WebRTC streaming endpoint on the backend:

    • Create a unique WebRTC session for each "Phone" camera.
    • This endpoint accepts incoming WebRTC video streams from phones.

  4. 📱 Generate a mobile streaming link with QR code:

    • Generate a URL: [https://<local_ip>:<port>]/remote_cam/[webrtc_id]
    • This page allows the phone to:
      • Select its camera using getUserMedia
      • Stream the video to the backend using WebRTC

  5. 🖥️ Frontend UI behavior:

    • While no stream is active, show the QR code in place of the camera feed preview.
    • Once the phone connects and starts streaming to the backend:
      • Show the live video preview in the miniature card on the dashboard

  6. 🔁 Stream routing from phone to frontend:

    • Phone streams video → Backend via WebRTC
    • Backend forwards (relays or rebroadcasts) the video to the frontend
    • Use either:
      • A WebRTC relay to forward the stream to the desktop browser
      • Or transcode into MJPEG or low-latency HLS/WebSocket for frontend preview
    • Keep latency low and reuse existing libraries where possible

Implementation constraints:

  • Use existing libraries for:
    • WebRTC signaling
    • QR code generation
    • Local HTTPS (e.g., mkcert, selfsigned, flask-tls)
  • Support multiple simultaneous phone camera streams
  • Keep code modular and simple

Optional:

  • Expire inactive sessions
  • Show connection status and stream quality

Related PRs

@nicolas-rabault nicolas-rabault mentioned this pull request Jun 19, 2025
Open
cursor[bot]
cursor bot reviewed Jun 19, 2025
View reviewed changes
Copy link
@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Placeholder Script Committed

The file scripts/start_webrtc_backend.py was accidentally committed as an incomplete or placeholder file, containing only a single space character. It should either be removed or properly implemented.

scripts/start_webrtc_backend.py#L1-L1

leLab/scripts/start_webrtc_backend.py

Line 1 in 0ac38c1

Fix in Cursor

Bug: Incorrect Uvicorn Launch for Phone Camera Demo

The scripts/start_phone_camera_demo.py script incorrectly launches uvicorn with app.main:app instead of app.main:main_app. The main_app object (defined in app/main.py) is the FastAPI application wrapped with Socket.IO to provide WebRTC signaling capabilities, which are essential for phone camera support. By using app.main:app, the script starts the plain FastAPI app, which lacks this crucial WebRTC functionality, rendering the phone camera features non-functional.

scripts/start_phone_camera_demo.py#L43-L52

leLab/scripts/start_phone_camera_demo.py

Lines 43 to 52 in 0ac38c1

# Start backend with SSL certificates
backend_cmd = [
"uvicorn",
"app.main:app",
"--host", "0.0.0.0",
"--port", "8000",
"--ssl-keyfile", "certs/key.pem",
"--ssl-certfile", "certs/cert.pem",
"--reload"
]

Fix in Cursor

Bug: WebRTC Cleanup Task Not Triggered

The start_cleanup_task() function is defined but never called, preventing the cleanup of expired WebRTC sessions. This leads to memory leaks as session data accumulates indefinitely in active_sessions, session_clients, active_streams, and stream_buffers.

app/webrtc_signaling.py#L335-L341

leLab/app/webrtc_signaling.py

Lines 335 to 341 in 0ac38c1

# Periodic cleanup task
async def start_cleanup_task():
"""Start periodic cleanup of expired sessions"""
import asyncio
while True:
await asyncio.sleep(300) # Check every 5 minutes
await cleanup_expired_sessions()

Fix in Cursor

BugBot free trial expires on July 22, 2025
You have used $0.00 of your $50.00 spend limit so far. Manage your spend limit in the Cursor dashboard.

Was this report helpful? Give feedback by reacting with 👍 or 👎

@nicolas-rabault
Copy link
Owner Author
nicolas-rabault commented Jun 19, 2025

@jurmy24 @DavidLMS I worked on implementing this very interesting feature, which was the main focus of our hackathon.
I managed to build a dedicated camera configuration interface for recording, and successfully generated a QR code on the miniature. Scanning it with a phone opens an HTTPS page that is trying to starts streaming the phone’s camera to the backend via WebRTC.
However, I'm still facing some connection issues that I haven’t been able to resolve.
Could you have a look.

@DavidLMS DavidLMS mentioned this pull request Jun 19, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cursor cursor[bot] cursor[bot] left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@nicolas-rabault
0