Stream Real-Time Video to a Vision Language Model

Install and verify Ollama

Install Ollama as the local VLM backend:

curl -fsSL https://ollama.com/install.sh | sh
ollama --version

Confirm that the Ollama service is running and its OpenAI-compatible endpoint is available:

sudo systemctl enable --now ollama
curl http://localhost:11434/v1/models

Expected output includes a JSON response with a data array. The array can be empty until you download a model.

Download a vision model

Download the model used by this workflow:

ollama pull gemma3:4b
curl http://localhost:11434/v1/models

The second command should list gemma3:4b. The model download can take several minutes depending on network speed.

Install Live VLM WebUI

Install pipx, add its application directory to your shell path, and install Live VLM WebUI in an isolated Python environment:

sudo apt update
sudo apt install -y pipx
pipx ensurepath
source ~/.bashrc
pipx install live-vlm-webui

Verify the installation:

command -v live-vlm-webui

Expected output should show the path to the live-vlm-webui executable.

Start Live VLM WebUI

Launch the application:

live-vlm-webui

The application generates a self-signed certificate and starts an HTTPS server on port 8090. Keep this terminal open while using the interface.

In another terminal, find the reachable address of your hardware platform:

hostname -I | awk '{print $1}'

From a browser with a webcam, open:

https://<HARDWARE_IP>:8090

Use https://, because browsers require a secure context for webcam access. Accept the self-signed certificate exception only after confirming that the address belongs to your hardware platform, then grant camera permission.

Configure the VLM backend

In the VLM API Configuration section:

1. Set API Base URL to http://localhost:11434/v1.
2. Refresh the model list.
3. Select gemma3:4b.
4. Leave Max Tokens at its default for the first test.
5. Set Frame Interval to 60 if you want less frequent analysis.

Verify real-time analysis

Select a camera and start VLM analysis. Confirm that the interface shows:

• A live video feed
• Generated analysis for sampled frames
• The selected model name and inference latency
• GPU, CPU, and memory utilization

Try the Scene Description preset, then enter a custom prompt such as:

Describe the most important objects and activity in one sentence.

The next processed frame should use the updated prompt.

Compare another model

To compare supported models, first select one from Find model recipes in the Overview tab. Download it with Ollama:

ollama pull <MODEL_NAME>

Refresh the model list in the interface, stop the current analysis, select the new model, and restart analysis. Compare response quality, inference latency, and memory utilization.

Stop or remove the application

Press Ctrl+C in the terminal running Live VLM WebUI to stop it.

To remove only Live VLM WebUI:

pipx uninstall live-vlm-webui

Ollama and downloaded models remain available. To remove an individual model without uninstalling Ollama:

ollama rm <MODEL_NAME>