Skip to content
Wallpaper Server Wallpaper Server Docs

Local semantic wallpaper search

This advanced feature is under development in PR #42. It is optional: the Wallpaper Server service does not start a model, download weights, transmit pictures to an AI provider or require Ollama for normal image browsing, uploads or rotation.

How it works

Section titled “How it works”

The CLI uses existing wallpaper metadata and thumbnail APIs. When you explicitly run semantic search, it retrieves up to a bounded number of indexed JPEG thumbnails, asks a locally installed multimodal embedding model to represent both your text query and the thumbnails as vectors, then ranks those images by cosine similarity. It does not alter the server index, rename images or update rotation history.

This is genuine visual/text embedding search—not keyword expansion over filenames or tags. The quality depends on the installed model. It is not a substitute for full-library indexing: the initial implementation searches only the first --max-candidates visible images returned by the API.

Prepare a local model

Section titled “Prepare a local model”

Install Ollama from the official project and pull a model supporting both image and text embeddings in its POST /api/embed implementation. The image-input protocol is confirmed in Ollama v0.40.16; older text-only embedding APIs are not compatible. The CLI requires both embedding and vision capabilities from /api/show before sending your query or thumbnails. A normal vision chat model or a text-only embedding model is not sufficient. A compatible local model is an external prerequisite; protocol tests do not establish its retrieval quality.

Run Ollama on 127.0.0.1:11434 or [::1]:11434. Local model discovery is checked before any image is sent. Cloud-backed models reported by the local Ollama instance are refused.

Section titled “Search”
Terminal window
wallpaper-server semantic search \
  --model YOUR_LOCAL_MULTIMODAL_EMBEDDING_MODEL \
  --query "misty pine forest under a midnight sky" \
  --max-candidates 30 --limit 10


# Machine-readable ranking:
wallpaper-server semantic search \
  --model YOUR_LOCAL_MULTIMODAL_EMBEDDING_MODEL \
  --query "minimal blue abstract landscape" --json


# Use a remote wallpaper library, but keep inference on this machine:
wallpaper-server semantic search \
  --server https://wallpapers.example.com \
  --ollama http://127.0.0.1:11434 \
  --model YOUR_LOCAL_MULTIMODAL_EMBEDDING_MODEL \
  --query "snowy mountains" --max-candidates 20

Results include an image ID, filename and cosine similarity score. Scores are relative similarities, not calibrated probabilities. JSON output also includes the model name and number of successfully embedded images.

Security and resource limits

Section titled “Security and resource limits”
  • The Ollama endpoint must be an HTTP numeric loopback IP, with no embedded credentials, proxy or cross-origin redirects.
  • The CLI checks the installed model list and rejects entries advertising remote/cloud backing. Do not configure the local Ollama daemon to proxy inference to an untrusted service.
  • The thumbnails are obtained through the wallpaper server’s existing API. The client sends them only to its configured local loopback model endpoint; no original source paths or full-resolution image files are required.
  • Each thumbnail has a 4 MiB bound. The initial scan is restricted to 1–100 candidate images, queried in batches of at most four. Output is limited to 1–50 matches and all HTTP requests have bounded timeouts.
  • No vector database or external account is required. The initial CLI search intentionally does not persist its embeddings between invocations.
  • The wallpaper server can still be remote: use HTTPS or a trusted tunnel, and protect admin tokens as described in the security guide.

A model without multimodal support, a missing local model, a malformed embedding or a dimensional mismatch causes an explicit error rather than silently presenting keyword results as semantic matches. Thumbnail retrieval failures reduce coverage instead of breaking unrelated gallery features.

Next-stage optimisation

Section titled “Next-stage optimisation”

Persistent per-model thumbnail embeddings, automatic invalidation on source revision changes, background indexing, local model auto-discovery and full-library search are future optimisations. Model/version pinning and persistent index concurrency must be verified before shipping them.