[go: up one dir, main page]

{ const container = $el; // The div with overflow const item = document.getElementById('sidebar-current-page') if (item) { const containerTop = container.scrollTop; const containerBottom = containerTop + container.clientHeight; const itemTop = item.offsetTop - container.offsetTop; const itemBottom = itemTop + item.offsetHeight; // Scroll only if the item is out of view if (itemBottom > containerBottom - 200) { container.scrollTop = itemTop - (container.clientHeight / 2 - item.offsetHeight / 2); } } })" class="bg-background-toc dark:bg-background-toc fixed top-0 z-40 hidden h-screen w-full flex-none overflow-x-hidden overflow-y-auto md:sticky md:top-16 md:z-auto md:block md:h-[calc(100vh-64px)] md:w-[320px]" :class="{ 'hidden': ! $store.showSidebar }">
Contact support

Models

Requires: Docker Compose 2.38.0 and later

The top-level models section declares AI models that are used by your Compose application. These models are typically pulled as OCI artifacts, run by a model runner, and exposed as an API that your service containers can consume.

Services can only access models when explicitly granted by a models attribute within the services top-level element.

Examples

Example 1

services:
  app:
    image: app
    models:
      - ai_model


models:
  ai_model:
    model: ai/model

In this basic example:

  • The app service uses the ai_model.
  • The ai_model is defined as an OCI artifact (ai/model) that is pulled and served by the model runner.
  • Docker Compose injects connection info, for example AI_MODEL_URL, into the container.

Example 2

services:
  app:
    image: app
    models:
      my_model:
        endpoint_var: MODEL_URL

models:
  my_model:
    model: ai/model
    context_size: 1024
    runtime_flags: 
      - "--a-flag"
      - "--another-flag=42"

In this advanced setup:

  • The service app references my_model using the long syntax.
  • Compose injects the model runner's URL as the environment variable MODEL_URL.

Attributes

  • model (required): The OCI artifact identifier for the model. This is what Compose pulls and runs via the model runner.
  • context_size: Defines the maximum token context size for the model.
  • runtime_flags: A list of raw command-line flags passed to the inference engine when the model is started.

Additional resources

For more examples and information on using model, see Use AI models in Compose