llm-server-aio

An all in one llm server configured from a docker compose file

This guide was made to be used on a debian server

Prerequisites

These are commands i think worked for me, i currently didn't have time to try again a clean install of the nvidia drivers

apt update
apt install linux-headers-$(uname -r)

# https://docs.docker.com/engine/install/debian/
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh ./get-docker.sh

Clone the repo :

git clone https://github.com/AArbey/AIO-LLM.git
cd AIO-LLM

Copy the .env.template and edit it with your own info.

cp .env.template .env
nano .env

Run with docker compose

docker compose -f docker-compose-aio.yml up -d

Name		Name	Last commit message	Last commit date
Latest commit History 724 Commits
.github		.github
core		core
example/custom_actions		example/custom_actions
pkg		pkg
services		services
tests/e2e		tests/e2e
webui		webui
.dockerignore		.dockerignore
.env.template		.env.template
.gitignore		.gitignore
.goreleaser.yml		.goreleaser.yml
Dockerfile.realtimesst		Dockerfile.realtimesst
Dockerfile.sshbox		Dockerfile.sshbox
Dockerfile.webui		Dockerfile.webui
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
docker-compose-aio.yml		docker-compose-aio.yml
docker-compose.amd.yaml		docker-compose.amd.yaml
docker-compose.intel.yaml		docker-compose.intel.yaml
docker-compose.nvidia.yaml		docker-compose.nvidia.yaml
docker-compose.yaml		docker-compose.yaml
go.mod		go.mod
go.sum		go.sum
jsconfig.json		jsconfig.json
main.go		main.go
slack.yaml		slack.yaml
start_realtimesst.sh		start_realtimesst.sh