Documentation || Examples || Cloud
Geniusrise is a modular, loosely-coupled MLOps framework designed for the era of Large Language Models, offering flexibility and standardization in designing networks of AI agents.
It defines components and orchestrates them providing observability, state management and data handling, all while supporting diverse infrastructures. With its modular and unopinonated architecture, Geniusrise empowers teams to build, share, and deploy AI across various platforms.
Geniusrise is powered by its components:
- geniusrise-text: Text components offerring:
- Inference APIs
- Bulk inference
- Fine-tuning
- geniusrise-audio: Audio components offerring:
- Inference APIs
- Bulk inference
- Fine-tuning
- geniusrise-vision: Vision components offerring:
- Inference APIs
- Bulk inference
- Fine-tuning
- geniusrise-listeners: Streaming data ingestion
- geniusrise-databases: Bulk data ingestion
The easiest way to use geniusrise is to host an API over a desired model. Use one of the examples from text, vision or audio.
Say, we are interested in running an API over a vision / multi-modal model such as bakLlava:
pip install torch
pip install geniusrise
pip install geniusrise-vision # vision multi-modal models
# pip install geniusrise-text # text models, LLMs
# pip install geniusrise-audio # audio models
Create a YAML file named genius.yml
:
version: '1'
bolts:
my_bolt:
name: VisualQAAPI
state:
type: none
input:
type: batch
args:
input_folder: ./input
output:
type: batch
args:
output_folder: ./output
method: listen
args:
model_name: 'llava-hf/bakLlava-v1-hf'
model_class: 'LlavaForConditionalGeneration'
processor_class: 'AutoProcessor'
device_map: 'cuda:0'
use_cuda: True
precision: 'bfloat16'
quantization: 0
max_memory: None
torchscript: False
compile: False
flash_attention: False
better_transformers: False
endpoint: '*'
port: 3000
cors_domain: 'http://localhost:3000'
username: 'user'
password: 'password'
Navigate to the directory and do:
genius rise
MY_IMAGE=/path/to/test/image
(base64 -w 0 $MY_IMAGE | awk '{print "{\"image_base64\": \""$0"\", \"question\": \"<image>\nUSER: Whats the content of the image?\nASSISTANT:\", \"do_sample\": false, \"max_new_tokens\": 128}"}' > /tmp/image_payload.json)
curl -X POST http://localhost:3000/api/v1/answer_question \
-H "Content-Type: application/json" \
-u user:password \
-d @/tmp/image_payload.json | jq
See usage for more advanced examples. There are over 150 examples for text, vision and audio.
- Website: geniusrise.ai
This is the base geniusrise framework, also the genius
CLI.
It provides the base structure and orchestration primitives for Spout
s and Bolt
s, and also covers operations.
- Docs: docs.geniusrise.ai
- Examples: geniusrise/examples
- Cloud: geniusrise.com