LLMOps (Large Language Model Operations)

Adapting MLOps practices for the generative age.

The New Stack

Model Selection: Choosing between closed (GPT-4, Claude) vs open (Llama 3, Mistral) models.
Vector Databases: Pinecone, Weaviate, or pgvector for storing embeddings.
Orchestration: LangChain or LlamaIndex for chaining calls together.
Evaluation: How do you test a non-deterministic output? Using "LLM-as-a-Judge" or RAGAS scores.

Fine-tuning vs RAG

RAG (Context)

Giving the model a textbook. Best for adding new knowledge or reducing hallucinations.

Fine-tuning (Behavior)

Sending the model to school. Best for changing the format, tone, or style of response.