Architecture
Your documents
↓
Ollama (embeddings) → pgvector (storage)
↓
Query → pgvector (search) → Ollama (generation) → AnswerEverything runs on your VPS. No external APIs required.
What You Need
- VPS with 8 GB RAM
- Ollama installed with a 7B model
- Docker for PostgreSQL with pgvector
Step 1: Set Up PostgreSQL with pgvector
mkdir ~/rag && cd ~/rag
nano docker-compose.ymlversion: '3.8'
services:
postgres:
image: pgvector/pgvector:pg16
restart: always
environment:
POSTGRES_DB: rag
POSTGRES_USER: rag
POSTGRES_PASSWORD: changeme
ports:
- "5432:5432"
volumes:
- pgdata:/var/lib/postgresql/data
volumes:
pgdata:docker compose up -dStep 2: Install Python Dependencies
apt install -y python3-pip
pip3 install psycopg2-binary ollama --break-system-packagesStep 3: Create the Database Schema
# setup_db.py
import psycopg2
conn = psycopg2.connect(
host="localhost", database="rag",
user="rag", password="changeme"
)
cur = conn.cursor()
cur.execute("CREATE EXTENSION IF NOT EXISTS vector")
cur.execute("""
CREATE TABLE IF NOT EXISTS documents (
id SERIAL PRIMARY KEY,
content TEXT,
embedding vector(768),
source TEXT
)
""")
cur.execute("""
CREATE INDEX IF NOT EXISTS documents_embedding_idx
ON documents USING ivfflat (embedding vector_cosine_ops)
""")
conn.commit()
cur.close()
conn.close()
print("Database ready")python3 setup_db.pyStep 4: Index Your Documents
# index.py
import ollama
import psycopg2
conn = psycopg2.connect(
host="localhost", database="rag",
user="rag", password="changeme"
)
cur = conn.cursor()
def index_document(text, source):
response = ollama.embeddings(
model="nomic-embed-text",
prompt=text
)
embedding = response['embedding']
cur.execute(
"INSERT INTO documents (content, embedding, source) VALUES (%s, %s, %s)",
(text, embedding, source)
)
conn.commit()
# Pull the embedding model first
# ollama pull nomic-embed-text
# Example: index a document
with open("your-document.txt") as f:
content = f.read()
# Split into chunks (simple approach)
chunks = [content[i:i+500] for i in range(0, len(content), 500)]
for i, chunk in enumerate(chunks):
index_document(chunk, f"document-chunk-{i}")
print(f"Indexed chunk {i+1}/{len(chunks)}")
cur.close()
conn.close()Step 5: Query Your Documents
# query.py
import ollama
import psycopg2
conn = psycopg2.connect(
host="localhost", database="rag",
user="rag", password="changeme"
)
cur = conn.cursor()
def query(question):
# Get embedding for the question
response = ollama.embeddings(
model="nomic-embed-text",
prompt=question
)
q_embedding = response['embedding']
# Find similar documents
cur.execute("""
SELECT content FROM documents
ORDER BY embedding <=> %s::vector
LIMIT 3
""", (q_embedding,))
context = "\n\n".join([row[0] for row in cur.fetchall()])
# Generate answer
prompt = f"""Based on the following context, answer the question.
Context:
{context}
Question: {question}
Answer:"""
response = ollama.generate(
model="llama3.1:8b",
prompt=prompt
)
return response['response']
print(query("What are the main points of the document?"))Running this on NoctHost
The generation model sets the size: a 7B model plus Postgres and pgvector runs comfortably on the Pro plan (4 vCPU, 8 GB), while a lighter embedding-only pipeline fits the Standard plan (2 vCPU, 4 GB). NoctHost bills hourly from a prepaid, crypto-funded balance, no card and no KYC, so you can prototype a RAG stack and tear it down without a monthly bill.