Troubleshooting

Troubleshooting

This page covers common issues and their solutions.

Startup Issues

"Connection refused" to PostgreSQL

Symptom:

sqlalchemy.exc.OperationalError: connection refused

Solutions:

1. Check if PostgreSQL is running:

   docker compose ps postgres

2. Check the port in your .env:

   # Port 5432 for both Full Docker and Hybrid modes
   DATABASE_URL=postgresql+asyncpg://ontokit:ontokit@localhost:5432/ontokit

3. Wait for PostgreSQL to be ready:

   docker compose logs -f postgres
   # Look for "database system is ready to accept connections"

"Connection refused" to Redis

Symptom:

redis.exceptions.ConnectionError: Connection refused

Solution:

docker compose up -d redis
docker compose logs redis

"Address already in use"

Symptom:

OSError: [Errno 98] Address already in use

Solutions:

1. Find what's using the port:

   lsof -i :8000
   # or
   netstat -tlnp | grep 8000

2. Kill the process:

   kill <PID>

3. Or use a different port:

   uvicorn ontokit.main:app --port 8001

Authentication Issues

"Token validation failed"

Symptom:

{"detail": "Token validation failed: ..."}

Solutions:

1. Check Zitadel is running:

   curl http://localhost:8080/debug/ready

2. Verify ZITADEL_ISSUER in .env matches Zitadel URL:

   ZITADEL_ISSUER=http://localhost:8080

3. Token may be expired - get a new one

4. Clear JWKS cache by restarting the API server

"Not authenticated"

Symptom:

{"detail": "Not authenticated"}

Solutions:

1. Ensure the Authorization header is included:

   curl -H "Authorization: Bearer <token>" ...

2. Check the token format (should be Bearer <token>, not just <token>)

CORS Errors in Browser

Symptom:

Access to fetch at 'http://localhost:8000/...' from origin 'http://localhost:3000'
has been blocked by CORS policy

Solution:

Add your frontend URL to CORS_ORIGINS in .env:

CORS_ORIGINS=["http://localhost:3000","http://localhost:3001"]

Then restart the API server.

Database Issues

Migration Errors

Symptom:

alembic.util.exc.CommandError: Can't locate revision identified by '...'

Solutions:

1. Check current revision:

   alembic current

2. If database is ahead of code, downgrade:

   alembic downgrade <target_revision>

3. If completely out of sync, reset:

   # WARNING: This deletes all data
   docker compose down -v
   docker compose up -d
   alembic upgrade head

"Table already exists"

Symptom:

sqlalchemy.exc.ProgrammingError: relation "projects" already exists

Solution:

Stamp the database with the current revision:

alembic stamp head

"Column does not exist"

Symptom:

sqlalchemy.exc.ProgrammingError: column "new_column" does not exist

Solution:

Run pending migrations:

alembic upgrade head

API Errors

500 Internal Server Error

Symptom:

Internal Server Error

Solutions:

1. Check the server logs in your terminal

2. Enable debug mode in .env:

   DEBUG=true

3. Test the specific operation in Python:

   import asyncio
   from ontokit.services.project_service import ProjectService
   # ... debug interactively

422 Validation Error

Symptom:

{
  "detail": [
    {
      "loc": ["body", "name"],
      "msg": "field required",
      "type": "value_error.missing"
    }
  ]
}

Solution:

Check your request body matches the expected schema. See API Reference for required fields.

"Not Found" for New Endpoints

Symptom:

{"detail": "Not Found"}

Solutions:

1. Restart the server to pick up new routes:

   # Kill and restart uvicorn
   pkill -f uvicorn
   uvicorn ontokit.main:app --reload

2. Check the router is registered in ontokit/api/routes/

3. Verify the endpoint path is correct

Docker Issues

Containers Keep Restarting

Solution:

# Check logs for the failing container
docker compose logs zitadel

# Check health status
docker compose ps

Out of Disk Space

Solution:

# Remove unused Docker resources
docker system prune -a

# Remove only unused volumes
docker volume prune

Can't Connect to Container Services

Solutions:

1. Check container is running:

   docker compose ps

2. Check port mappings:

   docker compose port postgres 5432

3. Check container logs:

   docker compose logs <service>

Performance Issues

Slow Database Queries

Solutions:

1. Enable SQL logging:

   DEBUG=true

2. Check for missing indexes

3. Use EXPLAIN ANALYZE in psql:

   EXPLAIN ANALYZE SELECT * FROM projects WHERE owner_id = '...';

High Memory Usage

Solutions:

1. Check container resource usage:

   docker stats

2. Limit container memory in compose.yaml:

   services:
     postgres:
       deploy:
         resources:
           limits:
             memory: 512M

Zitadel Issues

"App not found" or "invalid_request"

Symptom:

{"error":"invalid_request","error_description":"Errors.App.NotFound"}

Solution:

The OAuth applications haven't been created in Zitadel. Run the setup script:

./scripts/setup-zitadel.sh --update-env

Then recreate the API container: docker compose up -d --force-recreate api worker

"Unknown error occurred" on Login

Symptom: You enter credentials correctly but get a generic error.

Cause: Zitadel logs show Errors.User.GrantRequired - the project has role checking enabled but users don't have grants.

Solution:

Disable project role checking:

# Get admin PAT
docker cp ontokit-zitadel:/zitadel-data/admin.pat /tmp/admin.pat
PAT=$(cat /tmp/admin.pat)

# Get project ID
PROJECT_ID=$(curl -s -H "Authorization: Bearer $PAT" \
  -X POST "http://localhost:8080/management/v1/projects/_search" \
  -H "Content-Type: application/json" \
  -d '{"queries":[{"nameQuery":{"name":"OntoKit","method":"TEXT_QUERY_METHOD_EQUALS"}}]}' \
  | python3 -c "import sys, json; print(json.load(sys.stdin)['result'][0]['id'])")

# Disable role check
curl -X PUT "http://localhost:8080/management/v1/projects/$PROJECT_ID" \
  -H "Authorization: Bearer $PAT" \
  -H "Content-Type: application/json" \
  -d '{"name": "OntoKit", "projectRoleAssertion": false, "projectRoleCheck": false}'

Login Page Shows Error

Solutions:

1. Check Zitadel logs:

   docker compose logs zitadel

2. Verify redirect URIs match exactly in Zitadel console

3. Clear browser cookies and try again

Can't Create Applications

Solution:

Ensure you're logged in as admin and have the correct organization selected.

Background Job Issues

Embedding Generation Stuck or Failed

Symptom: Embedding generation shows "pending" indefinitely or job status shows failure.

Solutions:

1. Check the ARQ worker is running:

   # In Full Docker mode
   docker compose logs worker
   
   # In Hybrid mode, start the worker manually:
   uv run arq ontokit.worker.WorkerSettings

2. Check Redis is available:

   docker exec ontokit-redis redis-cli ping

3. For OpenAI/Voyage providers, verify the API key is set in the embedding config via the API endpoint.

Normalization Job Fails

Symptom: Normalization job returns error status.

Solutions:

1. Check worker logs for the specific error:

   docker compose logs worker | grep normalization

2. Ensure the ontology file is valid Turtle/RDF by checking the lint endpoint first.

ARQ Worker Not Processing Jobs

Symptom: Jobs stay in "pending" state.

Solutions:

1. Verify the worker container is running:

   docker compose ps worker

2. Check Redis connection:

   docker compose logs worker | grep redis

3. Restart the worker:

   docker compose restart worker

Embedding / Semantic Search Issues

"pgvector extension not available"

Symptom: Semantic search or duplicate detection returns errors about pgvector.

Solution:

The pgvector extension must be installed in your PostgreSQL instance. The Docker Compose setup includes it by default. If using an external PostgreSQL:

CREATE EXTENSION IF NOT EXISTS vector;

Embedding Provider Errors

Symptom: Embedding generation fails with API errors.

Solutions:

1. For OpenAI: Verify your API key is valid and has sufficient credits
2. For Voyage: Verify your API key and model name
3. For local (sentence-transformers): Ensure sufficient memory for model loading (~500MB)
4. Check the embedding config: GET /api/v1/projects/{id}/embeddings/config

Getting Help

If you can't resolve an issue:

1. Check the FastAPI documentation
2. Check the SQLAlchemy documentation
3. Search existing GitHub issues
4. Create a new issue with:
   • Steps to reproduce
   • Expected behavior
   • Actual behavior
   • Error messages/logs
   • Environment details (OS, Python version, etc.)

Quick Reference

Restart Everything

docker compose restart
pkill -f uvicorn
uvicorn ontokit.main:app --reload

Reset Database

docker compose down -v
docker compose up -d
# Reconfigure Zitadel applications and update .env files
./scripts/setup-zitadel.sh --update-env
# Recreate API/worker containers for new credentials
docker compose up -d --force-recreate api worker
# Run migrations
docker compose exec api alembic upgrade head

View All Logs

docker compose logs -f

Check Service Health

# API
curl http://localhost:8000/health

# Zitadel
curl http://localhost:8080/debug/ready

# PostgreSQL
docker exec ontokit-postgres pg_isready

# Redis
docker exec ontokit-redis redis-cli ping