Skip to main content
Enterprise Only

This section is only relevant to Enterprise customers who acquired an on-prem license.

Troubleshooting Guide

Solutions for common issues you might encounter with Permit Platform.

Pre-Installation Issues​

Git Repository Access​

❌ Problem: "Permission denied (publickey)" when testing Git access

Test Git connection:

ssh -T git@github.com -i ./permit-policy-key

Solutions:

  1. Verify public key is added to repository:

    • Check GitHub/GitLab/Bitbucket settings
    • Ensure "write access" is enabled for deploy key

  2. Check private key permissions:

    chmod 600 permit-policy-key
    ls -la permit-policy-key  # Should show -rw-------

  3. Verify repository URL format:

    # ✅ Correct formats:
    git@github.com:yourorg/permit-policies.git
    git@gitlab.com:yourorg/permit-policies.git

    # ❌ Incorrect formats:
    https://github.com/yourorg/permit-policies.git
    github.com:yourorg/permit-policies.git

Deploy Key Issues​

⚠️ Problem: "Deploy key already in use" (GitHub)

Solution: Each GitHub deploy key can only be used by one repository.

# Generate a new key pair
ssh-keygen -t rsa -b 4096 -f permit-policy-key-new -N ""

# Add permit-policy-key-new.pub to your repository
# Use permit-policy-key-new in installation

Server Requirements​

❌ Problem: Server doesn't meet minimum requirements

Check server resources:

# Check CPU cores
nproc

# Check memory
free -h

# Check disk space
df -h

# Check OS version
cat /etc/os-release

Minimum requirements:

  • CPU: 4 cores
  • RAM: 8GB
  • Storage: 50GB available
  • OS: Ubuntu 18.04+, RHEL/CentOS 7+

Installation Issues​

Permission Errors​

❌ Problem: "Permission denied" during installation

Solution:

# Always run installer with sudo
sudo ./install.sh

# Check if you have sudo access
sudo whoami  # Should output "root"

Disk Space Issues​

❌ Problem: "No space left on device"

Check disk usage:

# Check available space
df -h

# Check largest directories
du -sh /* | sort -hr | head -10

# Clean up if needed
sudo apt-get clean  # Ubuntu/Debian
sudo yum clean all  # RHEL/CentOS

Port Conflicts​

❌ Problem: "Port already in use" (80, 443, 6443)

Check what's using ports:

# Check port 80
sudo netstat -tulpn | grep :80
# or
sudo lsof -i :80

# Check port 443
sudo netstat -tulpn | grep :443

# Check k3s API port
sudo netstat -tulpn | grep :6443

Solutions:

  1. Stop conflicting services:

    sudo systemctl stop apache2  # or nginx, httpd
    sudo systemctl disable apache2

  2. Use different ports (advanced):

    sudo ./install.sh --http-port 8080 --https-port 8443

k3s Installation Failures​

❌ Problem: k3s fails to start or install

Check k3s logs:

# Check k3s service status
sudo systemctl status k3s

# View k3s logs
sudo journalctl -u k3s -f

# Check k3s node status
sudo k3s kubectl get nodes

Common solutions:

  1. Restart k3s:

    sudo systemctl restart k3s

  2. Reinstall k3s:

    sudo /usr/local/bin/k3s-uninstall.sh
    sudo ./install.sh

  3. Check system requirements:

    # Ensure sufficient memory
    free -h

    # Check if swap is disabled (recommended)
    sudo swapoff -a

Post-Installation Issues​

Cannot Access Web Interface​

❌ Problem: Cannot reach the web interface

Diagnostic steps:

  1. Check platform status:

    ./scripts/status.sh

  2. Test local connectivity:

    # Test from server itself
    curl -k https://localhost
    curl -k https://$(hostname -I | awk '{print $1}')

  3. Check ingress configuration:

    ./scripts/logs.sh ingress
    ./scripts/status.sh --ingress

  4. Verify DNS/networking:

    # If using custom domain
    nslookup permit.yourcompany.com

    # Test from client machine
    telnet <server-ip> 443

Service Startup Issues​

❌ Problem: Services failing to start or crashing

Check service health:

# Check all services
./scripts/status.sh --detailed

# Check specific service logs
./scripts/logs.sh backend
./scripts/logs.sh celery
./scripts/logs.sh opal-server

# Check resource usage
./scripts/status.sh --resources

Common solutions:

  1. Restart services:

    ./scripts/restart.sh

  2. Check resource constraints:

    # Check if out of memory
    dmesg | grep -i "killed process"

    # Check disk space
    df -h

  3. Scale down if resource-constrained:

    ./scripts/scale.sh backend --replicas 1
    ./scripts/scale.sh celery --replicas 1

Database Connection Issues​

❌ Problem: Services cannot connect to database

Check database status:

# Check PostgreSQL pod
./scripts/status.sh database

# Check database logs
./scripts/logs.sh postgres

# Test database connectivity
./scripts/test-database.sh

Solutions:

  1. Restart database:

    ./scripts/restart.sh postgres

  2. Check database resources:

    ./scripts/status.sh postgres --resources

  3. Verify database initialization:

    ./scripts/logs.sh postgres | grep -i "database system is ready"

Git Sync Issues​

❌ Problem: Policy sync failing from Git repository

Test Git connectivity:

# Test SSH connection
ssh -T git@github.com -i ./permit-policy-key

# Check policy sync logs
./scripts/logs.sh policy-sync

# Test Git from within platform
./scripts/test-git.sh

Solutions:

  1. Verify Git credentials:

    # Recreate Git configuration
    ./scripts/update-git-config.sh \
      --repo "git@github.com:yourorg/permit-policies.git" \
      --key "./permit-policy-key"

  2. Manual sync:

    ./scripts/policy-sync.sh --force

  3. Check repository permissions:

    • Verify deploy key still exists in repository
    • Ensure write access is still enabled

Performance Issues​

Slow Response Times​

⚠️ Problem: Platform responding slowly

Diagnostic steps:

  1. Check resource usage:

    ./scripts/status.sh --resources
    htop  # or top

  2. Check service response times:

    # Test API response time
    time curl -k https://localhost/health

    # Check database performance
    ./scripts/metrics.sh --database

  3. Scale up if needed:

    ./scripts/scale.sh backend --replicas 3
    ./scripts/scale.sh celery --replicas 2

High Resource Usage​

⚠️ Problem: High CPU or memory usage

Investigation:

# Check top resource consumers
./scripts/metrics.sh --top-consumers

# Check for memory leaks
./scripts/metrics.sh --memory-analysis

# Review service resource limits
./scripts/status.sh --resource-limits

Solutions:

  1. Optimize resource allocation:

    ./scripts/resources.sh backend --memory 2Gi --cpu 1000m

  2. Enable auto-scaling:

    ./scripts/scale.sh --auto-enable

Recovery Procedures​

Emergency Recovery​

If the platform is completely down:

# 1. Check system health
./scripts/emergency.sh --diagnose

# 2. Try emergency restart
./scripts/emergency.sh --restart

# 3. If restart fails, try safe mode
./scripts/emergency.sh --safe-mode

# 4. Last resort: reinstall
sudo ./scripts/cleanup.sh
sudo ./install.sh

Data Recovery​

If you need to recover from backup:

# List available backups
./scripts/backup.sh --list

# Restore from specific backup
./scripts/restore.sh --backup backup-20241201-143022.tar.gz

# Verify restoration
./scripts/status.sh --verify

Getting Support​

Before Contacting Support​

  1. Collect comprehensive logs:

    ./scripts/collect-logs.sh --include-system
    # Creates: permit-platform-logs-YYYYMMDD-HHMMSS.tar.gz

  2. Gather system information:

    ./scripts/system-info.sh > system-info.txt

  3. Document the issue:

    • What were you trying to do?
    • What happened instead?
    • When did it start happening?
    • Any recent changes?

Contact Information​

  • 📧 Email: support@permit.io
  • 💬 Slack: #support channel
  • 🕐 Response time: Within 24 hours

What to Include​

Always attach:

  • ✅ Logs bundle from collect-logs.sh
  • ✅ System information
  • ✅ Platform version and build ID
  • ✅ Detailed description of the issue
  • ✅ Steps to reproduce (if applicable)

Still having issues? Our support team is here to help. Don't hesitate to reach out with your logs and system information.

Contents