GitHub Plugin

Comprehensive GitHub integration that syncs repositories, issues, pull requests, commits, releases, and workflow data to your local PostgreSQL database with real-time webhook support.

Overview

The GitHub plugin provides complete synchronization between GitHub and your local database. It captures all aspects of your GitHub workflow including:

  • Faster Queries - Query your entire GitHub history without API calls or rate limits
  • Cross-Repository Analytics - Aggregate data across all repos in a single SQL query
  • Custom Dashboards - Build engineering metrics dashboards with your synced data
  • Offline Access - Your data is always available, even without internet
  • Real-Time Updates - Webhooks keep your local data in sync as changes happen
  • Historical Analysis - Track trends over time with full historical data

Installation

# Install the plugin
nself plugin install github

# Verify installation
nself plugin status github

Configuration

Environment Variables

# Required - GitHub Personal Access Token
# Generate at: https://github.com/settings/tokens
# Required scopes: repo, read:org, workflow, read:user
GITHUB_TOKEN=ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

# Required - PostgreSQL connection string
DATABASE_URL=postgresql://user:password@localhost:5432/nself

# Optional - Webhook signing secret
# Get from: Repository Settings > Webhooks > Edit > Secret
GITHUB_WEBHOOK_SECRET=your_webhook_secret

# Optional - Organization to sync (syncs all accessible repos if not set)
GITHUB_ORG=your-organization

# Optional - Specific repos to sync (comma-separated)
# Format: owner/repo,owner/repo2
GITHUB_REPOS=acamarata/nself,acamarata/nself-plugins

# Optional - Server configuration
PORT=3002
HOST=0.0.0.0

# Optional - Sync interval in seconds (default: 3600)
GITHUB_SYNC_INTERVAL=3600

Token Permissions

When creating a Personal Access Token (PAT), enable these scopes:

ScopePurpose
repoFull access to repositories (issues, PRs, commits)
read:orgRead organization data (teams, members)
workflowAccess to GitHub Actions workflow data
read:userRead user profile data
admin:repo_hookRequired for webhook management

Fine-Grained Tokens

For fine-grained PATs, select: Repository access (All or specific), and Permissions: Issues (read), Pull requests (read), Contents (read), Metadata (read), Workflows (read), Deployments (read).

Usage

Initialize Database Schema

# Create all required tables
nself-github init

# Or via nself CLI
nself plugin github init

Sync Data

# Sync all data from GitHub
nself-github sync

# Sync specific resources
nself-github sync --resources repos,issues,prs

# Incremental sync (only changes since last sync)
nself-github sync --incremental

# Sync a specific repository
nself-github sync --repo acamarata/nself

Start Webhook Server

# Start the server
nself-github server

# Custom port
nself-github server --port 3002

# The server exposes:
# - POST /webhook - GitHub webhook endpoint
# - GET /health - Health check
# - GET /api/* - REST API endpoints

CLI Commands

Repository Commands

# List all synced repositories
nself-github repos list

# List with details
nself-github repos list --details

# Search repositories
nself-github repos search "keyword"

# Get repository details
nself-github repos get owner/repo

# Show repository statistics
nself-github repos stats owner/repo

Issue Commands

# List all issues
nself-github issues list

# Filter by state
nself-github issues list --state open
nself-github issues list --state closed

# Filter by repository
nself-github issues list --repo owner/repo

# Filter by labels
nself-github issues list --labels bug,urgent

# Filter by assignee
nself-github issues list --assignee username

# Get issue details
nself-github issues get owner/repo 123

Pull Request Commands

# List all pull requests
nself-github prs list

# Filter by state
nself-github prs list --state open
nself-github prs list --state merged
nself-github prs list --state closed

# Filter by repository
nself-github prs list --repo owner/repo

# Filter by author
nself-github prs list --author username

# Get PR details
nself-github prs get owner/repo 456

Release Commands

# List all releases
nself-github releases list

# Filter by repository
nself-github releases list --repo owner/repo

# Get latest release
nself-github releases latest owner/repo

Workflow Commands

# List workflow runs
nself-github actions list

# Filter by status
nself-github actions list --status success
nself-github actions list --status failure

# Filter by repository
nself-github actions list --repo owner/repo

# Get run details
nself-github actions get 12345678

Status Command

# Show sync status and statistics
nself-github status

# Output:
# Repositories: 25
# Issues: 1,234 (456 open)
# Pull Requests: 789 (23 open)
# Commits: 45,678
# Releases: 156
# Workflow Runs: 3,456
# Last Sync: 2026-01-24 12:00:00

Webhook Setup

Repository Webhooks

  1. Go to Repository > Settings > Webhooks
  2. Add webhook:
    • URL: https://your-domain.com/webhooks/github
    • Content type: application/json
    • Secret: Your GITHUB_WEBHOOK_SECRET
  3. Select events:
    • Push
    • Pull requests
    • Issues
    • Workflow runs
    • Releases

Organization Webhooks

For org-wide webhooks:

  1. Go to Organization > Settings > Webhooks
  2. Same configuration as repository webhooks

Data Synchronization

ResourceSynced DataIncremental Sync
RepositoriesAll metadata, settings, topicsYes
IssuesFull issue data with labels, assigneesYes
Pull RequestsPR data, reviews, commentsYes
CommitsCommit history with diffsYes
ReleasesRelease versions with assetsYes
BranchesBranch list and protection rulesYes
TagsTag list with commit refsYes
MilestonesMilestone trackingYes
LabelsAll repository labelsYes
Workflow RunsGitHub Actions run historyYes
Workflow JobsIndividual job detailsYes
Check SuitesCheck suite resultsYes
Check RunsIndividual check resultsYes
DeploymentsDeployment historyYes
TeamsOrganization team dataYes
CollaboratorsRepository collaboratorsYes

Database Schema

Tables

TableDescription
github_repositoriesRepository metadata
github_issuesIssues with labels, assignees
github_pull_requestsPRs with merge info
github_commitsCommit history
github_releasesRelease tags
github_branchesBranch information with protection status
github_tagsGit tags
github_milestonesMilestone tracking
github_labelsRepository labels
github_workflow_runsGitHub Actions runs
github_workflow_jobsIndividual workflow job details
github_check_suitesCheck suite results
github_check_runsIndividual check run results
github_deploymentsDeployment records
github_pr_reviewsPull request reviews
github_issue_commentsIssue and PR comments
github_teamsOrganization teams
github_collaboratorsRepository collaborators
github_webhook_eventsWebhook event log

Analytics Views

ViewDescription
github_open_itemsOpen issues and PRs by repository
github_recent_activityLast 7 days activity
github_workflow_statsWorkflow success rates and durations

Example Queries

-- Open issues by repository
SELECT
  r.full_name,
  COUNT(*) as open_issues
FROM github_issues i
JOIN github_repositories r ON i.repo_id = r.id
WHERE i.state = 'open'
GROUP BY r.full_name
ORDER BY open_issues DESC;

-- PR merge rate by author
SELECT
    author,
    COUNT(*) as total_prs,
    COUNT(*) FILTER (WHERE merged_at IS NOT NULL) as merged,
    ROUND(
        COUNT(*) FILTER (WHERE merged_at IS NOT NULL)::numeric /
        NULLIF(COUNT(*), 0) * 100, 1
    ) as merge_rate_percent
FROM github_pull_requests
GROUP BY author
HAVING COUNT(*) >= 5
ORDER BY total_prs DESC
LIMIT 20;

-- Workflow success rate by repository
SELECT
    r.full_name,
    COUNT(*) as total_runs,
    COUNT(*) FILTER (WHERE w.conclusion = 'success') as success,
    COUNT(*) FILTER (WHERE w.conclusion = 'failure') as failure,
    ROUND(
        COUNT(*) FILTER (WHERE w.conclusion = 'success')::numeric /
        NULLIF(COUNT(*), 0) * 100, 1
    ) as success_rate
FROM github_workflow_runs w
JOIN github_repositories r ON w.repository_id = r.id
WHERE w.created_at > NOW() - INTERVAL '30 days'
GROUP BY r.full_name
ORDER BY total_runs DESC;

-- Development velocity
SELECT
    DATE_TRUNC('week', merged_at) as week,
    COUNT(*) as merged_prs,
    SUM(additions) as lines_added,
    SUM(deletions) as lines_deleted,
    ROUND(AVG(
        EXTRACT(EPOCH FROM (merged_at - created_at)) / 3600
    ), 1) as avg_time_to_merge_hours
FROM github_pull_requests
WHERE merged_at IS NOT NULL
  AND merged_at > NOW() - INTERVAL '12 weeks'
GROUP BY week
ORDER BY week DESC;

GraphQL Access

query GetRepositoryStats {
  github_repositories(
    order_by: { stargazers_count: desc }
    limit: 10
  ) {
    full_name
    stargazers_count
    forks_count
    open_issues_count
    github_pull_requests_aggregate(
      where: { state: { _eq: "open" } }
    ) {
      aggregate {
        count
      }
    }
  }
}

Environment Handling

Development

ENV=dev
GITHUB_TOKEN=ghp_dev_token
GITHUB_REPOS=myorg/dev-repo

Production

ENV=prod
GITHUB_TOKEN=ghp_prod_token
GITHUB_ORG=myorg

Use different tokens or sync different repos per environment.

Uninstall

# Remove plugin and data
nself plugin remove github

# Keep database tables
nself plugin remove github --keep-data

Webhook Configuration

Setting Up Webhooks

  1. Go to your repository/organization Settings
  2. Click "Webhooks" > "Add webhook"
  3. Configure:
    • Payload URL: https://your-domain.com/webhook
    • Content type: application/json
    • Secret: Your GITHUB_WEBHOOK_SECRET
  4. Select events:
    • Individual events: Check all relevant events
    • Or: Select "Send me everything"

Supported Webhook Events

  • push - Pushes to the repository
  • issues - Issue opened, edited, deleted, closed, reopened
  • pull_request - PR opened, closed, merged, edited
  • pull_request_review - PR review submitted
  • release - Release created, edited, published
  • workflow_run - GitHub Actions workflow completed
  • workflow_job - GitHub Actions job completed
  • check_suite - Check suite completed
  • check_run - Check run completed
  • deployment - Deployment created
  • deployment_status - Deployment status changed
  • create - Branch or tag created
  • delete - Branch or tag deleted
  • star - Repository starred
  • fork - Repository forked
  • And more...

Troubleshooting

Authentication Errors

Error: Bad credentials

Solution: Verify your token:

curl -H "Authorization: token $GITHUB_TOKEN" https://api.github.com/user

Rate Limiting

Error: API rate limit exceeded

Check your rate limit status:

curl -H "Authorization: token $GITHUB_TOKEN" https://api.github.com/rate_limit

Solutions:

  • Use incremental sync instead of full sync
  • Reduce the number of repositories being synced
  • Wait for rate limit to reset

Webhook Signature Failed

Error: Webhook signature verification failed

Solutions:

  1. Verify GITHUB_WEBHOOK_SECRET matches the webhook settings
  2. Ensure you are using the raw request body
  3. Check the Content-Type header is application/json

Debug Mode

LOG_LEVEL=debug nself-github sync

Integration Examples

Project Dashboard

-- Create a project health dashboard view
CREATE VIEW project_health AS
SELECT
  r.full_name,
  r.stargazers_count AS stars,
  (SELECT COUNT(*) FROM github_issues i
   WHERE i.repo_id = r.id AND i.state = 'open') AS open_issues,
  (SELECT COUNT(*) FROM github_pull_requests p
   WHERE p.repo_id = r.id AND p.state = 'open') AS open_prs,
  (SELECT AVG(CASE WHEN w.conclusion = 'success' THEN 1 ELSE 0 END) * 100
   FROM github_workflow_runs w
   WHERE w.repo_id = r.id
   AND w.created_at > NOW() - INTERVAL '7 days') AS ci_success_rate
FROM github_repositories r;

CI/CD Monitoring

// Monitor workflow failures
async function checkWorkflowHealth(repoName: string) {
  const result = await hasura.query({
    query: `
      query WorkflowHealth($repo: String!) {
        github_workflow_runs(
          where: {
            repository: { full_name: { _eq: $repo } }
            created_at: { _gte: "now() - interval '24 hours'" }
          }
          order_by: { created_at: desc }
        ) {
          id
          name
          conclusion
          created_at
          html_url
        }
      }
    `,
    variables: { repo: repoName }
  });

  const failed = result.github_workflow_runs
    .filter(r => r.conclusion === 'failure');

  if (failed.length > 0) {
    console.log(`Warning: ${failed.length} failed workflows`);
  }

  return result.github_workflow_runs;
}

Support

Related

Last Updated: January 2026 | Plugin Version 1.0.0