Manage, monitor, troubleshoot, and develop with locally running Airbyte instances using abctl CLI and kubectl. Use when working with local Airbyte deployments for tasks like (1) diagnosing sync issues (long-running jobs, failed syncs, no output), (2) checking Airbyte service health and pod status, (3) investigating configuration problems, (4) monitoring job progress and logs, (5) verifying data persistence to destinations like S3, (6) debugging connection or authentication errors, (7) managing Airbyte lifecycle (install/restart/uninstall), or (8) any other local Airbyte operations requiring abctl, kubectl, or API interactions.
How this skill is triggered — by the user, by Claude, or both
Slash command
/airbyte-local-manager:airbyte-local-managerThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
Manage and troubleshoot local Airbyte instances autonomously using the abctl CLI, kubectl, and Airbyte Public API.
Manage and troubleshoot local Airbyte instances autonomously using the abctl CLI, kubectl, and Airbyte Public API.
When investigating issues, use this iterative pattern:
Never stop at the first observation. Always investigate deeper by checking logs, pod status, configurations, and external services.
# Airbyte status
abctl local status
# Pod health
kubectl --kubeconfig ~/.airbyte/abctl/abctl.kubeconfig --namespace airbyte-abctl get pods
# Connection and job status (if project has manage_pipeline.py)
python manage_pipeline.py status
Use the diagnostic script:
python scripts/diagnose_sync.py <connection-id>
This automated script checks:
python scripts/check_s3_sync.py <connection-id> --bucket <bucket-name> --prefix <prefix>
Confirms whether data is actually being written to S3.
Scenario: Sync shows "running" for 20+ hours, no output to destination.
Steps:
Run diagnostic script:
python scripts/diagnose_sync.py <connection-id>
Check if data is reaching destination:
# For S3 destinations
python scripts/check_s3_sync.py <connection-id> --bucket <bucket>
# Or directly
aws s3 ls s3://<bucket>/<prefix>/ --recursive --human-readable | tail -20
If no S3 data, check worker pod logs:
# Find worker pods
kubectl --kubeconfig ~/.airbyte/abctl/abctl.kubeconfig --namespace airbyte-abctl get pods | grep worker
# Check logs
kubectl --kubeconfig ~/.airbyte/abctl/abctl.kubeconfig --namespace airbyte-abctl logs <worker-pod> --tail=100
# Look for errors
kubectl --kubeconfig ~/.airbyte/abctl/abctl.kubeconfig --namespace airbyte-abctl logs <worker-pod> | grep -i "error\|exception\|fail"
Check pod events for crashes or restarts:
kubectl --kubeconfig ~/.airbyte/abctl/abctl.kubeconfig --namespace airbyte-abctl describe pod <pod-name>
# Look at Events section
Test credentials:
# For AWS destinations
aws sts get-caller-identity
# For S3 bucket access
aws s3 ls s3://<bucket>/
Based on findings:
For detailed troubleshooting steps, see: references/troubleshooting_playbook.md
Watch sync job in real-time:
# Monitor pods
watch -n 5 'kubectl --kubeconfig ~/.airbyte/abctl/abctl.kubeconfig --namespace airbyte-abctl get pods'
# Follow worker logs
kubectl --kubeconfig ~/.airbyte/abctl/abctl.kubeconfig --namespace airbyte-abctl logs -f <worker-pod>
# Check job status via API
curl -H "Authorization: Bearer $AIRBYTE_SECRET_ACCESS_TOKEN" \
"http://localhost:8000/api/public/v1/jobs?connectionId=<ID>&limit=5"
When pods are unhealthy or configuration changes require restart:
# Restart specific deployment
abctl local deployments --restart <deployment-name>
# Example: restart worker
abctl local deployments --restart airbyte-abctl-worker
# Full restart (preserves data)
abctl local uninstall && abctl local install
# Complete wipe (DELETES ALL DATA - use with caution)
abctl local uninstall --persisted && abctl local install
When escalating an issue or doing deep investigation:
# 1. Overall status
abctl local status > diagnostic_report.txt
# 2. Pod status
kubectl --kubeconfig ~/.airbyte/abctl/abctl.kubeconfig --namespace airbyte-abctl get pods >> diagnostic_report.txt
# 3. Recent events
kubectl --kubeconfig ~/.airbyte/abctl/abctl.kubeconfig --namespace airbyte-abctl get events --sort-by='.lastTimestamp' >> diagnostic_report.txt
# 4. Save all pod logs
mkdir -p airbyte-logs
for pod in $(kubectl --kubeconfig ~/.airbyte/abctl/abctl.kubeconfig --namespace airbyte-abctl get pods -o name | cut -d'/' -f2); do
kubectl --kubeconfig ~/.airbyte/abctl/abctl.kubeconfig --namespace airbyte-abctl logs $pod > airbyte-logs/$pod.log 2>&1
done
# 5. Connection and job details via API
curl -H "Authorization: Bearer $AIRBYTE_SECRET_ACCESS_TOKEN" \
http://localhost:8000/api/public/v1/connections/<ID> >> diagnostic_report.txt
# Status
abctl local status # Check installation status
abctl local credentials # Get login credentials
# Management
abctl local install # Install Airbyte
abctl local install --low-resource-mode # For systems < 4 CPUs
abctl local uninstall # Remove (keeps data)
abctl local deployments --restart <name> # Restart component
# Debug
abctl -v local status # Verbose output
# Kubeconfig path
KUBECONFIG=~/.airbyte/abctl/abctl.kubeconfig
# Pod status
kubectl --namespace airbyte-abctl get pods
kubectl --namespace airbyte-abctl describe pod <pod-name>
# Logs
kubectl --namespace airbyte-abctl logs <pod-name>
kubectl --namespace airbyte-abctl logs -f <pod-name> # Follow
kubectl --namespace airbyte-abctl logs <pod-name> --previous # After crash
# Interactive access
kubectl --namespace airbyte-abctl exec -it <pod-name> -- bash
# Authentication
export AIRBYTE_SECRET_ACCESS_TOKEN="<token>"
# List connections
curl -H "Authorization: Bearer $AIRBYTE_SECRET_ACCESS_TOKEN" \
"http://localhost:8000/api/public/v1/connections?workspaceIds=<ID>"
# Get connection details
curl -H "Authorization: Bearer $AIRBYTE_SECRET_ACCESS_TOKEN" \
http://localhost:8000/api/public/v1/connections/<CONNECTION_ID>
# List jobs
curl -H "Authorization: Bearer $AIRBYTE_SECRET_ACCESS_TOKEN" \
"http://localhost:8000/api/public/v1/jobs?connectionId=<ID>&limit=10"
# Trigger sync
curl -X POST \
-H "Authorization: Bearer $AIRBYTE_SECRET_ACCESS_TOKEN" \
-H "Content-Type: application/json" \
-d '{"connectionId":"<ID>","jobType":"sync"}' \
http://localhost:8000/api/public/v1/jobs
For complete command reference: references/abctl_commands.md For complete API reference: references/api_endpoints.md
Common pod patterns:
airbyte-abctl-server-* - API serverairbyte-abctl-worker-* - Sync workerairbyte-abctl-workload-launcher-* - Job launcherairbyte-abctl-db-* - PostgreSQL databaseairbyte-abctl-webapp-* - Web UIsource-* - Source connector (temporary, per-sync)destination-* - Destination connector (temporary, per-sync)Comprehensive automated diagnostics:
python scripts/diagnose_sync.py <connection-id>
Checks:
Outputs:
Verify data persistence to S3:
python scripts/check_s3_sync.py <connection-id> --bucket <bucket-name> [--prefix <prefix>] [--hours 48]
Checks:
Outputs:
Step-by-step procedures for:
Load when: Facing specific error patterns or need detailed diagnostic steps.
Complete command reference for:
Load when: Need specific command syntax or exploring available options.
Airbyte Public API documentation:
Load when: Need to interact with Airbyte API programmatically or inspect configurations.
Don't accept the first symptom as the full story:
When investigating issues, check from top to bottom:
abctl local status)kubectl get pods)/jobs)kubectl logs)The diagnostic scripts automate common patterns:
diagnose_sync.py as first step for any sync issuecheck_s3_sync.py to verify destination persistenceAirbyte uses UUIDs for all resources:
Always confirm IDs before operations. Get them from:
explore_airbyte.py scriptInstead of assuming what's wrong:
# Airbyte API token
export AIRBYTE_SECRET_ACCESS_TOKEN="<token>"
# AWS credentials (for S3 destinations)
export AWS_ACCESS_KEY_ID="<key>"
export AWS_SECRET_ACCESS_KEY="<secret>"
export AWS_REGION="us-west-2"
Add to ~/.bashrc or ~/.zshrc:
# Kubeconfig
export KUBECONFIG=~/.airbyte/abctl/abctl.kubeconfig
# kubectl shortcut
alias kab='kubectl --namespace airbyte-abctl'
# Common commands
alias airbyte-status='abctl local status && kab get pods'
alias airbyte-logs='kab logs'
alias airbyte-pods='kab get pods'
npx claudepluginhub berrydev-ai/berrydev-plugins --plugin airbyte-local-managerManages and troubleshoots Apache Airflow using af CLI: lists DAGs, triggers runs, reads task logs, diagnoses failures, checks connections, variables, pools, and health.
Diagnoses why a data warehouse sync is failing and recommends the right recovery action. Covers source-level vs schema-level failures, stuck Running states, credential and schema-drift errors, and incremental-field misconfigurations.
Manages MongoDB Atlas Stream Processing pipelines: provisions workspaces, configures Kafka/Atlas/S3/Lambda connections, operates processors, debugs issues, sizes tiers.