Production Debugging with HTTPS Visibility

Get HTTPS traffic visibility in 30 seconds for urgent production debugging - no permanent installation needed.

Who This Is For

Use this guide when you:

  • Have a production issue RIGHT NOW that needs immediate investigation

  • Need to see inside HTTPS traffic without certificates or proxies

  • Want to debug authentication failures (401/403), API errors, or rate limiting

  • Need temporary visibility without permanent installation

This IS for:

  • Emergency troubleshooting and incident response

  • Short-term traffic capture (hours to days)

  • One-off debugging sessions

  • Proof that Qtap works in your environment

This is NOT for:

  • Long-term production monitoring (use Complete Guide for that)

  • Selective capture with complex rules (use Complete Guide for rulekit)

  • Multi-environment visibility (consider Qplane)

Time to results: 30 seconds (demo mode) or 5 minutes (full install)

🔒 Your Data Stays Local

This guide uses standalone Qtap with stdout output—all captured traffic stays on your server. No data is sent to Qpoint or any external service. Everything you capture remains in your terminal or log files under your control.

Want centralized visibility across a fleet with alerting? That's Qplane (our cloud control plane), but this guide keeps everything local and private.

Temporary Installation

The fastest way to see HTTPS traffic. No installation, no configuration file—just one command that downloads qtap to /tmp and starts capturing immediately.

One-Line Demo

SSH into your production server and run:

Prefer not to pipe curl to sudo? We get it. See manual installation instructions to download and verify the binary yourself before running.

Qtap will:

  1. Download the latest qtap binary to /tmp (no permanent installation)

  2. Start capturing all HTTP/HTTPS traffic with full visibility

  3. Display readable console output in real-time (access_logs plugin, full mode)

What You'll See

Captured traffic appears immediately:

Visual Alternative: For interactive inspection with a Chrome DevTools-style interface, see the DevTools UI guide. Perfect for exploring traffic visually or troubleshooting via SSH tunnel from your laptop.

Stop the Demo

Press Ctrl+C to stop qtap. The binary remains in /tmp and will be cleaned up on system reboot.

Pro tip: The demo script uses a default configuration (access_logs, full mode, all traffic). For production-safe filtering and custom rules, continue to the sections below.

Full Installation

This option gives you full control with custom configuration. Perfect for production-safe debugging with filters and rules.

Prerequisites

  • SSH access to production server (Linux with kernel 5.10+)

  • Root/sudo privileges

  • Running application to debug

Step 1: Create Minimal Configuration

SSH into your production server and create /tmp/qtap-debug.yaml:

Step 2: Install and Start Qtap

Install qtap with a single command:

Prefer not to pipe curl to sudo? See manual installation instructions to download, extract, and verify the binary yourself.

Start qtap (runs in foreground, shows captured traffic in real-time):

Qtap displays captured HTTPS traffic in real-time. Output is also saved to /tmp/qtap-output.log for later analysis.

Step 3: Generate Test Traffic

Open a second SSH session to your server (keep qtap running in the first session), then generate test traffic:

Or simply wait for your production application to generate traffic—you'll see it appear in real-time in your first SSH session.

Step 4: What You'll See

Readable console output (example 401 error from httpbin):

Key indicators of successful capture:

  • ✅ Process identified (/usr/bin/curl)

  • ✅ Request headers visible (including Authorization token!)

  • ✅ Response body readable (no base64 encoding)

  • ✅ Complete HTTPS traffic despite encryption

Step 5: Cleanup

When debugging is complete, press Ctrl+C in the qtap session to stop it, then:

Production Debugging Configuration

The quick start captures everything at full detail. For production use, this single configuration works for all common debugging scenarios—just adjust what you look for in the output.

Need advanced capture options? This guide uses the access_logs plugin for simplicity. For selective capture with rules, custom filtering, or other plugins (like http_capture), see Traffic Processing with Plugins.

Base Configuration

Create /tmp/qtap-debug.yaml:

Start capturing:

Now use a second SSH session to generate traffic (or just wait for your production app to make or receive requests).

Common Debugging Scenarios

Same config, different things to look for in the captured output.

Scenario 1: Authentication Failures

Problem: Third-party API returning 401/403 errors

Generate test traffic:

What to look for in qtap output:

  • Status codes: 401 Unauthorized or 403 Forbidden

  • Authorization header: Check if token is present and formatted correctly

  • Response body: Error message often explains the issue (expired token, wrong scope, etc.)

  • Process: Which service is making the failing request (exe: /usr/bin/python3, container name)

Scenario 2: Slow API Responses

Problem: External APIs taking too long to respond

What to look for in qtap output:

  • Duration: Response time shown in milliseconds at top of each capture

  • Bytes Sent/Received: Large payloads can cause slowness

  • Process: Which service is making slow requests

  • Host: Which external API is slow

Pro tip: Use mode: summary instead of full if you don't need bodies—just timing data.

Scenario 3: Rate Limiting

Problem: Hitting API rate limits (429 responses)

Generate test traffic:

What to look for in qtap output:

  • Status code: 429 Too Many Requests

  • Rate-limit headers: X-RateLimit-Remaining, X-RateLimit-Reset, Retry-After

  • Request frequency: Count how many requests to the same host in a short time window

  • Process: Which service is making too many requests

Scenario 4: External API Errors

Problem: Third-party API integration suddenly failing

What to look for in qtap output:

  • Status codes: 500, 502, 503, 504 (server errors)

  • Error response bodies: Often contain specific error messages or codes

  • Request bodies: Verify payload format is correct

  • Headers: Check required headers are present (Content-Type, API version headers)

  • Process: Which service is affected

Reducing Noise with Filters

In busy production environments, you'll capture a lot of traffic. Use filters to focus on what matters.

Filter by Process

Ignore noisy health checks, monitoring agents, or specific services:

Common filters:

  • Health checks: /usr/bin/health-check, /usr/bin/curl (if used for monitoring)

  • Monitoring agents: .*datadog.*, .*newrelic.*

  • Internal tools: /usr/local/bin/metrics-collector

Filter by Domain

Capture only specific domains by setting the default stack to none and overriding for specific endpoints:

How it works: The default http.stack applies to all traffic. Specific domains under endpoints override the default. With stack: none as the default, only the listed domains are captured.

When to use domain filtering:

  • Debugging a specific third-party integration

  • High-volume environments where you need to focus

  • Investigating issues with a particular external service

Adjust Capture Level

Start with less detail, increase as needed by changing the mode value in your config:

Available modes:

  • summary: Method, URL, status, timing - no headers or bodies (minimal overhead)

  • details: Summary + all headers - no bodies (auth/rate-limit debugging)

  • full: Everything including headers and bodies (complete debugging)

Tip: Start with summary, look at the list of requests, then restart qtap with full mode to dig into specific issues.

Filter by Response Code

Focus on errors, authentication failures, or rate limiting by capturing only specific HTTP status codes:

How it works: All traffic is captured at summary level (minimal overhead), but requests with status codes ≥ 400 are automatically upgraded to full mode with complete headers and bodies.

Common response code filters:

  • Authentication issues: http.res.status == 401 || http.res.status == 403

  • Rate limiting: http.res.status == 429

  • Client errors: http.res.status >= 400 && http.res.status < 500

  • Server errors: http.res.status >= 500

When to use response code filtering:

  • High-volume production debugging where you only care about failures

  • Authentication troubleshooting (capture 401/403 fully, ignore successful 200s)

  • Rate limit investigation (capture 429 responses with all headers)

  • Error diagnosis without capturing thousands of successful requests

Understanding Captured HTTPS Data

Qtap's eBPF hooks capture traffic at the kernel level before TLS encryption, giving you complete visibility into HTTPS traffic without proxies or certificate management.

What You Can See

Request details:

  • HTTP method (GET, POST, PUT, DELETE, etc.)

  • Full URL (scheme, host, path, query parameters)

  • All request headers (Authorization, Content-Type, custom headers)

  • Request body (when mode is full)

Response details:

  • Status code (200, 401, 500, etc.)

  • All response headers (Content-Type, rate-limit headers, cookies)

  • Response body (when mode is full)

  • Response timing (duration in milliseconds)

Process attribution (unique to eBPF-based capture):

  • Executable path (/usr/bin/curl, /usr/bin/python3, etc.)

  • Process ID (PID)

  • Container name (Docker/Kubernetes)

  • Container/pod labels

TLS information:

  • TLS detected (is_tls: true)

  • TLS library used (tlsProbeTypesDetected: ["openssl"])

  • Protocol version (http1, http2)

Capture Levels Explained

Level
What's Captured
Use Case

summary

Method, URL, status, timing, process info

High-level monitoring, minimal overhead

details

Summary + all headers (no bodies)

Debug headers, auth tokens, rate limits

full

Everything (headers + request/response bodies)

Complete debugging, error diagnosis

Example Captured Request (Full Mode)

This is what you'll see with mode: full and format: console:

Production Safety

Performance

  • Start with mode: summary for minimal overhead

  • Use process or domain filters to reduce volume

  • Typical overhead: 1-3% CPU

Sensitive Data

Captured data may contain API keys and PII. For quick debugging:

  • Data stays on your server in /tmp

  • Remove logs when done: rm /tmp/qtap-*.log

Cleanup

Need help? For additional resources:

PreviousDebugging & TroubleshootingNextAdvanced Use Cases

Last updated