Local

The qpoint.yaml file is used to configure Qtap. This guide explains how to create, deploy, and maintain a local configuration file for Qtap.

Configuration File Structure

The qpoint.yaml file consists of four main sections:

1. Version and Global Settings

2. Service Configurations (event stores and object stores)

3. Stacks (plugin collections)

4. Traffic Capture Rules

Here's the basic structure:

version: 2

services:
  event_stores:
    # Event storage configuration
  object_stores:
    # Object storage configuration

stacks:
  default_stack:
    plugins:
      # Default plugin configurations
  debug_stack:
    plugins:
      # Debug-specific plugin configurations

tap:
  # Global traffic capture settings
  endpoints:
    # Endpoint-specific rules

Creating a Basic Configuration

Let's walk through creating a simple configuration file step by step.

Step 1: Set Version

Start by setting the configuration version:

version: 2

Step 2: Configure Data Storage

Next, define where captured data will be stored:

services:
  # For connection metadata (anonymized)
  event_stores:
    - id: console_stdout
      type: stdout
  
  # For actual request/response content
  object_stores:
    - id: minio
      type: s3
      endpoint: 127.0.0.1:9000
      bucket: qpoint
      region: us-east-1
      access_url: http://localhost:9000/{{BUCKET}}/{{DIGEST}}
      insecure: true
      access_key:
        type: env
        value: S3_ACCESS_KEY
      secret_key:
        type: env
        value: S3_SECRET_KEY

This configuration:

• Outputs events (connection metadata) to the console for debugging

• Stores objects (request/response content) in a locally running MinIO S3-compatible store

• Uses environment variables for S3 credentials (recommended for security)

Step 3: Set Up Processing Plugins

Define how captured data will be processed:

stacks:
  default_stack:
    plugins:
      - type: debug
        config:
          mode: summary
      - type: detect_errors
        config:
          rules:
            - name: "Server Errors"
              trigger_status_codes:
                - '5xx'
              report_as_issue: true
              record_req_headers: true
              record_req_body: true
              record_res_headers: true
              record_res_body: true

This configuration:

• Creates a stack named "default_stack" with two plugins

• The debug plugin provides summary-level debug information

• The detect_errors plugin captures detailed information for 5xx server errors

Step 4: Configure Traffic Capture

Finally, set up what traffic to capture and how to process it:

tap:
  direction: egress
  ignore_loopback: false
  audit_include_dns: true
  http:
    stack: default_stack

This configuration:

• Captures all outgoing (egress) traffic

• Includes loopback traffic

• Includes DNS information in audit logs

• Applies the default_stack to HTTP traffic

Complete Example

Putting it all together:

version: 2

services:
  event_stores:
    - id: console_stdout
      type: stdout
  object_stores:
    - id: minio
      type: s3
      endpoint: 127.0.0.1:9000
      bucket: qpoint
      region: us-east-1
      access_url: http://localhost:9000/{{BUCKET}}/{{DIGEST}}
      insecure: true
      access_key:
        type: env
        value: S3_ACCESS_KEY
      secret_key:
        type: env
        value: S3_SECRET_KEY

stacks:
  default_stack:
    plugins:
      - type: debug
        config:
          mode: summary
      - type: detect_errors
        config:
          rules:
            - name: "Server Errors"
              trigger_status_codes:
                - '5xx'
              report_as_issue: true
              record_req_headers: true
              record_req_body: true
              record_res_headers: true
              record_res_body: true

tap:
  direction: egress
  ignore_loopback: false
  audit_include_dns: true
  http:
    stack: default_stack

Deploying with Local Configuration

To use your local configuration file with Qtap, you'll need to pass it to the agent during startup.

Docker Deployment

docker run \
  --user 0:0 \
  --privileged \
  --cap-add CAP_BPF \
  --cap-add CAP_SYS_ADMIN \
  --pid=host \
  --network=host \
  -v /sys:/sys \
  -v "$(pwd):/app/config" \
  -e S3_ACCESS_KEY=your_access_key \
  -e S3_SECRET_KEY=your_secret_key \
  -e TINI_SUBREAPER=1 \
  --ulimit=memlock=-1 \
  us-docker.pkg.dev/qpoint-edge/public/qpoint:v0 \
  tap \
  --config=/app/config/qpoint.yaml

This command:

• Mounts your current directory to /app/config in the container

• Sets required environment variables for S3 credentials

• Specifies the path to your configuration file with the --config flag

Kubernetes Deployment

For Kubernetes, you'll need to create a ConfigMap with your configuration:

kubectl create configmap qpoint-config --from-file=qpoint.yaml -n qpoint

Then reference it in your Helm values:

# values.yaml
config: |
  version: 2
  services:
    # Your config here...

# Or use the config from a file
helm install qpoint-tap qpoint/qpoint-tap \
  -n qpoint \
  --create-namespace \
  --set-file config=./qpoint.yaml

Binary Deployment

If you're running the Qpoint binary directly:

export S3_ACCESS_KEY=your_access_key
export S3_SECRET_KEY=your_secret_key
./qpoint tap --config=./qpoint.yaml

Configuration Sections in Detail

Event Stores

Event stores handle anonymized metadata about connections, such as timestamps, bandwidth usage, and basic request/response information.

services:
  event_stores:
    - id: console_stdout
      type: stdout

Available event store types:

• stdout: Outputs events to the console (useful for debugging)

• pulse: Sends events to a Qpoint Pulse service (usually for cloud-connected mode)

Object Stores

Object stores handle actual request and response content, including headers and bodies, which may contain sensitive information.

services:
  object_stores:
    - id: minio
      type: s3
      endpoint: 127.0.0.1:9000
      bucket: qpoint
      region: us-east-1
      access_url: http://localhost:9000/{{BUCKET}}/{{DIGEST}}
      insecure: true
      access_key:
        type: env
        value: S3_ACCESS_KEY
      secret_key:
        type: env
        value: S3_SECRET_KEY

S3-compatible storage options require:

• endpoint: The hostname:port for the S3 service

• bucket: The S3 bucket name

• region: The S3 region (if no region, use us-east-1)

• access_url: Template URL for accessing stored objects

• insecure: When true, allows insecure (HTTP) connections

• access_key and secret_key: Authentication credentials

Stacks and Plugins

Stacks are collections of plugins that define how Qtap processes captured traffic.

stacks:
  default_stack:
    plugins:
      - type: debug
        config:
          mode: summary
      - type: detect_errors
        config:
          rules:
            - name: "API Errors"
              trigger_status_codes:
                - '4xx'
                - '5xx'
              report_as_issue: true
              record_req_headers: true
              record_req_body: true
  
  verbose_stack:
    plugins:
      - type: debug
        config:
          mode: details

Common plugins include:

Debug Plugin

Controls how much information is logged for debugging:

- type: debug
  config:
    mode: summary  # or "details" for more verbose output

Error Detection Plugin

Configures when to capture detailed information based on response status:

- type: detect_errors
  config:
    rules:
      - name: "Server Errors"
        trigger_status_codes:
          - '5xx'
        report_as_issue: true
        record_req_headers: true
        record_req_body: true
        record_res_headers: true
        record_res_body: true
      
      - name: "Not Found Errors"
        trigger_status_codes:
          - '404'
        report_as_issue: false
        record_req_headers: true
        record_req_body: false

Status codes can be:

• Individual codes: '404', '500'

• Code ranges: '4xx', '5xx'

Traffic Capture Settings

The tap section controls what traffic is captured and how it's processed.

tap:
  direction: egress
  ignore_loopback: false
  audit_include_dns: true
  http:
    stack: default_stack
  filters:
    groups:
      - kubernetes
      - app
  endpoints:
    - domain: api.payment-processor.com
      http:
        stack: payment_stack

Key settings:

• direction: Controls which traffic is captured

  • all: All traffic

  • ingress: Only incoming traffic

  • egress: All outgoing traffic

  • egress-external: Only traffic to external networks

  • egress-internal: Only traffic within internal networks

• ignore_loopback: When true, excludes traffic on the loopback interface

• audit_include_dns: When true, includes DNS information in audit logs

• filters: Controls which processes to monitor

  • groups: Process groups to include

• endpoints: Domain-specific rules

  • domain: The domain pattern to match

  • http.stack: The stack to apply for matching traffic

Advanced Configuration

Credential Management

For security reasons, it's best to use environment variables for sensitive values:

access_key:
  type: env
  value: S3_ACCESS_KEY  # Name of environment variable

This approach keeps credentials out of configuration files and allows for easier rotation.

Multiple Stacks for Different Traffic

You can create multiple stacks for different types of traffic:

stacks:
  default_stack:
    plugins:
      - type: debug
        config:
          mode: summary
  
  payment_stack:
    plugins:
      - type: debug
        config:
          mode: details
      - type: detect_errors
        config:
          rules:
            - name: "Payment Errors"
              trigger_status_codes:
                - '4xx'
                - '5xx'
              record_req_headers: true
              record_req_body: false  # Don't record payment details

tap:
  http:
    stack: default_stack
  endpoints:
    - domain: api.payment-processor.com
      http:
        stack: payment_stack

This configuration applies detailed monitoring to payment processor traffic while using summary-level monitoring for everything else.

Validating Configuration

Before deploying, validate your configuration file:

qpoint validate --config=./qpoint.yaml

This command checks for syntax errors and configuration problems.

Troubleshooting

If you encounter issues with your configuration:

1. Check Syntax: Ensure your YAML is correctly formatted

2. Validate Config: Use the validation command shown above

3. Increase Logging: Set the debug plugin to "details" mode

4. Console Output: Use a stdout event store to see events in real-time

5. Check Environment Variables: Ensure all referenced environment variables are set

Common Configuration Patterns

Development Setup

version: 2
services:
  event_stores:
    - id: console_stdout
      type: stdout
  object_stores:
    - id: console_stdout
      type: stdout
stacks:
  default_stack:
    plugins:
      - type: debug
        config:
          mode: details
tap:
  direction: egress
  ignore_loopback: false
  audit_include_dns: true
  http:
    stack: default_stack

This configuration provides maximum visibility for development.