# Exporting HTTP Headers to OpenTelemetry

Qtap can promote specific HTTP headers into OpenTelemetry attributes, making them available for filtering, grouping, and alerting in your observability backend. This is useful for identity headers, custom metadata, or any request/response header you want to query on.

## Why Promote Headers?

When qtap captures HTTP traffic, it records standard metadata like method, URL, status code, and duration. But sometimes you need more — the identity of the authenticated user, the originating IP, or a custom tracking header.

With header promotion, those values appear as first-class OTel attributes in your backend. Instead of digging through raw payloads, you can:

* **Filter dashboards** by authenticated user email
* **Group metrics** by upstream service or region
* **Alert on specific header values** (e.g., missing auth headers)
* **Correlate requests** across services using custom trace headers

## How It Works

The `report_usage` plugin accepts a `headers` configuration that specifies which HTTP headers to extract. When qtap observes matching headers in captured traffic, it includes them as `customHeaders` on the reported event. In an OTel backend, these are flattened into dotted attributes.

Headers are prefixed by direction:

* `req.<Header-Name>` — from the HTTP request
* `res.<Header-Name>` — from the HTTP response

In your OTel backend, these appear as `customHeaders.req.<Header-Name>` and `customHeaders.res.<Header-Name>` due to struct flattening.

## Configuration

### Step 1: Configure an OTel Event Store

Add an `otel` event store to your qtap config. If you already have one from the [OpenTelemetry Integration guide](https://docs.qpoint.io/guides/qtap-guides/observability-and-integration/sending-qtap-events-to-opentelemetry), you can use your existing configuration.

```yaml
services:
  event_stores:
    - type: otel
      endpoint: "localhost:4317"
      protocol: grpc
      service_name: "qtap"
      environment: "production"
      tls:
        enabled: false
```

### Step 2: Add the report\_usage Plugin with Headers

Add `report_usage` to your stack with the `headers` list. Each entry is a header name to extract from observed traffic.

```yaml
stacks:
  default:
    plugins:
      - type: report_usage
        config:
          headers:
            - Cf-Access-Authenticated-User-Email
            - X-Forwarded-For
            - X-Request-Id
            - Content-Type
```

{% hint style="info" %}
Header promotion works with any event store — including `stdout` for local debugging. You don't need an OTel backend to test the feature.
{% endhint %}

### Step 3: Configure the Tap

Point your tap at the stack containing the `report_usage` plugin:

```yaml
tap:
  direction: egress
  ignore_loopback: true
  http:
    stack: default
```

### Complete Example

Here is a full `qtap.yaml` that captures HTTP traffic and promotes identity headers to OpenTelemetry:

```yaml
version: 2

services:
  event_stores:
    - type: otel
      endpoint: "localhost:4317"
      protocol: grpc
      service_name: "qtap"
      environment: "production"
      tls:
        enabled: false

  object_stores:
    - type: stdout

stacks:
  default:
    plugins:
      - type: http_metrics
      - type: access_logs
        config:
          mode: summary
          format: console
      - type: report_usage
        config:
          headers:
            - Cf-Access-Authenticated-User-Email
            - X-Forwarded-For
            - X-Request-Id
            - Content-Type

tap:
  direction: egress
  ignore_loopback: true
  http:
    stack: default
```

## What You Get in Your Backend

When qtap observes traffic containing the configured headers, they appear as dotted OTLP attributes on log records:

```
customHeaders.req.Cf-Access-Authenticated-User-Email: "user@example.com"
customHeaders.req.X-Forwarded-For: "203.0.113.50"
customHeaders.req.X-Request-Id: "abc-123-def"
customHeaders.res.Content-Type: "application/json"
```

These attributes are queryable in any OTel-compatible backend — filter dashboards, build alerts, or group by header value.

## Datadog Integration

If you use Datadog as your observability backend, promoted headers appear as log attributes that you can use in Log Explorer, dashboards, and monitors.

### Viewing Headers in Datadog

1. Open **Log Explorer** in Datadog
2. Search for `@customHeaders.req.Cf-Access-Authenticated-User-Email:*` to find all logs with that header
3. Click a log entry to see the full attribute breakdown under `customHeaders`

### Creating Facets

To use promoted headers in dashboard widgets and monitors, create a facet:

1. In Log Explorer, expand a log entry containing the header
2. Click the gear icon next to `@customHeaders.req.Cf-Access-Authenticated-User-Email`
3. Select **Create facet**
4. Give it a display name like "Authenticated User"

Once faceted, you can:

* **Group by** authenticated user in dashboard widgets
* **Filter** logs to a specific user
* **Create monitors** that alert when a specific user hits error rates

### Example Datadog Query

```
service:qtap @customHeaders.req.Cf-Access-Authenticated-User-Email:"anil@example.com" @event.summary.response_status:>=500
```

This finds all 5xx errors for a specific authenticated user.

## Common Headers to Promote

### Identity & Authentication

| Header                               | Description                          |
| ------------------------------------ | ------------------------------------ |
| `Cf-Access-Authenticated-User-Email` | Cloudflare Access authenticated user |
| `X-Forwarded-For`                    | Client IP from reverse proxies       |
| `X-Forwarded-User`                   | Authenticated user from auth proxies |
| `X-Real-Ip`                          | Original client IP                   |

### Request Tracking

| Header             | Description               |
| ------------------ | ------------------------- |
| `X-Request-Id`     | Unique request identifier |
| `X-Correlation-Id` | Cross-service correlation |
| `X-Amzn-Trace-Id`  | AWS trace identifier      |
| `Traceparent`      | W3C trace context         |

### Application Metadata

| Header                  | Description           |
| ----------------------- | --------------------- |
| `Content-Type`          | Response content type |
| `X-Cache`               | CDN cache hit/miss    |
| `X-Ratelimit-Remaining` | Rate limit status     |

{% hint style="warning" %}
**Sensitive headers:** Be careful promoting headers like `Authorization`, `Cookie`, or `Set-Cookie`. Their values appear in plaintext in your OTel backend. Only promote sensitive headers if you understand the security implications and your backend has appropriate access controls.
{% endhint %}

## Important Notes

* **Case-insensitive matching** — Header names are matched case-insensitively. `Cf-Access-Authenticated-User-Email` and `cf-access-authenticated-user-email` both work.
* **Only present headers are included** — If a configured header is not present in the actual request or response, it is simply omitted from the OTel output. No empty attributes are created.
* **Direction prefixing** — Request headers get the `req.` prefix, response headers get `res.`. This lets you distinguish the same header name appearing in both directions.
* **Works with any event store** — While this guide focuses on OTel, header promotion works with any event store including `stdout`. Use `stdout` for local debugging to verify headers are being captured correctly.
* **Works with any OTel backend** — When using an OTel event store, promoted headers use standard OTLP attributes, so they work with Datadog, Grafana, Honeycomb, New Relic, and any other OTel-compatible backend.

## Next Steps

* [OpenTelemetry Integration](https://docs.qpoint.io/guides/qtap-guides/observability-and-integration/sending-qtap-events-to-opentelemetry) — Full OTel setup guide
* [Traffic Processing with Plugins](https://docs.qpoint.io/getting-started/qtap/configuration/traffic-processing-with-plugins) — All available plugins and configuration
* [Traffic Capture Settings](https://docs.qpoint.io/getting-started/qtap/configuration/traffic-capture-settings) — Control what traffic qtap observes