Send your ngrok traffic as logs to Datadog

June 12, 2023

min read

Today, we’re excited to announce ngrok’s integration with Datadog Logs. You can now send logs of your ngrok traffic to Datadog with just a few clicks.

We are committed to making ngrok the best choice to serve your production websites and APIs. First-class observability is a key requirement for production workloads. This release continues that investment by expanding upon our logging integrations to include Datadog Log Management alongside AWS CloudWatch Logs, Kinesis and Firehose.

The Datadog integration is simple to configure, yet powerful. With it, you can:

Capture detailed logs at the TCP or HTTP layer
Introspect ngrok middleware behaviors like the authenticated user on each HTTP request
Filter which events are sent to Datadog with CEL predicates

The Datadog Log Management integration is available in the free tier up to 10,000 events per month, so all ngrok developers can add observability to their apps in minutes.

Try it out

Quickstart Guide: A detailed, step-by-step version of the video below.
Events Documentation: How ngrok’s event logging system works
Events Reference Docs: All event types and fields supported by the event logging system‍

Configuring the Datadog event destination takes only 3 steps: create a Datadog API key, set up an event subscription in ngrok, and then test the connection:

How it Works

ngrok captures traffic to your applications as it flows through our ingress platform and sends those logs to Datadog. You don’t have to change your application. A Datadog API key you specify during configuration is used to authenticate with your Datadog account.

Traffic Introspection

The Datadog Log Management integration allows you to customize the observability of your ngrok traffic to the level of detail you need. You may choose to capture traffic events as either TCP Connections (Layer 4) or HTTP Requests (Layer 7).

{
  "id": "AgAAAYiS_WwPbFrGPQAAAAAAAAyYTdmM2E2NjAy",
  "content": {
    "timestamp": "2023-06-06T23:14:21.839Z",
    "tags": [ "source:ngrok", "environment:production", "datadog.submission_auth:private_api_key" ],
    "service": "ngrok-logs",
    "attributes": {
      "event_id": "ev_2Qqu6v8RmxLbTTTebpnnjlBRArn",
      "event_type": "http_request_complete.v0",
      "service": "ngrok-logs",
      "event_timestamp": "2023-06-06T23:14:20Z",
      "status": "info",
      "object": {
        "conn": {
          "start_ts": "2023-06-06T23:14:20.472173436Z",
          "server_name": "sso-demo.sudobinbash.com",
          "server_ip": "172.10.14.172",
          "client_ip": "71.192.143.219",
          "server_port": 443
        },
        "http": {
          "request": {
            "method": "GET",
            "url": {
              "path": "/",
              "scheme": "https",
              "query": "",
              "host": "sso-demo.sudobinbash.com",
              "raw": "https://sso-demo.sudobinbash.com/"
            },
            "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36",
            "body_length": 0
          },
          "response": {
            "status_code": 304,
            "body_length": 0
          }
        },
        "tls": {
          "cipher_suite": "TLS_AES_128_GCM_SHA256",
          "version": "1.3"
        }
      }
    }
  }
}

Because ngrok is a unified ingress platform, you don't need to correlate logs between multiple proxies. All of the standard fields you’d expect from access logs like HTTP method, path, bytes sent+received and source IP address are available. When ngrok applies middleware like OAuth, mTLS or Load Balancing, your traffic log events will also include observability on the behaviors of those modules.

For instance, you may choose to include data for each request sent to Datadog about the authenticated OAuth user, whether a request was rejected by Circuit Breaker middleware, or how many bytes were saved by gzip compression.

"ip_policy": {
  "decision": "allow"
},
"circuit_breaker": {
  "decision": "allow"
},
"compression": {
  "bytes_saved": 192830,
  "algorithm": "gzip"
},
"oauth": {
  "decision": "allow",
  "user": {
    "name": "Jane Doe",
    "id": "102387230973973"
  },
  "app_client_id": "70012i08iu28uw-jsp8ue918uw10u8w2082wj2iw.apps.googleusercontent.com"
}

Filtering

ngrok enables you to filter exactly which events will be sent to Datadog as well as select exactly which fields within traffic event logs are captured. This enables you to cut through the noise and focus on the traffic that matters as well as to only pay for what you need to collect.

Traffic filtering is as easy as writing a simple CEL expression to define which events you want to send to Datadog. For example, if you were only concerned with errors for a particular customer, you might want to explicitly log just their HTTP errors:

ev.oauth.user.name == ‘customer@example.com’ && ev.http.response.status_code >= 400

Visualization

If you require finer-grain control over your visualizations, the Datadog allows you to configure both the tags and service name associated with the log messages so that you can better filter, aggregate, and compare events in Datadog visualizations. Tags and service names enable you to uniquely identify your ngrok events among other logs and craft queries, monitors and visualizations that target them without having to learn a complex or Datadog-specific query language.

Service name and event tags from ngrok help you uniquely identify and target events

Audit events are logs too

ngrok’s integration with Datadog is not limited to logging just the traffic flowing to your applications. You can use the same system to satisfy the requirements of your security and compliance teams that need to record changes in configuration and security posture by recording audit events of changes to your ngrok account in Datadog Log Management.

ngrok’s audit events are comprehensive of all CRUD operations against your ngrok account resources (API Keys, Authtokens, Domains, IP Policies, etc). Audit events include who made those changes and are captured regardless of whether those changes were made via ngrok’s API or the dashboard.

API

Like all ngrok resources, the Datadog integration can be configured programmatically via API. The API resource is documented here and our API client libraries in all major languages have been updated to support the integration. You can try it out provisioning the Datadog integration quickly via our API with the following request:

curl --location --request POST 'https://api.ngrok.com/event_destinations' \
--header 'Authorization: Bearer API_KEY_FROM_NGROK' \
--header 'Content-Type: application/json' \
--data-raw '{
    "description": "datadog log",
    "format": "json",
    "target": {
        "datadog": {
            "api_key": "API_KEY_FROM_NGROK",
            "ddtags": "environment:production",
            "service": "ngrok-service",
            "ddsite": "US5"
        }
    }
}'

Where else can you send your ngrok logs?

In addition to Datadog, you can also send your ngrok traffic logs to AWS services including CloudWatch Logs, Kinesis and Firehose. Want to send your traffic logs elsewhere? Let us know!

Get started

The Datadog Log Management integration is available in our free tier up to 10,000 events per month, so all ngrok developers can add observability to their apps. The following resources will help you get started:

Quickstart Guide: Follow the datadog integration guide to get started with ngrok events.
Events Documentation: How ngrok’s event logging system works
Events Reference Docs: All event types and their fields supported by the event logging system‍

‍

Share this post

Alan Shreve

Alan Shreve is no stranger to building distributed systems at scale. He organically grew ngrok from zero to 5 million users before raising $50 million Series A.