Data processing in REDS

There is a vast amount of data included in Raw Event Data Stream (REDS) feeds, and every partner has the ability to modify which columns are included from each stream. To gather the schema specific to your REDS feed efficiently, the platform offers a suite of REDS API endpoints that allow retrieving the definitions of all your REDS feeds and the enumeration maps for all impression-feed facets.

For a complete list of streams, see REDS feeds.

REDS primary streams processing tips

The following sections provide processing tips for each of the primary REDS streams, with guidance on how to interpret and join the data.

Impressions stream

All won impression events are tracked in the Impressions stream. It records the users (cookies or device IDs) to whom an impression was served, along with their associated metadata.

The following section describes how data is processed in an impression stream.

• The impression ID is an identifier of an impression event in a partner account, but it is not considered to be the 'unique identifier' for every event.
• Due to how we process our logs, some data that is captured near the beginning of an hour may be logged in the previous hour's folder. For example, an impression event that occurred at 14:01:03 UTC may be filed in the folder for hour 13 instead of hour 14.

Clicks stream

All recorded click events are tracked in the Clicks stream. It records the users (their cookies or device IDs) that clicked a creative, along with their associated metadata.

The following section describes how to process data in an clicks stream by using the impression ID.

• To join Clicks to Impressions, use the impression ID. Since there is a direct link between clicks and impressions, you can also join on the bid request ID or display impression ID.
• Though very rare, there can be instances in which a click event cannot be attributed to an Impressions event, but can be tracked back to a bid. This can happen when the feedback call gets modified before we have an opportunity to record it. This is just an anomaly of the Internet. Certain browsers with an ad blocker can change the query parameters on our feedback pixel, disabling our ability to parse and record impressions; however, the click generated from the impression is perfectly valid.

Video events stream

The Video events stream is a post-impression event feed with data pertaining to viewability, video events, and video-quartile metrics.

NOTE: This event feed also contains audio event data.

The following section describes how to process data in an video event stream by using the impression ID.

• Impression IDs included within the Video events feed can be joined to an impression ID within the Impressions feed.
• Not every impression will have a post-impression event recorded in this feed. Multiple records are expected for a given impression ID, as each record will define the tracked activity. For example, you will receive one record if the video was in view, and a separate record if the video started playing, etc.
• We track viewability on a sample of your total impressions. If you would like the sample rate increased, contact with your Account Manager.

Conversions stream

The Conversions includes unprocessed conversion data that records events that can and cannot be attributed to paid media. This is particularly useful for clients who apply advanced attribution models and attempt to understand the incremental impact of paid media. In other words, with this feed, our partners can analyze what proportion of the audience converts due to the impact of paid advertising, compared to the proportion of the audience who would have converted without advertising (considered to be loyal customers).

Unlike Impressions, Conversions don't have any direct relationship to Clicks, you must join them on the identifiers (combined identifier). So, for all queries of the data set, be sure to include log entry time and TDID in your output.

TIP: To ensure the identifier a vendor originally used to target a user, such as a UID2, RampID, or DAID, is preserved in reporting, even if the impression was purchased using a different identifier, use Originating IDs.

To perform last-touch conversion attribution (click or non-click) with raw event data, complete the following steps:

1. Get all conversion events from your preferred tracking tag ID. The tracking tag is a conversion pixel associated to your campaign that captures your conversion events.
2. Dedupe conversions that have the same order ID (namely, if the same user converts within your conversion deduplication window). Define this list as "potential conversion events."
3. Retrieve all impressions and clicks for the campaign and specified date range for which you are running your attribution model.
4. Join Impressions to Conversions, where both of the CombinedIdentifier and CombinedIdentifierType fields in the Conversions feed are equal to those in the Impressions feed. Define this list as "potential view-through."
5. Join Clicks to Conversions, where both of the CombinedIdentifier and CombinedIdentifierType fields in the Conversions feed are equal to those in the Impressions feed, and click DisplayImpressionId is equal to an impression ImpressionId. Define this list as "potential click-through."
6. Order the lists retrieved in steps 2, 4, and 5 by timestamp. This aggregated list provides you with a chronological view of a user's activity across the various feeds.
7. In the new aggregated list, discard all records for a user (CombinedIdentifier and CombinedIdentifierType fields) if the impression or click-through to a conversion falls outside the defined conversion attribution window for the advertiser.
8. Assign conversion credit based on the following rules:
   • For a given CombinedIdentifier field, if a conversion happens before an impression or click, discard the conversion and associated impression or click events.
   • For a given CombinedIdentifier field, if a click happens before the conversion event, attribute that as a click conversion. If there are multiple clicks for the same TDID, attribute the conversion only to the last click.
   • If there are only impressions before the conversion (no clicks), then attribute that as a view-through conversion. If there are multiple impressions, only attribute the conversion to the last impression.

Additional feeds processing tips

The following sections provide processing tips for each of the additional REDS streams you can enable in addition to the primary streams.

Viewability and post-events feeds

The Viewability and post-events feeds are third-party post-impression event feeds that provide viewability-related metrics, such as the trackability of the creative and content risk scores.

The following section describes how to process data in each viewability and post event feed by using the impression ID.

• Impression IDs included within the viewability feed can be joined to an impression ID within the impressions feed.
• Not every impression will have a post-impression event recorded in this feed. Multiple records are expected for a given impression ID, as each record will define the tracked activity. For example, you will receive one record if the video was in view, and a separate record if the video started playing, etc.
• We track viewability on a sample of your total impressions. If you would like the sample rate increased, contact with your Account Manager.

GDPR consent stream

The GDPR consent stream is provided in addition to the REDS event feeds. It can be used to access GDPR consent strings associated with events provided in the Impressions, Clicks, and Video Events streams and to inform your company’s specific legal bases for processing the event data. For data scrubbing and retention details, see REDS data practices.

IMPORTANT: GDPR requires consent to be specific, and while the Trade Desk may have legal bases for processing data, this is not a guarantee that you will have such legal bases.

The following section describes how data is processed in the GDPR Consent Stream.

• Certain fields in the GDPR Consent Stream may be modified to comply with regional regulations.
• You can associate data in the GDPR Consent stream to Impressions, Clicks, and Video Event by joining on an impression ID.
• To check the relevant purposes for which the corresponding impressions can be used, use the standard specified in the GDPRConsentStandard field to open the consent string.

For details on the consent string, see Transparency and Consent String with Global Vendor & CMP List Formats.