PII normalization, hashing, and encoding

Personally identifiable information (PII) refers to personal data, such as email addresses, phone numbers, credit card numbers, social security numbers, and other types of data that can be used to identify a specific user. For example, Unified ID 2.0 (UID2), a privacy-conscious identity solution, is based on email addresses and phone numbers. See also Unified IDs.

IMPORTANT: This section is for cleaning your PII data and not for creating UID2s. To create a UID2, use UID2 endpoints.

The following sections provide information and examples to help you correctly prepare your PII for uploading to The Trade Desk platform.

Requirements

Here's what you need to know about uploading CRM data to The Trade Desk platform:

• The platform can normalize email addresses or you can do it yourself.
• You must always normalize phone numbers before uploading:
  • For phone numbers, the platform removes only non-numerical symbols.
  • If the country code is missing, the platform cannot accurately normalize phone numbers.
• To normalize phone numbers (required) or if you prefer to normalize, hash, and encode email addresses yourself, refer to the following sections based on the type of PII:
  • Email addresses
  • Phone numbers

TIP: To test the normalization and hashing of email addresses and phone numbers, use the UID2 Hashing Tool. It provides the Base64-encoded value.

Email addresses

Here's what you need to know about uploading email addresses:

• Raw email addresses are normalized automatically. This means that if you upload unhashed (raw) email addresses, you don't need to normalize them.
• To send hashed email addresses, you must normalize them before hashing and encoding them.

Valid input for email addresses

If you're sending email addresses to CRM, the input data must be in one of the formats shown in the following table.

Email address normalization

The platform automatically normalizes uploaded raw email addresses. If you want to hash the email addresses yourself before sending them, you must normalize them before you hash them.

IMPORTANT: Normalization before hashing ensures that the generated ID value will always be the same, and the data can be matched accurately.

Here is some key general information relating to normalization and encoding for UID2:

• Raw UID2s, and their associated UID2 tokens, are case sensitive. When working with UID2, it's important to pass all IDs and tokens without changing the case. Mismatched IDs can cause ID parsing or token decryption errors.
• If you miss any of the required steps—for example, you hash without first normalizing—the result will not be the correct valid UID2 value for the input data.
  For example, let's say a data provider wants to generate a UID2 from JANESaoirse@gmail.com. This normalizes to janesaoirse@gmail.com, and the hashed and Base64-encoded value is ku4mBX7Z3qJTXWyLFB1INzkyR2WZGW4ANSJUiW21iI8=.
  The publisher, with the same email address, by mistake does not normalize. The hashed and Base64-encoded value for the un-normalized email, JANESaoirse@gmail.com, is VpLXEp5N1bj/V1WzjgZsC+FfuYdntAOywSVIO00FD/E=. These two different values result in two different UID2s. The first, processed correctly, matches other instances generated from the same original data. The second, incorrectly processed, does not.
  In this scenario, because the UID2 does not match other instances for the same user, the publisher misses the opportunity to benefit from targeted advertising.

To normalize an email address, complete the following steps:

1. Remove leading and trailing spaces.
2. If there are uppercase characters, convert them to lowercase.
3. In gmail.com addresses only:

   1. If there is a period (.) in the address (ASCII decimal code 46/UTF-8 hexadecimal code 2E), remove it.

      For example, normalize jane.doe@gmail.com to janedoe@gmail.com.

   2. If there is a plus sign (+) with an additional string after it, before the @gmail.com, remove the plus sign (+) (ASCII decimal code 43/UTF-8 hexadecimal code 2B) and all subsequent characters.

      For example, normalize janedoe+home@gmail.com to janedoe@gmail.com.

IMPORTANT: Make sure that the normalized email is UTF-8, not another encoding system such as UTF-16.

Email address hashing and encoding

An email hash is a Base64-encoded SHA-256 hash of a normalized email address. The email address is first normalized, then hashed using the SHA-256 hashing algorithm, and then the resulting bytes of the hash value are encoded using Base64 encoding. Note that the Base64 encoding is applied to the bytes of the hash value, not the hex-encoded string representation.

The following table shows an example of a simple input phone number, and the result as each step is applied to arrive at a secure, opaque value.

The final value, the hex to Base64 encoded representation of the SHA-256 hash, is the value to provide to the UID2 Operator endpoint.

IMPORTANT: When applying Base64 encoding, be sure to use a function that takes a hex value as input. If you use a function that takes text as input, the result is a longer string which is invalid for the purposes of UID2.

IMPORTANT: When applying Base64 encoding, be sure to Base64 encode the raw bytes of the hash or use a Base64 encoder that takes a hex-encoded value as input.

Normalization examples for email

The following table shows examples of original email addresses and the normalized and hashed values.

Some of the examples show email addresses that include the plus sign (+) with different domains, or the period in the first part of the email (for example, My.Email@example.com). For gmail addresses, the plus sign and following characters, up to the @ sign, are ignored in normalization, and the period in the first part of the email is also ignored. For other domains, these characters are included in the normalized value.

Phone numbers

Here's what you need to know about uploading phone numbers:

• You must normalize phone numbers before sending them in a request, regardless of whether you send them hashed or unhashed in a request.
• To upload normalized, hashed, and encoded CRM data, you must send phone numbers as Base64-encoded SHA-256 hashes of the normalized phone numbers.

Valid input for phone numbers

The platform removes all non-numerical symbols from phone numbers, but cannot normalize to correct values if the country code is missing. Because of this, normalizing phone numbers is always the responsibility of the uploader.

If you're sending phone numbers to CRM, the input data must be in one of the formats shown in the following table.

Phone number normalization

Whether you want to upload raw or hashed phone numbers, you must normalize them.

IMPORTANT: Normalization before hashing ensures that the generated ID value will always be the same, and the data can be matched accurately.

Here's what you need to know about phone number normalization requirements:

• The UID2 Operator accepts phone numbers in the E.164 format, which is the international telephone number format that ensures global uniqueness.
• E.164 phone numbers can have a maximum of 15 digits.
• Normalized E.164 phone numbers use the following syntax: [+][country code][subscriber number including area code] (with no spaces, hyphens, parentheses, or other special characters). Here are a few examples:
  • US: 1 (234) 567-8901 is normalized to +12345678901.
  • Singapore: 65 1243 5678 is normalized to +6512345678.
  • Australia: mobile phone number 0491 570 006 is normalized to add the country code and drop the leading zero: +61491570006.
  • UK: mobile phone number 07812 345678 is normalized to add the country code and drop the leading zero: +447812345678.

WARNING: Make sure that the normalized phone number is UTF-8, not another encoding system such as UTF-16.

Phone number hashing and encoding

A phone number hash is a Base64-encoded SHA-256 hash of a normalized phone number. The phone number is first normalized, then hashed using the SHA-256 hashing algorithm, and then the resulting bytes of the hash value are encoded using Base64 encoding. Note that the Base64 encoding is applied to the bytes of the hash value, not the hex-encoded string representation.

The following table shows an example of a simple input phone number, and the result as each step is applied to arrive at a secure, opaque value.

IMPORTANT: When applying Base64 encoding, be sure to use a function that takes a hex value as input. If you use a function that takes text as input, the result is a longer string which is invalid for the purposes of UID2.

FAQs

Here are some commonly asked questions about PII normalization.

Can I use the data output as a UID?

No. You can call the UID2 API to create a UID. The data outputs in this section here are to normalize, hash, or encode forms of PII data for more data security. The outputs cannot be used as a UID or generate UID Tokens.

Do I have to normalize, hash, and encode PII data when sending it to the platform?

No. You can upload raw data to the platform, and we will normalize, hash, and encode it for you.

Is it safe to send raw data?

Yes. We store PII in a separate, secure domain (called The Trade Desk Secured Environment) only temporarily and immediately delete it after UIDs are created.

What if I don't normalize my data before hashing it?

Not normalizing data before hashing might result in a different IDs. Mismatched data will reduce the effectiveness of targeted advertising.

Do I apply Base64 encoding to the SHA-256-encoded string or the normalized PII?

You must apply Base64 encoding to the raw 32-byte SHA-256 hash of your normalized PII, not to the normalized PII string or the 64-character hex-encoded representation of the SHA-256 hash provided in the examples.

TIP: To test the normalization and hashing of email addresses and phone numbers, use the UID2 Hashing Tool. It provides the Base64-encoded value.