User Scoring Custom Algorithms

User Scoring allows any buying partner to answer the question: How can I value an individual user by assigning them a unique base bid?

By default, an ad group assigns all users in an audience the same base bid for each impression. User Scoring provides an opportunity to assign a discrete bid value for each user, allowing a partner to override an ad group's base bid when creating data segments in The Trade Desk platform. This functionality combined with the data available in our Raw Event Data Stream solves the problem of determining the value of users that haven't been seen before.

The following diagram illustrates the workflow process.

Initial configuration

Step one - identify a data source

Once a business problem has been identified and the relevant Key Performance Indicator (KPI) selected, the right data set needs to be identified to correctly build the model. The table below shows the most common data sources depending on the main KPI:

Step two - choose an ad group

Once you've decided on a data source, you should next consider which ad group will be used to build a model for User Scoring predictions. Whichever ad group or groups you select, remember that their configuration will act as a statistical experiment to generate the sample data that will be used for modeling.

What to consider:

• Targeting Strategy: The model will end up reflecting the targeting strategy of the chosen ad group. For example, if an ad group bid adjusts on certain inventory sources, geolocations, or times, then the historical log data associated with the ad group will reflect those tactics.
• Number of Ad Groups: You can also consider building the model to reflect multiple ad groups, if they all use the same strategy or bid adjust in the same way on the same vectors.
• Flight Duration: The longer the time frame for data collection (how long will the chosen ad group be active?), the more accurate the model's predictions can become. However, longer time frames also mean more data, which requires more computation power and storage capacity.

Best practices for choosing duration

Test Shorter Time Frames First: Consider modeling on shorter time frames, such as one full day of raw log data, and note the amount of computing resources and storage used.

Decide on Balance of Time and Accuracy: Shorter time frames may result in less accurate data, while longer ones may result in greater accuracy. For example, if you need realistic conversion rates at the user level to create an initial set of user base bids, it may be best to choose a longer time frame, since this will result in more realistic rates.

Understand How Time and Conversions Interact: The type of conversion event can influence how long the time frame should be (e.g., it will take longer to collect car purchase data than service subscriptions). And the time frame can affect how many users don't convert (this problem is known as "right censored data"). By reasonably assuming that the non-converters could have converted given more time, there are mathematical methods to impute missing rates or user base bid scores.

Identifying and ingesting data

Prior Knowledge Needed and Call Outs

• Before reading further, please familiarize yourself with the background information and best practices on our Raw event data stream (REDS).
• Using the BaseBidCPM parameter in our POST /data/advertiser is an API-only feature and UI users will not know this feature is being used. Please coordinate with your trading teams to ensure they know which data elements are adopting User Scoring.
• Using cross-device targeting for the ad groups tied to cookie-scoring segments will have unintended consequences.
  • For example, if you uploaded 2 cookies with different BaseBidCPM values to your segment and a cross-device vendor determines that those 2 cookies are associated with the same user, we will overwrite the BaseBidCPM of all associated cookie IDs using the value of the last user that was uploaded.

Once you've set up your data source and ad group, you will be able to begin identifying and ingesting the data.

Data retrieval

Download the Python code sample for managing log-level data. The section below references this code sample.

Below you will find a Python code snippet that connects to the Amazon s3 API via a Boto package (2.x version) to retrieve two types of data events: impressions and conversions.

Each new data logs' rows are appended to a single CSV file, which occurs inside the function ingestGzipLogs.

The final result includes two master CSV data files:

• One file for impressions
• One file for conversions

You can reconfigure this code snippet to allow for more frequent retrievals of the data, as opposed to a one-day batch retrieval, depending on your individual needs.

REDS ingest

BUCKET_PATH = 'thetradedesk-useast-partner-datafeed'
FILTER_PATH = 'PARTNER_ID/redf5/date=DATE/hour=' 
conn = boto.connect_s3('USER_KEY', 'ACCESS_KEY')
mybucket = conn.get_bucket(BUCKET_PATH)

...

def ingestGzipLogs(dateFilter):
    for key in mybucket.list(FILTER_PATH + str(dateFilter)):
        logName = key.name
    if "impressions" in logName:
        key = ReadOnce(key)
        gz_file = gzip.GzipFile(fileobj=key, mode='rb')
        for line in gz_file:
            newLine = line.decode('utf-8').split('\t')
            with open(impCSV, "a") as output:
                writer = csv.writer(output)
                writer.writerow(newLine)
    elif "conversions" in logName:
        key = ReadOnce(key)
        gz_file = gzip.GzipFile(fileobj=key, mode='rb')
        for line in gz_file:
            newLine = line.decode('utf-8').split('\t')
            with open(convCSV, "a") as output2:
                writer = csv.writer(output2)
                writer.writerow(newLine)
    else:
        continue
...

Preliminary data analysis and transformations

In this section, we'll be using the R language, with its quick prototyping advantages, to look at the main ideas when performing data analysis and transformation.

Download the code sample for analyzing user data using r. The section below reference this code sample.

Reading CSV files

To start, you can use the fread function from the data.table library package (as a faster alternative to read.table or read.csv) to quickly read your two CSV files from the initial retrieval.

You may use functions such as dim(impData) or summary(impData) to get a general understanding of the dimensions of the data set, basic statistics along each column, and whether there are missing values.

impData <- fread("./Impressions.csv", header = FALSE)
colnames(impData) <- c("logentryTime", "impID", "partnerID", "advertiserID", "campaignID", "adgroupID", "privateContractID", "audienceID", "creativeID", "adformat", "frequency", "ssp", "publisherID", "dealID", "site", "referrerCatList", "foldPosition", "userHourOfWeek", "userAgent", "IPAddress", "TDID", "country", "region", "metro", "city", "deviceType", "OSFamily", "OS", "browser", "recency", "languageCode", "mediaCost", "feeFeatureCost", "dataUsageCost", "TTDCost", "partnerCost", "advertiserCost", "lat", "long", "deviceID", "zipcode", "processedTime", "deviceMake", "deviceModel", "renderingContext", "carrierID", "tempCelsius", "tempBucketStartCelsius", "tempBucketEndCelsius","placementID") #provide header for impressions data
convData <- fread("./Conversions.csv", header = FALSE)
colnames(convData) <- c("logentryTime", "convID", "advertiserID", "convType", "TDID", "IPAddress", "referrerURL", "monetaryValue", "monetaryValueCurrency", "orderID", "TD1", "TD2", "TD3", "TD4", "TD5", "TD6", "TD7", "TD8", "TD9", "TD10", "processedTime") #provide header for conversion data

Filtering to data subset

Next, you will have have to filter down to the appropriate subset of data. In the following code, the filter is placed at the advertiser ID level, but the filter can be adjusted to be at the campaign or ad group level.

subsetImps <- impData[impData[,impData$advertiserID=='ADVERTISER_ID'],]

There are other important things the code is doing. By only including view-through conversions, you will have to take the impressions data and conversion data, and perform a join across the TDID key.

joinedData <- merge(impDedupData, convData, by.x = 'TDID', by.y = 'TDID', all.x = TRUE)

Assigning attribution credit

Next, it is important to check for last-touch attribution, and assign attribution credit to the appropriate set of impressions. The idea is to take the difference between the log-entry times of matched impressions by TDID and legitimate conversions to match the conversions to the most recently served impressions.

Note that the following code snippet measures the difference in seconds.

summary(abs(difftime(matchedConvData$logentryTime.x ,matchedConvData$logentryTime.y)) <= attWin)

Formatting your target column

Before training a machine learning model, you need to initialize your target column – in this case, your user base bid scores.

The following R code shows a sequence of data transformation actions piped together by the %>% operator under the dplyr package. This carefully constructs the probability of a user to convert (i.e., individual user conversion rate), and multiplies it by:

• A "0.005" constant, which compensates for raw impression data gathered over a short time frame (e.g., one day) that would otherwise result in unrealistic and inflated conversion probabilities. * This brings us back to the trade off between accuracy (tied to time frame) and computational power. To avoid inaccuracy, you can ingest data over a longer time frame, but this requires more computing resources.
• The ad group goal target, expressed as an integer (Key Performance Indicator)
• "1000" (to convert into CPM units)

testGrouping <- joinedData2 %>% select(TDID.x, conv) %>% add_count(TDID.x) %>% group_by(TDID.x) %>% mutate(convSum = sum(conv)) %>% mutate(convRate = convSum/n) %>% mutate(userBaseBidCPM = 0.005*convRate*40*1000)

Once your user scores have been initialized, you can create a boxplot of the score distribution by using the boxplot function in R. The following snippet is one illustration of how to use it:

boxplot(filterGroupsWithCPM$userBaseBidCPM, data=filterGroupsWithCPM$userBaseBidCPM,main="User Base Bid Distribution",col=c("red","green"), notch=F, ylab="User Base Bid CPMs")

Gathering additional columns

Once you are satisfied with initializing the user base bid CPM values that will train your model, you will need to join back this new data to the impression details data to gather additional columns such as:

• SSP
• Publisher ID
• Site
• Placement ID

We can summarize the combination of these fields as SPSP.

Why these fields? For one approach to the problem, the unique combinations of these four values could serve as a rough proxy for user behavior.

trainingData <- joinedData2 %>% select(TDID.x, ssp.x, publisherID.x, site.x, placementID.x)
trainingData <- merge(filteredData, trainingData, by.x = 'TDID.x', by.y = 'TDID.x')

After identifying all possible impression data associated with the conversion pool of TDIDs, you will need to transpose the unique combinations of SPSP into individual column features. You may use the dplyr package for this transformation.

testGrouping2 <- trainingData %>% select(TDID.x, SPSPID) %>% group_by(TDID.x, SPSPID) %>% count(SPSPID) %>% filter(n < 20) %>% spread(SPSPID, n, fill = 0)

Possible issues

Encountering IDs with no base bid cpms

You will eventually encounter a situation where a subset of unique TDIDs did not see any conversions take place (verified through last-touch attribution). These TDIDs will not have any user base bid CPMs associated with them.

This usually happens because the data's time frame was not long enough to see a conversion occur – in other words, for IDs that have no conversion data, it may just be a result of not giving the users enough time to convert.

If you assume that these TDIDs have some probability of conversion (i.e., the users would have converted at some point in the future), you could use data imputation techniques to approximate what those missing CPM values should be. One popular technique is K-Nearest Neighbor (KNN) Imputation, which has the following advantages:

• Can handle variables or features with multiple missing values
• Can be used against both qualitative and quantitative attributes

knnTrainingData <- modelTrainingData[which(!is.na(modelTrainingData$userBaseBidCPM)),]
knnTestData <- modelTrainingData[which(is.na(modelTrainingData$userBaseBidCPM)),]
trctrl <- trainControl(method = "repeatedcv", number = 10, repeats = 1)
knn_model <-train(userBaseBidCPM~.,data=knnTrainingData,method="knn", trControl=trctrl, preProcess = c("center", "scale"), tuneLength = 10)
print(knn_model)
test_userBaseBidCPM_preds <- predict(knn_model, newdata = knnTestData)

Data sets with too many columns

Before training your first machine learning model, you may need to deal with the potential issue of having too many columns in the training data set. This problem is typically referred to as the curse of dimensionality. To solve this issue, you can use Principal Components Analysis (PCA) to determine the number of variables needed to represent important data signals or variability.

An example of this is shown in the following code snippet that helps determine how many features are needed.

plot(cumsum(pcaData$sdev)/sum(pcaData$sdev)*100,main='Cumulative proportion of variance explained',ylab='cumulative variance')

This results in the following plot:

Training and deploying the model

Now you are ready to build your first trained model!

In order to showcase a portfolio of options for your own machine learning tech infrastructure, this documentation illustrates three ways you can start using your model as soon as possible.

R language

The R language contains a lot of popular packages that cover a wide swath of machine learning algorithms. Since our training data set has numerical features with a numerical target column (i.e., user base bid scores), we could easily choose a linear regression algorithm with the assumption that a linear approximation works for our use case. The following code snippet combines functionality from caret (essentially a wrapper for multiple algorithms) and glmnet.

trctrl <- trainControl(method = "repeatedcv", number = 10, repeats = 1)
#use GLMNET with Regularized Regression
paramGrid <- expand.grid(lambda = 10^seq(10, -2, length=100), alpha = c(0,1))
regularizedReg_model <- train(userBaseBidCPM ~., data = trainingDataWithPCA, method = "glmnet", trControl=trctrl, preProcess = c("center", "scale"), tuneGrid = paramGrid)
print(regularizedReg_model)

This results in 200 possible models (since the paramGrid looks at over 200 possible combinations of two tuning parameters: lambda and alpha) after K Cross Validation is applied for each model combination.

• Lambda controls how much regularization (protects against overfitting) there needs to be
• Alpha controls whether the regularization is done using the Lasso or Ridge technique

The final result includes the model with the best tuned parameters (lambda and alpha) for making score predictions.

Download the Python code sample for deploying Amazon ml. The section below reference this code sample.

Amazon machine learning

Amazon Machine Learning creates a simplified environment for the user to quickly deploy a machine learning model.

After making an AWS account, this process first requires that you upload a data set into S3 storage to create the data source that the machine learning model will use.

Next, you may create a model that points to the training data source with a narrow set of custom settings.

Note that none of the necessary data preprocessing that was done in R is available in Amazing Machine Learning.

Interestingly, Amazon allows you no control over the exact selection of the machine learning algorithm. It instead decides on a regression method for you, since the target column (user base bid scores) is numeric in nature.

You will eventually be prompted to create an evaluation of your model. The only metric used in this evaluation is Root Mean Square Error (RMSE). A model with perfect predictions would have a RMSE score of zero. For simplification purposes, the model uses 30 column features for training the model.

For further information on Amazon ML services, please see https://aws.amazon.com/machine-learning/

To deploy, you must navigate to the Prediction section of the model evaluation, where you can enable the desired endpoint. Enabling an endpoint for the model renders it live for prediction data retrieval via the API.

By taking advantage of the endpoint URL at https://realtime.machinelearning.us-east-1.amazonaws.com, you can easily integrate your deployed model into a larger ML pipeline for generating user base bid scores.

Any new observations that are sent as part of the JSON request could result in an output similar to the following structure:

{
    "Prediction": {
        "details": {
            "Algorithm": "SGD",
            "PredictiveModelType": "REGRESSION"
        },
        "predictedValue": 64.68316650390625
    }
}

Please see sample code for how to hit a deployed model API endpoint with Amazon ML (link sample code).

Download the Python code sample for deploying Azure ml. The section below reference this code sample.

Azure machine learning

Microsoft Azure Machine Learning Studio provides a visual interface for beginning and advanced users alike to control data preprocessing, feature engineering, model testing, and deployment.

Individual actions, such as importing data or filtering data, are taken in a module, and multiple modules can be wired together to produce a machine learning pipeline. As with R, you may use K Cross Validation to evaluate your training data with the chosen algorithm (in this case, Linear Regression with L2 regularization and the Stochastic Gradient Descent optimizer).

This method uses the full power of fast experimental testing, since the training data can be wired to a set of multiple machine learning algorithms, and, consequently, simultaneous evaluations can be done within a user interface.

After running the Azure ML modules to perform K Cross Validation, we can see the RMSE metric for each K fold.

Scrolling down in the same evaluation metrics window, you will see the mean RMSE metric across all K folds, which determines how good the model and its tuning parameters are.

For simplification purposes, the model uses 30 column features for training.

For further information on Azure ML Studio services, please see https://studio.azureml.net.

Making sure that the training experiment runs without error, you can then create a predictive experiment that is deployed as a web service.

This service effectively runs live behind a REST API endpoint, which can be integrated into a larger machine learning pipeline for real-time predictions.

Once you choose to deploy the predictive experiment as a web service, the UI will bring you to the endpoint management section, where you may choose to test the endpoint, view the input schema, copy the API key, etc.

By using the API key and REST endpoint, you may have the opportunity to hit a live deployed model to retrieve model predictions. A successful REST call would result in something similar like in the following:

{
   "Results":{
      "output1":[
         {
            "yieldmo_764343212726492773_www.cnn.com_764345183319563885":"0",
            "gumgum_11602_blend.oola.com_25465":"2",
            "pubmatic_79532_www.topixstars.com_1071234":"0",
            "appnexus_7519_www.trend-chaser.com_10552680":"1",
            "technorati_81529_www.kake.com_73975":"0",
            "userBaseBidCPM":"0",
            "gumgum_11645_www.findagrave.com_25466":"12",
            "rubicon_13344_issuu.com_828762":"0",
            "appnexus_3135_www.reuters.com_12251195":"0",
            "Scored Labels":"104.732474708338",
            "gumgum_11645_www.spanishdict.com_25465":"0",
            "federatedmedia_248396_smallbusiness.chron.com_487224":"1",
            "gumgum_1000006_www.thegamer.com_25465":"0",
            "appnexus_1613_ancient-origins.net_12483184":"0",
            "pubmatic_155959_www.spanishdict.com_800421":"0",
            "pubmatic_156078_eternallysunny.com_904294":"0",
            "pubmatic_79532_www.topixoffbeat.com_1130751":"1",
            "appnexus_1908_www.spanishdict.com_6448088":"1",
            "federatedmedia_83744_www.merriam-webster.com_378385":"0",
            "appnexus_1908_www.icepop.com_10251970":"2",
            "gumgum_239_toofab.com_25465":"0",
            "gumgum_12366_hellogiggles.com_25465":"10",
            "gumgum_12900_www.everylastbite.com_30471":"0",
            "appnexus_1613_allthatsinteresting.com_9116152":"0",
            "pubmatic_156078_zdnzwf5-a.akamaihd.net_904294":"0",
            "adconductor_812726633_www.geekycamel.com_171220103":"0",
            "appnexus_3135_www.findagrave.com_12573855":"0",
            "appnexus_1908_www.sparkpeople.com_11133351":"0",
            "federatedmedia_235520_list25.com_501113":"1",
            "pubmatic_156078_www.rawstory.com_904294":"0",
            "technorati_81858_stocktwits.com_79581":"1",
            "pubmatic_156636_www.cleveland.com_1281684":"0"
         }
      ]
   }
}

Please see sample code for how to hit a deployed model API endpoint with Azure ML.

Pushing audiences to The Trade Desk platform

Prior Knowledge Needed and Call Outs

• Before reading further, please familiarize yourself with the background information and best practices on Anonymous ID data integration.
• Using the BaseBidCPM parameter in our POST /data/advertiser is an API-only feature and UI users will have no insight into that this feature is being leveraged. Please coordinate with your trading teams to ensure they know which data elements are adopting cookie scoring
• Using cross device on your adgroups tied to cookie scoring segments will have unintended consequnces..
  • For example: You upload 2 cookies in your segment with different BaseBidCPM values. If a cross device vendor determines that those 2 cookies are the same persona, we will overwrite the BaseBidCPM of all associated cookieIDs using the value of the last user that was uploaded.

Once your model is deployed behind a live API endpoint, you may integrate it into a larger machine learning workflow, in which new data or new user observations are pushed to the API endpoint to retrieve user scoring predictions.

These new observations would initially come from your REDS data, but you also have the power to merge other data sets from outside of The Trade Desk to better improve the predictive power of your model. After a specific time period of testing performance, you can push these new score predictions into The Trade Desk Data API and activate the user base bids within an ad group in our platform.

Our POST /data/advertiser guide covers all the necessary integration considerations however in order to assign a user base bid cpm, you will need to include a new parameter in the JSON payload as Items.Data.BaseBidCPM. See below for details and an example.

Base bid CPM in Data API

{  
   "AdvertiserID":"abcdefg",
   "Items":[  
      {  
         "TDID":"f5320d1f-69f0-4cf7-bc2b-1f20eb591f110",
         "Data":[  
            {  
               "Name":"Segment 1",
               "TtlInMinutes":43200,
               "BaseBidCPM":4.32
            }
         ]
      }
   ]
}

Appendix

The following R libraries and versions were used for the sample code:

• glmnet_3.0-2
• randomForest_4.6-14
• caret_6.0-84
• tidyr_1.0.0
• dplyr_0.8.3
• data.table_1.12.2