Row weights and making predictions in H2O

[hasithjp]

Motivation

This technical note was inspired by the following question from a Data Science user:

I’ve trained a GLM and a RF using weights, but when I try to predict on a test data I get this error:

> h2o.predict(glm_model, test_df[1:10,])
Error in .h2o.doSafeREST(conn = conn, h2oRestApiVersion = h2oRestApiVersion,  : 
  Test dataset is missing weights vector 'weights' (needed because a response was found and metrics are to be computed).

Why does H2O need weights when it generates predictions?

Do the predictions depend on the weights?

Discussion

If the user goes through the effort to create weights for training, they most likely want to use weights for validation (often a holdout set). We abort to prevent a user mistake. Only a pure test set without a response is accepted without weights, since no metrics are to be computed. Metrics need to know the row weights.

It's easy to add a trivial weights column to a validation frame. Here is an example of how to do that in R.

validation_frame$weights <- 1

Alternately, remove the response column from the data set to be predicted. Then metrics will not be computed and the error will not be hit.

Note that the name of the weights column is specified by the user when the model is built, and can be any name, not just "weights".

Example

library(h2o)

h2o.init()

train_df = as.h2o(iris)
train_df$weights <- 2
summary(train_df)

model = h2o.gbm(y = "Species", x = c(1:4), training_frame = train_df, weights_column = "weights")

# Prediction fails
test_df_with_y_column = as.h2o(iris)
summary(test_df_with_y_column)
pred = h2o.predict(model, test_df_with_y_column)

# Prediction works
test_df_without_y_column = as.h2o(iris)
names(test_df_without_y_column)
drops <- c("Species")
keeps = setdiff(colnames(test_df_without_y_column), drops)
test_df_without_y_column = test_df_without_y_column[, keeps]
summary(test_df_without_y_column)
pred = h2o.predict(model, test_df_without_y_column)
head(pred)

JIRA Issue Migration Info

Jira Issue: TN-1
Assignee: TomK
Reporter: TomK
State: Resolved
Relates to: #14943