Handling time values

Time values play an important role in our model because time is both a feature and a target (value to be predicted) in our model. First, we need to convert the pickup and dropoff times into pandas datetime values to calculate the target value, which will be the natural log of the difference in time between dropoff and pickup in seconds:

df["trip_duration"] = np.log((df.Lpep_dropoff_datetime - df.lpep_pickup_datetime).dt.seconds + 1)

In the preceding line of code, we are adding 1 second to the trip duration to prevent an undefined error when a log transformation is applied over the value.

But why are we using natural log transformation over the trip duration? There are three reasons for this, as follows:

• For the Kaggle competition on New York taxi trip duration prediction, the evaluation metric is defined as the Root Mean Squared Logarithmic Error (RMSLE). When log transformation is applied and the RMSE is calculated over the target values, we get the RMSLE. This helps us compare our results with the best-performing teams. 
• Errors in log scale let us know by how many factors we were wrong, for example, whether we were 10% off from the actual values or 70% off. We will be discussing this in detail when we look at the Error metric section.
• The log transformation over the target variable follows a perfectly normal distribution. This satisfies one of the assumptions of linear regression. The plot of the trip duration values (on a log scale) looks as follows: