Jacob: How are the student projects graded? Are they number graded, letter graded, or pass/fail?

Rachel: The NEAS statistics faculty give a letter grade to each student project.

Jacob: What are the criteria used by the NEAS faculty to grade the projects? Does a comment mean that the project has an error that lowers the grade below B-?

Rachel: That depends on the type of comment.

Jacob: What do you mean by choice vs mis-understanding?

Rachel: Some errors are choices that are not ideal. One candidate with a time series of 132 observations used 120 lags for the Box-Pierce Q statistic, presumably because this is the highest degrees of freedom in the table at the back of the textbook. This is not the ideal choice: the last 20% to 30% of the lags are not a good test of a white noise process, since the sample autocorrelation formula may cause low values at high lags. Ideally, we use between 15 and 40 lags for a time series of 132 observations. This is a minor item in the analysis. If the candidate uses the statistical techniques correctly, we ignore the non-optimal choice of K.

Other errors are more serious. One candidate thought that an AR(2) model regresses the current time series value on the value two periods back instead of on two previous values (1 period and 2 periods back). The candidate compared an AR(1) model with an AR(2) model (with just the two period back value) and found the AR(1) model to be superior. This misunderstands autoregressive processes.

Another candidate thought the AR(1) model regresses the current time series value y_t on the time period t, not on the lagged value y_t-1. The candidate found that the residuals from this model were not a white noise process; that is, the regression had serial correlation. This would be true for any random walk or autoregressive process; it is a serious error.

Jacob: What do you mean by the textbook explanation?

Rachel: The textbook has a good explanation of the sample autocorrelation function but a poor explanation of the partial autocorrelation function. Candidates using Excel have no easy way of forming the partial autocorrelation function. We expect student projects to consider the sample autocorrelations and compare them to the theoretical autocorrelations from the ARIMA process. We do not expect the student projects to make full use of the partial autocorrelation function. (Some student projects using Minitab have made good use of the partial autocorrelation function.)

The most common error in the student projects is the interpretation of the correlogram. The textbook says that a stationary model has a correlogram that declines fairly rapidly to zero, but it does not specify the exact meaning of declines fairly rapidly to zero.

Some candidates see a correlogram that declines slowly to zero over its full length, or a correlogram that declines to zero, becomes negative, and then rises slowly back to zero, and assume that these correlogram indicate a stationary series. They do not (except in rare cases), but it is hard to give precise rules. As long as the candidate also examines the first differences, this error does not require a revision of the student project. Several posted comments by the NEAS faculty discuss when a correlogram indicates stationarity.

The second most common error is taking too many differences or using too high an order for the ARIMA process. The textbook often examines both first and second differences.

If the first differences form a stationary process, we construct an ARIMA process on the first differences, even if the second differences have lower sample autocorrelations. Several postings discuss the errors in taking too many differences.

The proper order of an ARIMA model is disputed by statisticians. If an ARIMA(2,1,0) model has residuals that pass Bartlett’s test and give a reasonably low Box-Pierce Q statistic, we don’t use an ARIMA(12,2,8) model that performs slightly better. But the choice between ARIMA(2,1,0) and ARIMA(2,1,1) is less clear. The optimal model depends not only on the goodness-of-fit tests but also on the intuitive reasonableness of each model. Candidates using Minitab may form too many ARIMA models and not examine their reasonableness.

Jacob: What do you mean by the overall quality of the student project?

Rachel: Student projects must receive a grade of B- or higher for VEE credit. We encourage candidates to work on innovative projects. Designing an innovative project is hard, and few designs are error-free. Actuarial candidates have many responsibilities and little spare time. We do not require all student projects to receive A’s.

Jacob: What do you mean by whether a topic is discussed on the discussion board?

Rachel: As we review the student projects, we explain the most common errors on the discussion forum. We don’t expect candidates to read every posting. But once a topic has been discussed and the proper method is clear, we may direct candidates to the discussion forum postings for more guidance. Good independent work is difficult, and we don’t expect candidates to produce perfect projects without extensive guidance. But explaining all the statistical techniques clearly enough that candidates can apply them in their projects takes is equally difficult. It may take another few months before the postings are complete.