Estimating Derivatives for Samples of Sparsely Observed Functions, With Application to Online Auction Dynamics

Abstract

It is often of interest to recover derivatives of a sample of random functions from sparse and noise-contaminated measurements, especially when the dynamics of underlying processes is of interest. We propose a novel approach based on estimating derivatives of eigenfunctions and expansions of random functions into their eigenfunctions to obtain a representation for derivatives. In combination with estimates for functional principal component scores for sparse data, this leads to a viable solution of the challenging problem to recover derivatives for sparsely observed functions. We establish consistency results and demonstrate in simulations that the method is superior to alternative approaches (derivative estimation with random effects models based on B-spline bases, kernel smoothing, smoothing splines, or P-splines). Our study is motivated by an analysis of bidding histories for eBay auctions, for which bids are typically very sparse in the middle and somewhat more frequent near the beginning and end of an auction. We demonstrate the estimation of derivatives of price curves for individual auctions from the sparsely observed bidding histories and also derive a model-free first-order differential equation that applies in the case of Gaussian processes. This provides a data-driven dynamic model that we use to elucidate auction dynamics.