Generalized orthogonal linear derivative (GOLD) estimates were proposed to correct a problem of correlated estimation errors in generalized local linear approximation (GLLA). This paper shows that GOLD estimates are related to GLLA estimates by the Gram–Schmidt orthogonalization process. Analytical work suggests that GLLA estimates are derivatives of an approximating polynomial and GOLD estimates are linear combinations of these derivatives. A series of simulation studies then further investigates and tests the analytical properties derived. The first study shows that when approximating or smoothing noisy data, GLLA outperforms GOLD, but when interpolating noisy data GOLD outperforms GLLA. The second study shows that when data are not noisy, GLLA always outperforms GOLD in terms of derivative estimation. Thus, when data can be smoothed or are not noisy, GLLA is preferred whereas when they cannot then GOLD is preferred. The last studies show situations where GOLD can produce biased estimates. In spite of these possible shortcomings of GOLD to produce accurate and unbiased estimates, GOLD may still provide adequate or improved model estimation because of its orthogonal error structure. However, GOLD should not be used purely for derivative estimation because the error covariance structure is irrelevant in this case. Future research should attempt to find orthogonal polynomial derivative estimators that produce accurate and unbiased derivatives with an orthogonal error structure.

In this paper we investigate gradient estimation for a class of contracting stochastic systems on a continuous state space. We find conditions on the one-step transitions, namely differentiability and contraction in a Wasserstein distance, that guarantee differentiability of stationary costs. Then we show how to estimate the derivatives, deriving an estimator that can be seen as a generalization of the forward sensitivity analysis method used in deterministic systems. We apply the results to examples, including a neural network model.

We study the estimation of the mean function of a continuous-time stochastic process andits derivatives. The covariance function of the process is assumed to be nonparametric andto satisfy mild smoothness conditions. Assuming that n independent realizationsof the process are observed at a sampling design of size N generated by a positivedensity, we derive the asymptotic bias and variance of the local polynomial estimator asn,Nincrease to infinity. We deduce optimal sampling densities, optimal bandwidths, andpropose a new plug-in bandwidth selection method. We establish the asymptotic performanceof the plug-in bandwidth estimator and we compare, in a simulation study, its performancefor finite sizes n,N to the cross-validation and the optimalbandwidths. A software implementation of the plug-in method is available in the Renvironment.

Many applications of smoothed perturbation analysis lead to estimators with hazard rate functions of underlying distributions. A key assumption used in proving unbiasedness of the resulting estimator is that the hazard rate function be bounded, a restrictive assumption which excludes all distributions with finite support. Here, we prove through a simple example that this assumption can in fact be removed.

Given a parametric family of regenerative processes on a common probability space, we investigate when the derivatives (with respect to the parameter) are regenerative. We primarily consider sequences satisfying explicit, Lipschitz recursions, such as the waiting times in many queueing systems, and show that derivatives regenerate together with the original sequence under reasonable monotonicity or continuity assumptions. The inputs to our recursions are i.i.d. or, more generally, governed by a Harris-ergodic Markov chain. For i.i.d. input we identify explicit regeneration points; otherwise, we use coupling arguments. We give conditions for the expected steady-state derivative to be the derivative of the steady-state mean of the original sequence. Under these conditions, the derivative of the steady-state mean has a cycle-formula representation.