# Asymptotic normality for nonsmooth objective functions

Cross Validated Asked on December 10, 2020

Assume that $f ({bf x}; theta): mathbb{R}^p times Theta to mathbb{R}$, where ${bf x}$ is the vector of inputs (with some distribution) and $theta$ is the vector of parameters.

Also, assume that $E_{bf x}[f({bf x};theta)]$ is maximized at $theta^*$.

To estimate $theta^*$ I solve the following problem:
$$widehat{theta} = argmax_{theta} frac{1}{n} sum_{i=1}^n f ({bf x}_i; theta), qquad (1)$$
where ${bf x}_i$ are i.i.d. samples.

If $f$ is differentiable, under some regularity conditions, it is very well-known that $widehat{theta}$ is a consistent estimator of $theta^*$, and is also asymptotically Normal.

In my research I have encountered a problem where $f$ is discontinuous and non-differentiable (actually it is piece-wise constant), but its expected value $E_{bf x}[f({bf x};theta)]$ is continuous and differentiable.

I have shown that solving (1) gives a consistent estimator for $theta^*$, but cannot prove the asymptotic normality of the estimator.

To me this seems like a general problem which must have been addressed in the literature, but cannot find any reference.

Is there a paper that has addressed this problem?

Asymptotic normality results with non smooth objective functions has been addressed in the Econometric literature by Daniels (1961), Huber (1967), Pollard (1985), and Pakes and Pollard (1989).

Theorem.

Suppose that $hat{Q}_n(hat{theta}) ge sup_{thetainTheta}hat{Q}_n(theta) - o_p(n^{-1}), hat{theta}totheta_0$, and (i) $Q_o(theta)$ is maximized on $Theta$ at $theta_o$; (ii) $theta_0$ is an interior point of $Theta$, (iii) $Q_o(theta)$ is twice differentiable at $theta_0$ with nonsingular second derivative $H$; (iv) $sqrt{n}hat{D}to_d N(0,Omega)$; (v) for any $delta_nto0$, $sup_{||theta-theta_n|| le delta_n}|hat{R}_n(theta)/[1+sqrt{n}||theta-theta_0||]to_p 0$. Then $sqrt{n}(theta-theta_0)to_d N(0,H^{-1}Omega H^{-1})$.

Note: For reference you can refer to chapter on Large Sample Estimation and Hypothesis Testing by Newey and McFadden (Handbook of Econometrics).

Answered by A.K on December 10, 2020

## Related Questions

### Error metric to compare ratios derived from a binary prediction task

0  Asked on November 9, 2021

### Machine learning to select features to create predictive model

0  Asked on November 9, 2021 by atrag

### Summary statistics to collect for data that is too large in volume

0  Asked on November 9, 2021 by ryu576

### Endogeneity testing using correlation test

3  Asked on November 9, 2021 by sabiste

### find upper and lower bounds on average causal effect $theta$

1  Asked on November 9, 2021 by moneyball

### How to correct for unequal observation time in poisson regression

1  Asked on November 9, 2021 by thomas-richardson

### On gradient boosting and types of encodings

0  Asked on November 9, 2021

### RFE: Pre-define a specificity threshold

1  Asked on November 9, 2021

### Best software for image segmentation for time-series images?

1  Asked on November 9, 2021 by tshast2

### semantic value of a convolution neural network

1  Asked on November 9, 2021

### Getting Different R-squared for Same Model Equation

0  Asked on November 9, 2021

### How to maximize the steady state transition probability for a state in a Markov chain by altering that state’s outgoing transition probabilities?

0  Asked on November 9, 2021 by rob-goldstone

### Find out if increase in a series of values is statistically significant

1  Asked on November 9, 2021

### Covariance between sample mean of two simple random samples

0  Asked on November 6, 2021 by ana-pau-de-la-fuente

### Distribution of the maximum of two correlated normal variables

2  Asked on November 6, 2021

### How to formulate a classification problem with time series element

1  Asked on November 6, 2021 by rishu

### Asymmetric error measure for forecasts

1  Asked on November 6, 2021 by user235039

### How to model variance of a heteroskedastic dataset

1  Asked on November 6, 2021 by yilei-huang

### What analysis do I do? (Wage, education, gender)

2  Asked on November 6, 2021 by marvdste

### Is random seed a hyper-parameter to tune in training deep neural network?

1  Asked on November 6, 2021

### Ask a Question

Get help from others!