# Simulation envelopes and significance levels

Many spatial analysis packages provide Monte Carlo techniques to simulate upper and lower “envelopes” for a summary statistic as a function of distance (e.g. K-function, nearest neighbor, etc…). I’ve sometimes seen the significance of deviation from the envelope expressed as:

p = m * 1 / (n +1)

Where n is the number of simulations and m is the rank of the largest and smallest observation from a simulation sample for each distance r. For example, if m = 1, the first largest and first smallest values from the simulation sample are used to plot the upper and lower simulation envelopes. If m = 2, the second largest and second smallest values are used to plot the simulation envelopes.

My questions are:

1/ where does the + 1 in the denominator originate from? For instance, if we run 39 simulations, why divide by 40?

2/ Regardless of the number of simulations performed (i.e. 39 or 9999), m seems to have a disproportionate influence on the computation of the significance level. It would seem that taking the 2nd highest and lowest values of a simulation sample from 9999 simulations would have less of an impact on the width of the simulation envelope than from 39 simulations. I’m sure that there is a sound theoretical basis for this, but its logic escapes me. Is there an analogy in inferential (non-spatial) statistics that can help make this a bit more intuitive?

3/ How should one present the results of a simulation envelope? I sometimes see a p value defined for a simulation, but how is one to know if the presented p is for m=1 or a more stringent m=3?

Edit: Per whuber's comment here is some clarification:

1/ The type of simulation pertains to testing the null hypothesis such as complete spatial randomness (CSR)

2/ Example of a package that computes simulation envelopes is spatstat (R). The function is called envelope and nrank is the parameter m described in the above equation.

• Some clarification of the context would help here, Manny. There are two common applications of this approach: (1) Monte-Carlo simulation of a null distribution and (2) a bootstrapped sample distribution. In the former, the envelope is simulated according to the null hypothesis (such as CSR for a point process) whereas in the latter it is simulated with resampling from the data. The former can produce valid p-values but the latter cannot. Which situation are you considering? Also, could you provide links to "packages" that vary m, so we can see exactly what they are trying to do? – whuber Nov 9 '11 at 20:53
• Thanks @whuber, I clarified the context per your suggestion. – MannyG Nov 9 '11 at 21:25