Avatar

Concept of model validation and outlier detection in JAG3D (Software)

Micha ⌂, Bad Vilbel, Sonntag, 16. April 2017, 11:51 (vor 103 Tagen) @ E_merlet

Hello Etienne

I would like to read these books but they are not in free access in internet.

The concept of the B-method is given in the postulate of Baarda's A testing procedure for use in geodetic networks. Šidák correction can be found at e.g. Wikipedia.

Then it computes a global test.
This test tells us if there is/are outlier(s) in the observation(s) but it doesn't tell us where. Therefore Jag3D computes individual tests for each observations.
If the value of the individual test is near from 0, it means the observation is not an outlier and if the value of the test is big, it means the observation is an outlier. Right?

JAG3D estimates the unknwon parameters by least-squares adjustment. A so-called overall model test, the global test, is preformed to check the stochastic model as well as the functional model. If the global test fails, it is not possible to identify the reason of the rejection. An outliers is nothing else than a wrong functional model, but the reason for the rejection can also be a too optimistic stochastic model, i.e. if your distance measurement can be carried out with an uncertainty of e.g. 2 cm but you select 2 µm, the global test (as well as a lot of individual tests) will rejected.

The individual test adds an additional parameter to each observation (one at a time) and checks the model conformity or benefit of w.r.t. related uncertainties Q∇∇, e.g.

T_{prio,i} = \frac{\mathbf{\nabla_i^TQ_{\nabla\nabla,i}^{-1}\nabla_i}} {m\sigma_0^2

If Tprio exceeds a specific threshold, is significant. In case of a reliable stochastic model, can be interpreted as an outlier.

Does the expression Tprio~Fm,∞ means that the test follows a Fischer-distribution?

Yes, it follows the Fisher distribution F(f1,f2) with parameters f1=m and f2=∞. Due to f2=∞, it is equivalent to χ2-test.

Why does Jag3D compute an individual test a-priori and an individual test a-posteriori?

Remember my distance example (2 cm vs. 2 µm): Tprio based on your stochastic model. If you select an inadequate stochastic model, this test will fail many times and you run in a type I error. Tpost based on the estimated variance of the unit weight and are - in some cases - more suitable than Tprio but with less power.

And if we already know where the outliers are thanks to individual tests, why do we use B-method or Sidak correction?

What will you do, if the global test fails but no individual test identifies an outlier? Šidák correction and B-method are concepts to minimizes this confusion: The global test should be rejected, if at least one individual test was rejected and vice versa.

What do you mean with "They manipulate the probability value in a specific way"

Take a look at the Šidák correction

\alpha' = 1 - (1 - \alpha_G)^{1/r}

here αG and α' represents the probability value of the global test and the individual test, respectively, an r is the number of hypothesis. For B-method an online calculator is available:

\lambda(\alpha', 1-\beta, 1) = \lambda(\alpha_G, 1-\beta, r)

Here α is adapted in a way, that both test statistics have the same non-centrality parameter λ.

regards
Micha

--
kostenlose Scripte und Software nicht nur für Geodäten || Portal für Geodäten mit angeschlossenem Forum-Vermessung

Tags:
JAG3D, Model, Stochastic test, Outlier, Šidák, Baarda, B-method, Fisher, χ2


gesamter Thread:

 RSS-Feed dieser Diskussion