Concept of model validation and outlier detection in JAG3D (Software)

E_merlet, Mittwoch, 12. April 2017, 17:10 (vor 44 Tagen)
bearbeitet von Micha, Mittwoch, 12. April 2017, 17:17

Hello everyone,
I hope someone can help me. Sorry I don't speak german very well.
I was reading the informations about Java Graticule 3D (thanks google translate) and the section about model failure (modelstörung) and I wondered what is the design matrix B and what is the model failure ∇. What does it look like?
thank you all, Etienne

Avatar

Concept of model validation and outlier detection in JAG3D

Micha ⌂, Bad Vilbel, Mittwoch, 12. April 2017, 17:42 (vor 44 Tagen) @ E_merlet
bearbeitet von loesler, Mittwoch, 12. April 2017, 18:20

Hi Etienne,

I wondered what is the design matrix B and what is the model failure ∇. What does it look like?

The parameters in vector contains additional parameters that are introduced to the functional model. Sometimes, these parameters are called "estimated gross errors". A simple example may be a levelling network. In this case, the functional model reads

\delta h_{i,j} = H_j - H_i
\delta h_{i,k} = H_k - H_i
\delta h_{i,l} = H_l - H_i
\delta h_{j,l} = H_l - H_j
...

To preform an outlier test, you extend (at least one equation of) the functional model e.g. δhik

\delta h_{i,k} = (H_k - H_i) + \nabla_{i,k}

For ∇ = 0 the extension is unneeded because there is no benefit to estimate this additional parameter. If ∇ ≠ 0, the extended functional model improves your solution or is more suitable. In this case, we assume that the observation, which is related to the modified equation, is an outlier.
It will be an exception that ends up to zero, if no outlier is presented. In general, ∇ ≈ 0 and a stochastic test is used for the decision.

B is a design matrix, which contains the partial deviation w.r.t. the unknown parameter ∇. In the example above, B is just a vector with zeros, that contains a one at the row that corresponds to the observation δhik

B^T = 
 \begin{bmatrix}
0 & \cdots & 0 & 1_{\delta h_{i,k}} & 0 & \cdots & 0
\end{bmatrix}

kind regards
Micha

--
kostenlose Scripte und Software nicht nur für Geodäten || Portal für Geodäten mit angeschlossenem Forum-Vermessung

Tags:
Modelstörung, Outlier, Matrix, Stochastic test

Concept of model validation and outlier detection in JAG3D

E_merlet, Mittwoch, 12. April 2017, 18:27 (vor 44 Tagen) @ Micha

To preform an outlier test, you extend (at least one equation of) the functional model e.g. δhik

\delta h_{i,k} = (H_k - H_i) + \nabla_{i,k}

Ok, and do I have to extend the model with \nabla by myself or does the software do it by itself?

For ∇ = 0 the extension is unneeded because there is no benefit to estimate this additional parameter. If ∇ ≠ 0, the extended functional model improves your solution or is more suitable. In this case, we assume that the observation, which is related to the modified equation, is an outlier.
It will be an exception that ends up to zero, if no outlier is presented. In general, ∇ ≈ 0 and a stochastic test is used for the decision.

So, it means there is always an error, for every observation. The stochastic test tells us if the error is too big and has to be corrected or not. Right?
∇ ≠ 0 means the observation is an outlier. Does it not mean it has to be removed instead of being improved?

B is a design matrix, which contains the partial deviation w.r.t. the unknown parameter ∇. In the example above, B is just a vector with zeros, that contains a one at the row that corresponds to the observation δhik

So, B is like the jacobian matrix A but with the parameter \nabla in the partial deviation.

And what is E in EPi = (E-Ri)Bi\nabla i in the section Influence on the position of a point?

Etienne

Avatar

Concept of model validation and outlier detection in JAG3D

Micha ⌂, Bad Vilbel, Mittwoch, 12. April 2017, 18:43 (vor 44 Tagen) @ E_merlet

Hi,

Ok, and do I have to extend the model with \nabla by myself or does the software do it by itself?

JAG3D estimates in the last iteration step and shows the results in one of the columns of the observation tables.

So, it means there is always an error, for every observation.

There is in most cases a discrepance to zero, yes. But remember, your measurement process is not perfect and some kind of noise influences your observation. Thus, can be an error.

The stochastic test tells us if the error is too big and has to be corrected or not. Right?

With an specific probability value, yes.

∇ ≠ 0 means the observation is an outlier. Does it not mean it has to be removed

... or correcting or something else - but yes, you have to handle dubios observations and gives you a hint about the size of the possible outlier.

So, B is like the jacobian matrix A

Yes. In the extended model, B is a part of the extended A, i.e. A := [A B].

And what is E

E represents the identity matrix.

regards
Micha

--
kostenlose Scripte und Software nicht nur für Geodäten || Portal für Geodäten mit angeschlossenem Forum-Vermessung

Tags:
JAG3D, Stochastic test, Outlier

Concept of model validation and outlier detection in JAG3D

E_merlet, Freitag, 14. April 2017, 17:28 (vor 42 Tagen) @ Micha

And what is E

E represents the identity matrix.

Ok, that's what I thought but I wasn't sure.

I also read the section about statistical tests.
If the global test respects the hypothesis 0, it means there are no errors in the model or in the observations, right? So, it means the global test never respects this hypothesis cause there are always errors in the observations. It always respects the alternative hypothesis.

And what is m in the individual tests? Is it always m=1?

Spezialfälle der in JAG3D implementierten Testverfahren für m=1

not sure about the translation

thanks

B-method and Šidák correction

E_merlet, Samstag, 15. April 2017, 18:03 (vor 41 Tagen) @ E_merlet

Hello Micha,

in the same section (statistical test), I also don't understand what the Baarda's test and the correction of Sidak do.
What is their aim? What do they compare?

best regards, Etienne

Avatar

Concept of model validation and outlier detection in JAG3D

Micha ⌂, Bad Vilbel, Samstag, 15. April 2017, 18:44 (vor 41 Tagen) @ E_merlet

Hi,

If the global test respects the hypothesis 0, it means there are no errors in the model or in the observations, right?

No. It is a stochastic test, so remember a type one error as well as type two error.

It always respects the alternative hypothesis.

No. If the residuals are normally distributed and the stochastic model as well as the functional model are valid, the null-hypothesis will (or should) not rejected.

And what is m in the individual tests?

The rang of the matrix Q∇∇.

Is it always m=1?

No, take a look to the table.

I also don't understand what the Baarda's test and the correction of Sidak do

They manipulate the probability value in a specific way. Please take a look to e.g.:

  • Aydin, Demirel (2004), Computation of Baarda’s lower bound of the non-centrality parameter
  • Šidák (1967), Rectangular Confidence Regions for the Means of Multivariate Normal Distributions.

/Micha

--
kostenlose Scripte und Software nicht nur für Geodäten || Portal für Geodäten mit angeschlossenem Forum-Vermessung

Tags:
JAG3D, Model, Stochastic test, Outlier, Šidák, Baarda, B-method, Type I error, Type II error

Concept of model validation and outlier detection in JAG3D

E_merlet, Sonntag, 16. April 2017, 01:37 (vor 41 Tagen) @ Micha

They manipulate the probability value in a specific way. Please take a look to e.g.:

  • Aydin, Demirel (2004), Computation of Baarda’s lower bound of the non-centrality parameter
  • Šidák (1967), Rectangular Confidence Regions for the Means of Multivariate Normal Distributions.


Hello Micha, thank you for the reply.
I would like to read these books but they are not in free access in internet.


I'm really sorry to disturb you but I read a lot of paper about the least squares method and Fischer-distribution, Student-t-distribution, chi-squared distribution? test of hypothesis but I'm completely lost.

Here is the thing as I understood it.
Jag3D computes the compensation with the least squares method. Then it computes a global test.
This test tells us if there is/are outlier(s) in the observation(s) but it doesn't tell us where. Therefore Jag3D computes individual tests for each observations.
If the value of the individual test is near from 0, it means the observation is not an outlier and if the value of the test is big, it means the observation is an outlier. Right?

Is it the right order of computation?

Does the expression Tprio~Fm,∞ means that the test follows a Fischer-distribution?
Why does Jag3D compute an individual test a-priori and an individual test a-posteriori?

And if we already know where the outliers are thanks to individual tests, why do we use B-method or Sidak correction?
What do you mean with "They manipulate the probability value in a specific way"

I read so many papers but it sounds like chinese and I'm close to give up but I don't want it.
So, once again sorry for disturbing you.

Best regards, Etienne

Avatar

Concept of model validation and outlier detection in JAG3D

Micha ⌂, Bad Vilbel, Sonntag, 16. April 2017, 11:51 (vor 40 Tagen) @ E_merlet

Hello Etienne

I would like to read these books but they are not in free access in internet.

The concept of the B-method is given in the postulate of Baarda's A testing procedure for use in geodetic networks. Šidák correction can be found at e.g. Wikipedia.

Then it computes a global test.
This test tells us if there is/are outlier(s) in the observation(s) but it doesn't tell us where. Therefore Jag3D computes individual tests for each observations.
If the value of the individual test is near from 0, it means the observation is not an outlier and if the value of the test is big, it means the observation is an outlier. Right?

JAG3D estimates the unknwon parameters by least-squares adjustment. A so-called overall model test, the global test, is preformed to check the stochastic model as well as the functional model. If the global test fails, it is not possible to identify the reason of the rejection. An outliers is nothing else than a wrong functional model, but the reason for the rejection can also be a too optimistic stochastic model, i.e. if your distance measurement can be carried out with an uncertainty of e.g. 2 cm but you select 2 µm, the global test (as well as a lot of individual tests) will rejected.

The individual test adds an additional parameter to each observation (one at a time) and checks the model conformity or benefit of w.r.t. related uncertainties Q∇∇, e.g.

T_{prio,i} = \frac{\mathbf{\nabla_i^TQ_{\nabla\nabla,i}^{-1}\nabla_i}} {m\sigma_0^2

If Tprio exceeds a specific threshold, is significant. In case of a reliable stochastic model, can be interpreted as an outlier.

Does the expression Tprio~Fm,∞ means that the test follows a Fischer-distribution?

Yes, it follows the Fisher distribution F(f1,f2) with parameters f1=m and f2=∞. Due to f2=∞, it is equivalent to χ2-test.

Why does Jag3D compute an individual test a-priori and an individual test a-posteriori?

Remember my distance example (2 cm vs. 2 µm): Tprio based on your stochastic model. If you select an inadequate stochastic model, this test will fail many times and you run in a type I error. Tpost based on the estimated variance of the unit weight and are - in some cases - more suitable than Tprio but with less power.

And if we already know where the outliers are thanks to individual tests, why do we use B-method or Sidak correction?

What will you do, if the global test fails but no individual test identifies an outlier? Šidák correction and B-method are concepts to minimizes this confusion: The global test should be rejected, if at least one individual test was rejected and vice versa.

What do you mean with "They manipulate the probability value in a specific way"

Take a look at the Šidák correction

\alpha' = 1 - (1 - \alpha_G)^{1/r}

here αG and α' represents the probability value of the global test and the individual test, respectively, an r is the number of hypothesis. For B-method an online calculator is available:

\lambda(\alpha', 1-\beta, 1) = \lambda(\alpha_G, 1-\beta, r)

Here α is adapted in a way, that both test statistics have the same non-centrality parameter λ.

regards
Micha

--
kostenlose Scripte und Software nicht nur für Geodäten || Portal für Geodäten mit angeschlossenem Forum-Vermessung

Tags:
JAG3D, Model, Stochastic test, Outlier, Šidák, Baarda, B-method, Fisher, χ2

RSS-Feed dieser Diskussion