# GRN Test Results

## Evaluation of submissions

Legend

*S*: substitutions; the predicted arc type is different from the reference arc type*D*: deletions; there is no predicted arc corresponding to the reference arc (false negative)*I*: insertion; there is no reference arc corresponding to the predicted arc (false positive)*M*: matches; the predicted and reference arcs have the same type*SER = (S + D + I) / N*, where*N*is the number of arcs in the reference*Recall = M / N**Precision = M / P*, where*P*is the number of predicted arcs (*P = S + I + M*)*F1*: harmonic mean of*Precision*and*Recall*

This is the main evaluation. The primary ranking criterion is the strict SER.

Evaluation algorithm

The evaluation algorithm operates pair by pair (of genes). For each pair, it tries to maximize *Matches*, then to maximize *Substitutions*. For instance, let's consider a pair of genes, where the reference says one *Inhibition* arc and one *Transcription* arc. Here is the error count for the following predictions:

*Inhibition*and*Regulation*: 1*Match*, 1*Substitution**Activation*and*Binding*: 2*Substitutions**Inhibition*,*Requirement*and*Binding*: 1 Match, 1*Substitution*, 1*Insertion**Regulation*: 1*Substitution*, 1*Deletion*

Relaxed scores

The relaxed scores are computed the same way except that *Substitutions* are considered as *Matches*. This is an attempt to score the predictions regardless of the arc types. However it does not take into account the redundancy of arcs. For interpretation purposes, the following table is more accurate.

## Network shape evaluation

Attention: team ranks are different

This evaluation has been done as if all arcs in the reference and in the prediction were of type *Regulation*. Redundant arcs were removed. Notice that, as expected, *Substitutions* is always equal to zero, and that relaxed scores are strictly equal to strict scores.

This evaluation gives the accuracy of the prediction regardless of the type of the arcs, more accurately than relaxed scores in the previous evaluation. A good score means that the prediction reproduces accurately the shape of the network. The gap between this evaluation and the previous one indicates the (in)accuracy of predicted arc types.

## Valued network evaluation

In the same way as the previous evaluation, the arcs of type *Binding* and *Transcription* have been turned into *Regulation*. Again, redundant arcs have been removed. In this way the network only contains *Regulation*, *Activation*, *Inhibition* and *Requirement* arcs. In fact interactions of the mechanism axis have been removed, leaving only interactions of the effect axis. Some Systems Biology applications only need this kind of information. The gap between this evaluation and the first one indicates the (in)accuracy of predicted arcs of type *Binding* and *Transcription*.