Reports 1-1 of 1 Clear search Modify search
AdV-COM (AdV commissioning (1st part) )
ruggi - 16:03 Thursday 14 November 2019 (47657) Print this report
NI patch

Since the NI failure is still a relevant cause of lockloss, the code has been modified in order to increase the ITF robustness during the failure. Up to now, the lockloss occurred because the AA control signal was no more received by the coil driver board from the master board, while the switch was still allowing the use of that signal. From now on, during the failure also the switch will go to zero, allowing the use of local control signal (which should remain reliable). If we are lucky, the ITF will remain locked up to the end of the black out, and then the control will be back to the normal state.

For data analysis, we must consider that during the failure some annoying transient could affect the data. The event can be easily traced because it correspond to a data loss from Sc_NI.

Comments to this report:
ruggi - 15:02 Monday 18 November 2019 (47705) Print this report

The failure happened twice. The first time it did not work; the second it worked. The transient is not so smooth, but the logic can be improved.

Images attached to this comment
ruggi - 14:38 Tuesday 19 November 2019 (47718) Print this report

The logic of the patch has been changed. Many other change of code have been implemented, regarding mainly the communication among boards. Several lines of code no more in use have been removed.

narnaud - 17:54 Tuesday 19 November 2019 (47724) Print this report

Actually it seems that the NI patch worked another time yesterday afternoon. The first even below is the one reported here but there was another one about two hours later.

(...)

* transition @ GPS 1258119499 [2019-11-18 13:38:01+00:00 UTC]
        - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 15.0761804581 NI_MAR_TY = -23.3557052612
        - Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0

* transition @ GPS 1258119505 [2019-11-18 13:38:07+00:00 UTC]
        - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
        - Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 2.09182810783 NI_MAR_TY = -3.07991695404

=> Lock in SCIENCE mode survived! Data back after 6 s

* transition @ GPS 1258125100 [2019-11-18 15:11:22+00:00 UTC]
        - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 28.0534667969 NI_MAR_TY = -43.1779594421
        - Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0

* transition @ GPS 1258125106 [2019-11-18 15:11:28+00:00 UTC]
        - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
        - Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 28.4932975769 NI_MAR_TY = -44.0734710693

=> Lock in SCIENCE mode survived! Data back after 6 s

(...)

-9999 is the value that particular monitoring code gives to a channel when missing in a frame.

masserot - 18:06 Tuesday 19 November 2019 (47725) Print this report

The 6 seconds is the time period  where no data are sent by the Sc_NI DSP or received by the DAQ ( see 47706 , 47697, 47658, 47643)

narnaud - 23:19 Thursday 21 November 2019 (47758) Print this report

Now, my monitoring script finds 10 events during which SCIENCE mode survived a temporary loss of data from Sc_NI. I won't make any further entry regarding that: we'll include this pattern into the list of offline CAT1 for O3b. Though, one will have to define how many seconds of data should be flagged bad after Sc_NI data are back.

gps_start = 1258070418 (2019-11-18 00:00:00+00:00), gps_stop = 1258409200 (2019-11-21 22:06:22+00:00), duration = 338782 s

* transition @ GPS 1258119499 [2019-11-18 13:38:01+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 15.0761804581 NI_MAR_TY = -23.3557052612
    -    Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
* transition @ GPS 1258119505 [2019-11-18 13:38:07+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
    -    Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 2.09182810783 NI_MAR_TY = -3.07991695404
=> Lock in SCIENCE mode survived! Data back after 6 s

* transition @ GPS 1258125100 [2019-11-18 15:11:22+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 28.0534667969 NI_MAR_TY = -43.1779594421
    -    Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
* transition @ GPS 1258125106 [2019-11-18 15:11:28+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
    -    Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 28.4932975769 NI_MAR_TY = -44.0734710693
=> Lock in SCIENCE mode survived! Data back after 6 s

* transition @ GPS 1258229532 [2019-11-19 20:11:54+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 7.01193523407 NI_MAR_TY = -10.7937746048
    -    Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
* transition @ GPS 1258229539 [2019-11-19 20:12:01+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
    -    Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 22.0676021576 NI_MAR_TY = -34.5200119019
=> Lock in SCIENCE mode survived! Data back after 7 s

* transition @ GPS 1258253428 [2019-11-20 02:50:10+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 11.8113393784 NI_MAR_TY = -18.3564739227
    -    Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
* transition @ GPS 1258253435 [2019-11-20 02:50:17+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
    -    Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 25.6923675537 NI_MAR_TY = -39.4061698914
=> Lock in SCIENCE mode survived! Data back after 7 s

* transition @ GPS 1258283915 [2019-11-20 11:18:17+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 28.2850456238 NI_MAR_TY = -44.4513893127
    -    Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
* transition @ GPS 1258283922 [2019-11-20 11:18:24+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
    -    Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 22.153093338 NI_MAR_TY = -33.6940765381
=> Lock in SCIENCE mode survived! Data back after 7 s

* transition @ GPS 1258295812 [2019-11-20 14:36:34+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 160 NI_MAR_TX = 27.9129943848 NI_MAR_TY = -43.6636047363
    -    Next: Mode = 1 ITF_LOCK = 160 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
* transition @ GPS 1258295819 [2019-11-20 14:36:41+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 160 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
    -    Next: Mode = 1 ITF_LOCK = 160 NI_MAR_TX = 13.7728157043 NI_MAR_TY = -20.7293167114
=> Lock in SCIENCE mode survived! Data back after 7 s

* transition @ GPS 1258315710 [2019-11-20 20:08:12+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 16.7752056122 NI_MAR_TY = -26.0260696411
    -    Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
* transition @ GPS 1258315717 [2019-11-20 20:08:19+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
    -    Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 24.9769821167 NI_MAR_TY = -38.8707809448
=> Lock in SCIENCE mode survived! Data back after 7 s

* transition @ GPS 1258335749 [2019-11-21 01:42:11+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 18.6267108917 NI_MAR_TY = -28.8386802673
    -    Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
* transition @ GPS 1258335756 [2019-11-21 01:42:18+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 165 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
    -    Next: Mode = 1 ITF_LOCK = 165 NI_MAR_TX = 38.8921852112 NI_MAR_TY = -57.5897216797
=> Lock in SCIENCE mode survived! Data back after 7 s

* transition @ GPS 1258344840 [2019-11-21 04:13:42+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 23.734167099 NI_MAR_TY = -36.6579246521
    -    Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
* transition @ GPS 1258344846 [2019-11-21 04:13:48+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
    -    Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 2.97830367088 NI_MAR_TY = -4.49181032181
=> Lock in SCIENCE mode survived! Data back after 6 s

* transition @ GPS 1258407818 [2019-11-21 21:43:20+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 20.8295040131 NI_MAR_TY = -32.9173431396
    -    Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
* transition @ GPS 1258407825 [2019-11-21 21:43:27+00:00 UTC]
    - Current: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = -9999.0 NI_MAR_TY = -9999.0
    -    Next: Mode = 1 ITF_LOCK = 170 NI_MAR_TX = 29.3635120392 NI_MAR_TY = -44.6247673035
=> Lock in SCIENCE mode survived! Data back after 7 s

=> Survived 10 temporary data losses
delta_gps: min = 6 s, max = 7 s, mean = 6.7 s, median = 7.0 s, std = 0.46 s

Search Help
×

Warning

×