Reports 1-1 of 1 Clear search Modify search
AdV-COM (automation)
bersanetti - 18:25 Monday 26 January 2026 (68573) Print this report
Wrong request to ITF_LOCK after an unlock when the node is managed

A long standing bug is what happens when ITF_LOCK is managed (in particular by ITF_CONDITIONS for the autorelock failsafe): after an unlock, the request is changed to something different to the last target state. Some references are v65815 and v67207.

The bug originally caused the wrong requests LOCKED_PRWI and LOW_NOISE_1, which is why we linked the behaviour to the management by the CALI node or, more properly, to the double-management from CALI and ITF_CONDITIONS (never happening at the same time, though).

The bug has periods of high, low and no occurrence, and things got better once we started to properly un-manage ITF_LOCK from CALI after use; however, lately it's in the high phase again. Differently from before, now the wrong request is LOW_NOISE_2, which hasn't been requested by CALI for a long time, if ever; so this points to the interaction with state requests in general, also the manual ones, as lately we've been using LOW_NOISE_2 a lot, differently than during the Run.

A typical behaviour can be seen from the META_ log file of the ITF_CONDITIONS and ITF_LOCK nodes. From the latter:

83482:2026-01-21T22:22:54.538Z USERINPUT REQUEST: olserver52.virgo.infn.it virgorun  old=LOW_NOISE_3_ALIGNED new=LOW_NOISE_2
83483:2026-01-21T22:22:54.548Z USERINPUT REQUEST: olserver52.virgo.infn.it virgod  old=LOW_NOISE_2 new=LOW_NOISE_3_ALIGNED
83484:2026-01-21T22:22:54.553Z USERINPUT REQUEST: olserver52.virgo.infn.it virgorun  old=LOW_NOISE_3_ALIGNED new=LOW_NOISE_2

In this example there are even two hijacks: one is immediately corrected, but also immediately overridden again.

A difference that was noticed is that, differently from any other node, ITF_CONDITIONS was still using the /virgoDev/ metatron implementation and it ran as virgorun, while all other nodes run as virgod and with the production Metatron in /virgoApp/.

The link between the bug and this other issue is not certain at all, but a good starting point is to align all the Metatron nodes to the same version and user. ITF_CONDITIONS was restarted today at 14:47 UTC to be aligned to all the other nodes. Hopefully the next nights with the autorelock failsafe engaged will give more data to analyze and then understand and eventually fix the bug.


EDIT: I reverted the change, as Nicolas Arnaud noticed that it had the side effect that two channels not generated by the node but by another one (DQ_META_ITF_Mode and DQ_META_ITF_LOCKED from ITF_STATUS) were not available anymore. The reason is unclear and will be investigated, the starting point is most probably the following error message:

2026-01-26 14h41m52 UTC Could not open file info:/dev/shm/VirgoOnline/FbmMain/LastFrameInfo-ITF_CONDITIONS.info no more attempt

 

Search Help
×

Warning

Error

The present report has been modified outside this window. Please check for its integrity in the main page.

Refreshing this page will move this report into drafts.

×