Virgo Logbook

Virgo Runs (O4b)

Sposito - 7:01 Sunday 27 October 2024 (65413)

Operator Report - Night shift

Today, I found the ITF in CALIBRATION.

After F. Gherardini performed the daily DF_DAILY calibration, we noticed that the Hrec_hoft and Hrec_BNS_range data were no longer being produced. To address this issue, I first contacted the DAQ on call, E. Pacaud; however, he wasn't familiar with this type of problem.

Subsequently, I reached out to the Commissioning Coordinator, M. Was, as well as the Run Coordinator, N. Arnaud, to inform them of the issue. Following a discussion with Nicolas, we managed to get in touch with Hrec expert D. Verkindt, who identified that the problem stemmed from the CALnoise process.

D. Verkindt attempted to reload the CALnoise configuration; unfortunately, this did not resolve the issue. He noted that during a similar incident in the past, restarting the CALnoise process had corrected the timing discrepancies. Thus, we decided to contact A. Masserot to ensure our approach to resolving the situation was appropriate.

After a thorough analysis conducted by D. Verkindt, A. Masserot, E. Pacaud, and N. Arnaud, we proceeded to stop and restart the CALnoise process. Fortunately, this action did not cause any problems, and we were able to restore the Hrec_hoft and Hrec_BNS_range data, which became available again.

Throughout this process, the ITF remained locked in LOW_NOISE_3_SQZ. At 23:08, the ITF was successfully returned to SCIENCE mode.

ITF remained in SCIENCE for the rest of the shift.

Guard tour (UTC):
21:17 - 21:47
23:12 - 23:43
01:02 - 01:32
04:17 - 04:47

Oncall events

DAQ
(27-10-2024 21:15 - 27-10-2024 23:15) From remote
Status: Ended
Description: Data absence in Hrec_hoft and Hrec_BNS_range, after a DF_DAILY calibration.
Actions undertaken: stop and restart the CALnoise process

Images attached to this report

65413_1730008861_Screenshot from 2024-10-27 07-00-00.png

65413_1730008869_Screenshot from 2024-10-27 06-59-06.png

Comments to this report:

masserot - 2:04 Sunday 27 October 2024 (65414)

CALnoise issue - Acl 1r30p17

The problem is present only in the latest verion of Acl v1r30p17 . It is due to the use of the command "reloadConfig with reset " performed by the inject_checkHrecMir_script .

A simple workaround, instead of restarting the server, it's to perform a "reloadConfig with synchronization" .

Investigation in-progress

verkindt - 10:21 Sunday 27 October 2024 (65416)

Most of the relevant information were already provided by Esposito and Masserot logbook entries.
This comment is just to add that the origin of the problem was quite rapidly found to be CALnoise because, after the first CAL CheckHrec injection (that terminated with a Reload Config of CALnoise), the permanent lines injected by CALnoise of PCal, NE, WE, BS... were not present anymore.
Plot1 shows for instance the spectrogram of CAL_NE_MIR_Z_NOISE where can be seen the injections but not anymore the permanent lines that were present before the first injection.
A restart of CALnoise while the ITF was in LN3 succeeded in solving the problem and it did not unlock the ITF or introduced any timing problem (which I feared to occur on the basis of a problem we had some months ago when we did a CALnoise restart during a calibration shift).

Images attached to this comment