DRAFT  REPORT - shift in progress; the report will be finalized and convalidated at the end of the shift

ITF found in relocking phase, Calibration mode.

Hrec has two actuator models for each mirror NE,WE,BS: one model for LN1, one model for LN2.
When the flag V1:SAT_NE_LN2_P2 is 1, Hrec uses the LN2 model for NE
When the flag V1:SAT_WE_LN2_P2 is 1, Hrec uses the LN2 model for WE
When the flag V1:SAT_BS_LN2_P2 is 1, Hrec uses the LN2 model for BS
If those flags are missing, Hrec considers by default that ITF is in LN2.
If those flags are at 0, Hrec considers that we are in LN1.
Since the relock of this afternoon, the ITF was in LN2 but the flags were kept at 0, thus Hrec was using LN1 models for NE,WE,BS actuators.
That's why the sensitivity was more noisy (BNS range around 48 Mpc instead of 55 Mpc) and the adjustment of optical response was wrong (DCP frequency around 260 Hz instead of 180 Hz).
Waiting for the SAT flags to be put back to the correct values of 1, I have restarted Hrec arounf 21h45 UTC after having commented the lines SWITCH_LN2 and SET_ACTUATOR in the configuration Hrec_actuators.cfg (by default Hrec considers that ITF is in LN2). So, we are back to 55 Mpc.
Next time suspension's people wants to do an April fool with the coil drivers flags, they should tell us before the end of the day. We loose 6 hours in finding the origin of the wrong h(t) reconstruction.

After the maintenance yesterday, around 18h LT Michal realized that there was something strange in the reconstructed h(t), in particular the cavity pole was around 260 Hz instead of 180 Hz. Also the optical gains were not as usual (see figures 1 and 2). On the contrary, the cavity pole estimated by ISC was still ok (see last figure).

After a lot of investigations, and some injections for measurements of the optical responses and for actuator responses,  we found that the issue comes from the fact that the channels SAT_*_LN_P1 were all at 0, instead of being at 1, for BS, NE and WE (see figure 3).

We understood that the suspensions were in fact properly switched in their correct modes (by Metatron), but that the monitoring channels giving their modes were not correct. Hence Hrec was using the actuator models in HP for BS and LN1 for NE and WE, instead of LN1 for BS and LN2 for NE and WE. The use of incorrect actuator models directly impacts the estimation of the optical responses.

Note that the processes CoilsSb* were restarted around noon (see gray bands in figure 4) and the logs of these processes show a lot of error messages (python errors and ModBus errors), , to be checked if this is the source of the issue, or something else in the DSPs.  We called Valerio Boschi around 23h30 LT about this issue, to be checked tomorrow morning.

Around 23h45 LT, we restarted Hrec by temporarily forcing the mode of the suspensions to their real mode, without using the SAT monitoring channels. The optical responses are now back to nominal values (see very last part of figure 5).

_____________________

Around 23h50 LT (21h50 UTC), we finally  started the injections planned  for this evening/night: injections of lines at high frequency (range 1 kHz to 8 kHz) with the NE and WE PCal, to re-estimate the mechanical response of the PCal due to excitation of mirror internal (drum) modes.     (but we have skipped the injections for check hrec).

--> Injections should run to around 9h LT tomorrow morning.

___________________

To trigger this issue quickly next time, a flag must be added in the DMS to check that the suspensions are in the correct mode depending on the lock state.

Today, I found the ITF status in BAD_WEATHER.

15:00 UTC Commissioni break started.

During the shift, the lock was unstable; we managed to achieve a stable lock at 15:36 UTC. Around 16:00 UTC, the scheduled activities for the WEB acoustic injection started and ended around 16:50 UTC. After that, we needed to begin the scheduled calibration, but we encountered a problem with Hrec. M. Was, D. Bersanetti, A. Masserot, D. Verkindt, L. Rolland, and C. Grimaud started to investigate the issue. At 20:22, L. Rolland and C. Grimaud began the calibration. The activities are still in progress.

I'm not sure  ENV_NOISE_MAG_WEB was connected to DAC ch7

I think it was ch 6 instead - To Be Check

We measure the acoustic coupling at WEB before the interventions which will occurr during the commissioning break.

Note: during this measurement the hrec calibration is wrong about -20%

Setup: two amplified loudspeakers on North side, one in front of racks, one in front of cryotrap. Figure 1.

Injected noise: 8-2000Hz, level to dac is 0.004, used ENV_NOISE_MAG_WEB connected to DAC ch7

Times are in the attached log file.

Figures 2 and 3 show the observed effect in environmental sensors, B8 and Hrec. The rised noise structure look similar to those observed in January (elog 66031).

Figure 6 An extra noise in hrec is produced at at low frequency  not coherent with the injected noise.

As noticed before the shape of the structures depend on the level of microseism. Figures 4 and 5 record the wind and seismic noise at the time of this measurement.

Today I changed the Autoscience functionality of the ITF_CONDITIONS node: now it will ask to go to DQSTUDIES instead of SCIENCE, which will be the default during the Commissioning Break whenever we'll reach LOW_NOISE_3 and just take data.

DQSTUDIES was also added to the list of states from which the Automation will allow the automatic transition to SCIENCE (when we'll resume it, now it will default to itself).

This morning, the tuning of the polynomials used to drive the DAC outputs has been done by measuring in closed loop  the DAC voltage according the requested frequency, from 10Hz to 60Hz by a step of 10Hz.

Everything went well until the 60Hz frequency was requested,  for this frequency  the WEN rotor started to have some issues to extract the frequency .

The parameters were updated . The WWN rotor remains in operation while the WEN rotor has been stopped

This morning, the demodulation noises mitigation has been setup on the B1_PD2_56MHz

• the demodulation phase noise mitigation is done using the B1s_PD1_112MHz phase signal
• the demodulation amplitude noise mitigation is done using the B1_PD2_6MHz magnetude signal

For the B1p_PD1_56MHz only the demodulation phase noise mitigation is done using the B1s_PD1_112MHz phase signal

The B1_PD2_56MHz  and the B1p_PD1_56MHz demodulation noises  mitigation are engaged at at LOCKED_DC_READOUT and  disengaged at DOWN

ITF found in Science mode with Autoscience on.

At 6:00 UTC ITF in Maintenance mode, below the list of the activity communicated in control room:

• cleaning of the experimental areas;
• vacuum operations in CB;
• MC suspension checks (Valerio);
• B1p and B1 demodulation noise (Alain);
• works in CEB clean room;
• periodic tasks performed by the operator:
  • lock and scan of the OMC;
  • check of IMC working point;
  • TCS chillers refill;
  • TCS Thermal Camera Reference;
  • TCS power checks:

All the activities concluded at around 10:00 UTC; after Maintenance I set Bad_weather because the wind activity was very high; we could lock from 12:00 UTC to 12:14 UTC.

On March 18th PDU DPDT switches were installed on both NE (as reported in entry #66399) and MC. While NE setup has been tested on the 18th, MC devices have been tested successfully this morning.  

Today, I found ITF locked in CARM_NULL_1F. 13:37 UTC ITF back in SCIENCE with Autorelock.

At 15:00 UTC, scheduled Noise Injections and Calibration began. Here is the list of activities:

• 15:00-15:44 UTC: ENV Magnetic Noise Injections
• 15:45-15:59 UTC: ISC injections (inject_asc_test.py) - failed; ITF unlocked
• 16:44 UTC: ITF returned to LOW_NOISE_3
• 16:46-16:49 UTC: CALIBRATED_DF_SENSITIVITY - failed; ITF unlocked
• 17:37 UTC: ITF returned to LOW_NOISE_3
• 17:40-18:38 UTC: CALIBRATED_DF_SENSITIVITY

The CALIBRATED_DF_PCAL process was skipped due to insufficient time remaining.

18:40 UTF CALIBRATION ended

18:41 ITF back in SCIENCE

Guard Tour (UTC)
17:50 - 18:20
20:30 - 21:00

After some investigation, we found that the Tolm packets sent by the Acl tasks are not transmitted in the same order as the ACL tasks are executed:

• all the Tolm packets sent by an Acl task are transmitted in the same order as they are declared in the Acl configuration flle\
• today when a task is re-started, it is executed at the same rank as previously for the task order but its Tolm packets are put at the end in the packet list to transmit

As reminder, Monitoring information about emission and transmission times is available for each packet exchange using the TOLM-v2 format, meaning that today these informations are not available for the  Tolm packets sent to the suspension.

For the ISC_rtpc(rtpc20),

• Only the CALnoise, LSC_SUSP, LSC_Acl and ENVnoise Acl taks  send external Tolm packets to others devices (rtpcs, DSP, others  ). Related to the LSC_Acl ACL task, the Tolm packets related to the suspension are sent firstly and then in order  the one for ISYS named LSCToISYS,  to one for  SSFS named LSCToSSFS and as last the one to OMC named LSCToOMC
• the 2024-10-22: all the Acl taks were restarted with the latest Acl version. This plot shows the emited time  for each packet transmitted according to its Acl task , in brackets the current order of transmission
• the  2024-10-26, the CALnoise server was restarted . This plot show that after this operation the CALnoise Tolm packets are transmitted the last
• the 2025-02-18-16h45 the LSC_Acl server was restarted for unknown reasons. As consequence its Tolm packets, including the suspension ones, were sent the last , see this plot.

Today, the 2025-03-31 at 12h21m-UTC, as trial to recover the expected Tolm packet transmission order ,

• the LSC_SUSP, LSC_Acl and ENVnoise servers were restarted
• and the minimal elapsed time for the LSC_Acl task has been reduced for 24us to 23us
• this plot shows  the result of these operations: the expected order for the Tolm transmission has been recovered

ITF found in LOW_NOISE_3 in SCIENCE mode (AUTOSCIENCE_ON). 
Unfortunately the ITF was a bit unstable. We collected 3 unlocks events:

• 06:43 UTC due to PR F0 TY glitch. ITF autorelocked at 07:58 UTC at 2nd attempt. SCIENCE mode.
• 10:02 UTC due to end TM (MIR or MAR) correction saturation. ITF autorelocked at 10:53 UTC at 1st attempt. SCIENCE mode.
• 12:14 UTC (TBC). Taking advantage of the unlock, Masserot asked to retart some LSC processes and rtpc20. Activity performed at 12:21 UTC. Relocking in progress...

Ego truck passages around CEB (UTC):
07:55 - 08:45

SBE
08:42 UTC - SQB1 Position loop opened (TBC). Properly closed via VPM.

The following report has been submitted to the On-call interface.

On-call events -> GSS

Title: Surveillance camera restoration

Author(s): margarita

Details:

Surveillance camera restoration

* Note that any files attached to this report are available in the On-call interface.

A question for the design of O5 was if the green beam is visible on the detection benches.

Figure 1 shows the image of the camera looking at SDB1 with the infra-red filter turned on so that the camera provides color images instead of black and white. The image is during the lock acquisition with the arms locked on the green. The green is clearly visible all over the bench, and the light which has a slightly purple color is the infra-red beam (it was flashing when this image was taken, while the green was stable).

ITF found in LOW_NOISE_3 and in Science Mode. It unlocked at 21:04 UTC, back in Science Mode from 21:52 UTC.
ITF left locked.

Guard Tour (UTC)
20:55 - 21:32
23:16 - 23:46
1:50 - 2:23
4:07 - 4:44 

There was no noticeable difference in BNS range when the RFC was off resonance in LN3.

Figure 1. Comparing data with RFC off-resonance (blue) and on resonance (purple), there is no change in spectrum at any ports of the interferometer. The only difference is for the RFC itself, where there is no light in transmission, and the light reflection becomes an imperfect indepent measure of the laser intensity noise. The clearance between the 1501Hz PSTAB and the noise level looks the same for the RFC reflection and for the PSTAB out-of-loop channel (PDd).

Figure 2. Looking at the OMC length spectrum as expected the noise is much higher without the RFC loop (blue), especially at 30mHz. The only exception is a bump at ~3Hz which is present when the RFC loop is closed (purple), I guess this is the frequency at which the RFC control loop of CARM has the phase going through zero, creating gain peaking.

Today, I found ITF in SCIENCE. 

• 05:12 UTC, ITF unlocked. At 06:39 UTC, ITF relocked with autorelock and returned to SCIENCE.
• 11:45 UTC, ITF unlocked again. NE misaligned after the unlock, so I manually realigned to restart the lock.

ITF left in relocking 

Guard Tours (UTC):

• 06:00-6:40
• 08:10-08:50
• 10:30-11:10

ITF found at CARM_NULL_1F, back in Science Mode from 23:24 UTC. It kept the lock for the rest of the shift.

Guard Tour (UTC)
22:14 - 22:55
1:25 - 2:02 
3:03 - 4:37

This afternoon the ITF unlocked once at 16:27UTC, it relocked at the third attempt, science mode started at 17:35UTC; daily calibration and ISC noise injections from 18:30UTC to 19:03UTC; unlocked again at 21:11UTC, relock in progress.

-guard tours (UTC):
16:45 --> 17:15
20:00 --> 20:30

Calibration
- 18:30UTC: CALIBRATED_DF_DAILY;

ISC
- 18:43UTC: ASC injections (inject_asc_test.py);

Today, I found ITF in SCIENCE. 

• 06:34 UTC, ITF unlocked. At 07:21 UTC, ITF relocked with autorelock and returned to SCIENCE.
• 07:57 UTC, ITF unlocked again. At 08:46 UTC, ITF relocked with autorelock and returned to SCIENCE.
• 10:03 UTC, ITF unlocked again. At 10:49 UTC, ITF relocked with autorelock and returned to SCIENCE.
• 12:58 UTC, ITF unlocked again. 

iTF left in relocking acquisition 

Guard Tours (UTC):
06:30 - 07:10
11:15 - 11:55
14:15 - 14:45

ITF found in relocking phase; it went in Science mode at 22:41 UTC.

It unlocked at 1:35 UTC;  it went in Science mode at 2:24 UTC.

It unlocked at 4:24 UTC;  it went in Science mode at 6:09  UTC.

Software

BacNet server restarted at 22:35 UTC because it was providing flat data.

Guard tours (time in UTC)

23:00-23:40; 1:10-1:45; 3:45-4:20