In order to perform the optical losses measurements at WE around 8h-10h UTC, we have stopped the WE PCal permanent lines. During these measurements, we have seen glitches on the power sensors.

We initially though it may be related to the SAT activities to install a PDU on a rack close to the calibration rack, and in particular related to switching on the motor drivers plugged on this PDU. However, the conclusions are not clear finally.

Here are some plots as a reminder:

• figure 1: a short period with glitches during ~3 seconds. The peak-to-peak values of the power during the  period without noise are 1.299 to 1.301 W , it increases up to 1.295 to 1.305 W in the noise period.
• figure2: zoom on the same period.
• figure 3: another period, of about 1.5 minutes, with glitch rate increasing, and then they suddenly stopped.
• figure 4: around 9h14 UTC, the glitches seemed to appear at the time the SAT team started the motors. This very glitchy period lasted all along the motor tests, but did not stop at the end. 
• Around 9h48-9h54, we restarted the PCal processes and the glitches were not there after restart.
• Figure 5: At 9h56, we made a test: SAT team restarted the motor drivers, and glitches re-appeared with increasing rate. They were stop 1 or 2 minutes later, and the glitch rate decreased and suddenly they stopped.... however, glitches restarted a bit later, without particular action from SAT.
• FIgure 6: summary of the morning activities at WE from the trend of the channels, where the presence or absence of glitches is well visible looking at the min/max values.
• SAT team left WE building around 10h20-10h30.
• 10h30-10h40: stopped all PCal hardware (switch off the Hameg power supply and the DaqBox, ), and restarted all, to compare the noise of the WE and of the NE sphere until 11h30 (see previous logbook entry).
• Figure 7 : at 11h40, put back the permanent lines. The amplitude of the lines is much larger than the glitches (the bottom plot has been zoomed on y-axis to see some glitches before the calibration lines were restarted).

We did not notice such glitches during the run. However, they may have been present, but not visible since the calibration lines were always on.

In the data from this morning, it looks like the glitch rate was higher and over longer periods when there were activities around the calibration and SAT motor racks in the WE building, but it is not very conclusive.

The PCal power loop is using the single-ended output of the fastDAC. It is known that (at least some) fastDAC channels have some oscillation at ~MHz high frequency.  A past quick test at NE (logbook 66132) to use differential output was not useful about the noise level. But could it generate these small glitches?

To check if the situation is similar for NE PCal, it will be worth stopping the NE PCal lines also for a long period during the WE mirror replacement next week.

ITF found in Commissioning mode with locked arms; calibration activity in progress.

The calibration activity was concluded around 14:10 UTC; after that a strong earthquake (M 6.9 - Reykjanes Ridge)  prevented to relock.

At 16:16 UTC the ITF reached LN3; then we contacted Valerio to perform the WE kick. Unfurtunately the ITF unlocked after 1s.

A second kick was scheduled at 20:00 UTC but Valerio was not available; I did not understand if he will perform the kick later in the night...

For this reason I left the ITF in Commissioning mode with Autorelock_failsafe engaged.

SBE

SPRB opened because of the earthquake; properly closed.

WE PCal switched off today around 17h30 UTC. The laser driver has been switched off to prevent any restart of the PCal during the WE mirror replacement.

The DMS flags associated to the WE Pcal have been shelved  for two weeks  (the PCal ones around 17h30, the hrec bias ones based on the WE PCal around 19h50).

I have a done a reprocessing of Hrec for the time period where the SAT flags were disturbing h(t) reconstruction (see also elog66486), between 1427543600 (April 1 11h53 UTC) and  1427578996 (April 1 21h43 UTC).

The reprocessed data are here:
/data/prod/hrec/O4/hrepro/20250225-20250401/v1online.ffl

and the corresponding trend data are here:
/data/prod/hrec/O4/hrepro/20250225-20250401/v1online_trend.ffl

Plot1 shows a comparison of BNS range in online data (blue) and in reprocessed data (orange)
Plot2 shows a comparison of hoft_raw sensitivity curve in online data (blue) and in reprocessed data (orange).

Those reprocessed data will be then integrated to the AR frames production for chunk 20250225-20250401.

 

Plan for the day 

- Test of angle of incidence and mirror loss on reflection bench of NE PCal

- Test of angle of incidence and mirror loss on reflection bench of WE PCal

- Test of NE_Rx at WE PCal

Test of the angle of incidence and mirror loss on NE reflection bench mirror

1- mirror loss measurement
Around 7h20UTC we opened the reflection bench and placed the WSV sphere in front of the M3 mirror to measure the beam right after the viewport.
At 7h25 UTC, with laser at 1.3 W  we measured a power on WSV = 1.29882 +- 3.5e-4 W
At 7h36 we put WSV sphere at its initial position and measure a power on WSV =1.2966 +- 3.9e-4 W 

Those measurement are giving a value of mirror loss of 0.17% (comparable to LAPP measurement of 0.12% and last measurement made on site in June 2023 of 0.12% also)
This seem to indicate that most of the discrepency between NE and WE are coming from the viewport rather than the M3 mirror.

2- Visual inspection of the viewport

In order to visually inspect the viewport on the reflection bench we opened the diaphragm and shined a flash light onto it, the inspection was not conclusive as it seem to be some minimal scratch and dust but not particularly in the middle of the viewport.
After this we tried closing the diaphragm as it was before.

A first measurement made at 7h52UTC gave us a value of power on WSV =  1.29576 +- 3.5e-4 W

We tried closing the diaphragm more at 7h55 UTC and measured a power on WSV slightly lower.
We put the diaphragm back at it initial setting at 7h58UTC. 

At 8h05 UTC We closed NE Rx bench, with WSV at Rx spot and the laser at 1.3 W to redo a short calibration measurement for the photodiodes to check if touching the diaphragm changed the calibration.

Calibration AFTER playing with the Rx diaphragm. (PCal NE lines stopped)

THe background measurement is done using the same data than for the first calibration : 1427652018 for 3600s

The time period for this calibration measurement is from 10h10UTC to 13h39 : 1427710218 for 12540s

There is no significant change from the first calibration performed on the 2025-04-03 before touching the diaphragm. The plot of the voltages ratio from WSV to the photodiodes is in plot1.

Test of angle of incidence and mirror loss on reflection bench of WE PCal

We arrived at WE building at 8h25, and stoped the laser and Rx power supplu at around the same time.

1- Test of Rx blue box value when not plugged but still powered

8h30 : 15 V power supply stopped (blue box not stopped)
~8h33 :  Rx cable unplugged on Rx sphere
8h33 : blue box stopped, power supply switched on  → WE_Rx_PD1_DC goes up to 3.6 W constant . ADC saturation?
8h36: blue box switched on, still constant value.
8h37 : blue box and power supply off . Value back to ~0
8h40 : Rx sphere plugged, then power supply on, and blue box on-> value ~0.

The same effect was seen later with WSV when retrieving it from NE, in plot 2 we can see three steps of power value, first when the laser was on at 1.3W reading around -3.5 V.
Then when the laser was turned off and the WSV blue box turned off with a value around 0V and finally when the cable was disconnected from the sphere where we have a high reading of around 3 V (see plot 3) similar to the Rx case seen before.

2- Measurement of mirror loss
During this test we noticed some glitches in the data, more investigation will be done but not mirror loss measurement were performed

Measurement of Rx_NE noise at WE

8h43 : laser back to 1.3 W.  

8h45 : Stop PCal lines on WE.

10h45 : start of the noise level measurement of Rx_WE
10h47 : stoped laser and stop blue box of Rx_WE
10h50 : replace the Rx sphere of WE by the Rx sphere of NE to test the noise of Rx_NE.
10h58 -> 11h05 : start blue box, start laser and ctrl loop at 1.3 W. -> Channel PCAL_WE_Rx_PD1_DC is now reading the power using the Rx sphere from NE.
11h10 : WE_Rx sphere put back, laser switched on to 1.3 W
11h16 : put back calibration lines on NE and WE

In plot4 we can see the comparison of the noise level of the Rx_WE and Rx_NE sphere with the photodiodes as witness channels.
We see that the increased noise level for Rx_NE is also seen when reading it with Rx_WE electronics indicating that the higher noise is coming directly from the NE sphere itself.

Retrieving WSV from NE

Arrived at the NE building around 13h30 UTC, turned off laser around 13h35UTC

FInished removing WSV and its electronic at 14h05 UTC

Turned Rx and laser back on at 14h10 UTC

Related to the EDB server , the code for the EDB_OMC1 control has been added successfuly . It remains to test it

Cabling informations

• EDB_DBOX_DetLab - DaqBox SN52
  • mezzanine 0: DAC1955-SN46
    • ch02 - OMC1_PZT_dac : PZT driving signal
    • ch03 - OMC1_Peltier_dac : Peltier driving signal
  • mezzanine 2: ADC2378-SN06
    • ch02 - EDB_OMC1_PZT_out : PZT command readout
    • ch03 - EDB_OMC1_Peltier_out : Peltier command readout
    • ch04 - EDB_OMC1_Temp_out : OMC1 Temperature sensor readout
  • mezzanie 3 - SERVICE-SN10
    • ch00-01 - EDB_B1t _PD1 : B1t_PD1 Audio/DC readout
    • ch04-05 - EDB_B1s_PD1 : B1s_PD1 Audio/DC readout
• EDB_DBOX_DetLab2 - DaqBox SN113
  • mezzanine 3 - DEMOD-SN43
    • ch00 - EDB_B1t_PD : EBD_B1t RF readout
    • ch01 - EDB_B1s_PD : EDB_B1s RF  readout

The work has change the sign of the OMC error signal. To recover the locking  I have changed by pi the phase of B1_PD3 and B1 demodulation for the OMC length dither line. I have changed it using the external file, and also changed for the future the main file. After that change the OMC relocked at the first attempt.

Here are the extracted values of the voltage and charge of the mirrors extracted from the data for LSC_DARM and Hrec_hoft_20000Hz.

DARM:

Hrec:

These were extracted using the ASD values for f=19.9 Hz and 2f=39.8 Hz. The difference between DARM and Hrec is explained by the difference in the calibration factor at the 2 different frequencies.

The error values were extracted using the sqrt of the variance of the ASDs used for averaging the data.

Plan for the day

WE

- Installation of the WSV sphere on WE PCAL reflection bench at the place of the Rx Sphere

- ADC channel calibration for WSV channels

- WSV vs Tx_PD1/2 calibration measurement

- Retrieve WSV from WE

NE

-  Installation of the WSV sphere on NE PCAL reflection bench at the place of the Rx Sphere

- ADC channel calibration for WSV channels

- WSV vs Tx_PD1/2 calibration measurement during the night

Installation of WSV at WE

We arrived at WE building at approximately 7h UTC, switched off the laser and started the installation of WSV on the reflection bench.

ADC channel calibration WE

- Calibration for the power channel ch6 ADC 0 (8h UTC):
with a multifonction calibrator Time series connected to the input of the follower circuit for the PCAL_WE_WSV_DC channel

• Changing the channel gain to 2. and offset to 0. in the SWEB_dbox_rack config

• Signal injection from 0 to -6V with 1V steps with the Time Series

• Computing the calibration coefficient (gain and offset) with a linear regression

• updating the coefficient in the config at approximately 8h30 UTC

- Calibration for the temperature channel 5 ADC 0 (8h10 UTC):
with a multifonction calibrator Time series connected to the input of the follower circuit for the PCAL_WE_WSV_temp channelœ

• Changing the channel gain to 200 and offset to 0. in the SWEB_dbox_rack config

• Signal injection from 0 to -6V with 2V steps with the Time Series

• Computing the calibration coefficient (gain and offset) with a linear regression

• updating the coefficient in the config at approximately 8h30  UTC

Plot of the ADC calibration measurement are in plot1 (temperature) and plot2 (power)

* :-0.0875 found in practice, but not correct value in the ADC configuration)

WSV vs Tx_PD1/2 calibration measurement WE

• Start of background measurement at 9h25 (UTC) for 1h : 1427621118 for 3600s

• Laser turned on at 1.3W at 10h30 UTC  

• Start of calibration measurement at 11h30 UTC : 1427628618 for 13800s

The background measurement result are presented in plot 3 and plot 4
The result of the voltage ratio between WSV and the two photodiodes is presented in plot 5. From this ratio and using the WSV responsivity as rho = -2.6178694 V/W.

Retrieve WSV at WE

We arrived at WE around 15h20h UTC

Stopping laser control loop and laser around 15h25UTC

WSV and its electronic system was retrieved around 15h45

We turned the Rx sphere back on and the laser back on at 1.3 W to check that everything was ok and everything was normal. 

Installation of WSV at NE

We arrived at NE building at approximately 16h UTC, switched off the laser and started the installation of WSV on the reflection bench.

ADC channel calibration at NE

- Calibration for the power channel ch6 ADC 0 (16h40 UTC):
with a multifonction calibrator Time series connected to the input of the follower circuit for the PCAL_NE_WSV_DC channel

• Changing the channel gain to 2. and offset to 0. in the SNEB_dbox_rack config

• Signal injection from 0 to -6V with 1V steps with the Time Series

• Computing the calibration coefficient (gain and offset) with a linear regression

• updating the coefficient in the config at approximately 17h10 UTC

- Calibration for the temperature channel 5 ADC 0 (16h55 UTC):
with a multifonction calibrator Time series connected to the input of the follower circuit for the PCAL_WE_WSV_temp channel

• Changing the channel gain to 200 and offset to 0. in the SNEB_dbox_rack config

• Signal injection from 0 to -6V with 2V steps with the Time Series

• Computing the calibration coefficient (gain and offset) with a linear regression

• updating the coefficient in the config at approximately 17h10 UTC

Plot of the ADC calibration measurement are in plot 6 (temperature) and plot 7 (power)

WSV vs Tx_PD1/2 calibration measurement at NE - First calibration before Mirror losses measurement

• 18h UTC : background measurement for 1h 1427652018 for 3600s

• Laser turned on at 1.3W : 20h45 UTC 

• Data period for measurement from 22hUTC to 6hUTC 1427666418 for 28800s

The background measurement result are presented in plot 8 and plot 9
The result of the voltage ratio between WSV and the two photodiodes is presented in plot 10. From this ratio and using the WSV responsivity as rho = -2.6178694 V/W.

This morning we installed the new PDU at the WE building.

Start of the installation: 10:24 (local time)
End of the installation: 10:40

During the installation, ID, F7 and LC loops were kept closed during the whole process.

After the installation, we checked if the motors were working properly.
This operation took about one hour, due to some communication problem with the motors' server. After killing the process and rebooting, communication was restored and the tests on the motors were performed successfully.
We took the opportunity also to recenter the WE tower, both horizontally and vertically.

All the operations ended at 11:40 (local time).

NB: Shortly after the installation of the PDU, L. Rolland observed a series of glitches on the WE photon calibrator. Some checks have been performed to understand if there is a correlation between the glitches and the motors' cabling, but the results were inconclusive. We thus decided to temporarily unplug the motors, so that Rolland can continue to study these glitches.

Update of the Acl tasks order on the SDB_EDB_rtpc(rtpc1)

Today on the SDB_EDB_rtpc, the task repartition on the 3 RT_CPUs is shown in the following table

The proposal is to use the EDB Acl task to manage the EDB_OMC  lock and to use the  unused elapsed time for the EBD_OMC algorithms .

To allow this a the task repartition on the 3 RT_CPUs has been reorganized as follows

DAQ Boxes configuration: EDB_DBOX_DetLab and EDB_DBOX_DetLab2

• EDB_DBOX_DetLab (DBOX_SN51)
  • EDB_ADC (ADC2378_SN06) : add 3 new ADC channels at channel Id 2,3,4 for respectively EDB_OMC1_PZT_out_raw (100KHz), EDB_OMC1_Peltier_out_raw(100KHz) and EDB_OMC1_Temp_out_raw(10KHz ) and sent to the SDB_EDB_rtpc
  • EDB_MezzPD (SERVICE_SN10):  add  2 new photodiodes EDB_B1t_PD1 on channel 0-1 acquired at 100KHz and EDB_B1s_PD1 on channel 4-5 acquired at 100KHz . To receive correctly these new channels, the EDB_B1s1 signals read on channel 2-3 are now acquired at 10KHz instead of 100KHz previously
• EDB_DBOX_DetLab2 (DBOX_SN113)
  • EDB_DEMOD_B1s_B1t (DEMOD_SN43) add the 2 new photodiodes EBD_B1t _PD1 on channel id 0 and EDB_B1s_PD1 on channel id 1. These 2 RF photodiodes are then demodulated at F6MHz, 2xF6MHz, F56MHz and 2xF56MHz. To run
    • the sample and monitoring channels are sent to the SDB_EDB_rtpc(rtpc1). 
    • the data channels are sent to the SDB2_rtpc(rtpc13)

New channels readout

• The EDB_B1s_PD and EDB_B1t_PD RF channels are managed by the SDB2_Readout server where the last  part  of the digital demodulation has been setup.
• The EDB_B1s_PD and EDB_B1t_PD RF  Audio and DC channels  are managed by the EDB server, as well   the EDB_OMC1 part
  • the code related to the EDB_OMC1 part remains to be added in the EDB server

Note that for these 2 rtpcs, SDB_EDB_rtpc and SDB2_rtpc, we are close to the maximum data flow that can be handed over 1 single Tolm link : adding a second Tolm link  between the MxDx in the DAQ room and the ones in the CEB computing room could be helpfull

ITF found in LOCKED_ARMS_IR and COMMISSIONING Mode. The calibration activity at WE was concluded at around 16:00 UTC, after that the CAL Team moved to the NE Building.
At 17:20 UTC the preparatory work at NE was concluded, and I started to relock. After a brief manual realignment of PR and SR, the ITF was back in LOW_NOISE_3 at 18:17 UTC.
Under request of the crew, at 18:19 UTC I set the ITF in CALIBRATION mode and at 18:29 UTC I launched the python script inject_pcal_mechanical.py, which will keep measuring for the following 8 hours.
Calibration in progress, ITF left locked.

Other activities carried out during the shift:
From 13:00 to 13:50 UTC - Installation of new Power Supply for SQZ in DER (Montanari, Sposito)

We have tried to measure the noise of the IMC ramp-auto and the Ampli-EOM using monitoring channels of both electronic boxes.

Figure 1. Shows the check of the sensing noise of the spectrum analyzer. With a 50Ohm plugged on the input port we measure a noise level of 9 nV/rtHz. We have used the same settings for all other measurements, between 100kHz and 10MHz.

Figure 2 Shows the spectrum analyzer plugged on the "EO monit" output of the Ramp auto, which according to Jean-Pierre reads 1/20 of the output of the Ramp auto when a 50Ohm impedence device is used to read the EO monit output. The noise there is at a ~19 nV/rtHz level. After subtracting the measurement noise floor and multiplying by 20 this corresponds to an upper limit of ~300 nV/rtHz on the noise from main output of the Ramp auto. The EO monit is a lemo 3pin connector, so we used an addapter to plug it on the BNC input of the spectrum analyzer.

Figure 3. We have then measured the noise plugging the spectrum analyzer into the "EOM CORR MONIT" output of the Ampli EOM.

Figure 4. Shows the noise level with the IMC locked, ~260 nV/rtHz

Figure 5. Shows the noise level with the IMC unlocked, ~200 nV/rtHz.

Figure 6 Shows the noise level with no input from the ramp auto plugged into the Ampli EOM, ~335 nV/rtHz.

We forgot to make a measurement with the 50Ohm plug at the input of the Ampli EOM.

Figure 7 shows the noise at the output cable of the Ramp auto, the end of the cable that normally is plugged into the Ampli EOM, with the IMC unlocked. The noise was not stationary, with many bumps and lines sometimes small and sometimes large (see figure 8). The floor was ~40 nV/rtHz.

Figure 9. We wanted to plug the Ampl EOM corr monitoring into a digital demodulation channel. We found a cable in the atrium that was free and supposedly going to the Piscina, but we couldn't find afterwards the other end of it in the Piscina.

We plugged back the cable to the ampli EOM after this work.

Today, we installed the remote power supply for the Faraday units.

The SusDAQBridge process was restarted correctly (a reload was not enough) and we could check online that the SAT gains were being updated again both as INFO in the VPM and online through dataDisplay.

The Thread__init message appeared the first time at 2025-04-01-06h17m27-UTC, Below the details of the SusDAQBridge server

   2025-03-30-08h32m58-UTC>ERROR..-worker: Sa_MC - F0_DC_ENBL: Got NaN. Dsp #42103 status: EXIT_DSP_NOTCONNECTED
   2025-03-30-16h32m40-UTC>ERROR..-worker: Sa_WE - ACC1_Noise: Got NaN. Dsp #31013 status: EXIT_DSP_NOTCONNECTED
   2025-04-01-06h17m27-UTC>ERROR..-Thread.__init__() not called
   2025-04-01-06h17m37-UTC>ERROR..-Thread.__init__() not called
 

ITF found in relocking phase, Calibration mode.

At 6:52 UTC ITF in Commissioning mode and in DOWN; below the list of the activities communicated in control room:

• ALS CEB flip mirror test by Piernicola; completed.
• PLS MHz electronic noise measurement by M. Was; completed.
• calibration activities at the WE buildings by Grimaud and Rolland; in progress.

SBE FC

At around 10:00 UTC, under request of M. Vardaro, I switched off the FCIM motor crate and I switched on the FCEM motor crate.

As Franco explained, the CoilsSb* issue is long standing and I agree it can be misleading.

However, the actuation for NE/WE/BS was correct, as we have checked during the evening, so that means that the relays were set correctly.

The same can be said for the gain values inside the DSPs, as without them the actuation would have been wrong, and probably we would have unlocked given the quite big difference between the LN1 and LN2 actuators configurations (more than a factor 10 if I recall correcly).

I can infer then that the issue was that the values of the gains were correct, but the update was not sent to the DAQ, so maybe the hiccup was at the level of SusDAQBridge?

Looking into it, I can see loads of errors of the type

• 2025-04-02 12h46m57 UTC Thread.__init__() not called

happening once every 10 seconds, so there may be a problem there.

"Note that the processes CoilsSb* were restarted around noon (see gray bands in figure 4) and the logs of these processes show a lot of error messages (python errors and ModBus errors), , to be checked if this is the source of the issue"

If you check the log file in the past months you will see the same error, in spite of the reported error the actions requested to the relays are always completed correctly. This is a known problem for wich a dedicated merge request on PySb is pending:

https://git.ligo.org/virgo/virgoapp/PySb/-/merge_requests/3

The proper solution would be the update of the relay box firmware but we preferred not to do it during the run.

We have survived like this so far. We could decide to implement the merge request and proceed in silencing the error if becoming too misleading like in this case.

Profiting of the break, this morning we wanted to study the problem of the unlocks on the IMC caused by the flip mirror installed into the green generation box of the CEB.

At first, we reconnected the original flip mirror mount to reproduce the problems. Strangely enough, we couldn't in any way cause an unlock of the IMC (neither with the mount installed inside or outside the box).

One possible explanation is that the unlocks were due to some electrical interferences with the fiber EOM box, and that, after the test we did with it in the last months, we solved some ground shorcuts/electrical anomalies.

For the time being, we left the flip mirror installed in the box and connected. We also re-engaged the automatic flip-up of the mirror after the lock of the CITF in order to see if the problem appears again (and in case to better debug it) in the metatron ARMS_LOCK.py, line #1545.

Due to the use of a shaping filter in the coil driver boards of NE and WE, the injected voltage values differ from the settings. The injection values can be retrieve looking at the specific voltage channel of the coil.

Using the GPS of the injections, here are the values of the voltages as seen by the V1:Sc_XX_MIR_VOUT_YY (with XX: WE, WI, NE, NI and YY: DL, DR, UL, UR) channels:

Hrec has two actuator models for each mirror NE,WE,BS: one model for LN1, one model for LN2.
When the flag V1:SAT_NE_LN2_P2 is 1, Hrec uses the LN2 model for NE
When the flag V1:SAT_WE_LN2_P2 is 1, Hrec uses the LN2 model for WE
When the flag V1:SAT_BS_LN2_P2 is 1, Hrec uses the LN2 model for BS
If those flags are missing, Hrec considers by default that ITF is in LN2.
If those flags are at 0, Hrec considers that we are in LN1.
Since the relock of this afternoon, the ITF was in LN2 but the flags were kept at 0, thus Hrec was using LN1 models for NE,WE,BS actuators.
That's why the sensitivity was more noisy (BNS range around 48 Mpc instead of 55 Mpc) and the adjustment of optical response was wrong (DCP frequency around 260 Hz instead of 180 Hz).
Waiting for the SAT flags to be put back to the correct values of 1, I have restarted Hrec arounf 21h45 UTC after having commented the lines SWITCH_LN2 and SET_ACTUATOR in the configuration Hrec_actuators.cfg (by default Hrec considers that ITF is in LN2). So, we are back to 55 Mpc.
Next time suspension's people wants to do an April fool with the coil drivers flags, they should tell us before the end of the day. We loose 6 hours in finding the origin of the wrong h(t) reconstruction.

After the maintenance yesterday, around 18h LT Michal realized that there was something strange in the reconstructed h(t), in particular the cavity pole was around 260 Hz instead of 180 Hz. Also the optical gains were not as usual (see figures 1 and 2). On the contrary, the cavity pole estimated by ISC was still ok (see last figure).

After a lot of investigations, and some injections for measurements of the optical responses and for actuator responses,  we found that the issue comes from the fact that the channels SAT_*_LN_P1 were all at 0, instead of being at 1, for BS, NE and WE (see figure 3).

We understood that the suspensions were in fact properly switched in their correct modes (by Metatron), but that the monitoring channels giving their modes were not correct. Hence Hrec was using the actuator models in HP for BS and LN1 for NE and WE, instead of LN1 for BS and LN2 for NE and WE. The use of incorrect actuator models directly impacts the estimation of the optical responses.

Note that the processes CoilsSb* were restarted around noon (see gray bands in figure 4) and the logs of these processes show a lot of error messages (python errors and ModBus errors), , to be checked if this is the source of the issue, or something else in the DSPs.  We called Valerio Boschi around 23h30 LT about this issue, to be checked tomorrow morning.

Around 23h45 LT, we restarted Hrec by temporarily forcing the mode of the suspensions to their real mode, without using the SAT monitoring channels. The optical responses are now back to nominal values (see very last part of figure 5).

_____________________

Around 23h50 LT (21h50 UTC), we finally  started the injections planned  for this evening/night: injections of lines at high frequency (range 1 kHz to 8 kHz) with the NE and WE PCal, to re-estimate the mechanical response of the PCal due to excitation of mirror internal (drum) modes.     (but we have skipped the injections for check hrec).

--> Injections should run to around 9h LT tomorrow morning.

___________________

To trigger this issue quickly next time, a flag must be added in the DMS to check that the suspensions are in the correct mode depending on the lock state.

Today, I found the ITF status in BAD_WEATHER.

15:00 UTC Commissioni break started.

During the shift, the lock was unstable; we managed to achieve a stable lock at 15:36 UTC. Around 16:00 UTC, the scheduled activities for the WEB acoustic injection started and ended around 16:50 UTC. After that, we needed to begin the scheduled calibration, but we encountered a problem with Hrec. M. Was, D. Bersanetti, A. Masserot, D. Verkindt, L. Rolland, and C. Grimaud started to investigate the issue. At 20:22, L. Rolland and C. Grimaud began the calibration. The activities are still in progress.