ITF found in  TROUBLESHOOTING Mode and LOW_NOISE_3
All times are UTC.
14:26:50 Recentered SDB1 bench via VPM: SDB1_LC > Bench > Used fixed setpoints and set usage of floating setpoints at 15:03:28; 
15:30 CALIBRATION Mode set for CALIBRATION WITH ITF UNLOCKED (ITF DOWN after SHUTTER OPENING due to SDB2_B1p_DC high ~0.037 mA);
CALIBRATIONS WITH ITF UNLOCKED:
- From 15:31 to 15:43 CALIBRATED_DELAY;
- From 15:44 to 16:28 CALIBRATED_PRWI; 
- 16:56:20 ITF unlocked after shutter opening due to too much power on SDB2_B1p_DC;
At 17:08 I contacted Was to solve the issue during SHUTTER OPENING, at 17:11 I put CALI node in down interrupting the Calibration activity and requesting the ITF to go in CARM_NULL_1F as requested by Was to further investigate the origin of the issue. Was referred me that he set the python command "LSC.MICH_SET.AcConstChSet(75.0,600.0)", this action did not solved the issue because ITF correctly opened the SDB1 shutter but it unlocked when locking the OMC.
I noitced that the power of the TCS_NI_CO2_POWER_CH_PICKOFF was 0.550 W and I performed weekly TCS power checks, since CH values for WI and NI were not the desired one I contated the TCS oncall (Lumaca D.) which adjusted the powers (add entry when written) for both the NI/WI, activity ended at 18:39; 
CALIBRATED_DF_INtoEND_LN and ISC injections (inject_asc_test.py) SKIPPED.
From 19:20 to 19:54  Nardecchia asked to perform TCS checks with termocameras (#67992).
At 20:29 I called ISC oncall (Mantovani) due to several unlocks around CARM_OFFSET_STEP_3_OF_3. Oncall activiy ongoing.
At 21:06 Mantovani changed the MICH Gain for CARM_OFFSET_STEP_3_OF_3 from 0.3 to 0.4;
Relock in progress...

From 16:10 to 16:25 problems with internet connection (EGO Service adviced via mail);
Activity during TROUBLESHOOTING: WI CO2 Laser Tests (Nardecchia, Melo, Nocera, Spinicelli, #67991).

Unlocks during the shift:
- 13:43:12 (from LOW_NOISE_3);
- 13:43:12, 14:11:09, 14:46:15 (from SETTING_CARM_OFFSET_MID);
- 14:55:53 (from REDUCING_CARM_OFFSET_1_OF_3);
- 15:29:47, 16:56:20 (from LOCKING_OMC_DARM_B1_PD3;
- 17:05:58 (from  LOCKING_CARM_MC_IR);
- 18:49:22 (from LOCKING_ARMS_BEAT_DRMI_1F);
- 19:07:33 (from LOCKED_CARM_OFFSET_MID);
- 19:20:55 (from CARM_NULL_1F);
- 19:34:33 (from ACQUIRE_DRMI);
- 19:41:23 (from ALIGNING_DRMI);
- 19:49:00, 19:53:06, 19:56:25 (from LOCKING_ARMS_BEATING);
- 20:06:06 (from REDUCING_CARM_OFFSET_3_OF_3);
- 20:15:06 (from CARM_OFFSET_STEP_3_OF_3);
- 20:24:28 (from REDUCING_CARM_OFFSET_3_OF_3);
- 20:34:22 (from ACQUIRE_CARM_NULL_3F);
- 20:44:03, 20:54:19 (from REDUCING_CARM_OFFSET_3_OF_3);
- 21:05:12 (from REDUCING_CARM_OFFSET_3_OF_3);

Guard tours:
17:48 - 18:26
19:49 - 20:27

Given the recent lock instabilities, I performed a quick check of the WI DAS  with the thermal camera, to exclude the possibility that something may have happened during this afternoon’s intervention.

I compared the thermal images from 7 October 2025 with those taken this evening: no visible changes were observed(see attached figure).

Additionally, after the intervention, the CO₂ main laser has been behaving stably, with only two minor events recorded so far (see second figure).

ON SITE: Flavio - Piernicola - Suzanne

FROM REMOTE : Ilaria

Today,  we performed the following tests in the TCS room to investigate the problems on the WI CO2 main laser initially described in 67976. Here the lists of the tests:

1. 5 V TTL modules swap
We powered off both lasers and swapped the BNC cables that provide the 5 V TTL enable signals: WI connected to the NI module and vice versa.
Result: the WI laser continued to show on/off events. Therefore, the 5 V TTL signal is not the cause.

Actions  timeline:

• NI OFF: 12:28:26 UTC

• WI OFF: 12:28:52 UTC

TTL swap back. Then, 

• NI ON: 12:29:21 UTC

• WI ON: 12:30:19 UTC

2. RF driver swap and inspection
After excluding the 5 V TTL as the cause, we swapped the RF driver generators between the two CO2 lasers.
NI OFF: 13.36.54 UTC
WI OFF: 13.37.11 UTC
RF signal generators swapped. Then,
WI ON: 13.38.00 UTC
NI ON: 13.38.30 UTC  

3. INSPECTION ON THE BENCH
During this test, Flavio, Piernicola, and Suzanne visually inspected the lower part of the enclosure where the RF driver is located. We noticed that the WI laser switches off and on when touching the cables connecting the power supply to the RF driver at the junction (see the attached figure).
At this point, Flavio temporarily secured the cables to keep the laser on while deciding the next steps.

Notes:

• The BNC connections of the RF generators are still swapped — to be RE-SWAPPED

• Even in this case, touching these cables and turning the laser on/off produces simultaneous disturbances on the PR CHRoCC (see the second figure).

Since the intervention, no further crises have been observed in the CO₂ laser behavior.

We will continue to monitor the situation. In the meantime, the electronics crew has prepared the needs for soldering the cables, so that the repair can be performed during tomorrow’s maintenance.

It goes without saying, but the RF driver is covered with mouse droppings.

ITF found relocking (LOCKED_ARMS_BEAT_DRMI_3F) in PREPARE_SCIENCE mode. 05:36UTC - LOW_NOISE_3 acquired, ITF in SCIENCE mode at 05:39 (Autoscience ON); BNS Range ~50 Mpc. Unlocks (times in UTC):
08:13 - from LN3 due to DIFFp low gain? (pdf 1), 08:28 - from LOCKING_CARM_MC_IR, 08:33 - from LOCKING_ARMS_BEATING. 08:47 - from CARM_NULL_1F, causing SIB1 (IB) bench to drift from its orginal setpoint (pdf 2). 09:21 - Under guidance from Boldrini I manually unlocked ITF to allow for dithering loops to stabilize in LOCKED_ARMS_IR after recovering the position of IB by shifting PR_F0_X (+50 um). 09:56 - from LOCKING_OMC_DARM_B1_PD3 due to DIFFp_TY glitch? (pdf 3). 10:14 and 10:23 - from REDUCING_CARM_OFFSET_3_OF_3. 10:49 - from LOCKING_OMC_DARM_B1_PD3. 11:36 - from LOCKING_CARM_MC_IR, causing NE guardians and SNEB_SBE controls to open, properly closed at 11:56.
12:18UTC - ITF in TROUBLESHOOTING mode for intervention on WI CO2 Laser in TCS room, rescheduled to today as agreed with Melo, Nocera and Was. 12:44UTC - Automation relocked to LN3, unlocked again at 12:47UTC. Relock in progress.

ISC
07:03 - 08:05: ViolinModes briefly excited with high SeaActivity observed on DMS (fig.1)

Fig.1 shows an example of software saturation on the audio channel of the B1 photodiodes with a threshold set at 0.075 mW. In this example there is no sign of hardware saturation.

Fig.2 shows another example, where the analog saturation of the audio channel of the B1 photodiodes is visible at about -0.11 mW (predicted saturation value is 0.114 mW).

Accordingly we have decided to increase the saturation threshold in SDB2_Photodiodes.cfg at +/-0.10 mW.

The saturation threshold for the audio channels of the B1_PD1 and B1_PD2 photodiodes has been updated this morning at 08h50m48 UTC.

I am not sure whether this is relevant, but in correspondence with the power on/off events of the WI CO₂ laser and the glitches in the PR CHRoCC, simultaneous disturbances are visible on the UPS INF_CB_UPS1_RBT (see attached plot).

A force of 2N for the piezo motor is actually what one should expect from the specification. The rotator doesn't list a force, but the linear translation stage that use the same piezo-motor and the same driver has a holding force of 4N and has a force of 2N to move up and down a load: https://www.newport.com/f/agilis-piezo-motor-linear-stages

ITF found locked at LN3 in SCIENCE mode.
At 22:13UTC ITF unlocked (TBC). After the usual cross-alignment in ACQUIRE_DRMI, ITF relocked (only at 23:57 UTC after several unsuccessfully attempts). SCIENCE mode. 
At 04:30 UTC it unlocked again. Relocking in progress...

ISC
At 21.29 UTC the violin mode started to be excited (Value = 3.25721E-7) as happened the 14th Oct.

12:59 UTC ITF failed locking attempt at LN2
13:49 UTC ITF in SCIENCE
17:00 UTC after more glitches on WI CO2 and CHROCC simultaneously, that caused the range to drop to 30 Mpc for a few seconds, I contacted I. Nardecchia and P. Spinicelli and I went to TCS room to investigate the racks and the conditions of the room. I was not able to point out a possible common electric proximity, their electronics are on separated racks. I couldn't find any possible failures in the fans or any anomalies in the temperature of the electronics. The only thing I noticed is that PR CHROCC K.E.R.T., while giving the correct values for voltage and current, was making a soft continuous crackling sound that the other K.E.R.T.s on the adjacent rack weren't making. CO2 electronics showed nothing out of the ordinary.

Guard Tours
15:40 - 16:20
16:20 - 17:00  --> Guard reports inability to get out of the car and check North Terminal due to loose hunting dogs inside the perimeter
19:53 - 20:32

The only anomaly noticed after a second investigation is a soft continuous crackling sound coming only from PR CHROCC K.E.R.T. Other identical devices do not produce this sound.

After the manual power cycle reported by Francesco in his previous entry (67979), the WI CO2 laser appeared stable for some hours, until it switched off again and then restarted autonomously.

This happened at 17:38:30 UTC, after which the laser turned back on at 17:39:10 UTC.
A second event occurred at 17:51:50 UTC, with the laser restarting again at 17:52:50 UTC.

Cecilia briefly went to the TCS room to verify the hardware status, but did not observe anything abnormal.

Moreover, Francesco noticed that during every instability of the WI CO2 laser, the PR CHRoCC shows a simultaneous disturbance (see attached plot).
This correlation is quite puzzling, since the CHRoCC power supply is located on a completely different rack from the CO2 electronics.
For this night, there is not much we can do =(

ITF found in Science mode.

The ITF unlocked at 5:07 UTC, then was successfully relocked on the first attempt and set to Science Mode at 5:50 UTC. Unfortunately, around the same time, the WI CO₂ laser started exhibiting malfunctions. The ITF unlocked again at 6:30 UTC and could not be relocked afterward. At 7:57 UTC, I set Troubleshooting Mode and, following Ilaria’s instructions, went to the TCS room to perform a power cycle (switch off/on) of the WI CO₂ laser.

The ITF relocked and Science mode was set at 9:09 UTC; ITF unlocked again at 12:18 UTC.

Relock in progress.

ACS

The DMS was reporting a low-humidity issue in the DET LAB. At 10:00 UTC, I contacted Davide, who was already aware of the problem. An intervention to fix it is planned for the next maintenance session.

Guard tours (time in UTC)

5:30-6:10; 8:20-9:00

ITF found relocking (CARM_NULL_1F) in PREPARE_SCIENCE.
ITF back in SCIENCE mode at 13:29 UTC. Unfortunately ITF unlocked again at 13:36 UTC. I observed a decreasing of th B7 and B8 power just before the unlock. (TBC).
Relocked at first attempt at 13:48 UTC after the usual cross-alignment in ACQUIRE_DRMI. SCIENCE mode. 

15:26 UTC - S251018bi

19:32 UTC, after an unusual drop in the BNS range, CALIBRATION mode set
- 19:32 UTC - check hrec calibration
- 19:45 UTC - ISC injections (inject_asc_test.py)

ITF back in SCIENCE mode at 20:04 UTC.

Guard tour (UTC):
13:00 - 13:41
15:20 - 16:00
18:40 - 19:08

DAQ
15:04 UTC - SNAlert VPM process crashed. Restarted.

Today around 14:00 LT Francesco called me to report a malfunction of the WI CO2 laser (the one used for DAS).

From the first attached plot it appears that the laser started exhibiting anomalous behavior around 05:00 UTC, with repeated drops in output power followed by recoveries. These oscillations became progressively more frequent in the late morning, eventually leading to several short shutdowns (1–2 minutes each). The last one, the longest, lasted from 12:04 UTC to 12:11 UTC.

After this interruption, the laser restarted by itself without any action from either me or Francesco. The issue seems to have an electrical origin.

Despite the CO2 laser being off for about 7 minutes, the interferometer remained locked, although with an evident overall worsening of the signals. It eventually unlocked at 12:42 UTC.

I followed the lock acquisition again up to CARM NULL 1F and everything went well.

We will keep monitoring the laser behaviour.

ITF found in Science mode.

ITF unlocked at 8:32 UTC most likely for a problem of missing data visible in many channels and reported by INJ_Fb, SUSP_Fb, SDB_EDB_Fb (see attached plot).

ITF relocked at the first attempt, Science mode was set at 9:18 UTC.

Around 10:30 UTC , the CO₂ laser started exhibiting malfunctions; the effect was clearly visible on Hrec (see attached plot). At 12:12 UTC , I contacted the TCS on-call and I set Troubleshooting mode. The laser behavior returned to normal around 12:13 without any human intervention; at 12:28 I removed the Troubleshooting mode. Ilaria will monitor the situation.

The ITF unlocked at 12:42 UTC, relock in progress...

666Hz is the expected frequency of the motion of the rotator. The documentation (https://www.newport.com/p/AG-UC8) list for speeds, 5 steps per second, 100 step per seconds, 1700 steps per seconds and 666 steps per second. From what I remember from the rotator on SDB1, 666 steps per second is the default speed. It is very likely that the steps are done through some square wave applied on the piezo element that makes the rotation. So the measured line at 666Hz in the IMC length corresponds well to the expected mechanical motion created by the rotator as it moves.

ITF found in LOW_NOISE_3 and in Science Mode. It kept the lock for the whole shift.
ITF left locked.

I’ve quickly investigated these broad arches using the usual Scattered Light analysis method (ref. Figure 1), by correlating the frequency of the arches with the velocity data from all available position sensors.

The most correlated channels are those from the North End, starting from the Local Control of the suspension down to Filter-7. I’m attaching the list of most correlated channels, along with plots for some of those showing the highest Pearson correlation coefficients: Figure 2,3.
Further down the list, the first non-North End channel showing correlation is the Beam Splitter Filter-0 control. However, as shown in Figure 4, the frequencies do not always match perfectly.

Associated with this noise, there is also a seismic transient visible in the 0.03-0.1 Hz BRMS band of the seismometers at all terminal buildings. This may indeed correspond to the arrival of the seismic wave from the Indonesia, Papua New Guinea earthquake that occurred at 05:48:55 UTC.

The spectrum of IMC length when the rotator is moving is dominated by a huge and narrow line at 666 Hz, together with all the harmonics (fig 1). Such a narrow line is not present in the frequency noise at LN3 (red curve, see also the zoom in fig 2). There are bumps, excited during the experiment and present in the clean data: one of them is around the line. One can look also at the spectrogram (fig 3): when the rotator stops, the shake excites large structures (one quite close to the line, but not exactly coincident), that roll down quickly. This is not exactly what one would have expected if the narrow line were simply due to the mechanics of the dihedron. Anyway, a structure at that frequency is present. The Q is not to big, but in case it is the resonace of the dihedron, for sure it amplifies the effect of the noise injection.

Very interesting the shape of the time domain signal produced when the rotator is moving (fig 4): a perfect periodicity and a high non-linearity is visible also there. In order to better characterize that signal, I have divided the data (12 seconds) in 800 segments lasting 15 ms each, containing in such a way exactly 10 oscillations. Making the average of all the segments, the 10 oscillation are still perfectly visible. I show what happens if one does the same with the second derivative of the signal: a square wave appears (fig 5). The shape is clearly asymmetric:  1 ms up, 0.5 ms down, with a doubled amplitude with respect to the upper level. Everything seems too precise to be 'natural'. Hopefully the explanation of this shape can be found in the data sheet of the rotator.

The audio saturation threshold has been updated from 0.055 to 0.075 mW for the B1_PD1 and B1_PD2 photodiodes. The action was performed at 13h14m32 UTC.

Moreover we have updated the PD flag in the DMS to include a check of the saturation of the audio channels. DetMoni was restarted at 13h40m07 utc.

This morning, 2025-10-17, at 10H00-UTC there was a glitch in the BNS range . 

Performing the replay with 600s of data, without the logic related to the photodiode saturation,  with the B1_PD{1,2} channels it appeared that the glicthes were not present in the reprocessed channels  , even after the yesterday's tunings .

Looking in more details in the SDB2_Photodiodes Acl's code and in the SDB2_dbox_bench configuration, it appears that:

• at the DBOX level, the B1_PD{1,2}_{Audio,DC} channels are acquired at 100KHz with the suffix _raw but not stored in the DAQ 
• at the Acl level, the  B1_PD{1,2}_{Audio,DC} channels at 20KHz are computed, filtered and decimated and stored in the DAQ .
  • As consequence when the reprocessing is performed using B1_PD{1,2}_{Audio,DC}, its s done with the channels built by the SDB2_Photodiode . This exclude any possible issue with data delivered too late at the rtpc level .
• After further investigations, it appeared that the issue with the B1_DC_NULL_30_300_RMS are mainly due to  the "saturation" protection  of B1_PD{1,2}  channels (see this plot, zoom1 and zoom2 ) 
  • the saturation thresholds need to be adjusted

ITF found locked at LN3 in SCIENCE mode.

At 08:13 UTC ITF unlocked due to a glitch of SIB1. Relocked at 09:38 UTC after a realignment of PR (50 X-) and the usual cross-alignment in ACQUIRE_DRMI.
At 11:21 UTC, ITF unlocked again. Also Injection unlocked (TBC) Relocked at first attempt at 12:06 UTC.

If I get this correctly, the acceleration inferred from the calibrated signal is of the order of 0.02 m/s^2 (???: no units in the plot), so if we assume the whole SIB1 is moving, shaken by the rotator, a force of the order of F~100Kg * 0.02 m/s^2 = 2 N is applied by the rotator to the bench. I am no expert but this seems very large.

A different working hypothesis put forward in the past was that the rotation of the waveplate excites some resonance of the dihedron or its support, which is a much lighter mass. It could be important to discriminate between the two hypotheses in order to steer future upgrades. For instance, one could try to improve the mechanical isolation of the dihedron with respect to the SIB1, or to increase the damping of possible mechanical resonances of the dihedron, and then repeat the experiment.