ITF found in FMODERR_TUNED after recent unlock, lock acquisition in progress (Autoscience ON).

• Unlocked at 13:16UTC (pdf1) and again at 13:28UTC (pdf2) after reaching CARM_NULL_1F due to diverging SSFS correction.
• Relocked in LN3 at 14:21UTC, back in SCIENCE mode;  BNS Range ~49Mpc.
• Unlocked at 14:36UTC (pdf 3) again due to diverging SSFS correction; INJ on-call (Melo) notified.
• Automation stuck waiting for DET_MAIN in unknown state; fixed (as usual) by resetting node to INIT at 14:57UTC. Relocked at 15:30UTC.
• Unlocked at 16:01UTC (pdf 4) due to end TM (MIR or MAR) correction saturation possibly caused by high wind activity; speeds in excess of 50 km/h on site (as seen from DMS>WindActivity and MeteoStations red flags.)
• Relocked at 16:44UTC on first attempt despite above threshold wind speeds.
• Unlocked at 19:17UTC (pdf 5) due to multiple causes with yellow flag under DMS>SeaActivity>ENV_CEB_SEIS_V.
• Relocked at 20:04UTC on first attempt. Horizontal and Vertical QD1 B5 Galvo loops opened at 19:28 UTC, manually closed via VPM at 20:06UTC (fig. 1).

Guard tour (UTC)
18:06 -> 18:43
20:59 -> 

After the WEb UPS intervention, the Sc_WE glitch debugging channels have changed, meaning that the DSP codes are not exactly the same as before the UPS intervention

• before 2025-07-01-05h00m00s-UTC
• after 2025-07-01-16h00m00s-UTC
  • Sc_WE_txAA is not CAL_WE_MIR_Z_CORR delayed
  • Sc_WE_ErrPost is not CAL_WE_MIR_Z_NOISE delayed

At my arrival, locking acquisition was in progress. ITF had already experienced several unlocks

• 23:28UTC, back in Science

ALS
21:29UTC, I called the expert because the unlocks appeared to be due to low power in the WE ALS system
M. Gosselin realigned the green on WE. After that, D.Bersanetti tuned the thresholds in the automation.

QPD
22:28UTC I rearmed vBias of B1p_QD1 and B1p_QD2 

Guard Tours (UTC):
21:05 - 21:47
23:03 - 23:42
01:03 - 01:43
03:11 - 03:48

Afterwards, I had to change the thresholds in the automation:

lock_thr = 0.5
dc_trig_refl_hi = -0.5
dc_trig_refl_lo = -0.6

plot attached 

The following report has been submitted to the On-call interface.

On-call events -> Interferometer Sensing & Control

Title: ALS warm realignement

Author(s): gosselin

Details:

I have been called by Beatrice because there were some trouble to go through the lock acquisition. The West arm was unlocking.
I checked and see that there was some power missing on the signal WARM_BEAT_DC.
I first try to realign the beam in transmission, ie into the fiber on EIB but the improvement was very little.
I then reliagned the beam in the arm with the galvo on SWEB. I slightly changed the set points of the tilt X and tilt Y and recover a much greater signal in transmission.
The lock acquisition went through the green lock but eventually unlocked at CARM NULL 1F.

* Note that any files attached to this report are available in the On-call interface.

The activity started in the WE building with the NCal power supplies switched off at 6:21 UTC. Then, the WEN (box 08) and WWN NCal have been rotated by 180% to study the mirror/NCal vertical offset (see pictures). The power supplies have been turned back on at 6:39 UTC to check that the NCals were still working as usual. They have been switched off at 6:50 UTC.

Then the activity switched to the NE building.

The rotor R-10 which was removed on Jun 6 (see logbook entry) has been put back as NNN (see picture) after fixing it at IPHC. The rotor R-17 has been reinstalling on the west setup as NWN (reusing some of the old NEF rotor) and the cabling completed (see picture of the two DAQboxes used by the NCals). The East setup has also been completed with the NEN installed on the left part (see picture). The position of the suspended support has been adjusted to better than 0.1 mm.

The configurations of the SNEB_dbox_rack and NEB_NCal processes have been updated (and restarted) to support these new NCals. To get NEB_NCal working properly, the TCS HWS DAC channel (which is currently unused) has been commented out.

Finally, the new NWN and NEN rotors have been successfully tested for a short time around 16:43 UTC. They have been left off.

ITF found DOWN in MAINTENANCE mode. Recovery of WE concluded at 16:00 UTC. At 15:33 UTC I set COMMISSIONING mode.
Ncal activity at WE performed by Mours and Masserot up to 17:00 UTC.
PREPARE_SCIENCE set at 16:52 UTC.
ITF relocket at LN3 at 17:40 UTC. SCIENCE mode set. ATOSCIENCE engaged.

At 19:39 UTC ITF unlocked due to an unlock of IMC. DET Metatron node stuck in unknown state. Properly recovered. RFC took 10 minutes to relock. Bersanetti tuned again the gain of W ALS. ITF Relocking in progress after a realignment of North cavity and the usual cross-alignment in ACQUIRE_DRMI.

Guard tour (UTC):
17:55 - 18:35

Electricity
(01-07-2025 14:30 - 01-07-2025 15:00) On site
Status: Ended
Description: UPS_CB failure
Actions undertaken: Triggered by the DMS red flag of UPS_CB, that was in failure, I tried in vain to contact electricity responsible (OnCall not present in the schedule). So I contacted Andreazzoli to let him check the system. Upon he came on site, we realized that the UPS3 of CB was restored by someone else (who didn't communicated anything to the control room).

WE PCal switched on by Suzanne around 13h UTC

LASER

1435409988  01 Jul 2025 12:59:30 UTC  (01 Jul 2025 14:59:30 CEST) SWITCH ON

1435410007  01 Jul 2025 12:59:49 UTC  (01 Jul 2025 14:59:49 CEST) KEY ON

1435410084  01 Jul 2025 13:01:06 UTC  (01 Jul 2025 15:01:06 CEST) PUMP ON

1435410113  01 Jul 2025 13:01:35 UTC  (01 Jul 2025 15:01:35 CEST) REMOTE ENABLED

POWER SUPPLY

1435410166  01 Jul 2025 13:02:28 UTC  (01 Jul 2025 15:02:28 CEST) POWER SUPPLY ON

Then, at 13:08:00 UTC : enabled the laser from VPM and the control loops.

We then saw that there was extra-noise (see figure), and we realized that the LED flashing the IRIG-B in front of Tx_PD1 was enabled (red flag in the DMS, thanks Nicola for warning).  SInce WE_Tx_PD1 is in loop, the noise was seen on Tx_PD2 and the Rx sphere instead.

At 16h28UTC, we disabled the LED: PCal noise level back to normal.

The following report has been submitted to the On-call interface.

On-call events -> Electricity

Title: UPS3 problem

Author(s): andreazzoli

Details:

UPS3 power supply problem. Batteries in discharge from 14:20 to 14:40 ( UTC) . Arriving in site i confronted the operator.
The problem had been solved. Monitored ups operation by DMS. Batteries correctly charging at 54% (17:42 local time) and increasing

* Note that any files attached to this report are available in the On-call interface.

switch off

•  06:50 UTC open PSTAB and BPC at WEB
•  06:50 UTC switch-off 2W ampli
•  06:51 UTC Switch off PPLN crystal (temperature 117.63C)
•  06:52 UTC SOA current: from 69.5mA lowered to 0 and switch off (current unplugged at 06h53)
•  06:58 UTC unplug the corrections to the RF amplifier of the AOM (PSTAB cable 8 and VCO cable 6 unplugged)
•  07:03 UTC VCO alim L1.3 off 
•  07:10 UTC switch off the alim of the RF amplifier + photodiodes box ALS DCG L1 and 07:11 UTC L2+L3
•  07:13 UTC power supply alim L1, L2+L3 unplugged

switch on

• 13:10 UTC power supplies alim L1, L2+L3
• 13:10 UTC switch on the alim of the RF amplifier + photodiodes box ALS DCG L1, L2, L3 
• 13:11 UTC VCO alim L1.3
• 13:11 UTC plug the corrections to the RF amplifier of the AOM (PSTAB in can 8 e VCO in  can 6 )
• 13:12 UTC power supplies SOA and crystal without switch on 
• 13:12 UTC PPLN crystal oven and SOA switched on (current increased to 69.5mA)
• 13:13 UTC 2W amplifier on 
• 13:24 UTC PSTAB and BPC closed 

Relock of the green at around 15:05 UTC (we slightly changed the BPC WEB  tilt x setpoint from -0.001 to -0.135 to improve the alignment and tuned the thresholds in ALS_WARM.ini).

After some unlocks possibly due to the IMC cavity during the last few days, we decided to check the status of the IMC control and working point.

After experiencing some unlocks, possibly due to the IMC cavity, over the past few days, we decided to check the status of the IMC control and working point.

We started by measuring the IMC OLTF.  Compared to what we had few month ago, we measured a higher phase margin (~30⁰) but with a lower UGF (~100 kHz).

We then decided to increase the gain (-1 dB on the attenuator) in order to raise the UGF, and we eventually obtained ~125 kHz with a phase margin of ~23⁰ (see fig. 1).

We then checked the angular working point. Only minor corrections were needed, except for FF_V (+0.4 → +0.58). The new values have been applied and stored in the ACL.

Today, around 14h30 UTC, I have restarted on olserver117 (16 CPU) the process MdDQ which was running on olserver119 (4 CPU).
MdDQ starts each day at 6h30 the full reprocessing of the VIM plots of the previous day, so that they all start at 00h00 UTC (for the time plots) or are done at the max BNS range of the day (for the FFT plots). But, the number of scripts used in VIM has increased and this reprocessing was taking 100% of the 4 CPU and all the RAM of the olserver119 machine.
(see http://ganglia.virgo.infn.it/?r=month&cs=&ce=&m=load_one&c=New+Olservers&h=olserver119.virgo.infn.it)
and many scripts were killed during this daily reprocessing (as can be seen in /opt/w3/vim/util/reprocess_log.txt).
Thus, I have restarted MdDQ on olserver117, which has 16 CPU and much more memory. We will see if it solves the problem.

* PRELIMINARY *

Taking advantage of the UPS work at WEB (mains shutdown), we can compare the seismic, acoustic and magnetic noise levels of WEB (in purple all OFF, in blue standard conditions).

There is a huge difference in the triaxial magnetic fluxgate near the vacuum tank (ENV_WE_MAG_x), it seems that we have reached the sensor noise floor. To be further investigated.

Another interesting difference is the ALS microphone noise level (ENV_WE_ALS_MIC) for frequencies above 2 kHz.

HVAC was left always ON

* MORE DETAILED ANALYSIS WILL FOLLOW *

This morning around 12:40 UTC I realigned the MZI in SQZ box to recover some contrast, see picture.

I took the chance to check the optical power of the green beams:

- before MZI: 49 mW

- SQZ box output for FC: 32 mW

- after AOM first pass: 19 mW

- after AOM second pass: 12 mW

At  07:47:17 UTC, both ETM ring heaters were switched off.

At 07.59:44 UTC, Maria switched off the WE RH power supply.

========

At  12:38:09 UTC, WE RH was switched ON.

At  12:38:59 UTC, NE RH was switched ON.

The shift was dedicated to maintenance started at 6:00UTC, the main work of this morning was the replacement of WE UPS device; the switch off operations started at about 6:15UTC and they completed at around 8:00UTC; the UPS replacement completed at around 11:35UTC (#67178, #67178, #67182, #67181), now the equipments switch on restarted and it is ongoing; here a list of others activity reported to the control room:

- standard vacuum refill from 6:00UTC to 10:00UTC (VAC Team);
- cleaning of central building (Ciardelli with external firm: from 6:00UTC to 10:00UTC);
- DET: OMC Lock in single bounce configuration from 7:42UTC to 7:48UTC;
          OMC scan in single bounce configuration from 7:52UTC to 8:13UTC;
- TCS: chiller check and refill (Ciardelli, 9:25UTC);
          thermal camera reference 8:30UTC;
          power checks 9:35UTC:
                 CH        IN        OUT
          WI   0.305   0.042   0.499
          NI    0.665   0.025   0.289

west end recovering ongoing.

SBE
SWEB control loops opened at 6:06UTC.

SUSP
WE suspension control loops opened at 6:10UTC.

This morning, Suzanne has switch off the PCal laser driver and Pd power supply before the WE UPS intervention:

LASER:

1435389674  01 Jul 2025 07:20:56 UTC  (01 Jul 2025 09:20:56 CEST) PUMP OFF

1435389702  01 Jul 2025 07:21:24 UTC  (01 Jul 2025 09:21:24 CEST) KEY OFF

1435389717  01 Jul 2025 07:21:39 UTC  (01 Jul 2025 09:21:39 CEST) SWITCH OFF

POWER SUPPLY:

1435390544  01 Jul 2025 07:35:26 UTC  (01 Jul 2025 09:35:26 CEST) POWER SUPPLY OUTPUT OFF

1435390579  01 Jul 2025 07:36:01 UTC  (01 Jul 2025 09:36:01 CEST) POWER SUPPLY SWITCH OFF

Yesterday evening injection had some troubles to keep the lock. 
We suspected that it might come from the fact that the beam is quite on the edge of the FF QPD of the BPC (TY). 
We moved the PR by 40 um in vertical to recover a healthiest working point on the quadrant (see attached plot).
The beam was still center on the ITM.

Here is the sequence of switch off actions performed on SWEB and WE device in order to prepare the shutdown :

2025-07-01-06h06m14-UTC    info fabiog       'Bench loop:Open Angular Loop [-1:Disable,1:Enable=-1]' sent to SWEB_LC
2025-07-01-06h06m18-UTC    info fabiog       'Bench:Open Position Loop [-1:Disable,1:Enable=-1]' sent to SWEB_SBE

2025-07-01-07h16m11-UTC    info gouaty       Command 'B8_QD1:close shutter' executed
2025-07-01-07h16m13-UTC    info gouaty       Command 'B8_QD2:close shutter' executed
2025-07-01-07h16m23-UTC    info gouaty       Command 'B8_QD1:disable vbias' executed
2025-07-01-07h16m24-UTC    info gouaty       Command 'B8_QD2:disable vbias' executed

2025-07-01-07h17m56-UTC    info gouaty       Power camera SWEB_B8_Cam1 off from mezzanine SWEB_MezzCAM
2025-07-01-07h17m57-UTC    info gouaty       Power camera SWEB_B8_Cam2 off from mezzanine SWEB_MezzCAM
2025-07-01-07h17m59-UTC    info gouaty       Power camera SWEB off from mezzanine SWEB_MezzCAM
2025-07-01-07h18m00-UTC    info gouaty       Power camera SWEB_ElectronicBox off from mezzanine SWEB_MezzCAM
2025-07-01-07h18m02-UTC    info gouaty       Close shutter of camera SWEB_B8_Cam1 from mezzanine SWEB_MezzCAM
2025-07-01-07h18m04-UTC    info gouaty       Close shutter of camera SWEB_B8_Cam2 from mezzanine SWEB_MezzCAM
2025-07-01-07h18m38-UTC    info gouaty       Close shutter of photodiode SWEB_B8_PD1 from mezzanine SWEB_MezzPD_B8
2025-07-01-07h18m40-UTC    info gouaty       Power down photodiode SWEB_B8_PD1 of mezzanine SWEB_MezzPD_B8
2025-07-01-07h19m23-UTC    info gouaty       'PSD on/off [-1:Disable,1:Enable=-1]' sent to SWEB_PSD_sensing

2025-07-01-07h27m33-UTC    info gouaty       Disable output of PowerUnit13
2025-07-01-07h27m41-UTC    info gouaty       Disable output of PowerUnit12
2025-07-01-07h27m46-UTC    info gouaty       Disable output of PowerUnit19

PowerUnit13 switch off manually at 7h29 utc

PowerUnit20 switch off manually at 7h30 utc

PowerUnit19 switch off manually at 7h31 utc

SBE crate switched off at 7h37.

DAQ box 62 (SBE) switch off at 7h38 utc.

LVDT box (Nikhef voice coil control system) switch off at 7h38m30

DAQ box 43 (LC) switch off at 7h39 utc.

DAQ box 175 (NCal) switch off at 7h41

DAQ box 121 (TCS) switch off at 7h41m30

DAQ box 49 (PCAL) switch off at 7h42

DAQ box 50 (ALS) switch off at 7h44

SBE stepping motors and power electronic switch off at 7h45m30

The WRnode was powered  ON last week, 2025-06-24 but after this , it was impossible to reconfigure the DBox_SN173 using the timing signal porvided by the WRNode

This morning , the WRnode was reseted, thank the reset buttom . After this operation the  IRIGB was available at the DBox level and the DBox_SN173 was fully reconfigured .

The WR Timimg monitoring signals were restored in the TIMING_moni server . 

Operations completed at 2025-07-01-06h38m06-UTC

ITF found in LOW_NOISE_3 and in Science Mode. 
The ITF unlocked at 2:10 UTC and, after a period of instability of the Injection, it was possible to start the locking acquisition at around 2:22 UTC. After a series of attempts, the ITF was back in Science Mode from 3:25 UTC.
It unlocked again at 4:02 UTC, back in Science from 5:00 UTC.

Guard Tour (UTC)
20:55 - 21:38
23:18 - 23:56
1:18 - 1:57 
3:16 - 3:54

At my arrival, locking acquisition was in progress.

• 13:17UTC, ITF in Science
• 15:40UTC, Calibration mode to start the planned activity
  • 15:41UTC, ISC longitudinal noise injections (inject_lsc.py)
  • 15:56UTC, ITF unlocked, so I started with the other measurements planned with ITF unlocked
  • 15:58UTC, Measurement of the TF between CAL and Sc channels, and of PCal sensing delays
  • 16:19UTC, Measurement of actuators response for PR and BS mirrors
  • 16:53UTC, calibration failed due to an unlock of the injection
  • 16:57UTC, Troubleshooting mode to allow M. Gosselin and P. Spinicelli to realign the BPC
  • 18:02UTC, BPC alignment concluded and locking acquisition started
  • 18:48UTC, Troubleshooting mode to allow D.Bersanetti to restore the locking acquisition
  • 19:51UTC, back to Science

ISC On-Call
18:30UTC, ITF unlocked five times at Acquire DRMI step or before (first when I disaligned the PR to align the CITF, see 1st plot), so I called D.Bersanetti to check the status

VPM
16:49UTC, "FmRds_II" process restarted after an unexpected loss of connection

Guard Tours (UTC):
17:50 - 18:30
20:55 - 

Below the list of WE DSP glitches found  without unlock of the ITF

• 2025-06-28-16h33m18s-UTC
• 2025-06-28-22h18m46s-UTC
• 2025-06-29-20h26m56-UTC: all the channels filled with zero without any warning or error message reported by ISC_Fb
• 2025-06-29-23h18m54s-UTC
• 2025-06-30-11h33m10s-UTC
• 2025-06-30-15h46m47s-UTC