This morning, NEN and NWN frequencies have been swaped with the WEN and WWN frequencies. We have now:
- NEN_FREQ=18.18
- NWN_FREQ=18.20
- WEN_FREQ=18.02
- WWN_FREQ=18.04
The ITF was found in Troubleshooting and in locked arms with the commissioning team working on the ITF recovery, see Status of the ITF.
In parallel, Marco worked on FDS FD recovery; all the activities were stopped at 20:20 UTC.
At 19:54 UTC the ITF achieved a stable lock in CARM_NULL_1F; the ITF will be left in that state for the night.
This morning there were few problems with the lock acquisition regarding a couple of critical points:
1. The hand-off of PRCL from 3f to 1f. It seemed that when the error signal was making the hand-off, the demodulation phase was stil changing causing unlocks. We checked in the logs and indeed that phase change was sent with a ramp of 20s. We hardcoded a ramp of 1 second in the Automation.
2. The engagement of CARM slow loop is very agressive and it causes also some unlocks. We commented these lines and we managed to stay locked in CARM NULL 1f, even with the good phases of the DIFFp (and we fine tuned the phase of PRCL in 1f). We are currently working in the engagement of this loop, which seems to be our stopper.
A look at long averaged coherence between the ground current monitor and hrec_gated, up to 250Hz - Figures 1 and 2:
- some coherence at structures between 46 and 48Hz seen in Hrec, which should correspond to crossbar modes of TM payloads
- narrow lines coherence: 46.0 Hz (frequency of the inverter driving the supply fan of CEB hall HVAC), 80.5, 62.5
The two checked times are: 30 Nov 2025 9:00, 6 hrs (current probe at ground rod on the top of electrical panel in front of INJ cryotrap); 22 Feb 2026 2:00, 6 hrs (current probe at ground cable of the BS base).
Similar coherences are found also with magnetometers - Figures 3 and 4:
- the magnetometer on WI tower base sees enhanced coherence at 48.0 and 48.65 Hz: the latter is a known resonance of WI crossbar.
ITF found in LOCKED_ARMS_IR State and TROUBLESHOOTING Mode.
All times are UTC.
07:04 Unlock at REDUCING_CARM_OFFSET_1_OF_3.
07:14, 07:24, 07:36, 07:46, 08:11 Unlock at LOCKING_CARM_NULL_1F due to saturation of SDB2_B1p_PD2 causing Fast Shutter to close.
08:00 FDS FC recovery activity started (De Laurentis, Vardaro).
ITF left in LOCKED_ARMS_IR State and TROUBLESHOOTING Mode with ITF recovery still ongoing.
Yesterday, after the weekly maintenance, we continued working on the full recovery of the ITF.
During the night before, the ITF has been put in Locked arm due to troubles in locking even in carm null. The issue was due to the demodulation phase of the Diffp (mainly the TX DoF ). However, as the previous day, we could survive during the transient only using a "bad" phase (slightly different from two days ago), with most of the signal on the Q quadrature.
Moreover, looking at the phase camera, we saw some jumps of the sidebands, similarly to what we had when the PR Chrocc was mistuned. Looking at the usual PR Chrocc observarbles, both current and heater temperature probes were higher then the values before power outage (see fig. 1). Thus, we decided to perform a small step down (~0.05V ), at 14h12UTC.
During the whole afternoon, we could stably reach CARM_null_1f, but the step on chrocc didn't show any macroscopic improvement to the cleanness of the locking signals.
We tried few times to continue to LN2, with success. However, as the day before, the locks in LN2 lasted only ~10 minutes, with the interferometer unlocking without a clear reason (some of the longitudinal DoF drifting away).
Finally, the lock acquisition continued to have problem during the night. The work on it will continue also today.
ITF found in CARM_NULL_1F in Troubleshooting mode and SQZ team at work on FDS FC recovery (Vardaro, Zendri).
Activity concluded at 20:00 UTC with ITF locked at LN2.
Unfortunately, after the activity, the ITF was very unstable. It unlocked sistematically at LOCKING_CARM_NULL_1F.
After several unsuccessfully locking attempts, according with commissioners, I relocked at LOCKED_ARMS_IR.
SBE
16:00 UTC - SPRB vertical position recovered via stepping motors.
Today while locked in LN2 we found the MAR actuation of SR and PR payloads slighlty mistuned (former values pre-shutdown were not saved). We measured the coupling and corrected the driving.
DSP cards properly saved afterwards.
Today, on site, we
-Switched on the output of the NCal power supply (which were switched off due to the large general power supply failure of Feb 28th).
-Replaced the NWN controller. The removed one is suspect of being wrongly parametrised (slow frequency ramp up observed on 14 Oct 2025) and we will check this at IPHC.
With the replaced controller, the NWN frequency ramps up much faster than with the old one (see picture). The NWN phase control also looks better and needs to be tuned further which we will probably do next Tuesday during maintenance.
Time details of the changes:
10:30 utc in WEB:
Power on WEB NCal power and restart all rotors at default frequencies
11:00 utc in NEB:
Successfully test restart of NEB NCals before changing NWN motor controller
Replace NWN controller by new one (we bring back the replaced one to IPHC)
Restart all NEB NCal power supply and restart all NEB NCals with success.
Then we placed the magnetometers about 3 meters away from our ENV_WE_MAGxxx to perform a long measurement in coincidence. We decided to leave them there till next Friday.
ITF found in Troubleshooting mode with arms locked.
At 7:00 UTC ITF in Maintenance mode, below the list of the activties communicated in control room:
- cleaning of the experimental area;
- replacement of WI pump;
- tasks performed by the operator:
- lock and scan of the OMC in single bounce;
- TCS chiller refill;
- TCS thermal camera refernce;
- TCS power checks
| CH [W] | OUTER RING [W] | INNER RING [W] | |
| WI | 0.44 | 0.26 | 0.03 |
| NI | 0.61 | 0.59 | 0.09 |
At around 12:00 UTC we started to relcok the ITF; the commissioning team is now working at the lock recovery.
DET
SDB1 LC opened by the guardian; properly closed at 8:25 UTC.
The power emitted by the CO₂ WI auxiliary laser is shown in the attached figure.
As can be seen, the power emitted by the laser dropped to about half following the blackout.
Yesterday evening, taking advantage of Antonio’s intervention to replace the pump on the WI tower, I went to the TCS room to check whether a better operating point could be found by adjusting the duty cycle of the square wave (as we did in the past).
I tested different duty cycle settings and found that the duty cycle value that maximizes the power is 74%. However, the power level measured before the blackout could not be recovered.
This morning we replaced the failed pump on WI.
About the disturbances observed yesterday evening , the failed pump was offline but still connected to the controller (failure of the magnetic bearing controller, replaced just after the shutdown), we can't exclude noisy effects due to the faulty electronics.
The Imaging servers have been restarted with the release v17r0p7 aligned with the latest Fr(v8r49p0) and Fd(v8r65p1) librairies
Operations performed between 2026-03-10-06h17m33-UTC and 2026-03-10-06h20m37-UTC
After the WI pump failure, WI suspension is moving quite a lot, more with respect to the usual, fig.1.
I don't know if this is related to the failed locking attempts, tomorrow the problem will be better addressed.
ITF still in recovery - TROUBLESHOOTING at the beginning of the shift, which kept on going through the afternoon
Here the steps taken during the recovery and the edits to automation (Pinto, Spinicelli, Nardecchia, Was, Ruggi). LN2 was achieved three twice (18:30 UTC - vac pump failure, 19:57 UTC) lasting only a few minutes in both cases.
18:30 UTC Vacuum pump failure, fixed (Vacuum Team)
20:43 UTC ITF in LN2
20:56 UTC unlock, then unlock at LOCKING_CARM_NULL_1F (21:06 UTC), REDUCING_CARM_OFFSET_2_OF_3 (21:20 UTC), LOCKING_CARM_NULL_1F (21:30 UTC)
At the end of the shift, as instructed, I left the ITF in LOCKED_ARMS_IR - TROUBLESHOOTING, since it had not been possible to keep a stable lock in LN2 or to reach CARM_NULL_1F during the other locking attempts.
After the WI pump failure, WI suspension is moving quite a lot, more with respect to the usual, fig.1.
I don't know if this is related to the failed locking attempts, tomorrow the problem will be better addressed.
Today we continued the recovery of the ITF. We managed to reach LOW_NOISE 2.
At the beginning we noticed a lower power values on the 112 and 12 MHz sidebands, which was taken care of by Ilaria, acting on CH:
NI CH = 0.577W -> 0.708 W
WI CH = 0.324 W -> 0.416 W
At the end, we partially reversed the NI CH PWR increase: 0.708 W -> 0.630 W. (17.32 UTC finish of the actions).
After this, we had several issues in finding a good configuration of demodulation phases in order to get past the handoff from 3f to 1f in CARM null. In particular DIFFp was showing during the transient some misbehaviour (not entirely clear the rigth explanation), in which with good phases both I and Q (on TX DoF) were witnesses of some movement which ended up in a quick drift of the alignment and eventually the unlock. Curiously enough, the phase able to get past the initial transient was showing a ~45 deg mistuning. After several trials in which we noticed this behaviour, we implemented a double condition in the automation in which we use two phases, one 'wrong' for the transient and one good for the steady state (similarly to what we do for B4 56 MHz).
With this configuration we were able to lock the OMC, after a tuning of the OMC locking phases, and go up to LOWNOISE 2. However, after a while we unlocked due a to a WI tower pump failure. Antonio went to CEB to take care of it.
We leave the ITF locking on LOWNOISE2 for the night, to acquire some statistics on the acquisition.
N.B. the modification on the automation are the following:
ITF_LOCK.py: added lines 4681-82; 4685-86.
ITF_LOCK.ini, in the SHUTTER section: added the steady state DIFFp phases.
If the acquisition is robust enough, tomorrow we will proceed to fine tune all the several phases, if needed.
:)
Yesterday, after the weekly maintenance, we continued working on the full recovery of the ITF.
During the night before, the ITF has been put in Locked arm due to troubles in locking even in carm null. The issue was due to the demodulation phase of the Diffp (mainly the TX DoF ). However, as the previous day, we could survive during the transient only using a "bad" phase (slightly different from two days ago), with most of the signal on the Q quadrature.
Moreover, looking at the phase camera, we saw some jumps of the sidebands, similarly to what we had when the PR Chrocc was mistuned. Looking at the usual PR Chrocc observarbles, both current and heater temperature probes were higher then the values before power outage (see fig. 1). Thus, we decided to perform a small step down (~0.05V ), at 14h12UTC.
During the whole afternoon, we could stably reach CARM_null_1f, but the step on chrocc didn't show any macroscopic improvement to the cleanness of the locking signals.
We tried few times to continue to LN2, with success. However, as the day before, the locks in LN2 lasted only ~10 minutes, with the interferometer unlocking without a clear reason (some of the longitudinal DoF drifting away).
Finally, the lock acquisition continued to have problem during the night. The work on it will continue also today.
The failed pump will have to be replaced during the maintenance break
This morning we replaced the failed pump on WI.
About the disturbances observed yesterday evening , the failed pump was offline but still connected to the controller (failure of the magnetic bearing controller, replaced just after the shutdown), we can't exclude noisy effects due to the faulty electronics.
Also SC is aligned into the OPA. The heighest peak is 0.2% of TEM00
Profiting of M. D'Andrea's availability, around 10.15 UTC we went to check if anomalous Temperature and Humidity are the same on console. Since this is the case, we can exclude problems of transmission from console to data acquisition. A console shut-down-up apparently did not resolve the problem, and a serial port unplug - replug as well.
To increase the variety of weird functioning, Temperature returns to normal values and time evolution for a small amount of time at random time and for random periods. This could be due to transmission/encoding problems from sensors to console (to be investigated).
ITF found in LOCKED_ARMS_IR in TROUBLESHOOTING mode.
SIB2 position loop found opened. Properly closed.
ITF relocking up to CARM_NULL_3F after the usual cross-alignment in ACQUIRE_DRMI.
ITF phase tuning in progress up to CARM_NULL_1F.
This morning around 11 LT the Krakow POLGRAV team (Michal Ostrowski and Jerzy Kubitsz) installed two high sensitivity and low noise magnetometers near to the Virgo EXTternal magnetometer (ENV_EXT_MAG_N). The two magnetometers have been oriented along the geographic North, which in our location is approx +4 deg with respect to the magnetic North. Sensors are connectew with a local supply and DAQ synchronized with GPS.
It has been checked that the magnetometers are not saturating: their dynamic range is 19nT RMS, and the actual measured RMS is approx 5nT RMS.
The plan is to record 24 hrs of data and then move to another location. Before removing this installation some magnetic pulses will be injected locally in order to test the sensors synchronization.
ITF found in LOCKED_ARMS_IR State and TROUBLESHOOTING Mode.
ITF recovery ongoing after NI CO2 Laser RF Driver replacement (see #68263).
All times are UTC.
15:12 TCS Chillers Guardian restored with slightly higher flow (Menzione, Nardecchia).
19:07 Activity on ITF recovery concluded with ITF able to reach CARM_NULL_3F. ITF to be left in LOCKED_ARMS_IR (see #68841).
ITF left in LOCKED_ARMS_IR State and TROUBLESHOOTING Mode.
Fabio and Suzanne successfully replaced the RF driver of the NI CO2 laser (see Fig 1). To perform the intervention, the 220 V line was completely disabled from the electrical panel in the TCS room and re-enabled once the operation was completed.
The lasers (both CH and DAS) were switched on at 14:31 UTC, and the lost power was recovered (see Fig. 2).
[Nicola took care of re-engaging the Guardian system.]
A thermal image of the rings was acquired for a quick check, and the rings appear to have returned to their pre-blackout conditions (see Fig. 3).
After waiting for the strongest thermal transient to settle, and following a phase tuning performed by Manuel, the CITF locked and the sideband levels returned to values consistent with the pre-blackout conditions (see Fig. 3).
After that, Manuel and Diego started working on the lock acquisition (see entry 68841).
Today we continued with the recovery of the ITF.
From DRMI 1f we managed to reach carm null 3F, by tuning the demodulation phases of mostly 56 MHz and 169 MHz (MICH SRCL DARM and DIFFp), plus some less invasive tuning on PRAA and PRCL. We will continue the recovery next week.