Today we investigated the issue experienced yesterday with the ID of the WE.
We performed a measument in order to recompute the calibration for the accelerometeres. We found out that ACC_H1 was the one miscalibrated, but the correct compensator was the one used before the last update. One possibility is that the accelerometer switches between two states. The reason of this behaviour is not clear. Some investigation is needed.
In the meantime, we reloaded the older compensation filter, and we modified the DSP card adding one switch for the two compensators in order to have the possibility to move between one 'state' and the other.
This is a good reminder.
Figure 1. shows that SDB1 has changed orientation compared to where it was a few weeks ago, and during the recovery it has moved further away.
This morning I have disabled the floating set points (the bench learning and remembering on its own what are the set points), and put back the set points to the average values of the bench orientation in TX and TY from beginning of May.
We will see if helps, the interferometer locked on DRMI for 20 seconds, and the B1p quadrants galvo close during that time.
The work on newtonian calibrators, both on north end and west end, went on in the afternoon in parallel with the west arm lock recovery; the west arm has been realigned and relocked with the green lock too, during this work we noted a strange behavior of the WE suspension control with a slow oscillation that grow up, switching off the ID control it disappeared; I called Valerio to work on this problem, he didn't able to solve it postponing any other intervention to tomorrow morning, we continued to work with the WE controlled without the inertial damping; the work on newtonian calibrator stopped at around 18:15UTC and we continued with the lock recovery, we succeded to lock up to the DRMI but the lock was not good; we stopped here, I left the cavities locked on the infrared. Oncall events
Oncall eventsSUSP
(13-05-2025 15:15 - 13-05-2025 16:00) From remote
Status: Ended
Description: WE inertial damping not working
What we see looks similar to something that happened recently (#66258), and some interplay between SR and SDB1 alignment was the cause, I don't know if that is the case also now.
After the recovery of the West Arm and the end of Ncal operations, we started the recovery of the ITF without WE inertial damping.
The recovery proceeded smoothly until the lock of the DRMI, but we couldn't continue because the galvo loops would not stay closed. The DRMI lock itself is unstable, and does not last more than a few minutes, because the alignment loops are not working properly.
The beam on B1p camera looks reasonably well centered, so it might not be related to a bench misalignment.
It is not clear whether this is related to the absence of inertial damping on the WE, we contacted Diego and, as instructed, we leave the ITF as is.
The recovery will be continued tomorrow.
What we see looks similar to something that happened recently (#66258), and some interplay between SR and SDB1 alignment was the cause, I don't know if that is the case also now.
This is a good reminder.
Figure 1. shows that SDB1 has changed orientation compared to where it was a few weeks ago, and during the recovery it has moved further away.
This morning I have disabled the floating set points (the bench learning and remembering on its own what are the set points), and put back the set points to the average values of the bench orientation in TX and TY from beginning of May.
We will see if helps, the interferometer locked on DRMI for 20 seconds, and the B1p quadrants galvo close during that time.
Between 6 am and 10:30 am UTC, we did the following activities on West-End NCal Setup:
- we adjusted the microphone gains,
- we recabled the power supply of south and west position sensors,
- we prepared for the installation of WNN and WSN.
We resumed WNN and WSN installation at 5 pm UTC by fixing both rotors on the corresponding platines.
Installation to be continued.
Today we started the recovery of the interferometer by working on the West Arm.
We could see from the beginning two (one and its reflection?) white spots on the WE camera, which we could not remember (see Figure, WARM unlocked but flashing).
We looked for flashes on both B8 and, after misaligning NI, B4, without finding any, even with the "snail". We then checked the old position of WE MAR LCs and they were quite different with respect to the current ones. We recovered those and the snail could find flashes quite soon, first on the camera markers signals, then on the photodiodes.
We could easily relock the West Arm (no change in demodulation phase or anything), but the locks did not last much, and we could see a very big oscillation on the MAR correction at about 80 mHz. Fabio pointed out that the Inertial Damping of the WE was not working as expected, probably since quite a while.
Valerio had a look at the ID loop, but in the end we kept it open (Position Control) and somebody will look at it tomorrow morning.
With the West Arm locked we checked the drift control on SWEB and slowly discharged its memory; also, we re-engaged the BPC loop for the West ALS. Then also the ALS could be locked without issue.
We also did the initial tuning of FmodErr, which was quite off, without problems.
At this point the automation was put back together so that later in the evening we could do some tests of the full lock acquisition. The reverted changes are the same ones of last time, with a few minor differences:
The CEB_DBOX_LNFS and CEB_DBOX_ALS Timing error flags was red since 2025-05-12-15h33m38-UTC .
These 2 DBOXes were reconfigured between 07h34m48-UTC and 07h51m25-UTC (see the plot)
The ALS_CEB servers have been stopped and restarted and the BPC_{N,W}ARM_PZT_{x,y} have been restored to theirs previous values.
During the maintenance period, the Timing error flag became red for the EQB1_DBOX_03 DBox ,. It has been reconfigured at 12h49m19-UTC as well the EQB1_DEMOD_03 Demodulation mezzaine, the EQB1_HD_CC_DAC FAST DAC , and the SQZ_CC_DAC FAST DAC one to ensure the FAST DAC synchronization .
After these operations , the DMS Timing and Fast_DAC flags are green
ITF found in Upgrading mode, Down state with north cavity locked.
At 6:00 UTC ITF in Maintenance mode, below the list of the activities communicated in control room:
All the maintenance activities concluded at 10:51 UTC.
At around 12:30 UTC the commissioning crew started to realign the WEST arm; the NCal activity will continue in the afternoon.
Vacuum
The WE large valve has been opened in the morning.
ISC
At 7:29 UTC the WI Etalon loop has been re-enabled.
Suspensions / SBE
Due to the CAL activities at NE the NE ID loop and SNEB loop were opened by the guardians; I left them opened.
SWEB loop closed at 12:45 UTC.
WE mir z damper closed at 12:46 UTC.
Other
Grass cutting around the tunnel
After a preliminary investigation by Clement, the data looked good, so we decided to try and get more data with a longer scan time.
We tried yesterday, however, it was too windy and we couldn't keep ALS engaged in reallocation mode. Locking on IR was not a problem.
This morning, the wind seems less, so I opportunistically re-ran the script with longer parameters to get better data. Starting at 2025-05-13, 07:41:40, I re-ran the script with:
- offset = 0.7
- vFSR=2.0
- timescan=120
These parameters were determined emperically, I think it is about 1.5 FSR. The script is running now, I'll update this post when complete.
At scan 5, we recieved notice that staff would be entering the North and West end buildings to carry out essential work. I left the scan running, around scan 8, the IFO lost lock and couldn't relock - even on IR.
ITF found in DOWN in UPGRADING mode.
Here is the list of activities carried out during the shift:
Today I installed the Optris 640i thermocamera on the ZnSe WE viewport (the same it was used to take pictures with the FLIR thermocamera reported on LogE 66722) with the remote support of E. Kraja. The thermocamera is connected to a laptop left on the WE tower platform and it is possible to access this computer by remote through TeamViewer. Figs. 1 to 3 show the thermocamera installed. On Fig. 4 we can see the application's screen with the thermocamera image. To be noted that the payload is not central w.r.t the thermocamera image to avoid some burned pixels (white lines on Fig. 4) on the surface of the mirror. The idea is to perform measurements during the recovery of the ITF in order to continue the analysis the point absorbers (LogE 66696) after the intervention in tower by E. Majorana and H. Vocca (LogE 66729).
One issue with the thermocamera is that after a few minutes running, the image starts to get noisy until is not possible to see anything (Fig. 5), but it goes back to normal when the computer application is restarted. Therefore, it requires human action and control during the data taking.
Figure 1. Looking back in the data, the 50Hz harmonics I have seen in the DR coil after the WE replacement have disappeared on Apr 22, on the first day of the recovery a few hours after the gate valve was open. One the damper correction are started, a minute later the 150Hz, and higher odd harmonics of the 50Hz disappear. The 50Hz is masked by the high 50Hz correction of the damper correction.
Hence the issue with the 50Hz harmonics on the DR coil was never present once we had the interferometer, and this was a false leed for the questions of high magnetic coupling and 3Hz comb.
ITF found in DOWN in UPGRADING mode.
At 08:19 UTC Nardecchia, from remote, turned on WE RH.
Vacuum
07:25 UTC - WI valve opened (Pasqualetti)
Today, I closed the B2 quadrants shutters from 10h03m56 to 10h09m08 utc to measure the dark noise of these quadrants.
Figure 1. The increase in 50Hz harmonics on the WE DR coil started on Saturday April 12 in the morning around 07:16 UTC (9.16am LT).
Which after the final electrical checks: https://logbook.virgo-gw.eu/virgo/?r=66564
And at the beginning of the work of payload integration that happened on April 12, https://logbook.virgo-gw.eu/virgo/?r=66569
It could be that something got damaged during the integration, or could be a simple as the problem appeared only once the cables got connected to the right places. It would be nice to check the times when various tests were done last week, to see if the problem temporarily disappears during some of the test, when some of the cables are not connected.
Figure 2 During the check of injecting a square wave into the RH the 50Hz harmonics in the the DR coil have disappeared https://logbook.virgo-gw.eu/virgo/?r=66745
Figure 3 shows that there have actually been several changes in the past few weeks. In purple now after the ring heater was turned on, the problem is no longer present. In blue is the night before injecting a square wave into the RH, there is 50Hz and an extra low frequency noise, but no 150Hz (the extra 50Hz and noise are actualy present also on ather coils. And in red is the initial problems seen after installing WE in april.
As agreed with Diego, I switched on the WE RH at 8.0 W (17.25 V) at 08:19 UTC.
As reported in Michal's entry 66635, after the replacement of the WE mirror, a new bump around 178 Hz, among others, appeared in the sensitivity (see figure 1, blue curve).
It is worth reporting that, in the past, we already observed a similar structure around 178 Hz, excited both magnetically (by the near-field magnetic injections around SDB1, see https://tds.virgo-gw.eu/?r=23206, slide 10) and mechanically (see #61518).
Looking at the Hrec spectrogram, it seems that this noise structure slightly drifted during the first locks (see figures 1, 2, 3).
To be noticed, in the same period (April 24-29), some DoFs of the SDB1 bench seemed to drift too (see Figure 4), but it is hard to determine if there is a correlation.
1) I would say it is the “best” at least as far as NE's RH is concerned, otherwise we would have already had the same problem (we know that all power supplies of that kind create that interference). Maybe the same rationale can be applied to PR.
Other RHs should be “disabled” by default (if I am not mistaken), so they have no noise coming from the power supply.
2) Another power supply could be found that doesn't have that “comb”, but don't forget that the level of the comb is just above the level of the noise floor of the power supply itself, so you have to be careful to find a power supply that is also quiet enough.
Or you design a new one from scratch
ITF found in Upgrading mode, Down state with north cavity locked.
ITF found in Upgrading mode, Down state with north cavity locked.
The west end tower evacuation is still going on, others activities carried out during the day:
Today we tried injecting a signal on WE Ring-Heater, reading the signal on the mirror coils (SC_WE_MIR_VOUT).
This is a very relative test: it may give us indications of some asymmetry in the four coils, following the analysis made by Michal in entry #66736.
To perform this test, we used a balanced BF generator isolated from ground, so that it mimicked fairly well the behavior of the power supplies used to power the RH.
As a test signal we generated a 3 Hz square wave, with an amplitude of 1Vrms (and then 5 Vrms), but since the RH is a non-standard load (30 Ohm), the reading on the generator display is relative.
At first we sent the signal by leaving the connections of the two RH windings as we found them, and we were able to see the signal in the voltage monitors of the four coils, more or less equal between them.
At the end of the tests, we reconnected everything as at the beginning.
Preliminary conclusion:
1) we did not notice any obvious differences in the signal seen by the four coils under any condition of the test.
2) It is likely that, although care was taken at the beginning of the installation of the RH (years ago) to respect a certain phase necessary for noise reduction, some unintentional inversions occurred during the last interventions on the WE mirror (to be verified).
3) it seems the reduction of the noise is effective only up to 100 Hz , maybe for some "capacitive" asimmetries (just a guess) - we stress again on this relative measurement: the good one will be having again the ITF locked, and looking at Hrec.
It is worth noting that on the air side it is very easy to put things back in place with the good phase, because you only need to reverse the connection of one of the two windings on the power supply connector.
Now, at the first ITF relock, we should see the 3 Hz comb again; then reversing one of the two connections should reduce it.
Once the sensitivity of the ITF has been recovered, it would be worthwhile to do a new test of injecting a known signal into the RH while the ITF is locked, so as to understand the real coupling with Hrec.