Reports 1-1 of 1 Clear search Modify search
Detector Characterisation (Glitches)
mwas - 8:33 Saturday 27 April 2024 (64114) Print this report
New numerous glitches starting last night

Figure 1. From about 20:00 UTC yesterday there are many glitches in h(t) around 30Hz. The glitches between 18:00-19:00 UTC are normal, these are due to injections to measure coupling to h(t) of various of degree of freedom. What happens starting from ~20:00 UTC is in science mode, and needs understanding.

Figure 2 The SNR of these glitches has grown from ~15 to ~30 over 1 hour, and has been relatively steady since then. There were several unlocks since the problem started and the glitches have continued at the same SNR after the relocks.

Images attached to this report
Comments to this report:
narnaud - 10:04 Sunday 28 April 2024 (64119) Print this report

UPV + VetoPerf point to the following channels (without the _0 which is added in UPV to label the corresponding veto):

V0 → V1:ENV_WI_MAG_W_0, vetoed clusters: 454 (80.927 %)
V1 → V1:ENV_WI_MAG_N_0, vetoed clusters: 452 (80.570 %)
V2 → V1:ENV_WI_MAG_V_0, vetoed clusters: 452 (80.570 %)
V3 → V1:TCS_WI_CO2_ISSIN_0, vetoed clusters: 172 (30.660 %)
V4 → V1:ENV_CEB_ELECTRIC_0, vetoed clusters: 121 (21.569 %)

See https://scientists.virgo-gw.eu/DataAnalysis/DetCharDev/users/narnaud/UPV/20240428_glitches/V1:Hrec_hoft_16384Hz/perf/vp.html#V0 -> Click here to expand -> Time-frequency trigger distribution:    before/after veto.

In attachment the Omicron plots of h(t) and V1:ENV_WI_MAG_W, for comparison.

 

 

Images attached to this comment
mwas - 13:03 Sunday 28 April 2024 (64120) Print this report

Nice find. Figure 1 and 2 made using omicron-plot show that these glitches in the WI MAG started at the same time as the glitches in h(t) around 20:00 UTC, and that they have not been happening the day before, and have been happening since then. There are magnetic glitches between 150Hz and 200Hz that have stopped a few hours before the glitches at ~30Hz started. The two might be related.

Figure 3 shows that this glitches are not visible on the NI MAG. So it is likely that the issue is close to the WI.

Next steps should be:

  • Use the veto based on the WI MAG channel between 10Hz and 40Hz for GW analysis
  • Try to find the source of the WI MAG glitches. Is there a piece of hardware that is starting to fail near WI
Images attached to this comment
direnzo - 23:33 Sunday 28 April 2024 (64124) Print this report

These new glitches seem to originate from some noise source activating for ~2 minutes and off for ~4, similar to a square wave.

First of all, I identified the new glitch family selecting the omicron triggers with peak frequency lower than 40 Hz and SNR between 15 and 60. Then, plotting the temporal distances in the occurrences of consecutive glitches (similarly to what has been done for the 25-minute glitches), I noticed that these alternate between ~115 seconds and 250 or 225 seconds in the examined time interval: figure 1.

Figure 2 and 3 show the spectrograms of two consecutive glitches in hrec, and their whiten time series. The latter show opposite behaviours, with a spike up in one glitch, followed by a spike down in the next one.

Similar plots for the magnetometers at the West Input, show a step-like behaviour: figure 4 and 5.

The (non-whitened) time series of this magnetometer channel shows more clearly the square wave behaviour in the correspondence of each pair of glitches: figure 6.

The next task is finding what source is activating with a similar timing.

 

Images attached to this comment
swinkels - 8:34 Monday 29 April 2024 (64126) Print this report
The on-off pattern of the glitches sounds very similar to something we saw 4 years ago. This was eventually traced to one of the power supplies that drives the heating belt for the etalon control, which periodically went into thermal overload and shut itself down. Plotting the same signals as then, it seems we indeed have the same problem again.
Images attached to this comment
fiori, tringali - 10:09 Monday 29 April 2024 (64128) Print this report

Glitches disappeared around 6:45 UTC. Has some action been taken? Indeed, square waves were seen also in the CEB UPS CURR T monitor, as for the old problem with WI heating belts that Bas pointed out.

Images attached to this comment
direnzo - 10:35 Monday 29 April 2024 (64127) Print this report

I have run a cross-correlation analysis using deltas for each glitch time, and all the _mean and _rms channels from the trend frame. I confirm what was highlighted by Bas in the previous comment:

1) Since April 26 20:00 UTC, the LSC_WI_HB_moni channel has started behaving oddly, with a sequence of square waves from zero to the set point. This is simultaneous with the appearance of the new glitches. The channel monitoring the electric potential, LSC_WI_HB_cmd_100Hz_FS, doesn't present similar behaviour. Figure 1 shows the glitchgram, with the new glitches showing up at the end of Apr 26, and the time series of the two channels monitoring the WI etalon.

2) The glitches in hrec are synchronous with the steps in the square wave visible in LSC_WI_HB_moni.

3) The correlation analysis with the _rms and the _mean trend channels has produced no other correlated channel with these glitches.

Images attached to this comment
mwas - 10:38 Monday 29 April 2024 (64129) Print this report

Glitches might have stopped because the WI etalon loop error signal has finally become negative. The loop is asking for cooling the WI tower, which means turning the heating belt off. This switch off didn't happen immediately, but with an on/off for 10 minutes, so it is most likely due to the loop itself and not from an intentional switch off.

Images attached to this comment
dattilo, cavalieri - 11:35 Tuesday 30 April 2024 (64139) Print this report
As a follow up of the recent investigations on the WI etalon problem, we made a check of the WI heating belt power supply.
At around 08:30 UTC we went in TCS room where the power supply is located, and also looking at its analog display, we saw that it was delivering intermittently, as indicated by the LSC_WI_HB_moni signal.
We replaced the power supply with a spare one of the same kind (Kert 420). During the replacement (from 08:34 to 08:40 UTC) the operator temporarily opened the WI etalon loop, as recommended by Maddalena.
Now the loop seems working regularly (attached fig).
Images attached to this comment
Search Help
×

Warning

×