ITF State: locked in the IR
Quick Summary: ISYS in standard mode; all suspension loops closed; all SBEs loops closed with the exception of SPRB (SPRB intervention not concluded yet); UPS2 NE in alarm; large timing error for SPRB DAQ Box.
No activity ongoing.
This afternoon we worked on SPRB bench to realign the telescope.
Unfortunately the realignment is taking more time than anticipated, thus we will continue tomorrow.
We leave the bench in air with temporary weights, and the minitower closed.
The main activity of this afternoon was the SPRB telescope tuning; the work started at 14:00UTC with Piernicola and Romain working on SPRB minitower opening and bench balancing; we took this opportunity to go on with the EIB/injection extra noise investigation (#57247, #57249) after that I relock the two cavities and at 17:00UTC I set the ITF in NI single bounce to allow the alignment work on SPRB, the work is still in progress...
Sub-system reportsOther
at 15:50UTC the SWEB position loop was opened by the guardian because of an earthquake, controls properly closed.
After having revover a normal behavior for EIB, we checked the signals of the injection. It appeared that the beam was quite low on the IBJM quadrant, also low on B1p. This makes sense since we had to lower it to recover the alignment.
We brought the EIB Tx set point back to -600 urad as it was before the noise could appear (it had been set to -525 urad on tuesday).
We slightly adjusted the setpoint from -600 urad to -596 urad to recover the same position on the IBJM_v quadrant. By taking a closer look now it seems that we can still improve it (figure 1). We may want to do it in the coming days.
The IMC followed and went back to the position it had before the noise could appear.
The excess of noise that could be seen in the last two days disappeared. See RFC signals before the noise on EIB and now (figure 2)
starting the test (that therefore was not adding any new information). The change in the noise level clearly happened when an oscillation of the control loop was triggered, likely, by the work of Antonino in the laser lab at 14:33:05 UTC. The oscillation stopped when opening the position loop at 14:33:35 UTC (see Plot1).
Plot 2 shows the difference in the LVDTs raw noise level before and after the oscillation. Plot3 shows the error signal of the EIB control after the lucky intervention, with normal operating conditions recovered. The behaviour of the EIB is clearly not reliable and further investigation is needed in the coming days.
All volumes are up and running at their normal speed.
If in your activities you observe any anomalies in terms of data access or performance in your work, send an email to service@ego-gw.it
The global inverted pendulum control, which should have been in operation, was found to be permanently off. The latest check was done several months ago, then it was used for a while, but for some reason was manually disengaged (probably by me) and forgotten.
The attempt to put it in operation again failed, because of a malfunctioning of BS control. The problem was related to the fact that GIPC error signals were not received correctly by the control board. A reload of the code fixed the issue and GIPC was engaged successfully.
Today there were not the good working condition to allow a complete and accurate check of GIPC performance: the only thing I can say is that the lock of CITF was acquired with and without GIPC, with no evident difference.
Starting from CARMtoMC state, GIPC configuration is different and requires a specific check. It was not possible to do it, so we leave GIPC off.
This morning I found the ITF locked on the IR thus I took the occasion to perform OSD calibrations:
- 6:01 UTC NI single bounce;
- 6:07 UTC WI single bounce;
- 6:34 UTC CITF.
Afterward, we attempted to lock the ITF, but it was still unstable; the commissioning crew (ISC, ISYS) investigated the problem (ALS fast gain test to suppress the noise) .
In parallel Paolo worked at the GIPC and Romain did some preparatory work for the next activity on SPRB.
At around 13:30 UTC the vacuum team started the venting on SPRB.
SBE
- SWEB found open; properly closed. It opened again at 13:28 UTC, properly closed.
- at 13:14 UTC SPRB loops manually open for the next activity (venting, SPRB telescope tuning).
Electricity
The DMS is reporting that the UPS2 is in alarm; experts have been informed by mail.
This test will be done tomorrow morning (if the ITF can lock). What I would like to test is not to acquire the lock with a misaligned SR but misalign the SR after having locked carm0 of at least 5 urad.
Since last Monday night we have an increase of noise coming from the INJ side.
Since then it is very difficult, almost impossible to lock. The CITF is lockable but it is much more difficult to engage its control, and we rarely manage to go further than DARM to IR step.
The noise is present, apart from the INJ channels, on LSC_CARM, LSC_PRCL, LSC_MICH and on the Z corrections of the input mirrors.
In order to understand if the coupling mechanism was passing trough the beam injected from the terminal benches (doubling of the main beam) and then on the CITF trough the Input mirror correction, we have changed the gain of the ALS fast loop.
Figure attached:
- purple: before the EIB failure
- red: today with standard gains for fast ALS
- green: NEB increased from 0.4 to 0.6
- blue: NEB @ 0.6 and WEB from 0.75 to 1.1
No improvement are visible, thus this noise should be coupled in the central area (this includes the parts of the green signals generation that happens at the EIB and Atrium).
NB it seems that the red curve is less noisy on LSC channels, but in that period also the INJ_IMC_TRA is less noisy
It would be interesting to misalign the SR mirror by a "large" angle (somewhere in the range 5 to 20 microrad) and try to lock the ITF in this configuration (so with no SRCL control).
By doing so we could observe the dark fringe with the hot interferometer without being confused by the SR mirror.
Such observation of the dark fringe could help considerably the tuning of the thermal actuators.
Given that the alignment of the B1p phase camera is good, it seems that the alignment is not what is causing the fringe pattern.
There is a hypothesis that it is caused by spurious beams, and it was suggested that the issue oculd be on the EDB bench leading up to the phase camera where some lensing effect is also occuring, since the beam is now larger than before. Another suggestion was to check the B1p cam in SDB2 to see if the pattern/spurious beam could also come from the new parking position of the PR mirror.
Looking at one of the latest single bounce occurences (Fig. 1) the fringe pattern is clear on the B1p phase camera, but the B1p cam on SDB2 sees a gaussian beam. This suggests that the spurious beam causing the fringing should be on the optics leading to the phase camera, and that the PR parking position shouldn't be the problem. However, there seems to be a small beam on B1p cam on the right of the main beam, not sure if that could be some spurius beam as well.
With the CITF locked and the SPRB under drift control (using the B4_QD2 112 MHz signals, see Fig.1), we checked the centering of the EPRB diaphragms. We adjusted very finely their position and close the diaphragms at minimum aperture to have a good reference for the PC alignment during the afternoon shift.
When this check was done, the SPRB angular position was the following : TX = 152 urad, TY = -655.5 urad.
Then we put the ITF in NI single bounce, and we checked that for the same angular position given by the SPRB LVDT, the B4 beam was still passing through the EPRB diaphragms.
In addition we recorded a picture of the two B4 cameras in single bounce: see attached file.
For B4_Cam1, the gaussian fit parameters are: wx = 334 um and wy = 705 um.
As it seems that the raw storage area is more reliable, it scan in the /opt/w3/vim/storage/vimstart.sh has been restored this morning.
This Tuesday Riccardo performed two mitigation actions on the NEB "step" dry pump: (1) added a second layer of soft rubber underneath the pump (Figure 1 - before, Figure 2 - after); (2) added a softer section of corrugated pipe (Figure 3). The overall effect was to reduce the seismic line at 29.15 Hz in the NEB hall as recorded by various sensors (Figure 4 - VIM spectrogram, Figure 5 - guralp and accelerometers ASD, Figure 6 - NN array). The line amplitude reduced by about a factor 2 to 3, depending on the sensor. It remains to understand which of the two actions produced the observed effect. This will be done next Tuesday.
Looking at the acoustic spectrogram of the North end building, we noticed, during windy days, an increase of the acoustic noise in the experimental hall, Figure 1. Looking in detail, this seems to occur coincident with the growth of the pressure (*PRES*OUT*) measured along the supply air duct in the AHU room when the velocity (METEO_WIND_SPD) of the wind is greater than ~10 km/h and its direction (METEO_WIND_DIR) is less than ~100 deg clockwise with respect to the geographic North, Figure 2. It is worth mentioning that the values of wind speed and direction are measured by the meteorological station located on the roof of technical building 2.
Figure 3 shows the effect of the wind speed&direction on the microphone of the NEB experimental room when the *PRES_OUT* increases. The analyzed NEB data set is before the slow-down actions of the NEB AHU fan inverter (#57154). The blu curve is the "quiet": the acoustic ASD during a day with low wind activity (v~4 km/h). We observe a broadband noise in the frequency range ~(10-100) Hz (yellow and violet curves) during the days with wind velocity above ~10 km/h. But what triggers this acoustic noise amplification effect is also the wind direction below 100 deg. Indeed, the red curve, despite having sustained wind activity (v~34.3 km/h) but a direction greater than 100 deg (dir~222.3 deg), shows an acoustic noise in the room almost comparable to the quiet. This effect might be related to how the wind impinges on the supply and return external air ducts of the NEB AHU system.
Regarding the West end building, this effect seems less evident: independently of the direction, only the high wind velocity (above ~15 km/h) produces a slight growth of the acoustic noise in the frequency band ~(10-50)Hz (green and yellow curves) but the effect is larger than NEB above 50 Hz, Figure 4.
Further investigation will follow.
To characterize this noise also from the point of view of glitches, I attach three plots of the time-frequency distribution of Omicron glitch triggers in V1: DAQ_FMOD_56MHz_row_mag_FS in the considered time interval.
Fig. 1 is the classical "glitchgram", where every dot represents the central frequency of the corresponding glitch trigger identified by Omicron. The color and the size represent its SNR.
Fig. 2 gives an alternative picture of the previous glitchgram, where the rectangles represent the duration and bandwidth of each glitch.
Fig. 3 glitchgram provides additional information on the class of the various triggers identified by Omicron. The classification has been obtained as follows. Scattered Light glitchs in V1: DAQ_FMOD_56MHz_row_mag_FS look, in the spectrograms, "approximately similar" to those in hrec. So, I used a pre-trained classifier, trained on O3 hrec Virgo glitches, to classify V1: DAQ_FMOD_56MHz_row_mag_FS triggers in this time interval. These are shown as red circles. The classification of the other glitch classes is not as accurate (for many hrec classes there is no equivalent in DAG_FMOD_56MHz) so I have collectively labelled them as "Others".
Ideas about further analyses: other time intervals can be inspected. The rate of Scattered Light glitches can be correlated with auxiliary sensors, such as seismometers, as done in the ExtENV paper.
Yesterday we put in operation the new trigger for the SDB1 fast shutter, based on LSC_DARM, as reasoned in #57166 and comments. We use three different sets of thresholds (shutter_thr_darm_hi,shutter_thr_darm_lo) in three different stages of the lock acquisition, using the ITF_LOCK node:
- DOWN: very relaxed thresholds (1e3, 1e-2), that should never trigger;
- CARM_OFFSET_STEP_2_OF_3 (200 mW arms power): thresholds are (2e-2, 1e-2);
- CARM_NULL_1F: more stringent thresholds (1.5e-3, 1e-3).
Parameters are stored respectively in the [ITF_LOCK], [CARM_SET_STEP2] and [CARM_NULL_1F] sections, the cm command used is:
SDB1_FAST_SHUTTER.LSC_DARM_ABS_HIGH.AcRelayChSet(0.0,1.0,-2,shutter_thr_darm_hi,shutter_thr_darm_lo)
Pictures of the Rx bench, and the rack taken during the installation.
In order to evaluate how much 3 urad and 5urad misalignment of SR are effecting the recycling in the SRC, two images of B1p are taken.
It is visible that also for 3urad the beam bounces are present.
It would be interesting to misalign the SR mirror by a "large" angle (somewhere in the range 5 to 20 microrad) and try to lock the ITF in this configuration (so with no SRCL control).
By doing so we could observe the dark fringe with the hot interferometer without being confused by the SR mirror.
Such observation of the dark fringe could help considerably the tuning of the thermal actuators.
This test will be done tomorrow morning (if the ITF can lock). What I would like to test is not to acquire the lock with a misaligned SR but misalign the SR after having locked carm0 of at least 5 urad.
ITF State: locked in the IR
Quick Summary: ISYS in standard mode; all suspension loops closed; all SBEs loops closed with the exception of SWEB; signals related to the Air Conditioning of the Mode Cleaner are missing
No activity ongoing.
The first part of the shift was dedicated to the remote intervention of Bertolini on EIB SBE to reduce the excess noise and studies to understand the possible noise source (see report #57229). At 17:30 UTC the intervention was completed and Spinicelli went to the Laser Lab to restore the Injection system. IMC and RFC were available again from 17:45 UTC and it we start the ITF recovery (carried out by Bersanetti).
It was possible to relock the Infrared cavities quite easily, but the strong wind and microseism prevented the stable lock of the CITF. We finally reached CARM_NULL_1F at 20:43 UTC; the lock lasted until 21:07 UTC.
Activity concluded at 21:15 UTC, infrared cavities left locked.
Other
Since this morning at around 9:10 UTC the signals related to the Air Conditioning of the Mode Cleaner are missing. I tried to restart the eurothermServer, but this action did not solve the issue. Looking at the log, it was possible to manually ping the unresponsive device, which is actually reachable.
ACS expert informed of this issue.
We started the recovery of the lock acquisition at around 17:45 UTC, after the intervention on EIB. Unfortunately the lock acquisition was still quite impacted, as also che weather conditions remained pretty bad, if not worse than yesterday. However, we could have a stable INJ system, although noisier, and we could move forward with the lock acquisition with respect to yesterday afternoon.
We could lock the DRMI a few times, although it took quite longer than usual and with a much lower stability, and we suffered several unlocks for several different reasons: LSC actuation saturating due to weather conditions, ALS system unlocks (evenly distributed among North and West) and CARM2MC handoffs (a lot of these).
Other notes:
- the ALS DC powers on EIB are ~ 25% lower with respect to yesterday after the Maintenance, but it is reasonable given the stress that EIB sustained; in any case, the ALS lock is completely on reflection signals, so this does not affect us;
- only when the PR mirror is aligned, we can see again the short wavelet-like bursts of noise in CARM (and, consequently, on all three DOFs of the DRMI) (Figure 1).
At the very end of the shift we managed to reach once CARM_NULL_1F (starting from 20:43:49 UTC), with the standard configuration; given the hour and the sub-optimal stability of the ITF, we just did a few checks on the configuration:
- BS drift control closed at 20:45:01 UTC for TX and 20:45:39 UTC for TY;
- COMMp drift control closed at 20:48:10 UTC for TX and 20:48:25 UTC for TY;
- clean data with no ASC offsets: 20:53:00 UTC + 180 s;
- clean data with DIFFp_TX_SET = -2e-3 and DIFFp_TY_SET = 6e-3: 21:03:35 UTC + 180 s.
We unlocked at 21:07:08 UTC due to a sudden oscillation, visibile on most probes; this was the first occasion to test the new SDB1 fast shutter logic, which has been automated (more in a separate entry) (Figure 2).
The noise seems to be responsible for the contamination of Tx and Tz channels; due to the high blending frequency (s=6) the roll-off of the LVDT blending filter is not sufficient to absorb it. The cause of this malfunctioning is unclear and it will require an intervention. INJ team also attempted to temporarily block the bench to check whether the noise was affected but no effect was observed.
In order to mitigate the problem, additional low pass filtering has been introduced in the standard 5-th order LVDT blending filter; the new filter can be further optimised but some reduction of the excess noise has been achieved (see Plot 3). The laser beam was unblocked around 17:30 UTC.
MdVim issue
The MdVim server is stuck waiting for the completion of the /opt/w3/vim/storage/vimstart.sh command scanning the raw ,raw_full, rds, trend, spectro storage area .
Due to the issues on the raw offline storage area its scan has been removed . As consequence the plots related to the Raw dara storage in the VIM are not anymore updated