This morning at 5h UTC, the laser of NE PCal has been switched off from remote for the intervention inside the tower.

the missing plot

The following report has been submitted to the On-call interface.

On-call events -> Injection System

Title: ISYS main chiller failure

Author(s): chiummo

Details:

DMS notified that the coolant temperature of the ISYS main chiller was way above its usual value (24C instead of 19C). By looking at the evolution of the temperatures of the appliances cooled by this chiller, it was clear that very soon the neovan amplifier safety would have tripped because of temperature of diode 4 reaching 35C. This happened actually in a few minutes.
On site inspection in the EERoom found the supposedly on-duty chiller ON but with blank display. No signs of liquid leakage. Pump was still working (flow of coolant was not decreased), so it is likely that the failure concerned the heat exchange part. A quick attempt was tried to switch off and then on again and this restored the chiller display, with apparent full functionality of the chiller. Of course this chiller was not trusted, so it was switched off and then the cooling circuit was swapped to the spare chiller.
The Neovan amplifier could be switched on again, PMC relocked and so the IMC.

With the spare chiller, we have anyway reduced performances of the cooling circuit. Coolant flow was measured in around 6l/min and is now 3.8l/min. Coolant temperature was around 19C and is now 21.3C.
Both these parameters affect the working temperatures of neovan head and neovan pumping diodes. We know that Neovan head increased temperature is a source of alignment drift at the input of the PMC.
To be monitored and possibly improved on monday.

* Note that any files attached to this report are available in the On-call interface.

This morning, we performed the near field sweep magnetic injections with the small coil :

• WI tower
  sweep (8-330)Hz, duration 600 sec with coil oriented along EW, NS and vertical direction
  sweep (8-330)Hz, duration 600 sec with coil oriented along EW, NS, vertical direction and positioned close to the tower base edge (x,y)=(53,76) cm
   
• WE tower
  sweep (8-330)Hz, duration 600 sec with coil oriented along EW, NS and vertical direction
   
• WE SAT rack
  sweep (8-330)Hz, duration 600 sec with coil oriented along EW, NS and vertical direction (see Figures)

The action details are reported in the attached file.

Upon arrival I found ITF locked on IR.
After the usual cross-alignment in ACQUIRE_DRMI I started the lock acquisition. After few attempts ITF has been locked in a stable way at LOW_NOISE_2.
The planned ENV-INF activity on near-field magnetic injections in CB and WE (carried out by Fiori and Tringali) went on for the whole shift without major problems.
At 12:23 UTC the activity has ended. I opened NE and SNEB loops. NE RH has been switched OFF from remote by TCS expert. 
Vacuum team allerted to start the NE venting after NE RH thermalization (at ~14:30 UTC). 

After the ENV morning activity, at 12.23 UTC NE RH has been switched OFF (Fig. 1), in view of NE tower venting.

The NE RH must be cold (INF_TCS_NE_RH_TE < 20°C) with a tower pressure below 10e-2 mbar before starting the venting, to avoid oxidation issues.

ITF State: LOCKED_ARMS_IR
ITF Mode: LOCKING
Quick Summary: SBE loops closed, Suspensions loops closed, ISYS in standard state.
Activities ongoing since this morning: No activities ongoing

ITF found in DOWN State. At the beginning of the shift, from 14:20 UTC to 14:35 UTC, we performed with Tournefier measurements with NARM Cavity locked and WARM fully misaligned. After completing this task, we started to relock for the planned SGD activity, ITF loss measurements, carried out from remote by Vardaro, Bonavena with the support of Gouaty.
During the locking acquistion we experienced various unlocks at LOCKING_CARM_MC_IR, so at 16:00 UTC we contacted the ISC Oncall (Mantovani). The recovery proceeded until 20:30 UTC, with ACQUIRE_LOW_NOISE_2 as the higher step reached. As agreed with the Crew and the Commissioning coordinator, we stopped the recovery at 20:30 UTC to allow the Squeezing Team to work at LOCKED_DC_READOUT.
Activity concluded at 22:25 UTC, ITF left in LOCKED_ARMS_IR.

ISC
(02-06-2023 16:00 - 02-06-2023 20:30) From remote
Status: Ended
Description: During the locking acquistion, the ITF kept unlocking at LOCKING_CARM_MC_IR, at around 40 mW for B7/B8.
Actions undertaken: We contacted the ISC Oncall that, from remote, tuned the locking acquisition parameters. We were able to reach ACQUIRE_LOW_NOISE_2 at 20:27 UTC (see report #60429). As agreed with the crew, we stopped the Oncall intervention at 20:30 UTC to proceed with the Squeezing activity.

The following report has been submitted to the On-call interface.

On-call events -> ITF Operation

Title: recovery of itf lock

Author(s): mantovani

Details:

The lock acquisition was not working anymore
I've tuned the lock acquisition parameters starting from 100mW (especially DARM gain and phase) at each step
I've also changed the target ugf for DARM @ 3f from 65 Hz to 55Hz
we have unlocked going at LN2
we leave in DC readout to let Marco recover the alignment before the opening of the tower

* Note that any files attached to this report are available in the On-call interface.

The noise produced by the dangling magnet, which has disappeared with the removal of the broken piece, matches very well a model of thermal noise on the peak. About the tail at low frequency, the expected slope is 1/sqrt(f), characteristic of a frequency indipendent loss angle, but even other models could be possible in principle. In fig 1 the main model and the model corresponding to viscous losses are shown. The second is much smaller at low frequency: an improvement of the sensitivity below 300 Hz would much less appreciable in that case.

An investigation has been performed, looking for the best data to be compared to the best data collected before the intervention on WI. Infortunately this week the sensitivity seems always affected by some extra noise: the floor at high frequency is higer and there is an excess of frequency noise in the critical frequency range. Anyway, picking in the higher range periods, it is possible to find an improved curve in the floor between 80 and 600 Hz (fig 2).

The data seem to provide a confirmation of the model 1/sqrt(f): if the 'internal' thn projection on h is re-added in quadrature to the spectrum, one can obtain a curve very similar to the good one before the intervention (fig 3 - to be noticed that the matching is better if also the floor at high frequency is compensated using a model of readout noise). The exercise doesn't work so well if the viscous model is added (fig 4).

There is nothing rigorous about this study: unknown noises come and go, adding uncertainty in any comparison between curves taken in distant periods in time. However, we have some good indications that our assessment of thermal noise seems correct.

Check Hrec measurements

During the calibration shift we did several Check Hrec injections in different configuration to test if we saw the same kind of variation as with the CheckHrec measurement of the 11/05/2023 (see this logbook entry)
For the first 3 sets injections (one set being 5 Mir injection and 5 Pcal injection) the line at 80 Hz (see this) was not injected.
Then we did 1 set of injection with the 80Hz present as well as an additional 10Hz line
plot 1,2,3 and 4 show the result for MirNE and WE and Pcal NE and WE without the 80Hz line and plot 5,6,7 and 8 the result for the injection with 80Hz and 10Hz line.

Then we did a check Hrec injection set right after doing a sensitivity measurement by injecting noise in the ITF.
Result are shown in plot 9,10,11 and 12 with the first 5 injections being before the sensitivity injections and the other 5 being right after.
No significant discrepancies are seen contrary to precedent measurements made during the 11/05/2023 calibration shift.

The plot 13 shows the supperpose averages for all the Check Hrec injections done during the 01/06/2023 shift.
The modulus values follow the same shape between all actuators, however there is a difference in phase MirNE and MIrWE value with a delay of 30 μs for NE and 45 μs for WE that is probably linked to additionnal delays found during the measurement of the transfer fonction between CAL_{WE,PR,NE}_Z_CORR and Sc_{WE,PR,NE}_Z_CORR channels,  with an additional delay of 50 μs for WE and PR and 25 μs for NE.
The Pcals don't have the same phase behavior and are quite constant over the frequency range.

In conclusion we can see with this measurements that Hrec is correct at a 5% precision in modulus.

As mentioned in the previous entry, this morning the demodulation phase of the OMC PZT changed again.

We had to retune the demodulation phases by +2.7 rad, as follows :

• Phase of B1_PD3 changed from -1.5 to 1.2 rad.
• Phase of B1_PD2 changed from to -0.64 to 2.06 rad
• Phase of EDB_B1s_QD1/2_RF channels (demodulated at PZT frequency) changed from -1.17 to 1.53 rad.

This is exactly the opposite correction to the one performed last wednesday morning.

The attached figure shows that the phase monitored with the channel EDB_OMC_PZT_phi is jumping each time we stop/start the SDB1_LC process. To be investigated.

I superpose the optical responses measured the 11th May with the optical response of the 1st of June. The optical response has changed a lot between these two dates

This morning the topic of the shift was the implementation of blended error signals (combination of the B5_QD2 DC signals and of the B1s_QD2 RF signals) for the drift control of the SDB1 bench.

Unfortunalely the goal was not reached (our attempts of engaging the blended signals in the loops failed), but we took some data that should hopefully help us to understand the issue.

We reload SDB1 filters after defining the blending filters at 07h19m17-UTC.

Fig.1 shows the comparison between the FFT of B5_QD2_H/V with the B1s_QD2_RF_56MHz_I_OMC_X/Y_I signals, and with the SDB1_LC_TX/TY signals, using the data taken last wednesday with the bench under local control in DC readout. On the figure B1s_QD2_RF_56MHz_I_OMC_X_I has been multiplied by 60 and B1s_QD2_RF_56MHz_I_OMC_Y_I by 20 to accound for the gains applied under SDB1_LC when reading these channels.

From Fig.1 we can extract the intercalibration between the channels by looking at the frequencies of the dither lines applied on SDB1 (2.1 Hz in TY, 4.3 Hz in TX):

At 2.1 Hz (dither line in TY), B5_QD2_H = 9.71 um, B1s_QD2 H = 0.0746 um which gives a ratio of 130,16. Ratio between B5_QD2_H and SDB1_LC_TY is 9.71/0.273 = 35.57

At 4.3 Hz (dither line in TX), B5_QD2_V = 12.75 um, B1s_QD2 V= 0.00196 um which gives a ratio of 6505. Ratio between B5_QD2_V and SDB1_LC_TX is 12.75/0.262 = 48.66

We take as reference the amplitudes of the lines in SDB1_LC_TX/TY:

B5_QD2_H must be multiplied by 1/35.57 = 0.02811. B1s_QD2_RF_56MHz_I_OMC_X_I (after the rescaling by 60) must be multiplied by 130.16/35.57 = 3.66

B5_QD2_V must be multiplied by 1/48.66 = 0.02055. B1s_QD2_RF_56MHz_I_OMC_Y_I (after the rescaling by 20) must be multiplied by 6505/48.66 = 133.7

We include these gains in the blending filters. Filters loaded at 08h35m07 UTC.

We tried to test the blended filters for SDB1 TX at 08h40m54-UTC, while the SDB1 bench was already under drift control with the usual error signals > ITF unlocked immediately. We think that the transition between the two modes of drift control may be problematic.

With the ITF in DOWN, we added some monitoring channels in the Full Stream data (B5_QD2_H and B5_QD2_V). We also added a relay "LC_B1_DARM_enbl" and a reset condition on the error signals B1_DARM_TX_err and B1_DARM_TY_err. SDB1_LC Process stopped and started at 08h56m42-UTC.

We wanted to test the ITF lock with the blended filter already enabled. Thus we enabled them with the external reload at 09h00m46 UTC. However this created problems to control the bench (corrections to TX and TY were no longer applied when the B5 drift control was engaged, which is actually the expected behaviour given the logic implemented in SDB1_LC). So we disabled the blended filter at  09h26m22 UTC.

SDB1_LC restarted at 09h41m53 UTC to store the state of the relay "LC_B1_DARM_enbl" in the data.

While we were trying to relock the OMC, we noticed that the B1_PD3 demodulation phase was mistuned again, this will be discussed in a separated entry. We updated the demodulation phase of B1_PD3 and B1_PD3 at 11h36m26 UTC. Then could lock the OMC. We then updated the demodulation phase for B1s QD1/2 RF signals (PZT line) at 11h56m53-UTC.

At this point we wanted to test the blended signals with the interferometer still locked in RF (hoping for a stronger lock robustness to the motion of SDB1). To this purpose we commented temporary the lines 4827, 4828, 4834, 4835 in ITF_LOCK.py to prevent the lock in DC readout.

At 12h00m54-UTC: open B5 beam drift. The bench is therefore under Local Control (but the blended channels are not computed).

We disable LC_TX/Y_err_base signals at 12h10m04-UTC, and put relay B1s beam drift on at 12h11m02-UTC: unfortunately this was not sufficient to compute the blended channels as the B5_QD2_enbl_safe was also resetting the blended signals.

ITF unlocked while we were trying to engage the blended error signals (with the bench under Local Control) at 12h20m42-UTC. The SDB1 LC signals at the time of the unlock are shown in Fig.2. To be analyzed.

Stop/start the SDB1_LC process at 12h25m48-UTC (we removed the relay B5_QD2_enbl_safe from the reset conditions of the blended signals).

OMC lock at 13h07 utc.

Open B5 beam drift at 13h11m35 UTC.

We disable the LC_TX/Y_err_base signals at 13h13m36-UTC and close the relay of the B1s drift control at 13h13m51 UTC to enable the computation of the blended channels (in reality the bench is still under local control).

Enable OMC dither lines at 13h20m16 UTC.  Collecting data in these conditions until 13h54 utc. Fig.3 shows a side by side comparison of the FFT of the B5/B1s error signals before and after the blending filters are applied. Fig.4 shows the corresponding transfer functions, which are approximatively what we wanted. Fig.5 shows a comparison of the B5/B1s/LC error signals after application of the blending filters. They are at similar level on the SDB1 dither lines at 2.1 and 4.3 Hz, but the B1s error signals are much larger at low frequency, in particular for TX.

We try to reduce the gain of the B1s error signal (*1/2 in TY and *1/20 in TX). Filter loaded at 13h54m57 UTC.

With the blended channels already computed, we tried to engage them in the loop at 13h58m43-UTC, but the ITF unlocked again. To be investigated.

At the end of the shift we restored the standard LC_TX/TY_err_base signals.

We restored the engagement of the DC readout on B1_PD3 by uncommenting the lines 4827, 4828, 4834, 4835 in ITF_LOCK.py

During the shift we increased the speed of the ramp used to acquire the lock of the OMC from 0.0001 to 0.0002 (speed doubled).

Sensitivity measurement

During the calibration shift, we did several injections on NE and WE to estimate the sensitivity h(f) directly with the measured TF between CAL{NE,WE}_MIR_Z_NOISE and DARM.
And we compared h(f) with the FFT of the hoft_raw signal provided by Hrec. Above 1000 Hz a fit of the TF is used. The fit is not accurate and shows a cavity pole frequency too high.
Anyway, at least below 1000 Hz, the comparison between h(f) and FFT(hoft_raw) is quite good. The main discrepancy is around 300 Hz
and is due to a notch around 309 Hz and a notch around 355 Hz that we used during our noise injection.
Plots 1 and 2 show the TF and h(f) for WE, plots 3 and 4 shows the TF and h(f) for NE.

Upon arrival I found ITF stable locked at LOW_NOISE_@ since two hours.
The planned DET acticity on OMC alignment error signals blending, carried out bu Gouaty, Was, Hui went on for the whole shift without particular issue, except for an unlock which opened SDB1 and SDB2 LC. Properly closed.
During the relock, accidentally, it has been request to the automation to go up to CENTERING_MEAS_DAS_ON. Upon realized the mistake we unlock ITF but unfortunately the flip mirrors opened. With Lumaca (contacted) we restored soon the standard conditions. I had also to reset TCS_MAIN metatron node (DOWN and INIT). 
The relocking went on quite smoothly with a cross-alignment in ACQUIRE_DRMI.
Activity still in progress with ITF DOWN.

TCS
(02-06-2023 10:00 - 02-06-2023 10:30) From remote
Status: Ended
Description: flip mirrors opened during the lock acquisition

The optical response of PR and SR has been measured 3 times during the shift of the 1st of June. The 3 measurements agree with each other, however, no fit function has been found yet.

Analysis of the optical responses of NE WE and BS

Two sets of measurements have ben done: first from gps time 1369680954 to 1369681320, second from gps time 1369682520 to 1369682886:

The two measurements are distant by 20 min, the gain of the optical responses has not change sinicatively from one measurement to another. However the pole frequency  change by ~10 Hz, probably due to fluctuations in the SR position.

The values of the parameters are on the plots

The unlock reported by the operator in this entry, which occured for "no clear reason" (on May 31st at 10:41) happened within a few minutes when Maria finished the work in the TCS room.

Looking at at TCS microphone the coincidence is quite clear. 

ITF State: LOW_NOISE_2
ITF Mode: LOCKED
Quick Summary: SBE loops closed, Suspensions loops closed, ISYS in standard state.
Activities ongoing since this morning: No activities ongoing

The shift was dedicated to the planned CAL Weekly Calibrations (see report #60411). From 14:00 UTC to 15:15 UTC Boschi worked in the NE, WE and PR DSP to reduce the communication delay (see report # 60387). Unfortunately, the CAL Team reported that the delay with the DSP is still present: in agreement with the crew and the Weekly coordinator, other possible interventions by Boschi were postponed to allow the planned activity.
The first part of the measurements were performed in DOWN State, then at 16:21 UTC we started to relock, LOW_NOISE_2 reached at 16:46 UTC.
The ITF was manually unlocked at 20:08 UTC to be relocked at LOW_NOISE_1 for further measurements, it reached the target state at 20:33 UTC.
Activity concluded at 21:35 UTC, ITF left in LOW_NOISE_2 with AUTORELOCK_FAILSAFE Engaged.

Other activities carried out during the shift:

• CEB-LEV1 data acquisition process restarted (see report #60412);
• From 16:00 UTC to 16:15 UTC, NARM Lock cavity with WARM misaligned (Tournefier with Operator support);

EnvMoni
Since this morning at 4:25 UTC, the device ENV-NI became unreachable. As a consequence, the signals related to the NI Supensions temperature are not available anymore.
I contacted De Rosa on this issue and, following his instructions, at 20:10 UTC I went to the DAQ Room to stop and restart the device. Unfortunately the action did not solve the issue, Experts updated on the problem.

1) Measurement of the transfer fonction between CAL_*_Z_CORR and Sc_*_Z_CORR channels for all mirrors and all marionettes

• While the ITF is in down state
• 15h18 UTC :  asking metatron for CALIBRATED_DELAY state, to perform measurement of the transfer fonction between CAL_*_Z_CORR and Sc_*_Z_CORR channels for all mirrors and all marionettes

Looking at the measurements we still see an additionnal delay of 50 μs between CAL_{WE,PR}_MIR_Z_CORR and Sc_{WE,PR}_MIR_Z_CORR and 25 μs between CAL_NE_MIR_Z_CORR and Sc_NE_MIR_Z_CORR

2) CheckHrec measurements

• 16h40 : ITF in LN2

CheckHrec injection without the 80 Hz line

• 16h53 : 5 CheckHrec injections on WE and NE MIR from 10Hz to 1300 Hz
• 17h03 : 5 CheckHrec injections on WE and NE PCAL from10Hz to 1300 Hz  with 2 from 10Hz to 600 Hz, 1 from 600Hz to 1000Hz and 1 from 1000Hz to 1300Hz
• 17h15 : 1 CheckHrec injection on WE and NE PCAL to check if the injection are done correctly 

Note : the sensitivity seemed a bit high during the first CheckHrec set of injections, maybe because we didn't wait enough after acquiring LN2 to start injecting lines.

• 17h17 : 5 CheckHrec injections on WE and NE MIR from 10Hz to 1300 Hz
• 17h27  : 5 CheckHrec injections on WE and NE PCAL from10Hz to 1300 Hz  : 2 from 10Hz to 600 Hz, 1 from 600Hz to 1000Hz, 1 from 1000Hz to 1200Hz and 1 from 1200Hz to 1300Hz
• 17h38 : 5 CheckHrec injections on WE and NE MIR from 10Hz to 1300 Hz
• 17h48  : 5 CheckHrec injections on WE and NE PCAL from10Hz to 1300 Hz  : 2 from 10Hz to 600 Hz, 1 from 600Hz to 1000Hz, 1 from 1000Hz to 1200Hz and 1 from 1200Hz to 1300Hz

CheckHrec injection with the 80 Hz line and with the 10 Hz line

• 18h02 : 5 CheckHrec injections on WE and NE MIR from 10Hz to 1300 Hz
• 18h12  : 5 CheckHrec injections on WE and NE PCAL from10Hz to 1300 Hz  with 2 from 10Hz to 600 Hz, 1 from 600Hz to 1000Hz, 1 from 1000Hz to 1200Hz and from 1200Hz to 1300Hz

Trying CheckHrec injection after sensitivity injection

CheckHrec injection are still with additionnal lines at 10Hz and 80 Hz

• 18h24 : Noise injection with inject_sensitivity
• 18h32 :  5 CheckHrec injections on WE and NE MIR from 10Hz to 1300 Hz
• 18h42  : 5 CheckHrec injections on WE and NE PCAL from10Hz to 1300 Hz  with 2 from 10Hz to 600 Hz, 1 from 600Hz to 1000Hz, 1 from 1000Hz to 1200Hz and from 1200Hz to 1300Hz

3) Optical response for PR and SR

• 18h55 : lines injection for NE, WE and SR and PR optical response, then noise injection for the sensitivity
• 19h21 : lines injection for NE, WE and SR and PR optical response, then noise injection for the sensitivity adding low frequency lines from 13Hz to 400 Hz
• 19h41 : lines injection only for SR and PR optical response adding low frequency lines from 7Hz to 400 Hz

4) CheckHrec injection only with High frequency line

• 19h51 : Check Hrec injection with only the 808 Hz line
• 19h54 : Check Hrec injection with only the 1258 Hz  line
• 19h58 : Check Hrec injection with only the 2308 Hz  line
• 20h01 : Check Hrec injection with only the 1258 Hz  line

5) Going to LN1 to measure WI optical response

going to down then going to LN1 from the CALI node
locking in LN1 with NI and WI mirror in HP

• 20h44 : In to End line injection to calibrate WI and NI in regard to WE and NE

We took a look at the unlocks of the green during the scans.
They seem to be triggered by an oscillation at around 56 kHz that suddenly appears on the error signal (see figure 1)
By taking a look at the spectrum and the fft time of the unlocks we can spot several lines (figure 2 and figure 3).
The 70 kHz line is the phase modualtion used to lock the arms.
There are two fixed lines at 13.8 kHz and 27.6 kHz. They might be generated by the beating between the 70 kHz and the 56 kHz.... or the countrary. 
We can also spot a line at around 6 kHz which is increasing in frequency. It seems that there is a beating between this line and 50 kHz (see figure 4)

We do not have clear explanations for those oscillations yet (fixed and wandering), could they be due to some mechanical mode of the WE mirror?

We quickly look an other unlock which shows a similar but different behavior (figure 5)

These are the FSR scans done yesterday. As mentioned by Fabio these were interrupted from time to time. Could experts check what was the problem with the green?