Today we continued the work on the IPATSiA setup (#67057).
We arrived around 18.15 LT at WE. At 18.30h LT we switched on the chiller and the laser and waited a bit for thermalization. The power was oscillating a bit between 30 mW and 45 mW.
Here a chronological list of the events (see Fig. 1):
At 19:42 LT the ITF locked in LN2:
At 20:17 we changed the position of the point absorber. The power was stabilized at 28 mW.
At 21:02 the ITF unlocked and we changed again the position of the PA. At 21:50 the ITF relocked in LN3.
At 22:15h we raised a little the emitted power of the CO2 laser to 41mW.
At 22:55h we swicthed off the CO2 laser and then the chiller at 22:57h. The power setting has been left to 41 mW (not the minimum).
ITF found in lock acquisition; at 13:35 UTC ITF in Science mode.
At around 14:50 UTC there was an event which opened the B1pQD1 galvo loop; we put the ITF in Adjusting mode trying to keep alive the lock but we couldn't manage. After this unlock it was necessary to perform the manual pre alignment of the SR, see B1p galvos check.
At 15:07 UTC ITF in Commissioning mode for the planned activity; still in progress.
Suspensions
Guard tours (time in UTC)
18:30-19:00; 20:49-
For tracing the WE DSP issue, (see the plot)
In operation since 2025-06-26-12h03m19-UTC
As suggested by Michal: 2 packets mixed at the DSP board level
Another event with similar values, in different position. The channel with the power could be probably removed, or at least reduced in amplitude before sending it to the DSP. I would also propose to remove the unused channels: MAR calib and a channel supposed to send some damping of violin modes, never used.
It is hard to imagine how a glitch at ~400 can be created out of a calibration noise channel that usually contains only small numbers. One possibility is if the issue is that the channel order become wrong for a few samples. So that the LSC_B8_DC (third channel in the packet) is received instead of LSC_CAL_WE_MIR_Z_NOISE (thirteenth channel in the packet). LSC_B8_DC has a mean value of ~535, so that could create a value around 400, if it is wrong for 3 out of 4 of the 40kHz samples used by the DSP and recorded back in the DAQ at 10kHz.
This afternoon I was contacted by the operator as, after an ITF unlock, it was not possible any more to close the loop of the B1p_QD1_V galvo.
Before this incident, the corrections were varying typically between -7 and +7V (Fig.1), so the galvo loop seemed to be reasonably tuned (a tuning was done at the last maintenance). The operator mentioned that the SR mirror had to be realigned, and indeed, it seems that the SR mirror was not at its usual position during the failed attempt of closing the B1p_QD1_V loop, so I suspect this is related.
Now the situation seems to have come back to normal after realigning the SR mirror.
As a side effect of the download of the master board, a few probes don't work anymore. They are the output signals of MIR_PSDF, the signals acquired by the board itself. The problem seems to be only on the probes, because the signals are used to generate MIR_Z, MIR_TX and MIR_TY, which don't show any issue. They are anyway signals not used for the control.
After the latest unlock, a few probes have been added from Sc_WE, in order to monitor more GLB input channels. The signals have been added to existing unused channels and the name of the probes have not been changed. The legend is:
Sc_WE_noise --> GLB00 (LOCK FLAG)
Sc_WE_txAA --> GLB0a (calib lines on MI)
Sc_WE_tyAA --> GLB0b (calib lines on MA)
Sc_WE_ErrPost --> GLB0c (calib lines on MI)
The signals from calib are added to the locking correction (MA or MI) before the latest high pass filter, compensating the analog filter after the DACs.
A glitch has been catched; all the new probes show an anomalous sample. In one of the signals, the bad data is large enough to explain the unlock.
ITF found during locking acquisition. During the shift the ITF was mostly unstable, with the longest lock lasting around 2 hours. Here a timeable of the events:
5:25 UTC - Science Mode;
6:03 UTC - Unlock;
6:51 UTC - Lock of LOW_NOISE_3 lasted only 1 minute;
7:45 UTC - Science Mode;
9:27 UTC - Unlock
10:36 UTC - Science Mode;
12:01 UTC - Unlock
12:46 UTC - Unlock at ACQUIRE_LOW_NOISE_3;
Relock in progress.
The new antenna gain looks perfect: now all the signals stay at maximum the half of the dynamic range
Figure 1 shows the spectrum of the order 2 mode for three different alignment: initial alignment, slight re-alignment to reduce the beam jitter peaks by factor 10, alignment after doing slight realingments on 9 different modes. That last curve has 5 times less power than the other two, as the alignment has drifted far after many realignments on very high order modes.
Figure 2 shows the same data in terms of RIN, with the expected level of shot noise and PD electronic noise quadratically subtracted. There is an underlying broadband noise that has a slope somewhere between -1 and -0.66, it is present in all three cases, but is higher for the large misalignment (yellow line)
Figure 3 shows the sensing noise subtracted RIN for modes 1, 2, 3, 4 and 5, in that order. Modes order 1 and 3 are more strongly affected by beam jitter peaks. All the spectra hint at a broadband noise with a slope between -1 and -0.66.
Figure 4 shows that low level of noise with this setup can be achieved. It shows a spectrum of a single bounce beam from two months, lock on the 56MHz LSB done the previous day, and the order 1 mode and order 2 mode after realignment. The single bounce beam had noise a factor 4 lower at 100Hz, and had a similar power to the order 1 mode and order 2 mode. The 56MHz LSB had 5 times lower power, so more sensing noise, it has a comparable RIN to the carrier HOM, but a different shape, more flatter.
Figure 5 shows modes 6, 7, 8, 9 horizontal, 9 vertical, 10 and mode 00 after the cumulative misalignment from many changes in alignments for each HOM. The three 20% higher spectra at 140Hz correspond to modes that had a mostly vertical shape (order 6, 7 and 9 vertical mode).
As the realignment to minimize the beam jitter peaks actually degraded the alignment for the TEM00 mode by a factor few, the latter measurements correspond to observing each time a different mix of higher order modes of the beam. As the EDB OMC decomposes the beam on a completely different basis in a misaligned state.
None of the modes was coherent with h(t).
/users/mwas/ISC/RINanalysis_20250624/analyseRIN.m
ITF found in Commissioning mode; commissioning activiry concluded at 14:41 UTC.
From 14:41 UTC to 15:02 UTC ITF in Science mode.
At 15:02 UTC ITF in Commissioning mode; activity concluded at 20:43 UTC.
Science mode set at 20:55 UTC.
Guard tours (time in UTC)
18:30-19:00; 20:54 -
Continued the study of the HOM noise in LN2. Went through all the modes from 1 to 9.
The attached figure show the mode shape, and the corresponding spectrum, taken during the measurement. Started to take the pictures only starting from mode order 3.
The OMC error signal demodulation phase might be badly tuned, with most of the signal on the q quadrature, so it is better to reconstruct the error signal from the demodulated signals instead of using directly the EDB_OMC1_err channel. I noticed it only when looking at mode order 9.
The data collected below needs to be analyzed.
15:49 UTC (5min) - locked on carrier order 1 mode (mostly horizontal)
15:55 UTC (5min) - increased OMC modulation depth to 0.5V
16:08 UTC (5min) - locked on carrier order 2 mode (mostly vertical)
tried to improvement alignmetn to reduce the beam jitter peaks at a few hundred Hz. It did work, used only the first mirror. The power of the mode also increased slightly
16:19 UTC (5min) - locked on carrier order 2 mode (mostly vertical)
16:25 UTC (5min) - increased OMC modulation depth to 0.5V
locked on mode order 3 and made small vertical realignment to reduce beam jitter peaks
16:37 UTC (7min) - locked on carrier order 3 mode (6 petals), but then around 16:38:40 1 minute jumped to mostly vertical, with higher beam jitter peaks
16:45 UTC (5min) - increased OMC modulation depth to 0.5V
locked on mode order 4 and made small vertical realignment to reduce beam jitter peaks
16:57 UTC (6min) - locked on carrier order 4 mode (mostly horizontal
17:04 UTC (5min) - increased OMC modulation depth to 0.5V
locked on mode order 5 and made small vertical realignment to reduce beam jitter peaks
17:20 UTC (5min) - locked on carrier order 5 mode (5 horizontal lobes, two vertical)
17:26 UTC (5min) - increased OMC modulation depth to 0.5V
locked on mode order 6, beam jitter peaks are already relatively small
17:38 UTC (5min) - locked on carrier order 6 mode (mostly vertical)
at 17:43:05 the registered temperature of the EDB OMC jupmed by 5mK (it is out of loop), also B1s increased by a few percent (PD with turned of Vbias)
17:46 UTC (5min) - increased OMC modulation depth to 0.5V
locked on mode order 7, realigned slightly in horizontal and vertical, beam jitter peaks almost completely disappeared and power of mode increased by factor 2
18:03 UTC (5min) - locked on carrier order 7 mode (mostly vertical)
at 18:07:40 the registered temperature of the EDB OMC jupmed by 5mK (it is out of loop), also B1s increased by a few percent (PD with turned of Vbias)
18:11 UTC (4min) - increased OMC modulation depth to 0.5V
18:15 unlock due to WE DSP glitch, the work around implemented so far does not help
relocked in LN2, locked OMC on moder orde 8, aligned slightly to reduce beam jitter peak and increase power of mode
18:51 UTC (5min) - locked on carrier order 8 mode (a wide blob in both directions), forgot to reduce modulation depth, so it is at 0.5V
another alignment improvement
19:00 UTC (5min) - reduced OMC modulation depth to nominal
locked on mode order 9, realigned slightly in horizontal and vertical, to reduce beam jitter, peaks sometimes disappear, but go up and down in height
19:15 UTC (5min) - locked on carrier order 9 mode (mostly horizontal)
19:22 UTC (5min) - increased OMC modulation depth to 0.5V
relocked onto the vertical mode 9, not trying to realign it, as it is the smaller mode from which it easy to jump into the horizontal one
19:34 UTC (5min) - locked on carrier order 9 mode (mostly vertical)
locked on order 10 mode
19:52 UTC (5min) - mostly horizontal
going to order 2 mode again, when crossing all the sideband and
carrier TEM00 I have noticed the 56MHz TEM00 had low power, the 56MHz
order 1 modes had similar power as the 56MHz TEM00, and there was a
high power TEM00 in between, is it a carrier created from the HOM?
20:13 UTC (5min) - order 2 mode, vertical, has 5 times less power than at the beginning
20:27 UTC (5min) - unexpected TEM00, jitter peaks are relatively high, but I haven't realigned for this mode
Most likely the realignments to reduce the beam jitter peak have
actually completely misaligned the OMC, so it mixes all the modes
together, including all of the HOM into a TEM00.
Will need to repeat the whole experiment, but this time aligning only
once on the 56MHz, and not touching the alignment afterwards, even if
the spectra are spoiled by beam jitter
Gain of the 3 channels is now set to 0x18 in configuration files, equivalent to a gain of 1.38
2025-06-25-13h39m35-UTC info letendre SPRB_dbox_rack saved - letendre: change VGA gain from 0x20 to 0x18 for RF antenna (rev. 262180)
2025-06-25-13h39m52-UTC info letendre Reconfigure DAQ_DEMOD_ENV in SPRB_dbox_rack
2025-06-25-13h40m43-UTC info letendre SNEB_dbox_rack saved - letendre: change VGA gain from 0x48 to 0x18 for RF antenna (rev. 262181)
2025-06-25-13h41m15-UTC info letendre Reconfigure NEB_DEMOD_ENV in SNEB_dbox_rack
2025-06-25-13h41m26-UTC info letendre Reconfigure NEB_DEMOD_ENV in SNEB_dbox_rack
2025-06-25-13h42m06-UTC info letendre SWEB_dbox_rack saved - letendre: change VGA gain from 0x48 to 0x18 for RF antenna (rev. 262182)
2025-06-25-13h42m16-UTC info letendre Reconfigure WEB_DEMOD
ITF found during locking acquisition, after a series of attemps it was back in LOW_NOISE_3 and Science Mode from 6:07 UTC.
It unlocked at 7:24 UTC, back in Science Mode from 8:14 UTC.
At 9:57 UTC, under request of Mantovani, I've changed the Etalon WI setpoint from 20.67 to 20.73 with a ramp of one day.
ITF switched to Commissioning Mode at 12:27 UTC, the ITF unlocked at 12:44 UTC.
Relock in progress.
This is a follow-up of the work started few months ago.
This time, we aimed to test the input power variation after the OMC lock, ideally in the low-noise configuration (see Fig. 1).
However, due to the DSP/RTPC communication issues this morning, we initially conducted the test in the high-power configuration (LN1) to avoid any unlocks caused by the Sc_{N,W}E cards.
In both LN1 and LN2, we applied a ±10% variation in input power (see Figs. 2–5). The interferometer remained locked without major issues.
Some points to note:
The PSTAB loop was left closed. As a result, it attempted to compensate for the power variation, causing a small glitch in the power. A possible future test could involve repeating the procedure with the PSTAB loop open to determine whether any residual power glitches remain.
No saturation was observed on PD1 or PD2.
In LN2, the power variation appears to be associated with a step or movement in the SR angular degrees of freedom. Interestingly, an SR_TX step seems correlated with an increase in power, while an SR_TY step is correlated with a decrease.
One possible explanation could be a difference in the resonant frequencies of the rotator in the two directions, which may interfere with the DCP line in DARM used for the SR AA error signal.
In LN1, with the SR AA loop open, no spurious SR movement was observed.
Below is the log of the actions performed:
2025-06-24-16h46m19-UTC>INFO...-Received move_rel message from LSC_script_spinicelli_20250624_160808
2025-06-24-16h46m19-UTC>INFO...-Parsed message*: [4, 1, 200, 50]
2025-06-24-16h46m19-UTC>INFO...-setting channel 4 axis 1 to 200 at speed 50
2025-06-24-16h46m19-UTC>INFO...-writing position -4750
2025-06-24-16h48m06-UTC>INFO...-Received move_rel message from LSC_script_spinicelli_20250624_160808
2025-06-24-16h48m06-UTC>INFO...-Parsed message*: [1, 1, 500, 50]
2025-06-24-16h48m06-UTC>INFO...-setting channel 1 axis 1 to 500 at speed 50
2025-06-24-16h48m06-UTC>INFO...-writing position 174559
2025-06-24-16h48m06-UTC>INFO...-Received move_rel message from LSC_script_spinicelli_20250624_160808
2025-06-24-16h48m06-UTC>INFO...-Parsed message*: [4, 1, 50, 50]
2025-06-24-16h48m06-UTC>INFO...-setting channel 4 axis 1 to 50 at speed 50
2025-06-24-16h48m06-UTC>INFO...-writing position -4700
2025-06-24-16h49m51-UTC>INFO...-Received move_rel message from LSC_script_spinicelli_20250624_160808
2025-06-24-16h49m51-UTC>INFO...-Parsed message*: [1, 1, 500, 50]
2025-06-24-16h49m51-UTC>INFO...-setting channel 1 axis 1 to 500 at speed 50
2025-06-24-16h49m51-UTC>INFO...-writing position 175059
2025-06-24-16h51m17-UTC>INFO...-Received move_rel message from LSC_script_spinicelli_20250624_160808
2025-06-24-16h51m17-UTC>INFO...-Parsed message*: [1, 1, -2500, 50]
2025-06-24-16h51m17-UTC>INFO...-setting channel 1 axis 1 to -2500 at speed 50
2025-06-24-16h51m17-UTC>INFO...-writing position 172559
2025-06-24-16h51m18-UTC>INFO...-Received move_rel message from LSC_script_spinicelli_20250624_160808
2025-06-24-16h51m18-UTC>INFO...-Parsed message*: [4, 1, -250, 50]
2025-06-24-16h51m18-UTC>INFO...-setting channel 4 axis 1 to -250 at speed 50
2025-06-24-16h51m18-UTC>INFO...-writing position -4950
2025-06-24-16h55m19-UTC>INFO...-Received move_rel message from LSC_script_spinicelli_20250624_160808
2025-06-24-16h55m19-UTC>INFO...-Parsed message*: [1, 1, 2500, 50]
2025-06-24-16h55m19-UTC>INFO...-setting channel 1 axis 1 to 2500 at speed 50
2025-06-24-16h55m19-UTC>INFO...-writing position 175059
2025-06-24-16h55m19-UTC>INFO...-Received move_rel message from LSC_script_spinicelli_20250624_160808
2025-06-24-16h55m19-UTC>INFO...-Parsed message*: [4, 1, 250, 50]
2025-06-24-16h55m19-UTC>INFO...-setting channel 4 axis 1 to 250 at speed 50
2025-06-24-16h55m19-UTC>INFO...-writing position -4700
2025-06-24-17h00m48-UTC>INFO...-Received move_rel message from LSC_script_spinicelli_20250624_160808
2025-06-24-17h00m48-UTC>INFO...-Parsed message*: [1, 1, -2500, 50]
2025-06-24-17h00m48-UTC>INFO...-setting channel 1 axis 1 to -2500 at speed 50
2025-06-24-17h00m48-UTC>INFO...-writing position 172559
2025-06-24-17h00m48-UTC>INFO...-Received move_rel message from LSC_script_spinicelli_20250624_160808
2025-06-24-17h00m48-UTC>INFO...-Parsed message*: [4, 1, -250, 50]
2025-06-24-17h00m48-UTC>INFO...-setting channel 4 axis 1 to -250 at speed 50
2025-06-24-17h00m48-UTC>INFO...-writing position -4950
If even saturated, the WEB RF antenna provides information that every 36000s +/- 200s there is a glitch in the environment, and this has been happening for years.
By searching all ENV sensors we can find a corresponding glitch in the UPS current (phase T only).
Does anyone know which device (connected to the UPS line, i.e. an experimental device) does something every 10 hours?
Gain of the 3 channels is now set to 0x18 in configuration files, equivalent to a gain of 1.38
2025-06-25-13h39m35-UTC info letendre SPRB_dbox_rack saved - letendre: change VGA gain from 0x20 to 0x18 for RF antenna (rev. 262180)
2025-06-25-13h39m52-UTC info letendre Reconfigure DAQ_DEMOD_ENV in SPRB_dbox_rack
2025-06-25-13h40m43-UTC info letendre SNEB_dbox_rack saved - letendre: change VGA gain from 0x48 to 0x18 for RF antenna (rev. 262181)
2025-06-25-13h41m15-UTC info letendre Reconfigure NEB_DEMOD_ENV in SNEB_dbox_rack
2025-06-25-13h41m26-UTC info letendre Reconfigure NEB_DEMOD_ENV in SNEB_dbox_rack
2025-06-25-13h42m06-UTC info letendre SWEB_dbox_rack saved - letendre: change VGA gain from 0x48 to 0x18 for RF antenna (rev. 262182)
2025-06-25-13h42m16-UTC info letendre Reconfigure WEB_DEMOD
The new antenna gain looks perfect: now all the signals stay at maximum the half of the dynamic range
Some recent unlocks of the ITF in LN3 look like Fig.1, most recently this morning at UTC 07:25:16.
The cause seems to be very low coherence of the DIFFp_TY UGF monitor line, which disables the servo acting on the loop gain: eventually the loop enters its high frequency instability regime and starts oscillating at ~4 Hz, saturating B1 PDs. Shortly after, the ITF unlocks.
I have tested a substantial increase in the monitor line's amplitude from 0.75e-3 to 1.8e-3. The test happened online, the automation has not been changed. The coherence of the line rised to ~0.7 and the servo started operating again.
Fig.2, Fig.3 show the effects on DARM and on B1 PDs, I cannot spot any significant negative consequence on them of this adjustment.
If this is confirmed, the new line amplitude can be introduced in the automation and it will help prevent a few unlocks in the future.
All three building RF antennas (ENV_xxx_RF) are clearly saturated (signals are cut off at 8191 counts).
Since we don't have a knob to adjust the analog side (antenna gain), is it possible to reduce the ADC gain until there is no more saturation?
I had a look at the unlocks after the LSC_Acl mimimun elapsed_time tuning performed the 2025-06-24-16h20m30s-UTC:
Remarks
As new trial, the LSC_Acl minimum elapsed_time has been set to 33us instead of 29us previously at 2025-06-25-05h29m20-UTC
Today, I found the ITF in SCIENCE. During the shift, we experienced some unlocks. Here is the list:
During the shift, I also noticed an unusual behavior of the ITF that after an unlock from Science, we experienced similar unlocks during relocking at this state: LOCKING_OMC_DARM_B1_PD3. Here is the list of these types of unlocks:
I had a look at the unlocks after the LSC_Acl mimimun elapsed_time tuning performed the 2025-06-24-16h20m30s-UTC:
Remarks
As new trial, the LSC_Acl minimum elapsed_time has been set to 33us instead of 29us previously at 2025-06-25-05h29m20-UTC
After the latest unlock, a few probes have been added from Sc_WE, in order to monitor more GLB input channels. The signals have been added to existing unused channels and the name of the probes have not been changed. The legend is:
Sc_WE_noise --> GLB00 (LOCK FLAG)
Sc_WE_txAA --> GLB0a (calib lines on MI)
Sc_WE_tyAA --> GLB0b (calib lines on MA)
Sc_WE_ErrPost --> GLB0c (calib lines on MI)
The signals from calib are added to the locking correction (MA or MI) before the latest high pass filter, compensating the analog filter after the DACs.
A glitch has been catched; all the new probes show an anomalous sample. In one of the signals, the bad data is large enough to explain the unlock.
As a side effect of the download of the master board, a few probes don't work anymore. They are the output signals of MIR_PSDF, the signals acquired by the board itself. The problem seems to be only on the probes, because the signals are used to generate MIR_Z, MIR_TX and MIR_TY, which don't show any issue. They are anyway signals not used for the control.
It is hard to imagine how a glitch at ~400 can be created out of a calibration noise channel that usually contains only small numbers. One possibility is if the issue is that the channel order become wrong for a few samples. So that the LSC_B8_DC (third channel in the packet) is received instead of LSC_CAL_WE_MIR_Z_NOISE (thirteenth channel in the packet). LSC_B8_DC has a mean value of ~535, so that could create a value around 400, if it is wrong for 3 out of 4 of the 40kHz samples used by the DSP and recorded back in the DAQ at 10kHz.
Another event with similar values, in different position. The channel with the power could be probably removed, or at least reduced in amplitude before sending it to the DSP. I would also propose to remove the unused channels: MAR calib and a channel supposed to send some damping of violin modes, never used.
For tracing the WE DSP issue, (see the plot)
In operation since 2025-06-26-12h03m19-UTC
As suggested by Michal: 2 packets mixed at the DSP board level