Reports 1-1 of 1 Clear search Modify search
AdV-COM (AdV commissioning (1st part) )
bersanetti, allocca, cohen, chiummo, pillant, de rossi, casanueva - 16:41 Friday 30 March 2018 (40958) Print this report
ITF Recovery: 0.1 Dark Fringe, increased frequency noise, IMC tuning

This morning we carried on with the recovery of the ITF: after tonight's earthquake only one arm could relock automatically, the reason is not clear; we restarted from scratch, and we got help from remote in recovering SDB2 and SPRB:

  • a tuning of FmodErr was performed, as we were again very much away from the correct position of the MC mirror (~750 um);
  • after that, we went through lock acquisition up to 0.1 fringe (state PRITF_MICH_OFFSET_0_1), but all the signals were very noisy (especially SSFS, but also DARM and PRCL); a stable lock at this stage was not possible, even with the loops' UGFs quite in place;
  • at this point we started some digging on the involved signals:
    • we played a bit with the crossing frequency of the two CARM branches (IMC at low frequency and SSFS at high frequency) by acting on the relative gain of the same error signal which is sent to the two loops (SSFS_GAIN_NORM, which lives in SSFS_Ctrl but it is not stored); in Fig.1 you can see the difference in the SSFS spectrum with such parameter = 1 (in gold), = 2 (in purple) and = 3 (in blue); notice that such parameter has always been in the [0.75, 1] range, while from the restart of the Commissioning we have been using it =2 in order to lock the SSFS; apparently this is not an issue, as today we managed to lock in all the three cases from scratch, or switch from one to the other in-lock. A too high parameter causes, as expected, an oscillation around 200-250 Hz (which can be already spotted in the blue plot), as the two branches become mis-balanced; I added the SSFS_GAIN_NORM channel in the DAQ as a slow (1 Hz) channel (from 10:33 UTC);
    • we compared the SSFS spectrum of this morning and a random lock at the end of the last Commissioning phase in the same state of the lock acquisition (LOCKED_SSFS); it can seen in Fig.2 that the behaviour is quite worse in the current situation; the spectrum has been measured with the same SSFS_GAIN_NORM value, MICH is present just for reference but in this stage it is NOT locked on the Q quadrature of the SSFS signal and doesn't show any worsening;
    • although more stable than in the past day, the Injection is still behaving in a strange way: INJ_IMC_TRA is fluctuating a lot (0.5 W peak-to-peak) even with the PSTAB loop closed; it must be noted that the Automatic Alignment is still in drift control mode, and it can be seen looking at the IMC_ref camera an oscillation in TY which can be correlated to the power fluctuation, which propagate to the arms (Fig.3); in Fig.4 and Fig.5 there is the zoom of the bottom right panel, where we can see also the appearance of several lines at multiples of 100 Hz; this can be correlated to the very high 50 Hz noise that has been observed lately;
    • looking at the alignment signals, IMC_TRA shows some coherence with the TY and X position of the EIB bench (Fig.6), although in a very odd frequency range; by the way, while making this last plot I noticed a strange behaviour of dataDisplay, where the coherence above 5 kHz of two signals sampled one at 10 kHz and one at 20 kHz makes no sense, flipping sign continuosly (see Fig.7);
    • after the re-measurement of the sensing matrix of the IMC alignment we tried to close the AA in full bandwidth; some oscillation on TX and TY are still present, but we could keep it closed; the SSFS did not show any particular improvement;
    • looking at the INJ signals, we can see "hops" on the PMC signals, which can be correlated with the +- 0.5W fluctuations of IMC_TRA (see Fig.8); in any case, keeping the AA in full bandwidth mitigates the effect, so I re-enabled it in the Automation;
    • after an increase of the gain of the MC locking loop, we tried to relock: we needed again SSFS_NORM_GAIN = 2, but the performance of the SSFS got immediately better (see Fig.9); it must be noted that changing the gain of the IMC loop changes a bit the meaning of the SSFS_GAIN_NORM parameter, as it changes only the gain of one branch. Probably the IMC AA sensing matrix needs to be re-measured, but in the meanwhile we kept the AA in full bandwidth; notice that with the same overall gain, the SSFS UGF increased (while in recombined) from 3.4 to 4.7 kHz.
  • towards the end of the shift we found out that all demodulation phases changed (+1rad for 6MHz, +4rad for 56MHz, -0.8rad for 8MHz, etc.), but the LNFS did not lose lock with the timing (see Fig.10).

The recovery activity will go on in the afternoon shift.

Images attached to this report
Comments to this report:
swinkels, bersanetti - 18:53 Friday 30 March 2018 (40965) Print this report
I had a better look at the unlock at 12:28:55 UTC. Zooming in on the signals of the LNFS, it appears that the raw phases do not show any discontinuity (DAQ_LNFS_6MHz_raw_phi), but the derotated one (DAQ_LNFS_6MHz_phi) does show a step at the exact integer second, see attached figure. The interferometer unlocks a few millisecond later. Looking in the logfile of the LNFS_Demod process, it seems that the sine generators used to correct for the small difference between nominal and actual demodulation frequency are reset, which might explain the jump:

2018-03-30-12h28m53-UTC>INFO...-AcAdcChBufLoad> Ok
2018-03-30-12h28m53-UTC>INFO...-AcMain> TolmProcessorDeviceUpdateTaskOrder done
2018-03-30-12h28m54-UTC>INFO...-AcSineWaveChSet> dAclock - coswT0 -0.84264 sinwT0 -0.538477 - lCoswDt 1 lSinwDt -3.86416e-05 at Time 1206448153s - Fast
2018-03-30-12h28m54-UTC>INFO...-AcSineWaveChSet> d6MHz - coswT0 -0.317199 sinwT0 -0.948359 - lCoswDt 0.999991 lSinwDt 0.0043027 at Time 1206448153s - Fast
2018-03-30-12h28m54-UTC>INFO...-AcSineWaveChSet> d8MHz - coswT0 0.691585 sinwT0 0.722295 - lCoswDt 0.999984 lSinwDt 0.00573692 at Time 1206448153s - Fast
2018-03-30-12h28m54-UTC>INFO...-AcSineWaveChSet> d12MHz - coswT0 0.325158 sinwT0 0.94566 - lCoswDt 1 lSinwDt 0.000822659 at Time 1206448153s - Fast
2018-03-30-12h28m54-UTC>INFO...-AcSineWaveChSet> d18MHz - coswT0 0.508986 sinwT0 -0.860775 - lCoswDt 0.999998 lSinwDt 0.00198349 at Time 1206448153s - Fast
2018-03-30-12h28m54-UTC>INFO...-AcSineWaveChSet> d56MHz - coswT0 -0.854599 sinwT0 -0.519288 - lCoswDt 1 lSinwDt 0.000416745 at Time 1206448153s - Fast
2018-03-30-12h28m54-UTC>INFO...-AcSineWaveChSet> d112MHz - coswT0 0.168376 sinwT0 -0.985723 - lCoswDt 1 lSinwDt 0.000833491 at Time 1206448153s - Fast
2018-03-30-12h28m54-UTC>INFO...-AcEngStateSet> state 3(4) - nload 7/124(0) - nLoadFake 117 - cntLoad 31
2018-03-30-12h28m54-UTC>INFO...-AcEngStateSet> Load(3)
2018-03-30-12h28m54-UTC>INFO...-AcCompGpsLoadSet> GPS1206448153/0 - stsGps 1
2018-03-30-12h28m54-UTC>INFO...-AcRootParamSet> done at GPS1206448152 without cmd - nLoad 0, nApply 0 received from Cm - Load expected at GPS1206448153/1206448152+1 , rtn Ok - setTime 0.000391s, retry 0

Not sure if this is caused by a Cm message or by something else. The only human intervention around that time is a restart of the EPRB_PC process 5 seconds earlier, but this seems unrelated. The full merged log messages of all processes during 20 seconds around the event is attached. To be better understood by the DAQ experts.
Images attached to this comment
Non-image files attached to this comment
swinkels - 20:26 Friday 30 March 2018 (40966) Print this report
Digging further, it seems that a phase jump in the phase of the LNFS at the same moment that the phase-camera process is restarted was not an isolated case, a similar coincidence happened around 2018-03-28-14h19m18-UTC. The EPRB_PC process was restarted a few times, the first of which triggers a jump in the phase, see figure 1. In another case, the phase jumps during the night in coincidence with some missing samples and a jump in the CfgChange counters, see fig 2.

Note that the EPRB_PC and LNFS_Demod processes run both on rtpc6. My guess is that for some reason the real-time processes are overloaded, which triggers a reset of the sine generator used for the phase derotation in LNFS_Demod, which changes the demodulation phase of all photodiodes ...

It would be good to understand this issue, but one possible way to avoid this problem entirely is to remove the phase derotation step, and directly use the measured phase of the LNFS (which then rotates by the frequency difference between the nominal and actual demod frequency) to derotate the signals of the photodiodes.
Images attached to this comment
Search Help
×

Warning

Error

The present report has been modified outside this window. Please check for its integrity in the main page.

Refreshing this page will move this report into drafts.

×