An optical density of about OD = 0.5 has been removed around 10:10 UTC before the newly installed RFC_TRA photodiode. The digital gain has been adjusted from -3.8 to -1.5 to roughly match the previous output and be compliant with the existing lock acquisition thresholds).

New Fbs release v8r16 - previous v8r15p4

This new release is now deployed  on all the Fbs servers: FbsMoni, FbsDet, FbsSusp, FbsISC, FbsVac, FbsAlp and FbsStorage.

A new parameter called "optional flag" and set as the last parameter of the SMS keyword has been defined  and means:

• if the server exist , to  try to collect its SMS data . The server state will be "golden" if the collection run as expected otherwise the server state will be "active"
• if the server does'nt exist,  It  will not be taken in account to define the server state

Using this new feature, the Fbs configurations have been upgraded  by adding the some stopped servers with the optional flag set to 1

SPRB_Photodiode server

For debugging, I added the computation of the B4_{PD1,PD2}_56MHz_mag in the SPRB_Photodiodes.cfg (lines 178, 368) These new channels are stored in  the DAQ with the  "_FS" suffix

Figure 1 shows the performance of the new photodiode. In purple is a time with no power on the photodiode, in brown with the RFC locked, and in blue with the RFC locked before photodiode change. The data between the two photodiodes is comparable because a digital gain of 3.8 was put in to have the same recorded DC voltage in the DAQ for the two photodiodes.

For the electronic noise, the ADC noise is nominally at 2.5e-7 V/rtHz, unshaping the signal lowers it by a factor 20 and applying the gain 3.8 increases it. So in total it is expected to be at 2.5e-7/20*3.8 = 4.8e-8 V/rtHz, slightly lower than what is measured at 6e-8 V/rtHz.

With light present the shot noise is at 1.2e-V/rtHz, which corresponds to a RIN of 2e-8 1/rtHz. This is much higher than I expected, and would be consistant with only 1mW detected by the photodiode instead of ~20mW. The explanation for it is two fold:

• There is a neutral density filter in front of the photodiode, the absorption is not known, but given that the voltage output is a factor 4 lower than expected, we can guess it absorbs 75% of the power. The neutral density filter will be removed to increase the power, it should also lower the coupling of beam jitter, as neutral density filters create large couplings of beam jitter due to non uniform absoption, and absorption aging due to the beam passing through them.
• The photodiode quantum efficiency is rather poor, with a responsitivity of 0.33 A/W whereas a perfect photodiode would have a responsitivity of 0.85 A/W, so the efficiency is only ~40%. During the discussion on the photodiode choice, it was known that the responsitivty is at 0.33 A/W, but I hadn't realized what is the meaning of that quantity, and that it translate to a low quantum efficiency.

This two combined could explain why the shot noise is a factor 4 higher than I expected.

By looking at the online noise budget, the current sensitivity seems spoiled by SIB1 scattering noise, probably due to the reported increased noise on the F0 of the related suspension. 

Upon arrival I found ITF in Science Mode with BNS Range 44 Mpc. The ITF kept the lock for the whole shift but the BNS Range decreased from 44 to 36 Mpc. The sensitivity curve is affected by a very high noise in particular in the low frequencies region (but not only) and I'm not sure that it depends from the bad weather conditions. The sea activity was high but not like in the past days (plot1); periods with better sensitivity (plot2).

05:30 UTC Calibration Mode set.

Guard Tour (UTC)

• 23:45 - 00:21
• 01:28 - 02:01
• 03:32 - 04:04
• 04:55 - 05:25

Calibration
Weekly Magnetic Injections started at 05:30 UTC under request of Paoletti, Fiori.

SUSP
During the night all Susp IP DMS flags became yellow with high oscillation of F0 along X and Z DOF (all X and Z loops are not working properly) the values of those oscillations grows by a factor 3 in ~all suspensions. In addition of that, it should be important to understand why there was a significative misalignments of F0 along Z for NE, PR, MC, BS, WI and F0 along X for WE, PR (plot3). To be investigate with Susp experts asap.

Other
(18-11-2019 14:00 - ) In case of unlock:
- call Valerio to reset the MC FO coils and communicate the situation of all susp along X and Z DOF.

ITF locked for all the shift, Hrec_Range_BNS ~ 46 Mpc. Adjusting Mode from 15:53 to 16:18  UTC due to the Neovan head's chiller failure.

ISYS
(18-11-2019 15:50 - 18-11-2019 16:18) Neovan head's chiller failure.

Other
(18-11-2019 14:00 - ) In case of unlock:
- call Valerio to reset the MC FO coils

This afternoon at 15:50 UTC we received an alert because the temperature of the neovan head was increasing.
We went there and found that, as saturday night, the chiller had a failure. Since the situation was not critical yet, we took time to analyze what was happening.
The water was still flowing into the tube (as it can be also seen on the flow monitor) but was not cooled down anymore. The failure seems to come from the electronics control of the chiller. 
We switched it on and off and it came back to a normal situation (see plot 1).

We took advantage of being there and having the interferometer in troubleshooting to bring the power at the input of the interferometer back to 26 W. The power was indeed slowly increasing since the intervention of Friday. It comes from the PMC that, for once, is getting a better transmission (see plot 2).

Since the 20191110 there is a lot of fluctuation on the latency of FbmAlp. These fluctuations are mainly due to the Metatron server SUS_BS (see the attached plots).

Maybe the situation could be improved by the Metatron/Tango Experts

The Sc_NI event saw by the DAQ

2019-11-18-13h38m03-UTC>INFO...-[TolmFrameBuilderSourceManager::Clean] Source Sc_NI (version 6) has stopped -> removed
2019-11-18-13h38m08-UTC>INFO...-[TolmFrameBuilderSource::AfterPageLoading] new source Sc_NI (v6) fully activated at (corrected) GPS time: 1258119506.953013500

The failure happened twice. The first time it did not work; the second it worked. The transient is not so smooth, but the logic can be improved.

ITF in Science mode for all the shift, Hrec_Range_BNS oscillataing around 45.5 Mpc.

SUSP
At 13:38 UTC we had an event of NI marionette control failure. This time the ITF did not unlock; the consequence was a drop in Hrec_Range (see attached plot).

Other
(18-11-2019 14:00 - ) In case of unlock:
- call Valerio to reset the MC FO coils
- call Antonino to enter in "Piscina"

The sensor used to monitor the flow of the chiller is installed on the supply line, immediately after the chiller output.

It is an hall sensor flowmeter produced by B.I.O.-TECH e.k. (serie DFM-Ms, model 150175) and the output signal is a square wave 80imp./L --> 1.33Hz/(liters/min).

In attachment the plot of the signal immediately after the Friday installation that measured a flow of about 3.6 liters/min.

A new commit of the DSP Code has been performed (revision 1359) and tagged as 'O3_14'. This release contains some minor changes to the code (Sc_NI patch described here).

The RFC-Tra monitor PD installed has been designed to be shot noise limited between 10 Hz and 1 kHz both with the current level of impinging power (estimated to be around 20 mW) and the future one (~40 mW).
The sensor used is a OSI Optoelectronics' PIN-6DI, whose Responsivity at 1064 nm is ~0.33A/W.
According to simulations, electronics noise is below shot noise by a factor that goes from 2 to 8 depending on power level and frequency.
The transimpedance is 666 ohm (to stay below ADC saturation, 10 V).
To make the shot noise level readable by the ADC used, whose noise floor is ~250nV/sqrt(Hz), a shaping filter having 1 zero at 1 Hz and a pole a 20 Hz has been added and both the design and the measured transfer function are visible in fig.1
The PD noise in dark conditions has been measured as well (fig.2)

It looks like the stabilization of the temperature of the neoVAN head has been beneficial also to the stability of the system (see attached plot). Currently, the PMC seems to keep its alignment as witnessed by the fact that there are no PSTAB huge glitch (second plot) if we exclude unlock periods.

The power sent to the ITF seems to have a slow increasing drift (current value around 26.7W) so it is perhaps worth adjusting it to the initial value of 26W at the first occasion.

Here the ISC_Fb report related to Sc_NI missing data:

2019-11-18-02h18m02-UTC>INFO...-[TolmFrameBuilderSourceManager::Clean] Source Sc_NI (version 6) has stopped -> removed
2019-11-18-02h18m08-UTC>INFO...-[TolmFrameBuilderSource::AfterPageLoading] new source Sc_NI (v6) fully activated at (corrected) GPS time: 1258078706.208513570

ITF found in Science Mode. It unlocked at 2:18 UTC due to NI marionette control failure ( see attached plot #1 ). Back in Science Mode from 3:14 UTC.
ITF Left locked.

Other
- Water Level of Scolmatore slowly started to decrease during the night ( see attached image #2 );
- Strong Sea Activity during the Shift ( see attached plot #3).

14:00 UTC ITF in Science Mode with BNS Range ~48 Mpc. ITF kept the Science Mode for the whole shift. BNS Range decreased due to the sea activity.

GRB
15:18 UTC - [gracedb] Fermi event created: E354868. Alert reset at 15:39 UTC

Guard Tour (UTC)

• 14:10 - 14:42
• 16:00 - 16:34

SQZ
SQZ disingaged from 18:35 to 1836 UTC.

Today at 14:00 UTC the authority in charge have opened the floodgates of the Scolmatore to mitigate the flow of the Arno river towords Pisa. Consequently in very few hours the level of the Scolmatore has grown of about 2.5 mt (figures and plots attached).
During the shift we looked at the level of the Scolmatore in front of the EGO site. At around 17:30 UTC the level was at about 2 mt below the banks, which is the same level of the road. At the consecutive checks, at 19:30 and 22:00 UTC, the level was essentially the same.
So, for the moment we don't consider the situation particularly warrying.
The situation is monitored by the operator in shift.

The issue that Benoit reports is that h(t) is corrupted 4 seconds before the bit 0 of the state vector goes down 12:54:18 UTC, h(t) is already bad at 12:54:14 UTC. The reason is that h(t) is not a time domain code, but a frequency domain one done in blocks of 8 seconds. If any data input data in that 8 seconds is not good, then all 8 seconds of h(t) are corrupted. So h(t) is not only bad after the unlock, but also before the unlock, because it doesn't know about the unlock. There are two solutions:

• Use better information in Hrec_hoft for knowing when an unlock happens. Hrec can't use the state vector as that is computed further down the DAQ chain. So probably additional checks would need to be implemented into Hrec.
• Acknowledge that Hrec can be corrupted in the last 8 seconds before an unlock, and remove the last 8 seconds instead of 5 seconds before an unlock using a CAT1 flag that is accessible to online pipelines (such as MBTA).

ITF stable locked for all the shift, BNS Range increased during the shift up to ~48 Mpc. I set the Adjusting Mode and I switched the ITF in the "strong cfg"  from 9:49 to 10: 25 UTC in order to prevent an unlock due to temporary strong worsening in weather conditions (strong wind).

Guard Tour (UTC)

• 06:15 - 06:44
• 08:10 - 08:44
• 10:24 - 10:53
• 12:15 - 12:45

As Michal said, the Metatron latency due to the need to pass on information at 1 Hz in between nodes is well-known. This is dealt with offline by the CAT1 flag V1:DQ_SCIENCE_FALSE that checks for inconsistencies between ITF_LOCK and ITF_Mode == 1 (SCIENCE) -- and also by V1:DQ_LOCK_LAST_05S that removes the last 5 seconds of each lock segment. I'll double-check tomorrow if adding the min values of LSC_ENABLE and SDB1_fast_shutter_sw removes more data.

That being said, I don't quite understand the problem reported by MBTA: from https://logbook.virgo-gw.eu/virgo/uploads/47687_1573925680_Unlock-Nov14-12h54.png, the bit 0 of the state vector goes down at 12:54:18 UTC, meaning that from that time onwards, h(t) is not correctly computed. So these data are bad. Pipelines should use data when bits 0, 1 and 10 are active.

META_ITF_LOCK_index should not be used to flag the status of the interferometer with second precision. Metatron runs is a slow process, with approximately 1 second long refresh time (but not constant). META_ITF_LOCK_index reports when Metatron learned that the interferometer has unlocked, which will always be 1 or 2 seconds after the fact.

Better channels to use to learn quickly that the ITF has unlocked are

• LSC_ENABLE - this channels is equal to 1 the interferometre is controlled, and drops to 0 when the real time control gives up on controlling the interferometer (switches off the feedback to the mirrors) after an unlock
• SDB1_fast_shutter_sw - this channels is at 1 when the shutter blocks the beam at the output of the interferometer and at 0 when the shutter is open, so whenever it is at 1 for certain there is no GW signal to look at. In some cases it is an earlier warning of the unlock (by a fraction of a second)

See figure 1 for an example.