The loops of all suspensions (including IB, MC and SDB1) are now open (DC only) for safety reasons.

The ISC and the INJ subsytems were already in a safe state thanks to the operations done on the Automation; in addition:

• the main ENABLE flags for LSC and ASC have been turned off;
• I put in pause INJ_MAIN, all the ISC nodes, the ALS nodes and the SUS nodes to avoid issues during the recovery and the flooding of logfiles.

Below the ENEL(IPS) and UPS current and voltage monitoring channels time plots related to the time slot of the Timing issue :

• The timing issue seem occurred when the IPS came back  but at the same time  one of the UPS phases the ENV_CEB_UPS_VOLT_S  shutdowned to zero (zoom) . Maybe there is not only the Timing rack not powered
• it s strange to sse that the UPS MC current went to zero during IPS shutdown

• The timing issue seem occurred when the IPS came back

Just to add a bit of information: this is not the ENEL IPS, but our "Diesel engines" generated IPS. The ENEL IPS return after more than an hour (see attached plot for event timing)

The CEB "S" UPS went to zero when starting the Diesel generator. This needs to be better understood (there is no a good reason).

• it s strange to sse that the UPS MC current went to zero during IPS shutdown

This could be explained if the MCB UPS Current sensors electronics is powered with the IPS mains. To be checked.

Franscesco found the rack15  magnetotermic switch in down position. He rearmed  it and all the rack15 devices restarted correctly .

We wait for the GPS receivers synchronisation and then we restarted the main and the spare Timing mezzanine .

Du to the long timing system shutdown we had to reconfigure all the Tolm devices (DaqBox, ADC7674 and Tolm) .  It toke 2 hours to recover  everything

The same thing has to be done for the PISA DSP boards .

At the end  the SBE and LC loops were closed successfully  for:  SPRB, SNEB, SWEB and SDB2. It failed for the SDB1 LC one maybe because the SDB1 IP is not closed

The same timimg recovery operation has to be done in the VirgoRD VPM domain
 

The timing recovery is done on the  VirgoRD VPM domain

We had to reconfigure the CEB_DBOX_LNFS , the SIB2, SPRB and SDB2 Daq boxes to have theirs timing errors at zero

These operations were done between 12h36UTC and 13h06UTC  today