As mentioned in this operator report on Friday 15th we were forced to switch off the air conditioning system of the INJ area due to a failure.
Here we report a few details on the failure and a check of related side effects.
The sequence of the failure is the following (Fig1 and zoom in Fig2, red plots):
- on Thursday 14th at 14:15 UTC a pressure sensor broke down (channel INJ_PRES_OUT) and its value dropped to 0;
- this triggered the correction of the pressure loop to increase from ~10% to the maximum value (100%);
- this required the increase of the output fan speed (channel INJ_FREQ_OUT) from ~22Hz to 60Hz to increase the air flux (channel INJ_FLUX_OUT) and pressure;
- this stressed the air ducts that started to leak along a few joints;
- the maximum speed was enough to compensate the leakages and keep the system at the nominal temperatures, so the failure was not immediately spotted;
- the day after (Fri 15th) the problem was detected due to the fact that the leakages and the maximum speed were producing disturbances to the operation of the INJ system.
- during the morning of Fri 15th a temporary repair (also with an external firm) of the most evident leaks was attempted, but not sufficient because other leaks are present further down on the air ducts, some of them in areas not easily accessible. Once it appeared evident that the repair is complicated and time consuming, on the late morning the system was switched off (because it was producing too many IMC unlocks).
- on Friday afternoon a temporary solution was adopted to cope with this issue: at 15:30 UTC the INJ lab was put in communication (by opening a door) with the CEB Clean Rooms to take advantage of its air conditioning system that is independent and in operation. We are still in this configuration that seems enough to avoid downtimes and ensure the scheduled commissioning activities.
Looking at Figs 1 and 2 it's evident that the stress generated on the air ducts by staying at fans maximum speed it's not to be considered anomalous because when in nominal flux condition (channel INJ_MODE_SET value set from 2 to 1) the speed of the fan is comparable. In Fig3 it can be seen that the fan speed, when in nominal state, is at ~50Hz (sept 2016) or ~55Hz (sept2015), not much lower with respect to the maximum one (60Hz), occurred at the time of the failure.
We think that this accident has simply accelerated a latent problem on the ducts that need to be fully investigated.
In Figs 4 and 5 are reported trends of a few signals were it is possible to see that the temporary solution above described is ensuring acceptable values of temperature fluctuations (although around another working point), humidity, dust contamination (Fig5) and environmental noise. In some frequency ranges the noise is a bit higher, in others (example in Figs 6 and 7) it seems even lower.
Concerning the dust contamination we have to check the situation when people are working in the laser lab; however if the workaround solution has to last for several days, we can bring a portable air blower (currently in TCS room) in the laser lab and switch on it when persons are working there.
Futher checks and inspection of the ducts are planned for the next Tuesday morning maintenance .