Reports 1-1 of 1 Clear search Modify search
AdV-DAQ (Data collection)
cortese - 16:56 Monday 21 October 2024 (65363) Print this report
Comment to DAQ : latency jumps since the 2024-10-14-16h57m-UTC (65361)

The DAQ glitches can be explain by the fact that since 4 October the input flux to the fs01 nfs fileserver has increased again after the pause of September.

We have understood that the FdIOServer thread that sends frames via Cm is the same that writes logs on /virgoLog  (served by fs01) and therefore an overload of fs01 causes the whole thread to lag.

 

The increase of the flux (to some of the /virgoLog, /virgoData, /olusers or /virgoDev filesystems) happened again in correspondence of a restart of some virgo processes triggered by the unavailablity of the /data/archive area on 4 October.

From an analysis of the traffic on fs01 it turns out that the major writers are currently olserver119 and olserver114 in this order.

I see that on both server run some Fm processes which as far as I understand write on /virgoData , which stays indeed on fs01 , so explaining at least part of the overload.

It is not possible from the OS side to measure which of the processes on those 2 servers have a role because of the risk of impacting on the performances.

It should also be explained why the writing flux had been lower during all September.

Images attached to this comment
Search Help
×

Warning

Error

The present report has been modified outside this window. Please check for its integrity in the main page.

Refreshing this page will move this report into drafts.

×