Reports 1-1 of 1 Clear search Modify search
AdV-DAQ (Data collection)
masserot, mours - 11:39 Friday 29 March 2024 (63783) Print this report
20 second missing data period, from GPS1395725958 to GPS1395725978

Today at 05h39m-UTC

  • all the TolmFrameBuidler servers reports that they were unable to transmit theirs frames built at 5Hz  to FbmFFE frame mergers
    • ALS_CEB_Fb: 2024-03-29-05h39m01-UTC>ERROR..-frame 1395725958 not send to FbmFFE: sending queue is full
    • ALS_NEB_Fb: 2024-03-29-05h39m04-UTC>ERROR..-frame 1395725961 not send to FbmFFE: sending queue is full
    • ALS_WEB_Fb: :2024-03-29-05h39m04-UTC>ERROR..-frame 1395725961 not send to FbmFFE: sending queue is full
    • INJ_Fb: 2024-03-29-05h39m03-UTC>ERROR..-frame 1395725961 not send to FbmFFE: sending queue is full
    • ISC_Fb: 2024-03-29-05h39m03-UTC>ERROR..-frame 1395725961 not send to FbmFFE: sending queue is full
    • SDB2_Fb: 2024-03-29-05h39m03-UTC>ERROR..-frame 1395725961 not send to FbmFFE: sending queue is full
    • SDB_EDB_Fb: 2024-03-29-05h39m03-UTC>ERROR..-frame 1395725961 not send to FbmFFE: sending queue is full
    • SIB2_Fb: 2024-03-29-05h39m03-UTC>ERROR..-frame 1395725961 not send to FbmFFE: sending queue is full
    • SNEB_Fb: 2024-03-29-05h39m03-UTC>ERROR..-frame 1395725961 not send to FbmFFE: sending queue is full
    • SPRB_Fb: 2024-03-29-05h39m03-UTC>ERROR..-frame 1395725961 not send to FbmFFE: sending queue is full
    • SQB1_Fb: :2024-03-29-05h39m03-UTC>ERROR..-frame 1395725961 not send to FbmFFE: sending queue is full
    • SQB2_Fb: 2024-03-29-05h39m05-UTC>ERROR..-frame 1395725958 not send to FbmFFE: sending queue is full
    • SQZ_Fb: 2024-03-29-05h39m01-UTC>ERROR..-frame 1395725958 not send to FbmFFE: sending queue is full
    • SSFS_Fb: 2024-03-29-05h39m04-UTC>ERROR..-frame 1395725961 not send to FbmFFE: sending queue is full
    • SUSP_Fb: 2024-03-29-05h39m03-UTC>ERROR..-frame 1395725961 not send to FbmFFE: sending queue is full
    • SWEB_Fb: 2024-03-29-05h39m03-UTC>ERROR..-frame 1395725961 not send to FbmFFE: sending queue is full
    • and that they flushed their queues of 20 frames with a dt of 0.2 s : meaning that 10s of date have been lost
      • output queue to FbmFFE has been flushed; Sending frame 1395725968
  • All the Front End DAQ  *_50Hz servers reports
    • 2024-03-29-05h39m16-UTC>WARNING-FdIOPutFrame: miss 11.8 seconds between 1395725961.8 and 1395725973.6
  •  The FDS_Img_Fb and Imaging servers reports that they were unable to transmit theirs frames built at 1Hz to FbmFE frame mergers
    • FDS_Img_Fb: 2024-03-29-05h39m02-UTC>ERROR..-frame 1395725959 not send to FbmFE: sending queue is full
    • DET_Img_CEB: 2024-03-29-05h39m07-UTC>ERROR..-Could not output frame at GPS=1395725965.000 iFr=4 listIn=1241424 0x7fdd3d4774f0
    • DET_Img_End: 2024-03-29-05h39m08-UTC>ERROR..-Could not output frame at GPS=1395725966.000 iFr=7 listIn=1241457 0x7ff338004dc0
    • FDS_Img: 2024-03-29-05h39m08-UTC>ERROR..-Could not output frame at GPS=1395725966.001 iFr=2 listIn=1441382 0x7fdcf8005010
    • INJ_Img: 2024-03-29-05h39m07-UTC>ERROR..-Could not output frame at GPS=1395725965.002 iFr=4 listIn=853654 0x7f8628bcf9f0
    • PAY_Img_50Hz::2024-03-29-05h39m08-UTC>ERROR..-Could not output frame at GPS=1395725966.002 iFr=5 listIn=1441435 0x7f7a40159e00
    • PAY_Img: 2024-03-29-05h39m08-UTC>ERROR..-Could not output frame at GPS=1395725966.002 iFr=4 listIn=1441454 0x7fa00c004680
  • The Frame File monitor servers  reports
    • FmStol01:  20s of missing data

      2024-03-29-05h40m43-UTC>ERROR..-CHANNEL> At Mar29,24-05:39:12 1395725970, V1:SDB2_B1p_PD1_Blended missing 20, V1:SDB2_B1p_PD2_Blended missing 20, V1:SDB2_B1_PD1_Blended missing 20, V1:SDB2_B1_PD2_Blended missing 20, V1:SNEB_B7_DC missing 20, V1:SWEB_B8_DC missing 20, V1:SPRB_B4_56MHz_Q missing 20, V1:SIB2_B2_8MHz_I missing 20, V1:CAL_NE_MIR_Z_NOISE missing 20, V1:CAL_WE_MIR_Z_NOISE missing 20, V1:Sc_BS_MIR_Z_CORR missing 20, V1:Sc_PR_MIR_Z_CORR missing 20, V1:Sc_NI_MIR_Z_CORR missing 20, V1:Sc_WI_MIR_Z_CORR missing 20, V1:Sc_NE_MIR_Z_CORR missing 20, V1:Sc_WE_MIR_Z_CORR missing 20, V1:Sc_BS_MAR_Z_CORR missing 20, V1:Sc_PR_MAR_Z_CORR missing 20, V1:Sc_NI_MAR_Z_CORR missing 20, V1:Sc_WI_MAR_Z_CORR missing 20, V1:Sc_NE_MAR_Z_CORR missing 20, V1:Sc_WE_MAR_Z_CORR missing 20,  are back

  • FmStol02: 20s of missing data

    2024-03-29-05h40m55-UTC>ERROR..-CHANNEL> At Mar29,24-05:39:12 1395725970, V1:SDB2_B1p_PD1_Blended missing 20, V1:SDB2_B1p_PD2_Blended missing 20, V1:SDB2_B1_PD1_Blended missing 20, V1:SDB2_B1_PD2_Blended missing 20, V1:SNEB_B7_DC missing 20, V1:SWEB_B8_DC missing 20, V1:SPRB_B4_56MHz_Q missing 20, V1:SIB2_B2_8MHz_I missing 20, V1:CAL_NE_MIR_Z_NOISE missing 20, V1:CAL_WE_MIR_Z_NOISE missing 20, V1:Sc_BS_MIR_Z_CORR missing 20, V1:Sc_PR_MIR_Z_CORR missing 20, V1:Sc_NI_MIR_Z_CORR missing 20, V1:Sc_WI_MIR_Z_CORR missing 20, V1:Sc_NE_MIR_Z_CORR missing 20, V1:Sc_WE_MIR_Z_CORR missing 20, V1:Sc_BS_MAR_Z_CORR missing 20, V1:Sc_PR_MAR_Z_CORR missing 20, V1:Sc_NI_MAR_Z_CORR missing 20, V1:Sc_WI_MAR_Z_CORR missing 20, V1:Sc_NE_MAR_Z_CORR missing 20, V1:Sc_WE_MAR_Z_CORR missing 20,  are back

 

The origin of the 20 second missing data period, from GPS1395725958 to GPS1395725978, is not obvious: investigations in progress

Comments to this report:
cortese, kraja - 15:10 Friday 29 March 2024 (63786) Print this report

Today from 6:38:34 to 6:38:54 LT the volume of the fs01 fileserver had a temporary problem causing the NFS filesystems /virgoLog, /virgoData, /virgoApp, /virgoDev, /olusers to suffer of an increased latency (up to 5 times in writing).

No errors/warnings have been registered by the rtpcs OS, but evidently the delay has been enough to overflow the DAQ transmit buffers.

The root cause of the volume access glitch is currently not yet understood.

Search Help
×

Warning

×