Reports 1-1 of 1 Clear search Modify search
AdV-DAQ (Data collection)
verkindt - 14:22 Wednesday 19 February 2025 (66218) Print this report
Stop of HrecN and reduction of raw data stream

Today, around 13:12 UTC, I have definitely stopped the process HrecN which was providing h(t) computed using FFTs of 4s instead of 8s.
This process provided, since the start of O4b, an h(t) with a BNS range approximately 1 Mpc lower than the BNS range of the h(t) provided by Hrec.
In addition, it provided several channels V1:HrecN* which were redundant with V1:Hrec* channels.

I took this occasion to investigate what we would gain in the raw data buffer if we removed the HrecN channels from the raw data frames:
the answer is 9 TB for one year.
According to Nicolas's presentation at today's commissioning meeting, this action would give about 3 additional days to the raw data buffer.

There could be more days to gain if we would excise more useless channels from the raw data storage, if we review the raw data content
and find a list of channels that could be excised. Up to concerned responsibles to decide if such a (big) work would be valuable.
As a first information, I did a non-exhaustive list of the channel families stored in the raw data , with their data flow:

V1:Sc_*                               : 8.47 MB/s --> 255 TB/year
V1:ENV_*                            : 3.38 MB/s --> 102 TB/year
V1:SW* V1:SN* V1:SUSP* : 2.91 MB/s --> 87 TB/year
V1:*CAL* V1:*PCAL*          : 1.42 MB/s --> 42 TB/year
V1:LSC_*                            : 1.22 MB/s --> 37 TB/year
V1:ASC*                             : 0.91 MB/s --> 29 TB/year
V1:Sa_*                              : 0.57 MB/s --> 17 TB/year
V1:Hrec_*                           : 0.53 MB/s --> 16 TB/year
V1:HrecN_*                        : 0.33 MB/s --> 9 TB/year
V1:DAQ*                             : 0.3 MB/s  --> 8 TB/year
V1:NCal*                             : 0.3 MB/s  --> 8 TB/year

Could there be some redundancy to track in the Susp, Env or Cal channels
(which represent about 500 TB per year over the total of 1323 TB of raw data per year) ?

Of course, this list being non-exhaustive, there are other families of channels where we could track redundancy or usefulness.
This is a big work that Alain Masserot already did successfully before the start of O4b and that could be continued if we decide it is useful to go further.

Comments to this report:
masserot - 10:49 Thursday 20 February 2025 (66224) Print this report

Below the detail for a 1s frame  , expressed in MB/s,  for the RAW stream and the RAW_FUL stream  the GPS1423958000_2025-02-18-23h53m02-UTC

Under the directory, /virgoData/DAQ/DataFlow],

  • one can find a script called TreatFrameSize.sh
  • this command coud be launched  with the following syntatx ./TreatFrameSize.sh -gps <gpstime of a frame file>  -ffl "raw full" -full where
    • -ffl "raw full":  means to analyse the frame files contents for the raw frame and the raw_full frame
    • -full : means to analyse all the defined sub-systems , to display the ordered channel size  and to store the result in a file called /virgoData/DAQ/DataFlow/GPS-<gps>
Images attached to this comment
Search Help
×

Warning

×