Reports 1-1 of 1 Clear search Modify search
AdV-SAT (Suspension control upgrade)
Boschi, Gennai, Ruggi - 12:37 Tuesday 01 October 2019 (47056) Print this report
Investigation of NI issues

Yesterday suspension activities were concentrated on the investigation of the communication issue between NI and DAQ. Many actions have been performed:

  • NI PSDf board (42114)  was replaced with a spare (42097) but the problems were still present
  • The transceiver connected to the board was then replaced but the issue was not solved.
  • Finally NI board was replaced with the master board installed at Sc_PR (43148), that now uses 42114 board.

This last test apparently solved the problem. 

Comments to this report:
Boschi, Masserot - 18:04 Monday 07 October 2019 (47130) Print this report

Since the Sc_NI DAQ communication problem reappeared a few days ago, we decided to make a test: since 12:23 LT, we are now sending data from Sc_NI to a different frame builder, ISC_Fb, that receives a much lower data flow.

Boschi, Gennai, Ruggi - 18:45 Tuesday 12 November 2019 (47622) Print this report

Today several tests have been made on Sc_NI DSP crate:

  • MCH was replaced with a spare.
  • DSP Communication on RapidIO bus and with DAQ have been tested with dedicated diagnostic code.
  • Timing signal between Sa_NI and Sc_NI has been swapped.

None of tests was conclusive. Further investigation is needed.

rolland - 20:30 Tuesday 12 November 2019 (47623) Print this report

Probably since this intervention, the timing of NI DSP is off by 50 µs, which has generated a red flag in the DMS in CalNE. The figure shows the coherence, modulus and phase for Sc_NE,NI,WI_MIR_Z_CORR/CAL_NE,NI,WI_MIR_Z_CORR at the permanent calibration line sent at 2012 Hz. They are all stable (when ITF is locked), except the phase of NI which has shifted by 0.62 rad around 16h UTC (see right plot of second line).

This is not an issue for h(t) reconstruction since NI is not used for controls in Science. However, this is an issue for calibration of the actuator, and this should be fixed (reconfiguring the timing of the NI DSP) before tomorrow's calibration.

Images attached to this comment
masserot - 9:33 Wednesday 13 November 2019 (47626) Print this report

After the Timing cable swap between Sa_NI and Sc_NI, Here the report of the missing data see by the DAQ

Sa_NI(SUSP_Fb)

  • 1st event

    • 2019-11-12-21h37m08-UTC>INFO...-[TolmFrameBuilderSourceManager::Clean] Source Sa_NI (version 6) has stopped -> removed

    • 2019-11-12-21h37m14-UTC>INFO...-[TolmFrameBuilderSource::AfterPageLoading] new source Sa_NI (v6) fully activated at (corrected) GPS time: 1257629852.750406000

  • 2nd event

    • 2019-11-13-05h57m30-UTC>INFO...-[TolmFrameBuilderSourceManager::Clean] Source Sa_NI (version 6) has stopped -> removed

    • 2019-11-13-05h57m35-UTC>INFO...-[TolmFrameBuilderSource::AfterPageLoading] new source Sa_NI (v6) fully activated at (corrected) GPS time: 1257659873.437305980

Sc_NI(ISC_Fb):

  • 1st event

    • 2019-11-12-20h44m42-UTC>INFO...-[TolmFrameBuilderSourceManager::Clean] Source Sc_NI (version 6) has stopped -> removed

    • 2019-11-12-20h44m48-UTC>INFO...-[TolmFrameBuilderSource::AfterPageLoading] new source Sc_NI (v6) fully activated at (corrected) GPS time: 1257626706.708678970

  • 2nd event in coincidence with  ITF unlock

    • 2019-11-12-22h58m51-UTC>INFO...-[TolmFrameBuilderSourceManager::Clean] Source Sc_NI (version 6) has stopped -> removed

    • 2019-11-12-22h58m56-UTC>INFO...-[TolmFrameBuilderSource::AfterPageLoading] new source Sc_NI (v6) fully activated at (corrected) GPS time: 1257634754.930479000

  • 3rd event in coincidence with ITF unlock

    • 2019-11-13-01h08m52-UTC>INFO...-[TolmFrameBuilderSourceManager::Clean] Source Sc_NI (version 6) has stopped -> removed

    • 2019-11-13-01h08m56-UTC>INFO...-[TolmFrameBuilderSource::AfterPageLoading] new source Sc_NI (v6) fully activated at (corrected) GPS time: 1257642554.427278940

  • 4th event

    • 2019-11-13-01h19m50-UTC>INFO...-[TolmFrameBuilderSourceManager::Clean] Source Sc_NI (version 6) has stopped -> removed

    • 2019-11-13-01h19m55-UTC>INFO...-[TolmFrameBuilderSource::AfterPageLoading] new source Sc_NI (v6) fully activated at (corrected) GPS time: 1257643213.536378710

  • 5th event

    • 2019-11-13-01h37m07-UTC>INFO...-[TolmFrameBuilderSourceManager::Clean] Source Sc_NI (version 6) has stopped -> removed

    • 2019-11-13-01h37m13-UTC>INFO...-[TolmFrameBuilderSource::AfterPageLoading] new source Sc_NI (v6) fully activated at (corrected) GPS time: 1257644251.946678530

  • 6th event in coincidence with a break of the ITF lock sequence

    • 2019-11-13-01h41m19-UTC>INFO...-[TolmFrameBuilderSourceManager::Clean] Source Sc_NI (version 6) has stopped -> removed

    • 2019-11-13-01h41m24-UTC>INFO...-[TolmFrameBuilderSource::AfterPageLoading] new source Sc_NI (v6) fully activated at (corrected) GPS time: 1257644502.739578740

  • 7th event in coincidence with ITF unlock

    • 2019-11-13-03h56m37-UTC>INFO...-[TolmFrameBuilderSourceManager::Clean] Source Sc_NI (version 6) has stopped -> removed

    • 2019-11-13-03h56m43-UTC>INFO...-[TolmFrameBuilderSource::AfterPageLoading] new source Sc_NI (v6) fully activated at (corrected) GPS time: 1257652621.451279010

  • 8th event in coincidence with a break of the ITF lock sequence

    • 2019-11-13-04h15m58-UTC>INFO...-[TolmFrameBuilderSourceManager::Clean] Source Sc_NI (version 6) has stopped -> removed

    • 2019-11-13-04h16m04-UTC>INFO...-[TolmFrameBuilderSource::AfterPageLoading] new source Sc_NI (v6) fully activated at (corrected) GPS time: 1257653782.178278710

  • 9th event

    • 2019-11-13-06h37m52-UTC>INFO...-[TolmFrameBuilderSourceManager::Clean] Source Sc_NI (version 6) has stopped -> removed

    • 2019-11-13-06h37m58-UTC>INFO...-[TolmFrameBuilderSource::AfterPageLoading] new source Sc_NI (v6) fully activated at (corrected) GPS time: 1257662296.718378730

For memory the Timing cable must be an Ethernet type cable category 5e at least or 6 with the shielded connector

Boschi - 15:47 Wednesday 13 November 2019 (47632) Print this report

This morning at 10:03 UTC Sc_NI crate has been rebooted in order to resynchronize master board with the timing signal.

masserot - 9:37 Thursday 14 November 2019 (47643) Print this report

Here the Sc_NI troubles at  the DAQ level

This one cause the ITF unlock

2019-11-14-01h58m49-UTC>INFO...-[TolmFrameBuilderSourceManager::Clean] Source Sc_NI (version 6) has stopped -> removed
2019-11-14-01h58m55-UTC>INFO...-[TolmFrameBuilderSource::AfterPageLoading] new source Sc_NI (v6) fully activated at (corrected) GPS time: 1257731953.125913510


2019-11-14-02h08m53-UTC>INFO...-[TolmFrameBuilderSourceManager::Clean] Source Sc_NI (version 6) has stopped -> removed
2019-11-14-02h08m57-UTC>INFO...-[TolmFrameBuilderSource::AfterPageLoading] new source Sc_NI (v6) fully activated at (corrected) GPS time: 1257732555.920612710


2019-11-14-03h56m06-UTC>INFO...-[TolmFrameBuilderSourceManager::Clean] Source Sc_NI (version 6) has stopped -> removed
2019-11-14-03h56m11-UTC>INFO...-[TolmFrameBuilderSource::AfterPageLoading] new source Sc_NI (v6) fully activated at (corrected) GPS time: 1257738989.552613610


2019-11-14-04h05m28-UTC>INFO...-[TolmFrameBuilderSourceManager::Clean] Source Sc_NI (version 6) has stopped -> removed
2019-11-14-04h05m33-UTC>INFO...-[TolmFrameBuilderSource::AfterPageLoading] new source Sc_NI (v6) fully activated at (corrected) GPS time: 1257739551.228513700

None for the Sa_NI DSP

Boschi - 14:21 Tuesday 19 November 2019 (47717) Print this report

This morning the timing cable connected to Sc_NI crate has been replaced with a CAT6 shielded Ethernet cable. Previous cable was unshielded CAT5e.

Boschi - 11:30 Tuesday 26 November 2019 (47793) Print this report

This morning the transceivers of Sa_NI and Sc_NI have been swapped, on the Mux/DeMux side, as suggested by A. Masserot.

Search Help
×

Warning

Error

The present report has been modified outside this window. Please check for its integrity in the main page.

Refreshing this page will move this report into drafts.

×