Blocking code in the indexing pipeline actor?

[fulmicoton]

Tokio-console suggest that the indexing pipeline actor is busy 100% of the time.
We shoudl investigate this.

[fulmicoton]

Could not reproduce.

[fulmicoton]

@PSeitz I tried locally and could not reproduce. Can you cargo clean, and rerun and confirm this is not a glitch?

[PSeitz]

I can still reproduce it

[PSeitz]

I added some logs and everything looks fine there

2021-12-22T07:30:02.615Z  INFO : quickwit_indexing::actors::indexing_pipeline: start initialize
2021-12-22T07:30:02.615Z  INFO :{index=hdfs gen=0}: quickwit_indexing::actors::indexing_pipeline: spawn-indexing-pipeline root_dir=/home/pascal/LinuxData/Development/quickwit-indices/hdfs4/hdfs merge_policy
=StableMultitenantWithTimestampMergePolicy { demux_enabled: false, demux_factor: 8, demux_field_name: None, min_level_num_docs: 100000, merge_enabled: false, merge_factor: 10, max_merge_factor: 12, split_nu
m_docs_target: 10000000 }
2021-12-22T07:30:02.615Z  INFO :{index=hdfs gen=0}:{actor=GarbageCollector}: quickwit_indexing::actors::garbage_collector: garbage-collect-operation
2021-12-22T07:30:02.617Z  INFO : quickwit_indexing::actors::indexing_pipeline: stop initialize
2021-12-22T07:30:02.617Z  INFO :{index=hdfs gen=0}:{actor=Indexer}:{msg_id=2}: tantivy::indexer::segment_updater: save metas
2021-12-22T07:30:02.659Z  INFO :{index=hdfs gen=0}:{actor=Indexer}:{msg_id=2}: quickwit_indexing::actors::indexer: new-split split_id=01FQGGR2QSR4D8Z1145H1H35DP
2021-12-22T07:30:03.617Z  INFO :{msg_id=7}: quickwit_indexing::actors::indexing_pipeline: pipeline process message enter Observe
2021-12-22T07:30:03.621Z  INFO :{msg_id=7}: quickwit_indexing::actors::indexing_pipeline: pipeline process message exit Observe
2021-12-22T07:30:03.621Z  INFO :{msg_id=8}: quickwit_indexing::actors::indexing_pipeline: pipeline process message enter Supervise
2021-12-22T07:30:03.621Z  INFO :{msg_id=8}: quickwit_indexing::actors::indexing_pipeline: pipeline process message exit Supervise
 Num docs  247526 Parse errs     0 PublSplits   0 Input size    87MB Thrghput 43.52MB/s Time 00:00:02
2021-12-22T07:30:04.621Z  INFO :{msg_id=14}: quickwit_indexing::actors::indexing_pipeline: pipeline process message enter Observe
2021-12-22T07:30:04.627Z  INFO :{msg_id=14}: quickwit_indexing::actors::indexing_pipeline: pipeline process message exit Observe
2021-12-22T07:30:04.627Z  INFO :{msg_id=15}: quickwit_indexing::actors::indexing_pipeline: pipeline process message enter Supervise
2021-12-22T07:30:04.627Z  INFO :{msg_id=15}: quickwit_indexing::actors::indexing_pipeline: pipeline process message exit Supervise
 Num docs  499372 Parse errs     0 PublSplits   0 Input size   175MB Thrghput 58.52MB/s Time 00:00:03
2021-12-22T07:30:05.628Z  INFO :{msg_id=21}: quickwit_indexing::actors::indexing_pipeline: pipeline process message enter Observe
2021-12-22T07:30:05.632Z  INFO :{msg_id=21}: quickwit_indexing::actors::indexing_pipeline: pipeline process message exit Observe
2021-12-22T07:30:05.633Z  INFO :{msg_id=22}: quickwit_indexing::actors::indexing_pipeline: pipeline process message enter Supervise
2021-12-22T07:30:05.633Z  INFO :{msg_id=22}: quickwit_indexing::actors::indexing_pipeline: pipeline process message exit Supervise
 Num docs  736732 Parse errs     0 PublSplits   0 Input size   259MB Thrghput 64.77MB/s Time 00:00:04
2021-12-22T07:30:06.633Z  INFO :{msg_id=28}: quickwit_indexing::actors::indexing_pipeline: pipeline process message enter Observe
2021-12-22T07:30:06.639Z  INFO :{msg_id=28}: quickwit_indexing::actors::indexing_pipeline: pipeline process message exit Observe
2021-12-22T07:30:06.639Z  INFO :{msg_id=29}: quickwit_indexing::actors::indexing_pipeline: pipeline process message enter Supervise
2021-12-22T07:30:06.639Z  INFO :{msg_id=29}: quickwit_indexing::actors::indexing_pipeline: pipeline process message exit Supervise
 Num docs  963428 Parse errs     0 PublSplits   0 Input size   338MB Thrghput 67.71MB/s Time 00:00:05
2021-12-22T07:30:07.640Z  INFO :{msg_id=35}: quickwit_indexing::actors::indexing_pipeline: pipeline process message enter Observe
2021-12-22T07:30:07.643Z  INFO :{msg_id=35}: quickwit_indexing::actors::indexing_pipeline: pipeline process message exit Observe
2021-12-22T07:30:07.643Z  INFO :{msg_id=36}: quickwit_indexing::actors::indexing_pipeline: pipeline process message enter Supervise
2021-12-22T07:30:07.643Z  INFO :{msg_id=36}: quickwit_indexing::actors::indexing_pipeline: pipeline process message exit Supervise

[fulmicoton]

ah funny I added the exact same logs yesterday :)

[fmassot]

I'm removing this issue from 0.2