SQS message consumed but without any trace

[achref-riahi]

Hi, while testing a SQS message consumer using RR jobs, i found that some messages are consumed but without any trace.

For example, for 500 messages, ~ 450 are consumed correctly, because i can see the job plugin log of them, but 50 disappeared without any trace. For 100 messages ~ 15 messages disappeared ...

My config,

version: "3"

rpc:
  listen: "tcp://127.0.0.1:6001"

server:
  command: consumer.php
  relay: pipe

sqs:
  key: key
  secret: secret
  region: us-east-2	
  endpoint: https://sqs.us-east-2.amazonaws.com
jobs:
  num_pollers: 20
  pipeline_size: 100000
  pool:
    num_workers: 20
    max_jobs: 0
    allocate_timeout: 60s
    destroy_timeout: 60s
    debug: false
  consume: ["queue-name"]
  pipelines:
    queue-name:
      driver: sqs
      config:
        prefetch: 20
        consume_all: true
        skip_queue_declaration: true
        queue: queue-name

logs:
  encoding: console
  level: debug
  mode: development
  err_output: stderr
  output: stderr

  channels:
    jobs:
      level: debug
    server:
      level: debug
      mode: raw

[rustatian]

Hey @achref-riahi 👋

1. What is your RR version?
2. Are you using AWS SQS or local dockered env?

[rustatian]

Hey @achref-riahi 👋
Did you resolve this problem?

[achref-riahi]

Hello @rustatian, the issue only pertains to the FIFO queue; the standard queue functions correctly.

[rustatian]

Hard to say, I've retested the case with FIFO and real SQS: https://github.com/roadrunner-server/sqs/blob/master/tests/jobs_sqs_fifo_test.go#L439, all logs received successfully. If that possible, could you please try your case with a simple request-response worker like this one: https://github.com/roadrunner-server/sqs/blob/master/tests/php_test_files/jobs/jobs_ok.php

[achref-riahi]

Yes, of course. I'll try it as soon as possible. Otherwise, could you share with me the settings of your queue, such as message duplication rules, ...? Thank you.

[rustatian]

Just default one, I didn't change/tune any options.