Random timeouts and no metric

[bzajacc]

Cześć!

I have a few applications deployed on RR. But there's a problem with just one of them: App A receives requests from the outside world, validates them or does other things, and then makes request to App B. Sometimes, the request to App B takes more than 10 seconds, times out, and causes the main request to App A to return a HTTP 500 error.
Like:

I've checked for differences between the applications and only this one (B) has this issue. Do you have any ideas what I could check more? I've already tried everything, but the problem occurs 5-10 times a day, on different endpoints. I even have a case where 2 requests were sent with a 100ms interval, and the first got a 200 status, while the second got a 500 for this exact reason.

Here is rr.yaml: is same for both applications:

version: "3"

server:
  command: "php public/index.php"
  env:
    - APP_RUNTIME: Baldinof\RoadRunnerBundle\Runtime\Runtime

http:
  address: 0.0.0.0:80
  pool:
    debug: false
    num_workers: 8
  access_log: true
  middleware: [ "http_metrics", "static", "gzip" ]
  uploads:
    forbid: [ ".php", ".exe", ".bat" ]
  static:
    dir: "public"
    forbid: [ ".php", ".htaccess" ]

logs:
  mode: production
  channels:
    http:
      level: debug # Log all http requests, set to info to disable
    server:
      level: info # Everything written to worker stderr is logged
      mode: raw
    metrics:
      level: error

rpc:
  listen: tcp://127.0.0.1:6001

metrics:
  address: "0.0.0.0:9181"  

status:
  address: 0.0.0.0:2114

Additionally, I've noticed 'holes' in the metrics of this App B - in the others, there's nothing like that. The metrics are configured the same way, Prometheus is configured the same way, and the others show a continuous graph, but this App B doesn't.

[rustatian]

Hejka @bzajacc 👋🏻

Could you please share your configuration as well? (rr.yaml)

[rustatian]

Ah, sorry, I see

[bzajacc]

Yes, forgot it and update first post 😅

[rustatian]

Hm, really strange case. I see you're using a third party integration, not sure, but it may cause a problem, since we don't have any single test for it.
Also, it'd be nice to see logs around this problem.

[bzajacc]

Logs from AppA, because AppB doesnt have any, like request get lost on the way, and thats why is timeouted. I guess it could be an infrastructure problem or something, but it's weird that it's only happening with this one app. So, I'm thinking first of all it might be a software issue.

Does RR have some option to show more logs not only this one wit httpinfo?

About this third part integration, I have 7ish other application on that exac config (integartion and rr.yaml), and works fine..

[rustatian]

Yeah, you may use just debug level without the channels.