Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High number of DNS queries #1365

Closed
1 task done
ishanjain28 opened this issue Feb 17, 2025 · 4 comments · May be fixed by #1294
Closed
1 task done

High number of DNS queries #1365

ishanjain28 opened this issue Feb 17, 2025 · 4 comments · May be fixed by #1294
Labels
awaiting triage This issue needs to be reviewed bug Something isn't working

Comments

@ishanjain28
Copy link
Contributor

Description

I don't fully understand the code base so please excuse me if this is just wrong.

As far as I understand, It is using fetch every where, setting up and tearing down connections when it's done. It is not using a shared http client and it is not using connection pools.

The problem with the current approach, Jellyseerr generates ridiculously high amount of DNS traffic. In the last 2 weeks, I now have 84k requests to themoviedb.

Version

2.3

Steps to Reproduce

  1. Install Jellyseer
  2. Use it for a while
  3. Take a look at the amount of DNS traffic generated by it

Screenshots

No response

Logs

DNS Traffic over the last 2 weeks 

      4 10.0.50.24 _http._tcp.security.debian.org
      4 10.0.50.24 _https._tcp.deb.nodesource.com
      4 10.0.50.24 objects.githubusercontent.com
      6 10.0.50.24 fcm.googleapis.com
      6 10.0.50.24 fonts.googleapis.com
     12 10.0.50.24 _http._tcp.deb.debian.org
     12 10.0.50.24 deb.nodesource.com
     14 10.0.50.24 github.com
     22 10.0.50.24 updates.push.services.mozilla.com
     32 10.0.50.24 debian.map.fastlydns.net
     32 10.0.50.24 telemetry.nextjs.org
     50 10.0.50.24 registry.npmjs.org
    154 10.0.50.24 api.pushover.net
    168 10.0.50.24 api.radarr.video
    234 10.0.50.24 79frdp12pn-dsn.algolia.net
   1091 10.0.50.24 gravatar.com
   4490 10.0.50.24 image.tmdb.org
  84173 10.0.50.24 api.themoviedb.org

Platform

desktop

Database

PostgreSQL

Device

Android, iPad Pro

Operating System

iPadOS 18, Android 15

Browser

Firefox, Safari

Additional Context

I believe, reusing connections and sharing the http client across all the places it is used should reduce the DNS traffic.

Code of Conduct

  • I agree to follow Jellyseerr's Code of Conduct
@ishanjain28 ishanjain28 added awaiting triage This issue needs to be reviewed bug Something isn't working labels Feb 17, 2025
@ishanjain28 ishanjain28 changed the title Use a common HTTP Client & Connection pools High number of DNS queries Feb 17, 2025
@gauthier-th
Copy link
Collaborator

Duplicate of #728
This is something related to the way Node.js works. #1294 may solve that.

@gauthier-th
Copy link
Collaborator

gauthier-th commented Feb 17, 2025

It is not using a shared http client and it is not using connection pools.

What do you mean a shared http agent? It's using the default Agent from Node.js, which has keepAlive enabled with a timeout of 5s.

Edit: my bad, we don't use the default Agent of Node.js anymore (because of this) since a custom agent was added for HTTP proxy. I'll make a PR to set back the default value of keepAlive.

@ishanjain28
Copy link
Contributor Author

It's using the default Agent from Node.js, which has keepAlive enabled with a timeout of 5s.

This is what I meant. Usually people create a customized http client and use that every where. I guess it's okay to use the global agent as well.

Thanks for looking into this and responding to it! I'll close it since it's duplicate

@ishanjain28
Copy link
Contributor Author

Edit: my bad, we don't use the default Agent of Node.js anymore (because of this) since a custom agent was added for HTTP proxy. I'll make a PR to set back the default value of keepAlive.

Thank you! Closing this for aforementioned reason

gauthier-th added a commit that referenced this issue Feb 17, 2025
The Node.js documentation mentions that the default keep-alive value is not used when creating a
global agent manually, which is done in customProxyAgent.ts.

re #1365
fallenbagel pushed a commit that referenced this issue Feb 17, 2025
* fix: assign the keep-alive value explicitly

The Node.js documentation mentions that the default keep-alive value is not used when creating a
global agent manually, which is done in customProxyAgent.ts.

re #1365

* fix: typo
RankWeis pushed a commit to RankWeis/jellyseerr that referenced this issue Feb 27, 2025
* fix: assign the keep-alive value explicitly

The Node.js documentation mentions that the default keep-alive value is not used when creating a
global agent manually, which is done in customProxyAgent.ts.

re fallenbagel#1365

* fix: typo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting triage This issue needs to be reviewed bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants