Release 0.81.0 #2988

sgoggins · 2025-02-12T02:28:36Z

Description

Key rotator improvements
API Key checking command line function added
Message Reviews for PRs speed up

Signed-off-by: Andrew Brain <andrewbrain2019@gmail.com>

Signed-off-by: Ulincsys <ulincsys@gmail.com>

- Add multiprocessing tests for publisher and client interfaces - Add platform and key list commands to publisher API - Add wait (IE: ACK) command to publisher API - Add more verbose logging - Remove redundant requests from expiration process - Update and expand documentation Signed-off-by: Ulincsys <ulincsys@gmail.com>

Signed-off-by: Ulincsys <ulincsys@gmail.com>

Speed up github events recollection

…ction Speed up pr review comment recollection

Fix github event recollection syntax errors

Fix gitlab reviews syntax error

Use key manager merger into DEV

Add command to get github api key statuses

github-actions

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

pylint

tests/key_manager.py|24 col 16| W0621: Redefining name 'platform' from outer scope (line 62) (redefined-outer-name)
tests/key_manager.py|28 col 4| W0621: Redefining name 'logger' from outer scope (line 69) (redefined-outer-name)
tests/key_manager.py|32 col 4| W0621: Redefining name 'key' from outer scope (line 63) (redefined-outer-name)
tests/key_manager.py|4| W0611: Unused current_process imported from multiprocessing (unused-import)
tests/key_manager.py|5| W0611: Unused PIPE imported from subprocess (unused-import)

github-actions · 2025-02-12T02:29:07Z

augur/application/cli/backend.py

@@ -27,6 +29,7 @@
 from augur.application.cli import test_connection, test_db_connection, with_database, DatabaseContext
 import sqlalchemy as s

+from keyman.KeyClient import KeyClient, KeyPublisher


[pylint] _{reported by reviewdog 🐶}
W0611: Unused KeyClient imported from keyman.KeyClient (unused-import)

github-actions · 2025-02-12T02:29:07Z

augur/application/cli/collection.py

@@ -25,6 +27,8 @@
 from augur.application.cli import test_connection, test_db_connection, with_database, DatabaseContext
 from augur.application.cli._cli_util import _broadcast_signal_to_processes, raise_open_file_limit, clear_redis_caches, clear_rabbitmq_messages

+from keyman.KeyClient import KeyClient, KeyPublisher


[pylint] _{reported by reviewdog 🐶}
W0611: Unused KeyClient imported from keyman.KeyClient (unused-import)

github-actions · 2025-02-12T02:29:07Z

augur/tasks/github/events.py

@@ -3,37 +3,47 @@
 import sqlalchemy as s


[pylint] _{reported by reviewdog 🐶}
W0611: Unused sqlalchemy imported as s (unused-import)

github-actions · 2025-02-12T02:29:07Z

augur/tasks/github/events.py


 from augur.tasks.init.celery_app import celery_app as celery
 from augur.tasks.init.celery_app import AugurCoreRepoCollectionTask
 from augur.application.db.data_parse import *
 from augur.tasks.github.util.github_data_access import GithubDataAccess, UrlNotFoundException
 from augur.tasks.github.util.github_random_key_auth import GithubRandomKeyAuth
+from augur.tasks.github.util.github_task_session import GithubTaskManifest


[pylint] _{reported by reviewdog 🐶}
W0611: Unused GithubTaskManifest imported from augur.tasks.github.util.github_task_session (unused-import)

github-actions · 2025-02-12T02:29:07Z

augur/tasks/github/events.py

 from augur.tasks.github.util.util import get_owner_repo
 from augur.tasks.util.worker_util import remove_duplicate_dicts
-from augur.application.db.models import PullRequestEvent, IssueEvent, Contributor, CollectionStatus
-from augur.application.db.lib import get_repo_by_repo_git, bulk_insert_dicts, get_issues_by_repo_id, get_pull_requests_by_repo_id, update_issue_closed_cntrbs_by_repo_id, get_session, get_engine
+from augur.application.db.models import PullRequestEvent, IssueEvent, Contributor, Repo


[pylint] _{reported by reviewdog 🐶}
W0611: Unused Repo imported from augur.application.db.models (unused-import)

github-actions · 2025-02-12T02:29:09Z

keyman/Orchestrator.py

+                listening for, just check against each
+                channel that we have actions prepared for.
+            """
+            if channel == self.ANNOUNCE:


[pylint] _{reported by reviewdog 🐶}
E1101: Instance of 'KeyOrchestrator' has no 'ANNOUNCE' member (no-member)

github-actions · 2025-02-12T02:29:09Z

keyman/Orchestrator.py

+            """
+            if channel == self.ANNOUNCE:
+                if "requester_id" in request:
+                    stdout = f"{self.ANNOUNCE}-{request['requester_id']}"


[pylint] _{reported by reviewdog 🐶}
E1101: Instance of 'KeyOrchestrator' has no 'ANNOUNCE' member (no-member)

github-actions · 2025-02-12T02:29:09Z

keyman/Orchestrator.py

+                        conn.publish(stdout, "")
+                        self.logger.info(f"ACK; for: {request['requester_id']}")
+                    elif request["type"] == "LIST_PLATFORMS":
+                        platforms = [ p for p in self.fresh_keys.keys() ]


[pylint] _{reported by reviewdog 🐶}
C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary)

github-actions · 2025-02-12T02:29:09Z

keyman/Orchestrator.py

+                    self.logger.exception("Error during ANNOUNCE")
+                    continue
+
+            elif channel == self.REQUEST:


[pylint] _{reported by reviewdog 🐶}
E1101: Instance of 'KeyOrchestrator' has no 'REQUEST' member (no-member)

github-actions · 2025-02-12T02:29:09Z

keyman/Orchestrator.py

+
+            elif channel == self.REQUEST:
+                self.refresh_keys()
+                stdout = f"{self.REQUEST}-{request['requester_id']}"


[pylint] _{reported by reviewdog 🐶}
E1101: Instance of 'KeyOrchestrator' has no 'REQUEST' member (no-member)

sgoggins

Looks ok

Remove update weight hotfix

github-actions · 2025-02-12T02:48:06Z

tests/key_manager.py

+    ]
+}
+
+def mp_consumer(platform):


[pylint] _{reported by reviewdog 🐶}
W0621: Redefining name 'platform' from outer scope (line 62) (redefined-outer-name)

github-actions · 2025-02-12T02:48:06Z

tests/key_manager.py

+    if platform not in keys:
+        raise ValueError(f"Platform not valid for testing keys dict: {platform}")
+
+    logger = AugurLogger(f"Keyman_test_consumer").get_logger()


[pylint] _{reported by reviewdog 🐶}
W0621: Redefining name 'logger' from outer scope (line 69) (redefined-outer-name)

github-actions · 2025-02-12T02:48:06Z

tests/key_manager.py

+    logger.setLevel(1)
+    client = KeyClient(platform, logger)
+
+    key = client.request()


[pylint] _{reported by reviewdog 🐶}
W0621: Redefining name 'key' from outer scope (line 63) (redefined-outer-name)

github-actions · 2025-02-12T02:48:06Z

tests/key_manager.py

+from keyman.KeyClient import KeyClient, KeyPublisher
+from augur.application.logs import AugurLogger
+
+from multiprocessing import Process, current_process


[pylint] _{reported by reviewdog 🐶}
W0611: Unused current_process imported from multiprocessing (unused-import)

github-actions · 2025-02-12T02:48:06Z

tests/key_manager.py

+from augur.application.logs import AugurLogger
+
+from multiprocessing import Process, current_process
+from subprocess import Popen, PIPE


[pylint] _{reported by reviewdog 🐶}
W0611: Unused PIPE imported from subprocess (unused-import)

Signed-off-by: Sean P. Goggins <s@goggins.com>

ABrain7710 and others added 25 commits September 1, 2023 20:03

Add method

0cbb54f

Signed-off-by: Andrew Brain <andrewbrain2019@gmail.com>

Add command to get github key data

1cba93b

Signed-off-by: Andrew Brain <andrewbrain2019@gmail.com>

Initial work on key manager

c257be6

Signed-off-by: Ulincsys <ulincsys@gmail.com>

implement remaining functionality

87bd311

Signed-off-by: Ulincsys <ulincsys@gmail.com>

Merge branch 'dev' into key-manager

74c40f9

Signed-off-by: Ulincsys <ulincsys@gmail.com>

Add startup entrypoints for orchestrator

7ed6460

Signed-off-by: Ulincsys <ulincsys@gmail.com>

Merge branch 'dev' into key-manager

9110b4e

Signed-off-by: Ulincsys <ulincsys@gmail.com>

speed up events collection

15a212f

speed up pr review message collection

57c3c80

fix session syntax error

b27f0f8

start using new key manager

93039e4

expire actually returns a key!

387ab23

Merge pull request #2974 from chaoss/speed-up-events-collection

67af21b

Speed up github events recollection

Merge pull request #2975 from chaoss/speed-up-pr-review-comment-colle…

8f838ff

…ction Speed up pr review comment recollection

fix syntax errors

bd76a1e

Merge pull request #2979 from chaoss/speed-up-events-collection

e06b9e5

Fix github event recollection syntax errors

Merge pull request #2978 from chaoss/fix-gitlab-reviews-syntax-error

61efa9c

Fix gitlab reviews syntax error

remove no gitlab keys exception

8748fb0

Add debug logs

e13f66c

Merge branch 'dev' into github-api-key-command

f7c7cda

Merge pull request #2985 from chaoss:use-key-manager

0e26d37

Use key manager merger into DEV

remove unused import

674c21b

Merge pull request #2986 from chaoss/github-api-key-command

9933ee7

Add command to get github api key statuses

comment out update weight task as it is no longer needed

c199f85

github-actions bot reviewed Feb 12, 2025

View reviewed changes

sgoggins commented Feb 12, 2025

View reviewed changes

Merge pull request #2989 from chaoss/remove-update-weight-hotfix

8302e87

Remove update weight hotfix

github-actions bot reviewed Feb 12, 2025

View reviewed changes

version bump

dc69497

Signed-off-by: Sean P. Goggins <s@goggins.com>

sgoggins changed the title ~~Release 0.80.2~~ Release 0.81.0 Feb 12, 2025

sgoggins added deployed version Live problems with deployed versions documentation Updates documentation database Related to Augur's unifed data model bug-fix Fixes a bug dependencies Pull requests that update a dependency file labels Feb 12, 2025

sgoggins merged commit 455b02a into main Feb 12, 2025
12 of 13 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release 0.81.0 #2988

Release 0.81.0 #2988

sgoggins commented Feb 12, 2025

github-actions bot left a comment

github-actions bot Feb 12, 2025

github-actions bot Feb 12, 2025

github-actions bot Feb 12, 2025

github-actions bot Feb 12, 2025

github-actions bot Feb 12, 2025

github-actions bot Feb 12, 2025

github-actions bot Feb 12, 2025

github-actions bot Feb 12, 2025

github-actions bot Feb 12, 2025

github-actions bot Feb 12, 2025

sgoggins left a comment

github-actions bot Feb 12, 2025

github-actions bot Feb 12, 2025

github-actions bot Feb 12, 2025

github-actions bot Feb 12, 2025

github-actions bot Feb 12, 2025

Release 0.81.0 #2988

Release 0.81.0 #2988

Conversation

sgoggins commented Feb 12, 2025

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot Feb 12, 2025

Choose a reason for hiding this comment

github-actions bot Feb 12, 2025

Choose a reason for hiding this comment

github-actions bot Feb 12, 2025

Choose a reason for hiding this comment

github-actions bot Feb 12, 2025

Choose a reason for hiding this comment

github-actions bot Feb 12, 2025

Choose a reason for hiding this comment

github-actions bot Feb 12, 2025

Choose a reason for hiding this comment

github-actions bot Feb 12, 2025

Choose a reason for hiding this comment

github-actions bot Feb 12, 2025

Choose a reason for hiding this comment

github-actions bot Feb 12, 2025

Choose a reason for hiding this comment

github-actions bot Feb 12, 2025

Choose a reason for hiding this comment

sgoggins left a comment

Choose a reason for hiding this comment

github-actions bot Feb 12, 2025

Choose a reason for hiding this comment

github-actions bot Feb 12, 2025

Choose a reason for hiding this comment

github-actions bot Feb 12, 2025

Choose a reason for hiding this comment

github-actions bot Feb 12, 2025

Choose a reason for hiding this comment

github-actions bot Feb 12, 2025

Choose a reason for hiding this comment