Implement Comprehensive Node Info System for Monitoring and Management

[hathbanger]

Description:

We need to implement a comprehensive system for gathering, storing, updating, and retrieving node information. This system will provide real-time insights into the status and performance of our network nodes, enabling efficient management and monitoring.

1. Data Collection and Storage

• Implement a system to find OR collect and store the following node information:
  • Node ID
  • Current status (active, inactive, maintenance)
  • Hardware specifications (CPU, RAM, storage capacity)
  • Operating system and version (only compatible w linux ubuntu)
  • Geographic location
  • Current resource utilization (CPU, memory, storage, network)
  • Processes
  • Replication factor
  • Uptime
  • Last seen timestamp
  • Performance metrics (e.g., response time)
  • Version of node software
  • Connected peers (quorum membership)
  • Roles/capabilities of the node (all nodes do all things currently)
  • Any active alerts or warnings

2. Node Status Monitoring Service

• Develop a scalable background service that:
  • Regularly pings all nodes to check their status
  • Updates node information in the database
  • Calculates and updates derived metrics (e.g., uptime percentage)
  • Implements configurable check intervals
  • Handles a large number of nodes efficiently
  • Implements retry logic for temporarily unresponsive nodes

3. API Endpoints

• Create RESTful API endpoints for:
  • Retrieving information for a specific node: GET /api/v1/nodes/{nodeId}
  • Listing all nodes with optional filtering: GET /api/v1/nodes
  • Updating node information: PUT /api/v1/nodes/{nodeId}
  • Reporting node status (for node self-reporting): POST /api/v1/nodes/{nodeId}/status

4. Authentication and Authorization

• Leverage PK in LASR Wallet to handle authentication and signatures for all protected endpoints.

5. Documentation

• Provide comprehensive API documentation
• Create system architecture documentation
• Include deployment and configuration guides

Technical Notes:

• Use TikV db for storage??
• time-series db for storing historical performance data
• Use WebSockets or server-sent events for real-time updates where applicable