xrx state machine draft

[mprast]

XRX State Machine

The purpose of this PR is to add a backing state machine to the xrx reasoning agent. A sample state machine has been fully integrated into shopify-app. With this change the agent (mostly) appears to be able to:

• Understand the state it's in and infer its current objective
• Understand what states it can transition into
• Transition to the next state when appropriate
• Guide the user back to the current objective if they try to go out of bounds
• Switch flows if the user indicates they want to do something different

Testing

No special setup is needed for the state machine; just pull the branches down for xrx-sample-apps and xrx-core and play around with shopify-app as usual. The agent will log what state it's in and will use a 'transition-state' node to transition when appropriate.

A flow is a graph of steps. Each flow has an 'initial' step, which is the step the agent starts in when it starts the flow. There are three sample flows in shopify-app/reasoning/app/agent/flows.yaml - one for buying a product from the store, one for submitting an app to be listed in the store, and one initial flow for figuring out what the user wants to do. The agent will move between these flows as necessary. It will abandon the flow it's on and start a new one if you ask it to.

Feel free to tinker with flows.yaml to add your own flows. The format should be self-explanatory, but if you have questions just shoot me a slack!

To demonstrate the capabilities of the state machine, I recorded four sample conversations I had with shopify-app using interactive-test.py. These are in shopify-app/reasoning/app/agent/sample_conversations. Feel free to replicate these yourself. If you can't, or if anything looks weird, please let me know!

TODOs & Cleanup Work

• Slim down the prompts: I kept adding stuff until I got to a point I liked; a lot of it can probably be removed
• Figure out how to pass the state machine to the client via session details: I rigged something that worked but I need to find out if it's the right way to do it
• Build general functionality to trim state machine info in interactive-test.py: the response logging was outputting the entire structure of the state machine - including all flows, states, and transitions - on every response, which made responses very hard to read. I rigged something to redact the state machine from the session variable on output; this can probably be made configurable if we want a general way to say "this variable is huge so don't output it please" (chris - what do you think?)
• remove various debug cruft (mostly pdb imports)

Next Steps

• Remember history when transitioning out of a flow and allow the user to return to where they were if they want
• Auto-gen initial flow so it doesn't have to be specified in flows.yaml
• More testing with more complicated state graphs
• Individual objectives for each transition
• Anything else I'm forgetting!

[mprast]

Extra notes: the agent appears to get "stuck" after applying guardrails a single time. That is to say, if you try to do something unrelated to the current state and the agent stops you, you can no longer switch between flows - the agent will stop you every time. I think it's getting too hung up on the conversation history.

I think the cleanest way around this may be to have a separate graph node that intercepts the output of the RespondToUser node and replaces it if the latest question and answer are unrelated to the objective of the current state machine node. We'd explicitly not include the rest of the conversation here to make sure the agent doesn't get thrown off.

@chrislott what do you think?

[mprast]

also - I think it's probably worth having a separate node type for the initial 'query' flow (the one that describes the options to the user and asks the user what they want to do). as it stands modeling this as its own flow seems to confuse the agent with its vagueness; the agent uses it to circumvent the state guardrails a lot

[alessandro-neri]

[Feature] Implement State Machine for Enhanced Conversation Flow

1. Overview

• What is the feature?
  Introduced a robust state machine to manage conversation flows within the xRx framework. This includes the addition of a new state_machine.py module that defines the StateMachine class for handling state transitions and flow management. Updated the __init__.py to integrate the state machine and modified the elevenlabs_tts.py to support the new state management functionalities.

• What changed?

  • Agent Framework: Added state machine utilities to handle complex conversation flows and state transitions.
  • Text-to-Speech (TTS) Module: Updated TTS implementation to work seamlessly with the new state management system.

2. Files Modified

File Name	Changes
__init__.py	• Context: Initializes the agent framework and integrates core utilities. • Changes [EDIT]: Added an import statement for StateMachine from the newly created utils/state_machine.py.
state_machine.py	• Context: Manages conversation states and transitions within the agent framework. • Changes [NEW]: Introduced the StateMachine class with methods for initializing session data, generating prompts based on current state, handling state transitions, and managing flow transitions. Configured logging and loaded flow definitions from flows.yaml.
elevenlabs_tts.py	• Context: Handles text-to-speech functionalities using ElevenLabs API. • Changes [EDIT]: Modified the is_open property to ensure compatibility with the new state machine by maintaining the open state accurately after state transitions.

3. Issues/Improvements

Security. Potential exposure of flow definitions.

- **Specific security concern:** Flow definitions in `flows.yaml` may contain sensitive logic that could be exposed if not properly secured. - **Specific mitigation needed:** Implement access controls and validation to ensure only authorized users can modify or access flow definitions.

Performance. Increased initialization time due to loading flows.

- **Specific performance impact:** Loading and parsing `flows.yaml` during session initialization may introduce latency. - **Specific optimization needed:** Cache parsed flow definitions or optimize the loading mechanism to reduce initialization time.

Maintainability. Complexity of state transitions.

- **Specific maintenance concern:** Managing numerous state transitions could lead to increased complexity and potential bugs. - **Specific improvement needed:** Implement comprehensive unit tests and documentation for state transitions to enhance maintainability.

Simplification. Redundant state transition logic.

- **Specific simplification opportunity:** Some state transition logic in `state_machine.py` may overlap with existing utilities. - **Specific refactoring needed:** Refactor `StateMachine` methods to eliminate redundancy and streamline state management processes.

---

[mprast]

you can now turn the state machine on or off using an env var. xRx will only use a state machine if STATE_MACHINE_ON is set to "true"

[sabbyanandan]

A general comment. I don't see STATE_MACHINE_ON as an env-var in this PR, and neither do I see it in the upstream code. I assume then it will be a flag at the App level?

[mprast]

A general comment. I don't see STATE_MACHINE_ON as an env-var in this PR, and neither do I see it in the upstream code. I assume then it will be a flag at the App level?

yes, the idea is it gets defined in the .env file that gets passed to docker-compose

[sabbyanandan]

If the YAML is corrupted, would we want to handle the error gracefully or let the developer know of it?

[sabbyanandan]

Maybe I am overthinking this.

[mprast]

I think it's a good idea - I'll add a handler

[mprast]

probably best to just say 'hey double-check your flows.yaml'. although maybe the exception is descriptive enough on its own...let me check

[mprast]

yeah the default error is not helpful; I'll add a better one

[mprast]

@sabbyanandan addressed (see readFlowsYaml above)

[sabbyanandan]