Merge pull request #81 from cagostino/chris/llm_funcs_cleaning

Chris/llm funcs cleaning

Author: cagostino

README.md

@@ -98,8 +98,8 @@ Here is an example of what the .npcshrc file might look like after this has been
 ```bash
 # NPCSH Configuration File
 export NPCSH_INITIALIZED=1
-export NPCSH_PROVIDER='ollama'
-export NPCSH_MODEL='llama3.2'
+export NPCSH_CHAT_PROVIDER='ollama'
+export NPCSH_CHAT_MODEL='llama3.2'
 export NPCSH_DB_PATH='~/npcsh_history.db'
 ```
 `npcsh` also comes with a set of tools and NPCs that are used in processing. It will generate a folder at ~/.npcsh/ that contains the tools and NPCs that are used in the shell and these will be used in the absence of other project-specific ones. Additionally, `npcsh` records interactions and compiled information about npcs within a local SQLite database at the path specified in the .npcshrc file. This will default to ~/npcsh_history.db if not specified. When the data mode is used to load or analyze data in CSVs or PDFs, these data will be stored in the same database for future reference.
@@ -142,6 +142,28 @@ For cases where you wish to set up a project specific set of NPCs, tools, and as
 └── assembly_lines/    # Project workflows
 ```
 
+## IMPORTANT: migrations and deprecations
+
+### v0.3.4
+-In v0.3.4, the structure for tools was adjusted. If you have made custom tools please refer to the structure within npc_compiler to ensure that they are in the correct format. Otherwise, do the following
+```bash
+rm ~/.npcsh/npc_team/tools/*.tool
+```
+and then
+```bash
+npcsh
+```
+and the updated tools will be copied over into the correct location.
+
+### v0.3.5
+-Version 0.3.5 included a complete overhaul and refactoring of the llm_funcs module. This was done to make it not as horribly long and to make it easier to add new models and providers
+
+
+-in version 0.3.5, a change was introduced to the database schema for messages to add npcs, models, providers, and associated attachments to data. If you have used `npcsh` before this version, you will need to run this migration script to update your database schema:   [migrate_conversation_history_v0.3.5.py](https://github.com/cagostino/npcsh/blob/cfb9dc226e227b3e888f3abab53585693e77f43d/npcsh/migrations/migrate_conversation_history_%3Cv0.3.4-%3Ev0.3.5.py)
+
+-additionally, NPCSH_MODEL and NPCSH_PROVIDER have been renamed to NPCSH_CHAT_MODEL and NPCSH_CHAT_PROVIDER
+to provide a more consistent naming scheme now that we have additionally introduced `NPCSH_VISION_MODEL` and `NPCSH_VISION_PROVIDER`, `NPCSH_EMBEDDING_MODEL`, `NPCSH_EMBEDDING_PROVIDER`, `NPCSH_REASONING_MODEL`, `NPCSH_REASONING_PROVIDER`, `NPCSH_IMAGE_GEN_MODEL`, and `NPCSH_IMAGE_GEN_PROVIDER`.
+- In addition, we have added NPCSH_API_URL to better accommodate openai-like apis that require a specific url to be set as well as `NPCSH_STREAM_OUTPUT` to indicate whether or not to use streaming in one's responses. It will be set to 0 (false) by default as it has only been tested  and verified for a small subset of the models and providers we have available (openai, anthropic, and ollama). If you try it and run into issues, please post them here so we can correct them as soon as possible !
 
 
 ## npcsh usage
@@ -477,12 +499,15 @@ npcsh> /sample What is the capital of France?
 
 
 ### Search
-Search can be accomplished through the `/search` macro. You can specify the provider as being "google" or "perplexity" or "duckduckgo".  The default is google.
+Search can be accomplished through the `/search` macro. You can specify the provider as being "perplexity" or "duckduckgo". For the former,
+you must set a perplexity api key as an environment variable as described above. The default provider is duckduckgo.
 
+NOTE: while google is an available search engine, they recently implemented changes (early 2025) that make the python google search package no longer as reliable.
+For now, we will use duckduckgo and revisit this issue when other more critical aspects are handled.
 
 
 ```npcsh
-npcsh!> /search -p google  who is the current us president
+npcsh!> /search -p duckduckgo  who is the current us president
 
 
 President Donald J. Trump entered office on January 20, 2025. News, issues, and photos of the President Footer Disclaimer This is the official website of the U.S. Mission to the United Nations. External links to other Internet sites should not be construed as an endorsement of the views or privacy policies contained therein.

examples/data_analyst.npc

@@ -2,7 +2,7 @@ name: data_analyst
 primary_directive: Assist users with data analysis tasks, including data loading, plotting, and statistical computations.
 model: llama3.2
 provider: ollama
-suggested_tools_to_use:
+tools:
   - data_loader
   - data_plotter
   - stats_calculator

examples/datacollector.npc

@@ -1,6 +1,6 @@
 name: DataCollector
 primary_directive: Collect user feedback regarding UI preferences.
-suggested_tools_to_use:
+tools:
   - feedback_gatherer
 model: claude-3-haiku-latest
 provider: anthropic

examples/market_analyst.npc

@@ -1,6 +1,6 @@
 name: market_analyst
 primary_directive: You are an AI assistant focused on monitoring and analyzing market trends. Provide de
-suggested_tools_to_use:
+tools:
   - market trend analysis
   - report generation
 model: llama3.2

examples/npc_team/factory/csv_generator.py

@@ -1,11 +1,11 @@
-import pandas as pd
+import pandas as Enter
 import random
 import os
 from faker import Faker
 
 fake = Faker()
 
-import pandas as pd
+import pandas as Enter
 import sqlite3
 from datetime import datetime
 from npcsh.npc_compiler import NPCCompiler, NPCSQLOperations, NPCDBTAdapter

examples/presenter.npc

@@ -1,6 +1,6 @@
 name: presenter
 primary_directive: Create a presentation deck summarizing the analysis.
-suggested_tools_to_use:
+tools:
   - deck_creator
 model: claude-3-haiku-20240307
 provider: anthropic