You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
llm_output = analyze_image_base(inputs['prompt']+ '\n\n attached is a screenshot of my screen currently.', file_path, filename, npc=npc)
1366
-
prompt:
1367
-
engine: "natural"
1368
-
code: ""
1369
-
postprocess:
1370
-
- engine: "natural"
1371
-
code: |
1372
-
Screenshot captured and saved as {{ filename }}.
1373
-
Analysis Result: {{ llm_output }}
1351
+
import time
1352
+
from npcsh.image import analyze_image_base, capture_screenshot
1353
+
1354
+
out = capture_screenshot(npc = npc, full = True)
1355
+
1356
+
llm_response = analyze_image_base( '{{prompt}}' + "\n\nAttached is a screenshot of my screen currently. Please use this to evaluate the situation. If the user asked for you to explain what's on their screen or something similar, they are referring to the details contained within the attached image. You do not need to actually view their screen. You do not need to mention that you cannot view or interpret images directly. You only need to answer the user's request based on the attached screenshot!",
1357
+
out['file_path'],
1358
+
out['filename'],
1359
+
npc=npc,
1360
+
**out['model_kwargs'])
1361
+
# To this:
1362
+
if isinstance(llm_response, dict):
1363
+
llm_response = llm_response.get('response', 'No response from image analysis')
0 commit comments