README

# A text summarizer using Phi 3

* default_llm_commands.json - default instructions, you can create another file to change the commands
* generation.args - you can adapt these to generate with other arguments


## General Runner Usage (TODO UPDATE)

    python3 llm_general_multi_process_gpu.py 
        --input_json test.json  
        --output_dir partitions 
        --gpu_devices 2 3 4 
        --processes_per_gpu 2 
        --model_id "microsoft/Phi-3.5-mini-instruct" 
        --first_summary_field "summary" 
        --summary_over_summary_field "abstract" 
        --artifact_text_field "text" 
        --sys_commands_file llm_commands.json 
        --generation_args_file generation_args.json 
        --debug 
        --id_field "file_path"

## Expected data

A json with a list of objects with

    * the artifacts text field
    * possible context

The names of these fields will be passed as arguments