Using non-NotagenX models causes Gradio to generate infinitely

[Epictyphlosion]

I have a standard 2060 with 6GB of VRAM, so generating with NotagenX is incredibly slow. I was able to change the model from NotagenX to one of the standard pretrain models. Doing this causes the output in Gradio to continuously generate music over and over while never outputting anything, even if it finishes a song.

I read in another issue that these pretrain models do not support the same prompting that NotagenX does, however there is no way to set any prompts which these models should accept.

[ElectricAlexis]

Hi, for now the gradio is not compatible with the generation of pre-trained models. But it's ok to let the pre-trained model to generate without prompts, or generate according to given tune headers.

1. Move to the inference folder: cd inference/
2. Change the parameters in config.py to comply with the setting of a pre-trained model. Here I use the NotaGen-medium:

import os

# Configurations for inference
INFERENCE_WEIGHTS_PATH = '../pretrain/weights_notagen_pretrain_p_size_16_p_length_2048_p_layers_16_c_layers_3_h_size_1024_lr_0.0001_batch_4.pth'               # Path to weights for inference
NUM_SAMPLES = 1                                               # Number of samples to generate (only for generate mode)
TOP_K = 9                                                       # Top k for sampling
TOP_P = 0.9                                                      # Top p for sampling
TEMPERATURE = 1.2                                                 # Temperature for sampling
ORIGINAL_OUTPUT_FOLDER = os.path.join('../output/original', os.path.splitext(os.path.split(INFERENCE_WEIGHTS_PATH)[-1])[0] + '_k_' + str(TOP_K) + '_p_' + str(TOP_P) + '_temp_' + str(TEMPERATURE))
INTERLEAVED_OUTPUT_FOLDER = os.path.join('../output/interleaved', os.path.splitext(os.path.split(INFERENCE_WEIGHTS_PATH)[-1])[0] + '_k_' + str(TOP_K) + '_p_' + str(TOP_P) + '_temp_' + str(TEMPERATURE))

# Configurations for model
PATCH_STREAM = True                                             # Stream training / inference
PATCH_SIZE = 16                                                # Patch Size
PATCH_LENGTH = 2048                                             # Patch Length
CHAR_NUM_LAYERS = 3                                             # Number of layers in the decoder
PATCH_NUM_LAYERS = 16                                           # Number of layers in the encoder
HIDDEN_SIZE = 1024                                               # Hidden Size

3. Run inference.py directly: python inference.py

If you are familiar with ABC notation, you can give specific tune header lines as prompt to let the model generate pieces. For example, in inference.py, you can change the last lines of the code to:

if __name__ == '__main__':

    inference_patch(
        prompt_lines=[
            '%%score { ( 1 2 ) | ( 3 4 ) }\n',
            'L:1/8\n',
            'Q:1/4=40\n',
            'M:4/8\n',
            'K:Db\n',
            'V:1 treble nm="Piano" snm="Pno."\n',
            'V:2 treble\n',
            'V:3 bass\n',
            'V:4 bass\n',
        ]
    )

Then the model will generate a small piano piece (though may not be that good as NotaGen-X).
I hope this solves your problem. 😄