got assertion error while loading the CodeT5+ models 16B-Instruct, 6B and 2B like this "AssertionError: Config has to be initialized with encoder and decoder config". how to solve this issue

[Tarak200]

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

checkpoint = "Salesforce/codet5p-2b"
device = "cuda" # for GPU usage or "cpu" for CPU usage

tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint,
torch_dtype=torch.float16,
trust_remote_code=True).to(device)

encoding = tokenizer("def print_hello_world():", return_tensors="pt").to(device)
encoding['decoder_input_ids'] = encoding['input_ids'].clone()
outputs = model.generate(**encoding, max_length=15)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

used the same code from huggingface, encountered the following error :
AssertionError: Config has to be initialized with encoder and decoder config, How to resolve this issue?

[Roisan]

Hi @Tarak200 , I am also getting the same error while inferencing using Codet5p 2B model. Were you able to find a solution? Thanks.

[angelocurti]

Hi @Roisan @Tarak200 , I am getting the same error. Did you find a solution?
Thanks!

[Roisan]

@angelocurti Hello, I haven't been able to find a solution. Any suggestions from your end will be highly appreciated.

Thank you.

[ardhiwiratamaby]

Creating the generation config solved my problem.

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from transformers import GenerationConfig
import torch

checkpoint = "Salesforce/codet5p-2b"
device = "cuda" # for GPU usage or "cpu" for CPU usage

Select a device (CPU if no GPU is available, otherwise the first GPU)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"Selected device: {device}")

tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint,
torch_dtype=torch.float16,
trust_remote_code=True).to(device)

python_function = "write code for matrix multiplication"
encoding = tokenizer(python_function, return_tensors="pt").to(device)
encoding['decoder_input_ids'] = encoding['input_ids'].clone()

# Create a GenerationConfig with custom settings
generation_config = GenerationConfig(
max_length=950
)

outputs = model.generate(**encoding, generation_config=generation_config)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

[Roisan]

@ardhiwiratamaby Thank you for providing the solution, I am now able to infer the 2B model.

[latuan1]

Hello, @ardhiwiratamaby,
I got assertion error while full fine-tuning CodeT5+ 2B:
File "/root/anaconda3/envs/train-model/lib/python3.11/site-packages/transformers/trainer.py", line 2245, in train
return inner_training_loop(
^^^^^^^^^^^^^^^^^^^^
File "/root/anaconda3/envs/train-model/lib/python3.11/site-packages/transformers/trainer.py", line 2627, in _inner_training_loop
self._maybe_log_save_evaluate(
File "/root/anaconda3/envs/train-model/lib/python3.11/site-packages/transformers/trainer.py", line 3103, in _maybe_log_save_evaluate
self._save_checkpoint(model, trial)
File "/root/anaconda3/envs/train-model/lib/python3.11/site-packages/transformers/trainer.py", line 3200, in _save_checkpoint
self.save_model(output_dir, _internal_call=True)
File "/root/anaconda3/envs/train-model/lib/python3.11/site-packages/transformers/trainer.py", line 3902, in save_model
self._save(output_dir)
File "/root/anaconda3/envs/train-model/lib/python3.11/site-packages/transformers/trainer.py", line 4006, in _save
self.model.save_pretrained(
File "/root/anaconda3/envs/train-model/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3337, in save_pretrained
misplaced_generation_parameters = model_to_save.config._get_non_default_generation_parameters()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/anaconda3/envs/train-model/lib/python3.11/site-packages/transformers/configuration_utils.py", line 1079, in _get_non_default_generation_parameters
default_config = self.class()
^^^^^^^^^^^^^^^^
File "/root/.cache/huggingface/modules/transformers_modules/Salesforce/codet5p-2b/0083d4d638746e6c9ee3dbd504e6dd68738e3c87/configuration_codet5p.py", line 78, in init
"encoder" in kwargs and "decoder" in kwargs
AssertionError: Config has to be initialized with encoder and decoder config.
How to solve this issue.