max_length not used

[lintangsutawika]

Using max_length=... in --model_args somehow is not used. This is suppose to fix that.

[baberabb]

Hey @lintangsutawika . This won't quite work as self.max_length is a property

lm-evaluation-harness/lm_eval/models/huggingface.py

Lines 405 to 408 in 9d36354

	@property
	def max_length(self):
	if self._max_length: # if max length manually set, return it
	return self._max_length

and would almost never be None.

Maybe a bug in the case of max_length being set in gen_kwargs?

[lintangsutawika]

This seems to work if I do

--model_args "pretrained=...,max_length=2048"

Should it work with the current version?

[baberabb]

This seems to work if I do

--model_args "pretrained=...,max_length=2048"

Should it work with the current version?

Sorry, I don't quite understand. you said this isn't working in the OP?