Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix infer task from model_name if model from sentence transformer #2151

Conversation

eaidova
Copy link
Contributor

@eaidova eaidova commented Jan 8, 2025

What does this PR do?

issue found during attempt to export https://huggingface.co/BAAI/bge-small-zh-v1.5 via optimum-intel without --task specification
The root cause is infer_library_from_model detects task as sentence-transformers even if --library transformers selected in cli and there is no additional condition for specification task for sentence transformers, as the result method raises error

KeyError: "The task could not be automatically inferred. Please provide the argument --task with the relevant task from image-text-to-text, audio-classification, zero-shot-image-classification, masked-im, audio-xvector, image-to-text, audio-frame-classification, depth-estimation, visual-question-answering, semantic-segmentation, image-to-image, image-classification, feature-extraction, image-segmentation, inpainting, sentence-similarity, object-detection, automatic-speech-recognition, multiple-choice, text-classification, fill-mask, text-generation, zero-shot-object-detection, text-to-image, reinforcement-learning, text2text-generation, token-classification, question-answering, mask-generation, text-to-audio. Detailed error: 'Could not find the proper task name for the model BAAI/bge-small-zh-v1.5.'

provided default task for sentence-transformers and added mechanism to explicitly set library_name (in case if user force --library_name in cli, need additional changes for enabling this behaviour in optimum-intel and other integrations that allow to select library_name) for avoiding mismatch between auto-detected and selected library (sentence transformers model can be exported with both transformers and sentence transformers library name and export configuration and exported model may be different for this case)

@eaidova
Copy link
Contributor Author

eaidova commented Jan 8, 2025

@IlyasMoutawwakil @echarlaix could you please take a look?

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@@ -1770,6 +1770,7 @@ def _infer_task_from_model_name_or_path(
revision: Optional[str] = None,
cache_dir: str = HUGGINGFACE_HUB_CACHE,
token: Optional[Union[bool, str]] = None,
library_name: Optional[str] = None,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you're adding library_name to the signature but shouldn't this argument be specified in the method's calls as well ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just want to get confirmation that it is reliable approach before doing that.
In case of optimum itself it also need to provide it to infer_task_from_model too https://github.com/huggingface/optimum/blob/main/optimum/exporters/onnx/__main__.py#L259

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah you can go ahead and add it where needed, it seems reliable.

Copy link
Contributor Author

@eaidova eaidova Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, added.

Only not sure about tflite exporter, as I can see for getting export config, it uses library_name="transformers" explicitly, so it means that other library are not supported, right? I also added library_name="transformers" for avoid model type mismatch (e.g. in case of sentence transformers described above, if it will not be provided, model will be loaded as sentence transformer and raises error in some steps later

@eaidova
Copy link
Contributor Author

eaidova commented Jan 10, 2025

failed test is not related to my changes, modernbert is available only on transformers master branch, there is no transformers package that contains this model code yet

@IlyasMoutawwakil IlyasMoutawwakil merged commit adcae38 into huggingface:main Jan 10, 2025
56 of 57 checks passed
@IlyasMoutawwakil IlyasMoutawwakil removed the request for review from echarlaix January 10, 2025 15:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants