[FLINK-37799][model] support deepseek、qwen

[qining-mj]

What is the purpose of the change

This PR adds an implementation to Flink's Model API. This implementation achieves prediction and inference by invoking DeepSeek 、Qwen based on its OpenAI SDK.
The DeepSeek、Qwen uses an API format compatible with OpenAI. By modifying the configuration, you can use the OpenAI SDK or softwares compatible with the OpenAI API to access the DeepSeek Qwen.


Brief change log

Adds Model Functions based on DeepSeek API.
Adds Model Functions based on Qwen API.

Verifying this change

The changes introduced in this PR is verified by test cases introduced in flink-models/flink-model-deepseek.

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: yes

Documentation

Does this pull request introduce a new feature? yes
If yes, how is the feature documented? docs

[flinkbot]

CI report:

• 1f224a1 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[qining-mj]

@davidradl Teacher David Radl, I have optimized and tested the code. Please review it again. Thank you for your guidance.

[qining-mj]

@Zakelly
Teacher Zakelly, I have optimized and tested the code. Please review it again. Thank you for your guidance.

[Zakelly]

@qining-mj Thanks for your contribution! My suggestion is that you may read the contribution guide and follow the process. Please create a jira ticket first

[davidradl]

@qining-mj please could you add the Jira number to this PR title

[qining-mj]

@qining-mj please could you add the Jira number to this PR title

Okay, Teacher David Radl, I will handle it as soon as possible. Thank you

[qining-mj]

Hi @twalthr @dawidwys mentor,Can you help me review the optimized PR?

bg: I am currently in the medical industry and we are using the Flink 2.1 model to anonymize and process medical data. So we internally adapted the Deepseek and QWEN models to the Models module. I hope the community can merge. We will continue to maintain it in the future.

[fsk119]

Thanks for your contribution. I just have one question: could you use the builtin openai sdk[1] to using qwen and deepseek model. As far as I know, most of the platform is compatible with openai sdk.

[1] https://github.com/apache/flink/blob/master/flink-models/flink-model-openai/src/main/java/org/apache/flink/model/openai/OpenAIModelProviderFactory.java

[qining-mj]

Hi @fsk119 mentor, it is entirely possible to use the built-in OpenAI SDK [1] to handle QWEN and DeepSeek models. This time, PR provided OpenAI SDK to access Deepseek and QWEN. Currently, our company is using the above code for the medical industry.

[qining-mj]

Thanks for your contribution. I just have one question: could you use the builtin openai sdk[1] to using qwen and deepseek model. As far as I know, most of the platform is compatible with openai sdk.

[1] https://github.com/apache/flink/blob/master/flink-models/flink-model-openai/src/main/java/org/apache/flink/model/openai/OpenAIModelProviderFactory.java

Hi @fsk119 mentor, due to different access methods, our company has encapsulated openai sdk into a Flink flink-model-openai-sdk-shaded public module, as we also use large models from companies such as Tencent internally

[fsk119]

Thank you for sharing.
If so, could you help us understand why new modules like DeepSeek and Qwen are being introduced into the Flink repository?

By the way, would you mind sharing how you use Flink to build ML pipelines at your company? Have you encountered any challenges that Flink currently cannot address?

Your feedback will help us continuously improve Flink’s AI capabilities — we really appreciate your insights!

[qining-mj]

Thank you for sharing. If so, could you help us understand why new modules like DeepSeek and Qwen are being introduced into the Flink repository?

By the way, would you mind sharing how you use Flink to build ML pipelines at your company? Have you encountered any challenges that Flink currently cannot address?

Your feedback will help us continuously improve Flink’s AI capabilities — we really appreciate your insights!

Thank you for your response.

In the healthcare industry, different hospitals and companies often deploy various AI models, such as those from DeepSeek, Qwen, OpenAI, as well as custom-trained models. In our data processing workflows using Flink and Paimon, we need to support multiple model providers. However, currently, Flink 2.1 only offers built-in support for OpenAI. To address this limitation, we extended Flink 2.1 by integrating support for DeepSeek, Qwen, and Tencent Hunyuan models.

After running these extensions in production for some time, they have proven to be highly stable. Therefore, we believe contributing this implementation back to the community would benefit others and make it easier to maintain moving forward.

Regarding the use of large language models (LLMs) with Flink in healthcare—such as model application scenarios, specific demands, and more—we have gathered substantial insights and requirements. We will share these details via Jira in the coming days to help further improve Flink’s AI capabilities.

[fsk119]

In the healthcare industry, different hospitals and companies often deploy various AI models, such as those from DeepSeek, Qwen, OpenAI, as well as custom-trained models. In our data processing workflows using Flink and Paimon, we need to support multiple model providers. However, currently, Flink 2.1 only offers built-in support for OpenAI. To address this limitation, we extended Flink 2.1 by integrating support for DeepSeek, Qwen, and Tencent Hunyuan models.

Although Flink only includes the OpenAI model in the repository, the OpenAI-style model provider can actually support various models, such as Qwen.
Given that, the current name might be too narrow or misleading.
How about renaming the provider to openai-compat to reflect that it's compatible with the OpenAI API format, rather than being limited to OpenAI itself?

[fsk119]

Regarding the use of large language models (LLMs) with Flink in healthcare—such as model application scenarios, specific demands, and more—we have gathered substantial insights and requirements. We will share these details via Jira in the coming days to help further improve Flink’s AI capabilities.

I am looking forward to receiving any response about Flink AI usage. Recently, we are preparing features like VECTOR_SEARCH, Flink AI Agent. Hoping these features can also help you to build your ML pipeline.

[qining-mj]

Regarding the use of large language models (LLMs) with Flink in healthcare—such as model application scenarios, specific demands, and more—we have gathered substantial insights and requirements. We will share these details via Jira in the coming days to help further improve Flink’s AI capabilities.

I am looking forward to receiving any response about Flink AI usage. Recently, we are preparing features like VECTOR_SEARCH, Flink AI Agent. Hoping these features can also help you to build your ML pipeline.

Thank you for the guidance. Renaming the provider to openai-compat is an excellent suggestion. It more accurately reflects its purpose of supporting any model compatible with the OpenAI API format, rather than being narrowly focused on OpenAI itself. This is a much clearer and more inclusive name.

Following this change, I would like to propose extending the new flink-openai-compat module to add support for other OpenAI-compatible model providers, such as DeepSeek, Qwen, and Tencent Hunyuan.

If the community agrees, I am prepared to take on this development work. I would be grateful if you could assign this task to me so I can submit a Pull Request for this enhancement.

I also hope to actively participate in community contributions.

I look forward to your confirmation.

[fsk119]

+1 to open a FLIP to rename the OpenAI model provider identifier.

[yunfengzhou-hub]

+1 for renaming the OpenAI model provider identifier.

[qining-mj]

Hi @fsk119 mentor,The current flink-model-openai module only supports OpenAI models, as the endpoint and MODEL parameters are currently defined solely for OpenAI compatibility. I aim to contribute to extending support for models like DeepSeek and Tongyi Qianwen based on the existing flink-model-openai framework. Please evaluate the feasibility of this approach. Thank you.

[fsk119]

Could you write a doc describe what do you want to change? As far as I know, you only need to change the package name or so.