apache · moiseenkov · Jan 28, 2025
diff --git a/docs/apache-airflow-providers-google/operators/cloud/automl.rst b/docs/apache-airflow-providers-google/operators/cloud/automl.rst
@@ -163,25 +163,21 @@ You can find example on how to use VertexAI operators here:
     :end-before: [END how_to_cloud_vertex_ai_delete_model_operator]
 
 .. _howto/operator:AutoMLPredictOperator:
-.. _howto/operator:AutoMLBatchPredictOperator:
 
 Making Predictions
 ^^^^^^^^^^^^^^^^^^
 
 To obtain predictions from Google Cloud AutoML model you can use
-:class:`~airflow.providers.google.cloud.operators.automl.AutoMLPredictOperator` or
-:class:`~airflow.providers.google.cloud.operators.automl.AutoMLBatchPredictOperator`. In the first case
+:class:`~airflow.providers.google.cloud.operators.automl.AutoMLPredictOperator`. In the first case
 the model must be deployed.
 
 
-Th :class:`~airflow.providers.google.cloud.operators.automl.AutoMLBatchPredictOperator` deprecated for tables,
-video intelligence, vision and natural language is deprecated and will be removed after 31.03.2024.
-Please use
+For tables, video intelligence, vision and natural language you can use the following operators:
+
 :class:`airflow.providers.google.cloud.operators.vertex_ai.batch_prediction_job.CreateBatchPredictionJobOperator`,
 :class:`airflow.providers.google.cloud.operators.vertex_ai.batch_prediction_job.GetBatchPredictionJobOperator`,
 :class:`airflow.providers.google.cloud.operators.vertex_ai.batch_prediction_job.ListBatchPredictionJobsOperator`,
-:class:`airflow.providers.google.cloud.operators.vertex_ai.batch_prediction_job.DeleteBatchPredictionJobOperator`,
-instead.
+:class:`airflow.providers.google.cloud.operators.vertex_ai.batch_prediction_job.DeleteBatchPredictionJobOperator`.
 You can find examples on how to use VertexAI operators here:
 
 .. exampleinclude:: /../../providers/tests/system/google/cloud/vertex_ai/example_vertex_ai_batch_prediction_job.py

diff --git a/docs/apache-airflow-providers-google/operators/cloud/dataflow.rst b/docs/apache-airflow-providers-google/operators/cloud/dataflow.rst
@@ -54,9 +54,6 @@ There are several ways to run a Dataflow pipeline depending on your environment,
     command-line tool to build and save the Flex Template spec file in Cloud Storage. See:
     :ref:`howto/operator:DataflowStartFlexTemplateOperator`
 
-- **SQL pipeline**: Developer can write pipeline as SQL statement and then execute it in Dataflow. See:
-  :ref:`howto/operator:DataflowStartSqlJobOperator`
-
 It is a good idea to test your pipeline using the non-templated pipeline,
 and then run the pipeline in production using the templates.
 
@@ -283,29 +280,6 @@ Also for this action you can use the operator in the deferrable mode:
     :start-after: [START howto_operator_start_flex_template_job_deferrable]
     :end-before: [END howto_operator_start_flex_template_job_deferrable]
 
-.. _howto/operator:DataflowStartSqlJobOperator:
-
-Dataflow SQL
-""""""""""""
-Dataflow SQL supports a variant of the ZetaSQL query syntax and includes additional streaming
-extensions for running Dataflow streaming jobs.
-
-Here is an example of running Dataflow SQL job with
-:class:`~airflow.providers.google.cloud.operators.dataflow.DataflowStartSqlJobOperator`:
-
-.. exampleinclude:: /../../providers/tests/system/google/cloud/dataflow/example_dataflow_sql.py
-    :language: python
-    :dedent: 4
-    :start-after: [START howto_operator_start_sql_job]
-    :end-before: [END howto_operator_start_sql_job]
-
-.. warning::
-    This operator requires ``gcloud`` command (Google Cloud SDK) must be installed on the Airflow worker
-    <https://cloud.google.com/sdk/docs/install>`__
-
-See the `Dataflow SQL reference
-<https://cloud.google.com/dataflow/docs/reference/sql>`_.
-
 .. _howto/operator:DataflowStartYamlJobOperator:
 
 Dataflow YAML

@@ -26,8 +26,7 @@ Google Cloud SDK installation
 -----------------------------
 
 Some operators, such as :class:`~airflow.providers.google.cloud.operators.kubernetes_engine.GKEStartPodOperator`,
-:class:`~airflow.providers.google.cloud.operators.dataflow.DataflowStartSqlJobOperator`, require
-the installation of `Google Cloud SDK <https://cloud.google.com/sdk>`__ (includes ``gcloud``).
+require the installation of `Google Cloud SDK <https://cloud.google.com/sdk>`__ (includes ``gcloud``).
 You can also run these commands with BashOperator.
 
 Create a new Dockerfile like the one shown below.

diff --git a/providers/src/airflow/providers/google/CHANGELOG.rst b/providers/src/airflow/providers/google/CHANGELOG.rst
@@ -27,6 +27,37 @@
 Changelog
 ---------
 
+13.0.0
+......
+
+.. note::
+  This release of provider is only available for Airflow 2.9+ as explained in the
+  `Apache Airflow providers support policy <https://github.com/apache/airflow/blob/main/PROVIDERS.rst#minimum-supported-version-of-airflow-for-community-managed-providers>`_.
+
+Breaking changes
+~~~~~~~~~~~~~~~~
+
+.. warning::
+  Deprecated classes, parameters and features have been removed from the Google provider package.
+  The following breaking changes were introduced:
+
+  * Operators
+
+    * Removed ``AutoMLBatchPredictOperator``. Please use the operators from ``airflow.providers.google.cloud.operators.vertex_ai.batch_prediction_job`` instead
+    * Removed ``DataflowStartSqlJobOperator``. Please ``DataflowStartYamlJobOperator`` instead
+    * Removed ``PromptLanguageModelOperator``. Please ``TextGenerationModelPredictOperator`` instead
+    * Removed ``GenerateTextEmbeddingsOperator``. Please ``TextEmbeddingModelGetEmbeddingsOperator`` instead
+    * Removed ``PromptMultimodalModelOperator``. Please ``GenerativeModelGenerateContentOperator`` instead
+    * Removed ``PromptMultimodalModelWithMediaOperator``. Please ``GenerativeModelGenerateContentOperator`` instead
+
+  * Hooks
+
+    * Removed ``GenerativeModelHook.prompt_multimodal_model_with_media()``. Please use ``GenerativeModelHook.generative_model_generate_content()`` instead
+    * Removed ``GenerativeModelHook.prompt_multimodal_model()``. Please use ``GenerativeModelHook.generative_model_generate_content()`` instead
+    * Removed ``GenerativeModelHook.get_generative_model_part()``. Please use ``GenerativeModelHook.generative_model_generate_content()`` instead
+    * Removed ``GenerativeModelHook.prompt_language_model()``. Please use ``GenerativeModelHook.text_generation_model_predict()`` instead
+    * Removed ``GenerativeModelHook.generate_text_embeddings()``. Please use ``GenerativeModelHook.text_generation_model_predict()`` instead
+
 12.0.0
 ......
 

diff --git a/providers/src/airflow/providers/google/cloud/hooks/vertex_ai/generative_model.py b/providers/src/airflow/providers/google/cloud/hooks/vertex_ai/generative_model.py
@@ -24,7 +24,7 @@
 from typing import TYPE_CHECKING
 
 import vertexai
-from vertexai.generative_models import GenerativeModel, Part
+from vertexai.generative_models import GenerativeModel
 from vertexai.language_models import TextEmbeddingModel, TextGenerationModel
 from vertexai.preview.caching import CachedContent
 from vertexai.preview.evaluation import EvalResult, EvalTask
@@ -100,186 +100,6 @@ def get_cached_context_model(
         cached_context_model = preview_generative_model.from_cached_content(cached_content)
         return cached_context_model
 
-    @deprecated(
-        planned_removal_date="January 01, 2025",
-        use_instead="Part objects included in contents parameter of "
-        "airflow.providers.google.cloud.hooks.generative_model."
-        "GenerativeModelHook.generative_model_generate_content",
-        category=AirflowProviderDeprecationWarning,
-    )
-    def get_generative_model_part(self, content_gcs_path: str, content_mime_type: str | None = None) -> Part:
-        """Return a Generative Model Part object."""
-        part = Part.from_uri(content_gcs_path, mime_type=content_mime_type)
-        return part
-
-    @deprecated(
-        planned_removal_date="January 01, 2025",
-        use_instead="airflow.providers.google.cloud.hooks.generative_model."
-        "GenerativeModelHook.text_generation_model_predict",
-        category=AirflowProviderDeprecationWarning,
-    )
-    @GoogleBaseHook.fallback_to_default_project_id
-    def prompt_language_model(
-        self,
-        prompt: str,
-        pretrained_model: str,
-        temperature: float,
-        max_output_tokens: int,
-        top_p: float,
-        top_k: int,
-        location: str,
-        project_id: str = PROVIDE_PROJECT_ID,
-    ) -> str:
-        """
-        Use the Vertex AI PaLM API to generate natural language text.
-
-        :param project_id: Required. The ID of the Google Cloud project that the service belongs to.
-        :param location: Required. The ID of the Google Cloud location that the service belongs to.
-        :param prompt: Required. Inputs or queries that a user or a program gives
-            to the Vertex AI PaLM API, in order to elicit a specific response.
-        :param pretrained_model: A pre-trained model optimized for performing natural
-            language tasks such as classification, summarization, extraction, content
-            creation, and ideation.
-        :param temperature: Temperature controls the degree of randomness in token
-            selection.
-        :param max_output_tokens: Token limit determines the maximum amount of text
-            output.
-        :param top_p: Tokens are selected from most probable to least until the sum
-            of their probabilities equals the top_p value. Defaults to 0.8.
-        :param top_k: A top_k of 1 means the selected token is the most probable
-            among all tokens.
-        """
-        vertexai.init(project=project_id, location=location, credentials=self.get_credentials())
-
-        parameters = {
-            "temperature": temperature,
-            "max_output_tokens": max_output_tokens,
-            "top_p": top_p,
-            "top_k": top_k,
-        }
-
-        model = self.get_text_generation_model(pretrained_model)
-
-        response = model.predict(
-            prompt=prompt,
-            **parameters,
-        )
-        return response.text
-
-    @deprecated(
-        planned_removal_date="January 01, 2025",
-        use_instead="airflow.providers.google.cloud.hooks.generative_model."
-        "GenerativeModelHook.text_embedding_model_get_embeddings",
-        category=AirflowProviderDeprecationWarning,
-    )
-    @GoogleBaseHook.fallback_to_default_project_id
-    def generate_text_embeddings(
-        self,
-        prompt: str,
-        pretrained_model: str,
-        location: str,
-        project_id: str = PROVIDE_PROJECT_ID,
-    ) -> list:
-        """
-        Use the Vertex AI PaLM API to generate text embeddings.
-
-        :param project_id: Required. The ID of the Google Cloud project that the service belongs to.
-        :param location: Required. The ID of the Google Cloud location that the service belongs to.
-        :param prompt: Required. Inputs or queries that a user or a program gives
-            to the Vertex AI PaLM API, in order to elicit a specific response.
-        :param pretrained_model: A pre-trained model optimized for generating text embeddings.
-        """
-        vertexai.init(project=project_id, location=location, credentials=self.get_credentials())
-        model = self.get_text_embedding_model(pretrained_model)
-
-        response = model.get_embeddings([prompt])[0]  # single prompt
-
-        return response.values
-
-    @deprecated(
-        planned_removal_date="January 01, 2025",
-        use_instead="airflow.providers.google.cloud.hooks.generative_model."
-        "GenerativeModelHook.generative_model_generate_content",
-        category=AirflowProviderDeprecationWarning,
-    )
-    @GoogleBaseHook.fallback_to_default_project_id
-    def prompt_multimodal_model(
-        self,
-        prompt: str,
-        location: str,
-        generation_config: dict | None = None,
-        safety_settings: dict | None = None,
-        pretrained_model: str = "gemini-pro",
-        project_id: str = PROVIDE_PROJECT_ID,
-    ) -> str:
-        """
-        Use the Vertex AI Gemini Pro foundation model to generate natural language text.
-
-        :param project_id: Required. The ID of the Google Cloud project that the service belongs to.
-        :param location: Required. The ID of the Google Cloud location that the service belongs to.
-        :param prompt: Required. Inputs or queries that a user or a program gives
-            to the Multi-modal model, in order to elicit a specific response.
-        :param generation_config: Optional. Generation configuration settings.
-        :param safety_settings: Optional. Per request settings for blocking unsafe content.
-        :param pretrained_model: By default uses the pre-trained model `gemini-pro`,
-            supporting prompts with text-only input, including natural language
-            tasks, multi-turn text and code chat, and code generation. It can
-            output text and code.
-        """
-        vertexai.init(project=project_id, location=location, credentials=self.get_credentials())
-
-        model = self.get_generative_model(pretrained_model)
-        response = model.generate_content(
-            contents=[prompt], generation_config=generation_config, safety_settings=safety_settings
-        )
-
-        return response.text
-
-    @deprecated(
-        planned_removal_date="January 01, 2025",
-        use_instead="airflow.providers.google.cloud.hooks.generative_model."
-        "GenerativeModelHook.generative_model_generate_content",
-        category=AirflowProviderDeprecationWarning,
-    )
-    @GoogleBaseHook.fallback_to_default_project_id
-    def prompt_multimodal_model_with_media(
-        self,
-        prompt: str,
-        location: str,
-        media_gcs_path: str,
-        mime_type: str,
-        generation_config: dict | None = None,
-        safety_settings: dict | None = None,
-        pretrained_model: str = "gemini-pro-vision",
-        project_id: str = PROVIDE_PROJECT_ID,
-    ) -> str:
-        """
-        Use the Vertex AI Gemini Pro foundation model to generate natural language text.
-
-        :param project_id: Required. The ID of the Google Cloud project that the service belongs to.
-        :param location: Required. The ID of the Google Cloud location that the service belongs to.
-        :param prompt: Required. Inputs or queries that a user or a program gives
-            to the Multi-modal model, in order to elicit a specific response.
-        :param generation_config: Optional. Generation configuration settings.
-        :param safety_settings: Optional. Per request settings for blocking unsafe content.
-        :param pretrained_model: By default uses the pre-trained model `gemini-pro-vision`,
-            supporting prompts with text-only input, including natural language
-            tasks, multi-turn text and code chat, and code generation. It can
-            output text and code.
-        :param media_gcs_path: A GCS path to a content file such as an image or a video.
-            Can be passed to the multi-modal model as part of the prompt. Used with vision models.
-        :param mime_type: Validates the media type presented by the file in the media_gcs_path.
-        """
-        vertexai.init(project=project_id, location=location, credentials=self.get_credentials())
-
-        model = self.get_generative_model(pretrained_model)
-        part = self.get_generative_model_part(media_gcs_path, mime_type)
-        response = model.generate_content(
-            contents=[prompt, part], generation_config=generation_config, safety_settings=safety_settings
-        )
-
-        return response.text
-
     @deprecated(
         planned_removal_date="April 09, 2025",
         use_instead="GenerativeModelHook.generative_model_generate_content",