Skip to content

Conversation

@gr2m
Copy link
Collaborator

@gr2m gr2m commented Nov 21, 2025

Background

generateText() / streamText() doesn't pass thought signature for image parts in response messages. That breaks multi-turn chats with gemini-3-pro-image-preview

Summary

  • Gemini now supports non-PNG images in assistant messages
  • Add thought signature to image parts in response.messages
  • spec update: LanguageModelV3File#providerMetadata (optional field)

Manual Verification

import { google } from '@ai-sdk/google';
import { generateText } from 'ai';
import 'dotenv/config';
import { presentImages } from '../lib/present-image';

async function main() {
  const step1 = await generateText({
    model: google('gemini-3-pro-image-preview'),
    prompt: 'Create an image of Los Angeles where all car infrastructure has been replaced with bike infrastructure, trains, pedestrian zones, and parks. The image should be photorealistic and vibrant.',
  });

  await presentImages(step1.files);

  const step2 = await generateText({
    model: google('gemini-3-pro-image-preview'),
    messages: [
      ...step1.response.messages,
      {
        role: 'user',
        content: 'Now create a variation of the image, but in the style of a watercolor painting.',
      },
    ]
  });

  await presentImages(step2.files);
}

main().catch(console.error);

Checklist

  • Tests have been added / updated (for bug fixes / features)
  • Documentation has been added / updated (for bug fixes / features)
  • A patch changeset for relevant packages has been added (for bug fixes / features - run pnpm changeset in the project root)
  • I have reviewed this pull request (self-review)

Related Issues

Fixes #10441
Closes #7979

@gr2m
Copy link
Collaborator Author

gr2m commented Nov 21, 2025

I verified that body.contents[0].parts[0].thoughtSignature (or body.contents[0].parts[0].thought_signature) needs to be set in the request body.

Problem is that this will need a spec change because language-model-v3-file.ts does not defined providerMetadata?: SharedV3ProviderMetadata; yet (unlike all other LanguageModelV3Content types)

but we would need to pass it via providerMetadata here

6078060/packages/google/src/google-generative-ai-language-model.ts#L268-L274

same way we pass it for function calls above

Which means we cannot backport it.

Or I'm missing something 🤔

…and set it in `convertToGoogleGenerativeAIMessages()`
@gr2m gr2m added bug Something isn't working as documented provider/google and removed provider/black-forest-labs Issues related to the @ai-sdk/black-forst-labs provider. https://bfl.ai labels Nov 22, 2025
@gr2m
Copy link
Collaborator Author

gr2m commented Nov 22, 2025

we basically ran into #7979

gr2m added 2 commits November 23, 2025 14:24
…-gemini-3-pro-image-preview-in-middle-of-a-conversation-breaks-thought_signature
@gr2m gr2m marked this pull request as ready for review November 23, 2025 22:57
@gr2m gr2m changed the title fix(google): gemini 3 pro image parts and thought signature fix(google): add thought signature to gemini 3 pro image parts Nov 23, 2025
@gr2m gr2m added the backport label Nov 23, 2025
@gr2m
Copy link
Collaborator Author

gr2m commented Nov 23, 2025

I will start the backport pull request, but won't merge until it's absolutely necessary, since the spec change is a breaking change und might cause friction to users unaffected by the bug this fixes.

@gr2m gr2m enabled auto-merge (squash) November 23, 2025 23:09
@gr2m gr2m merged commit db913bd into main Nov 23, 2025
22 checks passed
@gr2m gr2m deleted the 10441-switching-from-gemini-3-pro-preview-to-gemini-3-pro-image-preview-in-middle-of-a-conversation-breaks-thought_signature branch November 23, 2025 23:11
vercel-ai-sdk bot pushed a commit that referenced this pull request Nov 23, 2025
@vercel-ai-sdk vercel-ai-sdk bot removed the backport label Nov 23, 2025
@vercel-ai-sdk
Copy link
Contributor

vercel-ai-sdk bot commented Nov 23, 2025

⚠️ Backport to release-v5.0 created but has conflicts: #10508

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai/provider bug Something isn't working as documented provider/google

Projects

None yet

2 participants