Skip to content

[Fix] 제미나이 이미지 분석 에러 수정 #149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nomorefifa merged 2 commits into from
Dec 15, 2025
Merged

[Fix] 제미나이 이미지 분석 에러 수정 #149

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
55eb07a
chore: fix gemini error
nomorefifa Dec 15, 2025
fd45030
fix: gemini image error
nomorefifa Dec 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
204 changes: 108 additions & 96 deletions src/main/java/com/salemale/domain/item/service/ItemImageAiService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.web.reactive.function.client.WebClient;
import reactor.util.retry.Retry;
import software.amazon.awssdk.core.ResponseBytes;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
Expand All @@ -30,183 +31,196 @@
@Service
public class ItemImageAiService {

private final WebClient webClient; // WebClient를 인스턴스 필드로 재사용
private final WebClient geminiWebClient;
private final S3Client s3Client;
private final ObjectMapper objectMapper;

@Value("${gemini.api-key}")
private String apiKey;

@Value("${gemini.model}")
private String model;

@Value("${gemini.api-url}")
private String apiUrl;

@Value("${aws.s3.bucket}")
private String bucketName;

@Value("${aws.s3.region}")
private String region;

// 생성자에서 WebClient 설정 (DI로 생성)
public ItemImageAiService(WebClient.Builder webClientBuilder, S3Client s3Client, ObjectMapper objectMapper,
@Value("${gemini.api-key}") String apiKey,
@Value("${gemini.api-url}") String apiUrl) {
// 이미지 크기 제한 (Base64 인코딩 후 4MB)
private static final int MAX_IMAGE_SIZE = 4 * 1024 * 1024;
Comment on lines +47 to +48
Copy link

@coderabbitai coderabbitai bot Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

Gemini API maximum image size limit base64 encoded

💡 Result:

Short answer: when sending an image inline (base64) to Gemini the per-file limit is generally 7 MB. Uploading from Cloud Storage can allow larger files (e.g., up to 30 MB for some models). Also account for base64 overhead (~33% size increase). [1][2][3]

Sources:

  • Google Cloud/Vertex AI image limits (inline = 7 MB; GCS larger). [1]
  • Vertex AI / Gemini image-understanding page with per-model inline limits (7 MB) and larger GCS limits. [2]
  • Firebase/Vertex-AI input-file requirements (notes on inline/base64 encoding and scaling). [3]

🏁 Script executed:

# Examine the ItemImageAiService.java file to understand the context
find . -type f -name "ItemImageAiService.java" | head -1 | xargs wc -l

Repository: Team-SaleMale/BE

Length of output: 136

🏁 Script executed:

# Read the relevant sections of the file
find . -type f -name "ItemImageAiService.java" | head -1 | xargs cat -n | sed -n '40,90p'

Repository: Team-SaleMale/BE

Length of output: 1897

Update MAX_IMAGE_SIZE to 7 MB to match Gemini API specifications.

The Gemini API supports up to 7 MB for base64-encoded images sent inline. The current 4 MB limit is unnecessarily restrictive. Update line 48 to private static final int MAX_IMAGE_SIZE = 7 * 1024 * 1024; and the comment accordingly (also applies to line 47).

🤖 Prompt for AI Agents 
In src/main/java/com/salemale/domain/item/service/ItemImageAiService.java around
lines 47 to 48, the MAX_IMAGE_SIZE constant and its comment are set to 4 MB but
should be 7 MB to match Gemini API specs; update the comment to indicate "Base64
인코딩 후 7MB" and change the constant to use 7 * 1024 * 1024 so the limit aligns
with the API.


public ItemImageAiService(
S3Client s3Client,
ObjectMapper objectMapper,
@Value("${gemini.api-key}") String apiKey,
@Value("${gemini.api-url}") String apiUrl) {

this.s3Client = s3Client;
this.objectMapper = objectMapper;
this.apiKey = apiKey;
this.apiUrl = apiUrl;

// baseUrl과 기본 헤더 설정
this.webClient = webClientBuilder
// Gemini 전용 WebClient 설정
this.geminiWebClient = WebClient.builder()
.baseUrl(apiUrl)
.defaultHeader("x-goog-api-key", apiKey)
.defaultHeader("Content-Type", "application/json")
.build();
}

/**
* 이미지 URL을 받아 상품 정보를 분석합니다.
* @param imageUrl S3 temp 폴더의 이미지 URL
* @return AI가 분석한 상품 정보
* 이미지 분석
*/
public ProductAnalysisResponse analyzeProductImage(String imageUrl) {

// 1. temp URL 검증
if (!imageUrl.contains("/temp/")) {
throw new GeneralException(ErrorStatus.IMAGE_NOT_TEMP_URL);
}

// 2. S3에서 이미지 다운로드
byte[] imageBytes = downloadImageFromS3(imageUrl);
// 2. S3에서 이미지 다운로드 (MIME 타입 포함)
ImageData imageData = downloadImageFromS3(imageUrl);

// 3. Base64 인코딩
String base64Image = Base64.getEncoder().encodeToString(imageBytes);
// 3. 이미지 크기 검증
if (imageData.base64Data.length() > MAX_IMAGE_SIZE) {
throw new GeneralException(ErrorStatus.IMAGE_SIZE_EXCEEDED);
}

// 4. Gemini API 호출
String responseText = callGeminiApi(base64Image);
String responseText = callGeminiApi(imageData);

// 5. 응답 파싱 및 DTO 생성
// 5. 응답 파싱
return parseGeminiResponse(responseText);
}

/**
* S3에서 이미지를 다운로드합니다.
* S3에서 이미지와 MIME 타입 다운로드
*/
private byte[] downloadImageFromS3(String imageUrl) {
private ImageData downloadImageFromS3(String imageUrl) {
try {
// URL에서 S3 키 추출
String s3Key = extractS3KeyFromUrl(imageUrl);

// S3에서 이미지 가져오기
GetObjectRequest getObjectRequest = GetObjectRequest.builder()
.bucket(bucketName)
.key(s3Key)
.build();

ResponseBytes<GetObjectResponse> objectBytes = s3Client.getObjectAsBytes(getObjectRequest);
log.info("S3 이미지 다운로드 성공 - 크기: {} bytes", objectBytes.asByteArray().length);

// 실제 MIME 타입 추출
String contentType = objectBytes.response().contentType();
if (contentType == null || contentType.isEmpty()) {
contentType = detectMimeType(s3Key);
}

byte[] imageBytes = objectBytes.asByteArray();
String base64Data = Base64.getEncoder().encodeToString(imageBytes);

return objectBytes.asByteArray();
return new ImageData(base64Data, contentType);

} catch (S3Exception e) {
log.error("S3 이미지 다운로드 실패: {}", e.getMessage());
throw new GeneralException(ErrorStatus.IMAGE_DOWNLOAD_FAILED);
}
}

/**
* Gemini API를 호출하여 이미지 분석 결과를 받습니다.
* 파일 확장자로 MIME 타입 추정
*/
private String callGeminiApi(String base64Image) {
private String detectMimeType(String s3Key) {
String lowerKey = s3Key.toLowerCase();
if (lowerKey.endsWith(".png")) return "image/png";
if (lowerKey.endsWith(".jpg") || lowerKey.endsWith(".jpeg")) return "image/jpeg";
if (lowerKey.endsWith(".webp")) return "image/webp";
if (lowerKey.endsWith(".gif")) return "image/gif";
return "image/jpeg"; // 기본값
}

/**
* Gemini API 호출 (재시도 로직 포함)
*/
private String callGeminiApi(ImageData imageData) {
try {
Map<String, Object> requestBody = createGeminiRequestBody(base64Image);
Map<String, Object> requestBody = createGeminiRequestBody(imageData);

// 인스턴스의 webClient 사용
String response = webClient.post()
.uri("/" + model + ":generateContent") // baseUrl 이후 경로만 지정
String response = geminiWebClient.post()
.uri("/" + model + ":generateContent")
.bodyValue(requestBody)
.retrieve()
.bodyToMono(String.class)
.block(Duration.ofSeconds(30)); // 동기 호출 유지
.timeout(Duration.ofSeconds(30))
.retryWhen(Retry.backoff(3, Duration.ofSeconds(1))
.maxBackoff(Duration.ofSeconds(5))
.filter(throwable -> !(throwable instanceof GeneralException))
)
.block();

return response;

} catch (Exception e) {
log.error("Gemini API 호출 실패: {}", e.getMessage());
throw new GeneralException(ErrorStatus.GEMINI_API_ERROR);
}
}
Comment on lines +136 to 157
Copy link

@coderabbitai coderabbitai bot Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major

Refine the retry filter to avoid retrying non-transient errors.

The retry filter !(throwable instanceof GeneralException) may not work as intended because WebClient throws WebClientException or WebClientResponseException, not GeneralException. This could result in retrying 4xx client errors (e.g., bad request, unauthorized) that should fail immediately.

Consider filtering by HTTP status code:

-                    .retryWhen(Retry.backoff(3, Duration.ofSeconds(1))
-                        .maxBackoff(Duration.ofSeconds(5))
-                        .filter(throwable -> !(throwable instanceof GeneralException))
-                    )
+                    .retryWhen(Retry.backoff(3, Duration.ofSeconds(1))
+                        .maxBackoff(Duration.ofSeconds(5))
+                        .filter(throwable -> {
+                            if (throwable instanceof WebClientResponseException) {
+                                int status = ((WebClientResponseException) throwable).getStatusCode().value();
+                                return status >= 500; // Retry only 5xx server errors
+                            }
+                            return true; // Retry network errors
+                        })
+                    )

Don't forget to add the import:

import org.springframework.web.reactive.function.client.WebClientResponseException;


/**
* Gemini API 요청 본문을 생성합니다.
* Gemini API 요청 본문 생성 (최적화)
*/
private Map<String, Object> createGeminiRequestBody(String base64Image) {
// 프롬프트 설정
String prompt = """
이 상품 이미지를 분석해서 다음 정보를 JSON 형식으로 추출해:
{
"productName": "상품의 전체 이름 (브랜드 + 모델명 포함)",
"category": "카테고리",
"confidence": 신뢰도
}

중요: productName은 반드시 구체적으로 작성해
예시: "메리다 스컬트라 100", "삼성 갤럭시 S24", "나이키 에어맥스 270", "아이폰 17PRO"

카테고리 목록:
HOME_APPLIANCE, HEALTH_FOOD, BEAUTY, FOOD_PROCESSED, PET,
DIGITAL, LIVING_KITCHEN, WOMEN_ACC, SPORTS, PLANT,
GAME_HOBBY, TICKET, FURNITURE, BOOK, KIDS, CLOTHES, ETC

JSON 형식만 출력하면돼.
""";

// 이미지 파트
Map<String, Object> imagePart = new HashMap<>();
Map<String, String> inlineData = new HashMap<>();
inlineData.put("mime_type", "image/jpeg");
inlineData.put("data", base64Image);
imagePart.put("inline_data", inlineData);
private Map<String, Object> createGeminiRequestBody(ImageData imageData) {
// 단순하고 명확한 프롬프트
String prompt = "Analyze this product image and extract information in JSON format:\n" +
"{\n" +
" \"productName\": \"full product name with brand and model\",\n" +
" \"category\": \"one of: HOME_APPLIANCE, HEALTH_FOOD, BEAUTY, FOOD_PROCESSED, PET, DIGITAL, LIVING_KITCHEN, WOMEN_ACC, SPORTS, PLANT, GAME_HOBBY, TICKET, FURNITURE, BOOK, KIDS, CLOTHES, ETC\",\n" +
" \"confidence\": 0.0 to 1.0\n" +
"}\n" +
"Examples: \"Samsung Galaxy S24\", \"Nike Air Max 270\", \"iPhone 15 Pro\"\n" +
"Return ONLY the JSON object, no markdown, no explanation.";

// 텍스트 파트
Map<String, String> textPart = new HashMap<>();
textPart.put("text", prompt);

// 이미지 파트 (정확한 MIME 타입 사용)
Map<String, Object> imagePart = new HashMap<>();
Map<String, String> inlineData = new HashMap<>();
inlineData.put("mime_type", imageData.mimeType);
inlineData.put("data", imageData.base64Data);
imagePart.put("inline_data", inlineData);

// 컨텐츠 구성
Map<String, Object> content = new HashMap<>();
content.put("parts", List.of(textPart, imagePart));

// generationConfig 추가 (JSON 응답 강제)
Map<String, Object> generationConfig = new HashMap<>();
generationConfig.put("temperature", 0.4); // 일관성 높이기
generationConfig.put("topP", 0.8);
generationConfig.put("topK", 40);
generationConfig.put("maxOutputTokens", 512);
generationConfig.put("responseMimeType", "application/json"); // JSON 강제

// 최종 요청 본문
Map<String, Object> requestBody = new HashMap<>();
requestBody.put("contents", List.of(content));
requestBody.put("generationConfig", generationConfig);

return requestBody;
}

/**
* Gemini 응답을 파싱하여 DTO로 변환합니다.
* Gemini 응답 파싱
*/
private ProductAnalysisResponse parseGeminiResponse(String responseText) {
try {
// JSON 파싱
JsonNode rootNode = objectMapper.readTree(responseText);
JsonNode candidatesNode = rootNode.path("candidates");

if (candidatesNode.isEmpty()) {
throw new GeneralException(ErrorStatus.IMAGE_ANALYSIS_FAILED);
}

// 텍스트 추출
String text = candidatesNode.get(0)
.path("content")
.path("parts")
.get(0)
.path("text")
.asText();

// JSON 부분만 추출 (마크다운 코드 블록 제거)
// JSON 정제 (마크다운 제거)
String jsonText = extractJsonFromMarkdown(text);

// 상품 정보 파싱
Expand All @@ -219,16 +233,14 @@ private ProductAnalysisResponse parseGeminiResponse(String responseText) {
.build();

} catch (Exception e) {
log.error("Gemini 응답 파싱 실패: {}", e.getMessage());
throw new GeneralException(ErrorStatus.IMAGE_ANALYSIS_FAILED);
}
}

/**
* 마크다운 코드 블록에서 JSON 추출
* 마크다운 제거
*/
private String extractJsonFromMarkdown(String text) {
// ```json 또는 ``` 블록 제거
String cleaned = text.trim();
if (cleaned.startsWith("```json")) {
cleaned = cleaned.substring(7);
Expand All @@ -242,64 +254,64 @@ private String extractJsonFromMarkdown(String text) {
}

/**
* 문자열을 Category enum으로 변환
* 카테고리 파싱
*/
private Category parseCategory(String categoryStr) {
try {
return Category.valueOf(categoryStr.toUpperCase());
} catch (IllegalArgumentException e) {
log.warn("알 수 없는 카테고리: {}. ETC로 설정합니다.", categoryStr);
return Category.ETC;
}
}

/**
* URL에서 S3 키 추출
* S3 키 추출
*/
private String extractS3KeyFromUrl(String url) {
try {
// URL 디코딩
String decodedUrl = URLDecoder.decode(url, StandardCharsets.UTF_8);

log.debug("S3 URL 파싱 시작 - URL: {}", decodedUrl);

// 정규식을 사용한 안전한 파싱
// Virtual-hosted-style: https://bucket.s3.region.amazonaws.com/key
// Virtual-hosted-style
String virtualHostedPattern = "https?://" + Pattern.quote(bucketName) +
"\\.s3\\." + Pattern.quote(region) + "\\.amazonaws\\.com/(.+)";

Pattern pattern = Pattern.compile(virtualHostedPattern);
Matcher matcher = pattern.matcher(decodedUrl);

if (matcher.find()) {
String key = matcher.group(1);
log.debug("추출된 S3 Key: {}", key);
return key;
return matcher.group(1);
}

// Path-style: https://s3.region.amazonaws.com/bucket/key
// Path-style
String pathStylePattern = "https?://s3\\." + Pattern.quote(region) +
"\\.amazonaws\\.com/" + Pattern.quote(bucketName) + "/(.+)";

pattern = Pattern.compile(pathStylePattern);
matcher = pattern.matcher(decodedUrl);

if (matcher.find()) {
String key = matcher.group(1);
log.debug("추출된 S3 Key (Path-style): {}", key);
return key;
return matcher.group(1);
}

// 파싱 실패
log.error("URL 파싱 실패 - URL: {}, Bucket: {}, Region: {}",
decodedUrl, bucketName, region);
throw new GeneralException(ErrorStatus.INVALID_IMAGE_URL);

} catch (GeneralException e) {
throw e; // 이미 처리된 예외는 그대로 전달
throw e;
} catch (Exception e) {
log.error("URL 파싱 중 예상치 못한 에러: {}", e.getMessage());
throw new GeneralException(ErrorStatus.INVALID_IMAGE_URL);
}
}
}

/**
* 이미지 데이터 클래스
*/
private static class ImageData {
private final String base64Data;
private final String mimeType;

public ImageData(String base64Data, String mimeType) {
this.base64Data = base64Data;
this.mimeType = mimeType;
}
}
}