feat(genai): Add new GenAI SDK samples (WIP) #10095
Conversation
There was a problem hiding this comment.
Hello @msampathkumar, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
Summary of Changes
Hello! Gemini here, providing a summary of this pull request. This PR, titled "feat(genai): Add new GenAI SDK samples (WIP)", aims to introduce a collection of new sample code snippets for the Google Cloud Generative AI Java SDK. These samples demonstrate various functionalities of the SDK, including different input modalities (text, image, video), token counting, streaming responses, function calling, and controlled generation using schemas and system instructions. The PR description indicates this is a work in progress and includes a checklist relevant to adding new samples.
Highlights
- New GenAI Samples: Adds multiple new Java code samples demonstrating how to use the GenAI SDK for various tasks.
- Multi-modal Input: Includes samples showing how to generate content using combinations of text, images (from GCS and local files), and video.
- Token Counting: Provides samples for counting tokens for both text-only and multi-modal (text and video) inputs.
- Advanced Features: Demonstrates features like streaming content generation, function calling with function descriptions, controlled generation using enum schemas, and using system instructions.
- Integration Tests: Adds corresponding integration tests to verify the functionality of the new samples.
Changelog
Click here to see the changelog
- genai/snippets/src/main/java/genai/gemini/CountTokensWithText.java
- Adds a new sample demonstrating how to count tokens for text input using the GenAI SDK.
- genai/snippets/src/main/java/genai/gemini/CountTokensWithTextAndVideo.java
- Adds a new sample demonstrating how to count tokens for text and video input using the GenAI SDK.
- genai/snippets/src/main/java/genai/gemini/GenerateContentStream.java
- Adds a new sample demonstrating streaming content generation with text input using the GenAI SDK.
- genai/snippets/src/main/java/genai/gemini/GenerateContentWithEnumSchema.java
- Adds a new sample demonstrating controlled content generation using an enum schema with the GenAI SDK.
- genai/snippets/src/main/java/genai/gemini/GenerateContentWithFunctionDescription.java
- Adds a new sample demonstrating function calling with text input and a function description using the GenAI SDK.
- genai/snippets/src/main/java/genai/gemini/GenerateContentWithSystemInstruction.java
- Adds a new sample demonstrating content generation with a system instruction using the GenAI SDK.
- genai/snippets/src/main/java/genai/gemini/GenerateContentWithText.java
- Adds a new sample demonstrating basic content generation with text input using the GenAI SDK.
- genai/snippets/src/main/java/genai/gemini/GenerateContentWithTextAndImage.java
- Adds a new sample demonstrating content generation with text and image input (from GCS) using the GenAI SDK.
- genai/snippets/src/main/java/genai/gemini/GenerateContentWithVideo.java
- Adds a new sample demonstrating content generation with text and video input (from GCS) using the GenAI SDK.
- genai/snippets/src/main/java/genai/gemini/TextGenerationWithMultiImage.java
- Adds a new sample demonstrating content generation with text and multiple image inputs (from local file and GCS) using the GenAI SDK.
- genai/snippets/src/test/java/genai/gemini/SnippetsIT.java
- Adds integration tests for the newly added GenAI SDK samples.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
New samples arrive,
Showing AI's many modes,
Code helps us explore.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
This pull request introduces several new GenAI SDK samples, which is a valuable addition. The samples cover various functionalities like token counting, content generation with different modalities (text, image, video), streaming, function calling, and system instructions. The code is generally clear and follows a consistent structure for snippets.
I've identified a few areas for improvement, primarily concerning safe handling of Optional types and the brittleness of some test assertions. Addressing these will enhance the robustness and reliability of the samples and tests.
No specific style guide was provided, so feedback regarding style or maintainability is based on common Java best practices, generally aligned with principles from guides like the Google Java Style Guide.
Summary of Findings
- Unsafe Optional Handling: In
GenerateContentWithFunctionDescription.java,Optional<List<FunctionCall>>is accessed without proper checks, potentially leading to runtime exceptions. (Addressed in review comments) - Brittle Test Assertions: Tests in
SnippetsIT.javause hardcoded exact values for token counts, which can make tests fail if underlying models or tokenizers change slightly. (Addressed in review comments) - Hardcoded Values in Samples: Many samples use hardcoded values for
modelId, prompts, GCS URIs, and local file paths (e.g.,CountTokensWithText.java:17,GenerateContentWithTextAndImage.java:29). While theTODO(developer)comments address this, it's a common characteristic of these samples. For production code, these should be configurable. (Not commented directly due to review settings - low severity) - Unused Imports and Variables: Some files contain unused imports or variables:
GenerateContentStream.java: Unused importcom.google.genai.types.GenerateContentConfig(line 7).SnippetsIT.java: Unused importscom.google.gson.annotations.SerializedName(line 26),java.util.Base64(line 34).SnippetsIT.java: Unused constantsTARGET_LANGUAGE_CODE(line 55),TEXT_TO_TRANSLATE(line 56).
(Not commented directly due to review settings - low severity)
- Clarity of Output in Samples: Some samples print the entire response object (e.g.,
CountTokensWithText.java:36). It might be more illustrative for users if the samples printed specific, relevant fields from the response, aligning with the example output comments. (Not commented directly due to review settings - low severity) - Handling of Return Values in
mainMethods: Themainmethods in several snippets call functions that return values, but these return values are not always used or printed bymain. This is generally acceptable for snippets focused onSystem.out.printwithin the called method. (Not commented directly due to review settings - low severity) - Potentially Unused/Misconfigured Test Elements: In
SnippetsIT.java:
- The
GEMINI_FLASH_1_5constant (line 51) is identical toGEMINI_FLASH. If a different 1.5 model was intended (e.g., for the commented-out tests), this might be a misconfiguration. - The
readImageFilemethod (line 90) appears unused by the active tests in this file.
(Not commented directly due to review settings - low severity)
Merge Readiness
The pull request adds useful GenAI SDK samples. However, there are a few high and medium severity issues related to unsafe Optional handling and brittle test assertions that should be addressed before merging to ensure code correctness and test stability. Other minor points are noted in the summary for future consideration. I am unable to approve pull requests, but I recommend making the suggested changes before this PR is merged by an authorized reviewer.
| System.out.println(response.functionCalls().get(0)); | ||
| return response.functionCalls().toString(); |
There was a problem hiding this comment.
There are a couple of concerns here:
- Unsafe
OptionalAccess:response.functionCalls()returns anOptional<List<FunctionCall>>. Calling.get()withoutisPresent()check can lead toNoSuchElementException. Furthermore, if the list is present but empty,get(0)on the list will cause anIndexOutOfBoundsException. - Meaningfulness of Return Value:
response.functionCalls().toString()returns the string representation of theOptionalobject itself (e.g., "Optional[...]"), not the content of the function call. This might not be the most useful value if this method's result is to be used programmatically.
Consider checking for presence and emptiness before accessing elements, and returning a more direct representation of the function call data.
Optional<List<FunctionCall>> functionCallsOptional = response.functionCalls();
if (functionCallsOptional.isPresent() && !functionCallsOptional.get().isEmpty()) {
FunctionCall firstCall = functionCallsOptional.get().get(0);
System.out.println(firstCall);
return firstCall.toString(); // Or a more specific part of the FunctionCall if needed
} else {
System.out.println("No function calls returned by the model or the list was empty.");
return "No function calls returned"; // Indicate absence of function calls
}| @Test | ||
| public void testTokenCount() throws Exception { | ||
| int tokenCount = GetTokenCount.getTokenCount(PROJECT_ID, LOCATION, GEMINI_FLASH); | ||
| assertThat(tokenCount).isEqualTo(6); |
There was a problem hiding this comment.
Asserting an exact token count (isEqualTo(6)) can make this test brittle. Token counts might change slightly with model updates or minor tokenizer adjustments.
Could this assertion be made more flexible, for example, by checking if the token count is greater than zero, or within a small expected range? Alternatively, if this count is stable and well-understood for the specific prompt and model, adding a comment explaining why 6 tokens are expected could be helpful for future maintainers.
| @Test | ||
| public void testMediaTokenCount() throws Exception { | ||
| int tokenCount = GetMediaTokenCount.getMediaTokenCount(PROJECT_ID, LOCATION, GEMINI_FLASH); | ||
| assertThat(tokenCount).isEqualTo(16252); |
There was a problem hiding this comment.
Similar to the previous token count assertion, asserting an exact media token count (isEqualTo(16252)) can lead to a brittle test. These counts can be sensitive to model or processing changes.
Would it be possible to use a range or a 'greater than' assertion here? If the exact number is critical and expected to be stable, a comment explaining its derivation would add clarity.
Description
Fixes #
Note: Before submitting a pull request, please open an issue for discussion if you are not associated with Google.
Checklist
pom.xmlparent set to latestshared-configurationmvn clean verifyrequiredmvn -P lint checkstyle:checkrequiredmvn -P lint clean compile pmd:cpd-check spotbugs:checkadvisory only