Gemini API

Summary

Build with Gemini 2.0 Flash, 2.5 Pro, and Gemma using the Gemini API and Google AI Studio.

Release history

1. docs update Aug 30, 2026 · issue 011
   └──▷ USE IT
   Generate a high-resolution drone shot video at 1080p instead of the default 720p.
   json copy

   {
     "model": "gemini-omni-1.1-flash",
     "input": "A drone shot of a mountain landscape at sunrise.",
     "resolution": "1080p"
   }

   Create a video that transitions smoothly between two images using first-and-last-frame interpolation.
   json copy

   {
     "model": "gemini-omni-1.1-flash",
     "first_frame_b64": "<BASE64_IMAGE_1>",
     "last_frame_b64": "<BASE64_IMAGE_2>",
     "input": "A smooth cinematic transition from a lush green forest at sunrise to a snowy forest under a starry night sky."
   }

   Extend an uploaded video and introduce a new character from a reference image into the continued scene.
   json copy

   {
     "model": "gemini-omni-1.1-flash",
     "contents": [
       {"type": "document", "uri": "$VIDEO_URI"},
       {"type": "document", "uri": "$CHARACTER_IMG_URI"},
       {"type": "text", "text": "Extend this video: have the character shown in <IMAGE_REF_0> enter the scene and wave."}
     ]
   }

   • ›Adds resolution field to video generation config for gemini-omni-1.1-flash, supporting values 360p, 720p (default), 1080p, and 4K.
   • ›Adds first_frame_b64 / firstFrameB64 and last_frame_b64 / lastFrameB64 fields for first-and-last-frame video interpolation, animating a smooth transition between two images.
   • ›Adds video_config with extend mode to seamlessly continue an existing video (uploaded via Files API or generated in multi-turn) by 3–10 seconds per extension, up to a total of 40 seconds.
   • ›Supports reference media in video extension via the <IMAGE_REF_0> and <VIDEO_REF_0> prompt tags, allowing new characters or objects to be introduced into an extended clip (video references support up to 3 clips, up to 3 seconds each).
   • ›Adds explicit prompt tag syntax for declaring media roles: <FIRST_FRAME>, <LAST_FRAME>, <VIDEO_REF_N>, <VIDEO_0>, <PREVIOUS_VIDEO> inline, and structured [# Sources ...] / [# References ...] declarations for multi-input scenarios.
   +1 moreshow less

   • ›New gemini-omni-1.1-flash model is available for video generation.
2. docs update Aug 29, 2026 · issue 010
   └──▷ TRY IT
   Transcribe a recorded call with both speaker labels and word-level timestamps to build a searchable, attributed transcript.
   copy

   $ curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \
     -H "x-goog-api-key: $GEMINI_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "model": "gemini-3.5-transcribe",
       "input": [
         {
           "type": "audio",
           "uri": "YOUR_FILE_URI",
           "mime_type": "audio/mp3"
         }
       ],
       "generation_config": {
         "transcription_config": {
           "mode": {
             "type": "verbatim",
             "diarization_mode": "speaker",
             "timestamp_granularities": ["word"]
           }
         }
       }
     }'

   Transcribe a technical meeting recording with domain jargon and clean up filler words automatically for a readable summary.
   copy

   $ curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \
     -H "x-goog-api-key: $GEMINI_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "model": "gemini-3.5-transcribe",
       "input": [
         {
           "type": "audio",
           "uri": "YOUR_FILE_URI",
           "mime_type": "audio/mp3"
         }
       ],
       "generation_config": {
         "transcription_config": {
           "custom_vocabulary": ["Gemini", "Kubernetes", "BigQuery"],
           "mode": "smart"
         }
       }
     }'

   • ›New POST https://generativelanguage.googleapis.com/v1beta/interactions endpoint accepts audio input and returns transcriptions when model is set to gemini-3.5-transcribe.
   • ›New generation_config.transcription_config.language_codes field accepts BCP-47 codes (e.g. es-ES) to hint at spoken language, or an empty list for automatic detection across 85+ locales with dynamic code-switching support.
   • ›New generation_config.transcription_config.custom_vocabulary array accepts up to 1,000 domain-specific terms, brand names, or acronyms to bias recognition (optimal results with up to 100 terms).
   • ›New generation_config.transcription_config.mode.diarization_mode field (set to speaker) enables speaker diarization, tagging segments with labels like spk_1 / spk_2 for up to 8 speakers.
   • ›New generation_config.transcription_config.mode field supports two transcription modes: verbatim (default, exact word-for-word output) and smart (disfluency removal, inline self-corrections, automatic structured formatting into paragraphs/lists, and grammatical cleanup).
   +2 moreshow less

   • ›Transcription output is returned in interaction.output_text; when diarization_mode or timestamp_granularities is enabled, detailed word-level annotations are attached to the interaction content.
   • ›New gemini-3.5-transcribe model supports automatic language detection, diverse accents, background noise, and multi-language (code-switching) conversations without manual configuration.
3. changelog-20260828-455e7c2b Aug 28, 2026 · issue 009

   Gemini Omni Flash GA adds video extension, interpolation, and resolution control; Gemini 3.5 Transcribe GA brings streaming and non-streaming speech-to-text

   └──▷ TRY IT
   Generate a 1080p video continuation of an existing clip using the new extend task and resolution parameter.
   copy

   $ curl https://generativelanguage.googleapis.com/v1beta/models/gemini-omni-1.1-flash:generateContent \
     -H 'Content-Type: application/json' \
     -d '{
       "contents": [{"parts": [{"text": "Continue this video with a sunset scene"}]}],
       "generationConfig": {
         "task": "extend",
         "video_config": {
           "resolution": "1080p"
         }
       }
     }'

   Generate an interpolated video transitioning between two images using the image_to_video task on gemini-omni-1.1-flash.
   copy

   $ curl https://generativelanguage.googleapis.com/v1beta/models/gemini-omni-1.1-flash:generateContent \
     -H 'Content-Type: application/json' \
     -d '{
       "contents": [{"parts": [
         {"inline_data": {"mime_type": "image/jpeg", "data": "<first_frame_base64>"}},
         {"inline_data": {"mime_type": "image/jpeg", "data": "<last_frame_base64>"}}
       ]}],
       "generationConfig": {
         "task": "image_to_video",
         "video_config": {"resolution": "720p"}
       }
     }'

   • ›New resolution parameter in video_config for gemini-omni-1.1-flash supports 360p, 720p (default), and 1080p outputs (with upscaling for 1080p and 4K).
   • ›New extend task on gemini-omni-1.1-flash enables seamless video extension by generating continuations appended to an existing clip.
   • ›New image_to_video task with up to 2 images on gemini-omni-1.1-flash enables interpolation between a first and last frame to generate a transitioning video.
   • ›New gemini-3.5-transcribe-live model provides low-latency, bidirectional streaming speech-to-text over WebSockets via the Live API, supporting interim and finalized transcription events, Smart transcription mode, and multiple Voice Activity Detection (VAD) strategies.
   └──▷ BREAKING ON UPGRADE

   • !The existing (pre-GA) Gemini Omni endpoint will be deprecated on September 30, 2026.
4. docs update Aug 28, 2026 · issue 009

   Gemini API adds Gemini 3.7 Flash model with Java SDK support for configurable thinking levels

   • ›Adds ThinkingLevel enum (with values such as HIGH) to GenerationConfig in the Java SDK, enabling control over reasoning depth via com.google.genai.gaos.models.interactions.ThinkingLevel.
   • ›Introduces Gemini 3.7 Flash as a new available model for thinking/reasoning interactions.
   • ›New Java SDK classes CreateModelInteraction, InteractionsInput, GenerationConfig, Interaction, and CreateInteractionRequestBody under com.google.genai.gaos.models.interactions support structured reasoning API calls.
5. docs update Aug 28, 2026 · issue 009

   Gemini Live API adds real-time speech-to-text streaming with automatic language detection and custom vocabulary.

   • ›Adds Live Transcription: real-time, continuous speech-to-text streaming with automatic language detection and custom vocabulary, supporting use cases such as live subtitles, meeting transcription, voice dictation, and customer call logging.
   • ›Gemini 3.7 Flash is now available on the Live API.
6. docs update Aug 28, 2026 · issue 009

   Gemini API adds gemini-3.5-transcribe model with diarization, word timestamps, custom vocabulary, and smart transcription via POST /v1beta/interactions

   └──▷ TRY IT
   Transcribe a recorded meeting with per-word timestamps and speaker labels to attribute statements to individuals for downstream analysis.
   copy

   $ curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \
     -H "x-goog-api-key: $GEMINI_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "model": "gemini-3.5-transcribe",
       "input": [
         {
           "type": "audio",
           "uri": "YOUR_FILE_URI",
           "mime_type": "audio/mp3"
         }
       ],
       "generation_config": {
         "transcription_config": {
           "mode": {
             "type": "verbatim",
             "diarization_mode": "speaker",
             "timestamp_granularities": ["word"]
           }
         }
       }
     }'

   Transcribe a technical call with product-specific terminology, using custom vocabulary to improve recognition accuracy for jargon and brand names.
   copy

   $ curl -X POST "https://generativelanguage.googleapis.com/v1beta/interactions" \
     -H "x-goog-api-key: $GEMINI_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "model": "gemini-3.5-transcribe",
       "input": [
         {
           "type": "audio",
           "uri": "YOUR_FILE_URI",
           "mime_type": "audio/mp3"
         }
       ],
       "generation_config": {
         "transcription_config": {
           "custom_vocabulary": ["Gemini", "Kubernetes", "BigQuery"],
           "mode": "smart"
         }
       }
     }'

   • ›New POST https://generativelanguage.googleapis.com/v1beta/interactions endpoint accepting model: gemini-3.5-transcribe to convert speech in uploaded audio files to text.
   • ›Adds generation_config.transcription_config.language_codes field (BCP-47 list, e.g. ["es-ES"]) to hint or restrict language detection; omit or pass [] for automatic detection across 85+ locales with dynamic code-switching.
   • ›Adds generation_config.transcription_config.custom_vocabulary array (up to 1,000 terms, best results with up to 100) to bias recognition toward domain-specific terms, acronyms, and proper names.
   • ›Adds generation_config.transcription_config.mode.diarization_mode: "speaker" to tag each spoken segment with a speaker identifier (spk_1, spk_2, etc.); supports up to 8 speakers (3+ is experimental).
   • ›Adds generation_config.transcription_config.mode.timestamp_granularities: ["word"] to return exact start/end time offsets for every recognized word.
   +2 moreshow less

   • ›Adds generation_config.transcription_config.mode field accepting {"type": "verbatim"} (default, word-for-word with filler words preserved) or "smart" (removes disfluencies, resolves self-corrections, applies structured formatting such as numbered lists, currencies, and dates).
   • ›New gemini-3.5-transcribe model delivers automatic language identification, speaker diarization, word-level timestamps, and smart transcription in a single API call; interaction.output_text carries the full transcript.
7. docs update Aug 28, 2026 · issue 009

   Gemini Omni 1.1 Flash gains output resolution control (360p–4K), first/last frame interpolation, and video extension via prompt or Files API

   └──▷ TRY IT
   Generate a high-resolution video at 1080p when you need broadcast-quality output from a text prompt.
   copy

   $ curl -X POST https://api.gemini.example/v1/video:generate \
     -H 'Content-Type: application/json' \
     -d '{
       "model": "gemini-omni-1.1-flash",
       "input": "A drone shot of a mountain landscape at sunrise.",
       "resolution": "1080p"
     }'

   Animate a smooth transition between two keyframe images — useful for creating cinematic scene changes without manual frame-by-frame work.
   copy

   $ curl -X POST https://api.gemini.example/v1/video:generate \
     -H 'Content-Type: application/json' \
     -d '{
       "model": "gemini-omni-1.1-flash",
       "first_frame_b64": "'$FIRST_FRAME_B64'",
       "last_frame_b64": "'$LAST_FRAME_B64'",
       "input": "A smooth cinematic transition from a lush green forest at sunrise to a snowy forest under a starry night sky."
     }'

   • ›Adds resolution parameter to video generation requests, supporting 360p, 720p (default), 1080p, and 4K output values for the gemini-omni-1.1-flash model.
   • ›Adds first/last frame interpolation via first_frame_b64 and last_frame_b64 fields, letting you supply a start image and end image so the model animates a smooth cinematic transition between them.
   • ›Adds video extension capability: upload a clip via the Files API and prompt the model (e.g. 'Continue the scene.') to generate a seamless 3–10 second continuation appended to the end of the clip.
   • ›Supports reference media during video extension — supply additional images or video clips alongside your base video to introduce new characters or objects into the extended footage using <IMAGE_REF_N> and <VIDEO_REF_N> prompt tags.
   • ›Adds explicit prompt-based source/reference declaration syntax ([# Sources <FIRST_FRAME>@Image1 <LAST_FRAME>@Image2], [# References <VIDEO_REF_0>@Video1]) for multi-input video generation with precisely assigned roles.
   +3 moreshow less

   • ›Supports looping video generation by setting both <FIRST_FRAME> and <LAST_FRAME> to the same image.
   • ›Introduces <VIDEO_0> and <PREVIOUS_VIDEO> source tags for video editing and multi-turn extension respectively.
   • ›Video extension supports up to 40 seconds total length, extending in 10-second increments using the last 10 seconds of the original clip as context.
8. docs update Aug 28, 2026 · issue 009

   Gemini API adds Gemini 3.7 Flash and Gemini 3.5 Transcribe speech-to-text models with speaker diarization and word timestamps.

   • ›New gemini-3.5-transcribe model ID available for low-latency speech-to-text with utterance-based language detection, speaker diarization, word-level timestamps, and custom vocabulary biasing.
   • ›New gemini-3.5-transcribe-live model ID available for live/streaming speech-to-text with the same diarization and timestamp capabilities.
   • ›New gemini-omni-1.1-flash model ID available supporting fast video generation, editing, keyframe interpolation, and extension with native audio.
   • ›New Gemini 3.7 Flash model is now available.
9. docs update Aug 28, 2026 · issue 009

   Gemini API adds Gemini 3.7 Flash, Omni Flash video generation, 3.5 Transcribe speech-to-text, and a new Interactions API as the default interface.

   • ›Introduces the Interactions API as the new default interface for building with Gemini models and agents, replacing the now-legacy generateContent API; a step-by-step migration guide is available.
   • ›Adds Gemini Omni Flash, a new model for video generation and editing.
   • ›Adds native image generation and editing capability via Nano Banana.
   • ›Adds streaming support in the Interactions API for real-time tokens, incremental thoughts, and tool call events.
   └──▷ BREAKING ON UPGRADE

   • !The generateContent API is now considered legacy; the Interactions API is the default interface as of June 2026. Existing integrations continue to be supported but new projects should migrate.
10. docs update Aug 28, 2026 · issue 009

    Gemini API adds Gemini 3.7 Flash, Gemini 3.5 Transcribe, Omni Flash video model, and promotes Interactions API as the new default interface.

    • ›New Interactions API is now the default interface as of June 2026 for building with Gemini models and agents, superseding the generateContent API (now considered legacy); a migration guide and Interactions Overview are available.
    └──▷ BREAKING ON UPGRADE

    • !The generateContent API is now considered legacy; the Interactions API has become the default interface as of June 2026. Existing integrations continue to be supported but new projects should migrate.
11. August 27, 2026 Aug 27, 2026 · issue 009

    Gemini Omni Flash GA: video extension, frame interpolation, and 4K resolution control via gemini-omni-1.1-flash

    └──▷ USE IT
    Generate a 4K video from a prompt using the new resolution control parameter to get the highest-quality output.
    json copy

    {
      "model": "gemini-omni-1.1-flash",
      "contents": [{"parts": [{"text": "A drone flyover of a mountain range at sunset"}]}],
      "video_config": {
        "resolution": "4k"
      }
    }

    Interpolate between two images to create a smooth transition video using the image_to_video task.
    json copy

    {
      "model": "gemini-omni-1.1-flash",
      "task": "image_to_video",
      "contents": [
        {"parts": [{"inline_data": {"mime_type": "image/jpeg", "data": "<first_frame_base64>"}}]},
        {"parts": [{"inline_data": {"mime_type": "image/jpeg", "data": "<last_frame_base64>"}}]}
      ],
      "video_config": {
        "resolution": "720p"
      }
    }

    Extend an existing video clip by generating a continuation using the extend task.
    json copy

    {
      "model": "gemini-omni-1.1-flash",
      "task": "extend",
      "contents": [
        {"parts": [{"inline_data": {"mime_type": "video/mp4", "data": "<source_video_base64>"}}]}
      ],
      "video_config": {
        "resolution": "1080p"
      }
    }

    • ›Releases gemini-omni-1.1-flash, the GA fast conversational video generation and editing model.
    • ›Adds video extension capability: generate continuations at the end of an existing clip using the extend task or a direct prompt.
    • ›Adds frame interpolation via the image_to_video task with up to 2 images, generating a video that transitions between a first and last frame.
    • ›Adds a resolution parameter in video_config supporting 360p, 720p (default), 1080p, and 4k outputs (1080p and 4K use upscaling).
    └──▷ BREAKING ON UPGRADE

    • !The gemini-omni-flash-preview endpoint will be deprecated on September 30, 2026; callers must migrate to gemini-omni-1.1-flash.
12. August 26, 2026 Aug 26, 2026 · issue 009

    Gemini 3.5 Transcribe GA: two dedicated speech-to-text models with diarization, streaming, and 85+ language support

    • ›Adds gemini-3.5-transcribe model: high-accuracy, non-streaming speech-to-text with utterance-based language detection across 85+ languages, speaker diarization, word-level timestamps, and custom vocabulary biasing of up to 1,000 terms.
    • ›Adds gemini-3.5-transcribe-live model: low-latency, bidirectional streaming speech-to-text over WebSockets via the Live API, with interim and finalized transcription events, Smart transcription mode, and configurable Voice Activity Detection (VAD) strategies.
13. August 13, 2026 Aug 13, 2026 · issue 002

    Gemini 3.7 Flash (gemini-3.7-flash) is now GA, targeting coding, web dev, and agentic workflows.

    • ›Adds gemini-3.7-flash as a generally available model with improvements across software engineering, web development, and agentic workflows, available at an introductory price through December 31, 2026.
14. July 30, 2026 Jul 30, 2026 · issue 002

    Gemini Robotics ER 2 enters public preview with two new model endpoints for spatial reasoning and real-time robot streaming.

    • ›Adds gemini-robotics-er-2-preview endpoint supporting advanced spatial reasoning, agentic code execution, multi-step tool orchestration, video moment finding, progress classification, and multi-robot coordination.
    • ›Adds gemini-robotics-er-2-streaming-preview endpoint optimized for real-time text streaming via the Live API, enabling low-latency robot agents with bidirectional audio and video input.
    • ›Both gemini-robotics-er-2-preview and gemini-robotics-er-2-streaming-preview accept text, image, video, and audio inputs and support function calling with blocking behavior for physical robot actions.