Introduction to Function Calling with Gemini

Gemini

Gemini is a family of generative AI models developed by Google DeepMind.

 

Function Calling

Function calling (tool use) acts like a bridge between user and language model. Function calling in Gemini lets developer create a description of a function in their code, then pass that description to a language model in a request. The response from the model includes the name of a function that matches the description and the arguments to call it with.

 

Without function calling, getting consistent structured data from generative text model will be frustrating. When there is Function Calling, you don't have to hope for the best in a freeform text response from a generative model, you can define clear functions with specific parameters and data types. These function declarations tell Gemini model to structure its output.

 

How to create a function calling application 

Step 1. Submit a prompt and function declaration to the Gemini models.

Declare a `Tool` in a schema format that is compatible with the OpenAI schema.

PROJECT_ID=myproject
LOCATION=us-central1
MODEL_ID=gemini-2.0-flash-001

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  https://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/${LOCATION}/publishers/google/models/${MODEL_ID}:generateContent \
  -d '{
    "contents": [{
      "role": "user",
      "parts": [{
        "text": "What is the weather in Boston?"
      }]
    }],
    "tools": [{
      "functionDeclarations": [
        {
          "name": "get_current_weather",
          "description": "Get the current weather in a given location",
          "parameters": {
            "type": "object",
            "properties": {
              "location": {
                "type": "string",
                "description": "The city name of the location for which to get the weather.",
                "default": {
                  "string_value": "Boston, MA"
                }
              }
            },
            "required": [
              "location"
            ]
          }
        }
      ]
    }]
  }'

 

The following example is a model response to the user prompt "What is the weather like in Boston?".

The model proposes calling the `get_current_weather` function with the parameter `Boston, MA`.

candidates {
  content {
    role: "model"
    parts {
      function_call {
        name: "get_current_weather"
        args {
          fields {
            key: "location"
            value {
              string_value: "Boston, MA"
            }
          }
        }
      }
    }
  }
  ...
}

 

 

Step 2. Provide the API output to the model

Invoke the external API and pass the API output back to to the model.

PROJECT_ID=myproject
MODEL_ID=gemini-2.0-flash
LOCATION="us-central1"

curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
https://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/${LOCATION}/publishers/google/models/${MODEL_ID}:generateContent \
-d '{
"contents": [
{
  "role": "user",
  "parts": {
    "text": "What is the weather in Boston?"
  }
},
{
  "role": "model",
  "parts": [
    {
      "functionCall": {
        "name": "get_current_weather",
        "args": {
          "location": "Boston, MA"
        }
      }
    }
  ]
},
{
  "role": "user",
  "parts": [
    {
      "functionResponse": {
        "name": "get_current_weather",
        "response": {
          "temperature": 20,
          "unit": "C"
        }
      }
    }
  ]
}
],
"tools": [
{
  "function_declarations": [
    {
      "name": "get_current_weather",
      "description": "Get the current weather in a specific location",
      "parameters": {
        "type": "object",
        "properties": {
          "location": {
            "type": "string",
            "description": "The city name of the location for which to get the weather."
          }
        },
        "required": [
          "location"
        ]
      }
    }
  ]
}
]
}'

 

 

Refs

https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling#rest_1