Flutter 官方最近发布了 GenUI SDK for Flutter , 其理念就是通过 GenUI SDK 作为编排层，将用户与 AI Agent 间的交互从文本转变为丰富的交互式UI。
GenUI 如何工作？
GenUI 由多个组件组成：
- GenUiConversation： GenUI 的主要交互入口。它包含 GenUiManager 和 ContentGenerator 类，管理对话历史记录，并协调整个生成式 UI 流程。
- Catalog： CatalogItem 对象集合，定义了 AI 可以使用的组件集。每个 CatalogItem 都指定一个组件的名称（供 AI 引用）、其属性的数据模式以及用于渲染 Flutter 组件的构建器函数。
- DataModel： 一个集中式的、可观察的动态 UI 状态存储。组件绑定到此模型中的数据。当数据更改时，仅重建依赖于该特定数据的组件。
- ContentGenerator： 用于与生成式 AI 模型通信的接口。此接口使用流将 A2uiMessage 命令、文本响应和错误发送回 GenUiConversation。
- A2uiMessage： AI（通过内容生成器）向 UI 发送的消息，指示 UI 执行诸如 beginRendering、surfaceUpdate、dataModelUpdate 或 deleteSurface 之类的操作。
其基础流程就是，开发者通过 CatalogItem 定义一些 UI 组件，每一个 UI 组件 包含以下主要元素：
- name: CatalogItem 名称，在 json 返回时会用出字段作为 key
- dataSchema： 通过 json schema 描述的组件状态数据结构
- widgetBuilder： 如何基于模型根据 dataSchema 格式返回的数据来渲染组件
- 可选的 example
ContentGenerator 在发送请求到后端 LLM API 时，会将开发者定义的 CatalogItem 列表作为 surfaceUpdate function call 的参数定义，同时发送到后端。LLM 通过 functionCall 的方式调用 surfaceUpdate 来更新、定义 UI 组件以及状态，然后通过 beginRendering 的 functionCall 来触发 GenUiManager 渲染界面。而其中用到的数据结构就是 A2uiMessage。而用户在 UI 界面上的交互也会用 UserMessage 的方式将组件状态数据上报。
其中的关键是，GenUI SDK 并不仅仅是预定义若干固定的 UI 组件，而是尝试让 LLM 通过组合 UI 组件的方式去生成复杂的用户交互 UI。而其中的挑战是如何通过有限的 System Prompt 来让 LLM 根据需要组合出稳定可用的交互 UI，根据用户的交互确定用户目标。
GenUI 基础功能测试
GenUI SDK 提供 18 个预定义的 CoreCatalogItems：
static Catalog asCatalog() {
return Catalog([
audioPlayer,
button,
card,
checkBox,
column,
dateTimeInput,
divider,
icon,
image,
list,
modal,
multipleChoice,
row,
slider,
tabs,
text,
textField,
video,
]);
}
然而初步测试下来，虽然已经在 System Prompt 上做了不少控制，也难以使用 gemini-2.5-flash 模型生成稳定可用的 UI，特别是 button、textField 等用户交互类组件。因此只使用几个简单的展示类组件，以及自己编写一个代码高亮组件供 GenUI 使用：
final appCatalog = Catalog([
CoreCatalogItems.column,
CoreCatalogItems.text,
codeCatalogItem,
]);
参考官方教程 Get started with the GenUI SDK for Flutter , 对于国内用户需要实现自己的 GoogleGenerativeServiceInterface 来解决网络问题。目标是在 CycBox 中集成 AI 根据用户需求生成配置文件与 Lua 脚本功能。
CycBox Chatbot 效果截图：
总体而言， LLM 基本能理解用户意图生成可交互的组件组合，而由于 GenUiManager 帮忙接管了 UI 渲染，的确使得 AI Agent 的交互编程变得非常简单。
GenUI SDK 后端 API 请求与响应
最后，通过对 GenUI SDK 接口请求与响应数据的抓包，可以更直观地看到以上交互是如何实现的：
发送请求，带上组件定义：
{
"contents": [
{
"parts": [
{
"text": "You are a CycBox IoT debugging assistant specializing in Lua script generation. SystemPromptRemoved"
}
]
},
{
"parts": [
{
"text": "Could you help me debug PMS9103M Air Quality Sensor?"
}
],
"role": "user"
}
],
"model": "models/gemini-2.5-flash",
"toolConfig": {
"functionCallingConfig": {
"allowedFunctionNames": [
"surfaceUpdate",
"beginRendering",
"provideFinalOutput"
],
"mode": "ANY"
}
},
"tools": [
{
"functionDeclarations": [
{
"description": "Updates a surface with a new set of components.",
"name": "surfaceUpdate",
"parameters": {
"properties": {
"components": {
"description": "A list of component definitions.",
"items": {
"description": "Represents a *single* component in a UI widget tree. This component could be one of many supported types.",
"properties": {
"component": {
"description": "A wrapper object that MUST contain exactly one key, which is the name of the component type (e.g., 'Heading'). The value is an object containing the properties for that specific component.",
"properties": {
"Code": {
"properties": {
"code": {
"description": "The source code to display",
"type": "STRING"
},
"filename": {
"description": "Optional filename or description to display as header",
"type": "STRING"
},
"language": {
"description": "Programming language for syntax highlighting",
"enum": [
"dart",
"lua",
"yaml",
"javascript",
"python",
"json",
"bash",
"c",
"cpp",
"java",
"rust",
"go"
],
"type": "STRING"
}
},
"required": [
"code",
"language"
],
"type": "OBJECT"
},
"Column": {
"properties": {
"alignment": {
"description": "How children are aligned on the cross axis. ",
"enum": [
"start",
"center",
"end",
"stretch",
"baseline"
],
"type": "STRING"
},
"children": {
"description": "Either an explicit list of widget IDs for the children, or a template with a data binding to the list of children.",
"properties": {
"explicitList": {
"items": {
"type": "STRING"
},
"type": "ARRAY"
},
"template": {
"properties": {
"componentId": {
"type": "STRING"
},
"dataBinding": {
"type": "STRING"
}
},
"required": [
"componentId",
"dataBinding"
],
"type": "OBJECT"
}
},
"type": "OBJECT"
},
"distribution": {
"description": "How children are aligned on the main axis. ",
"enum": [
"start",
"center",
"end",
"spaceBetween",
"spaceAround",
"spaceEvenly"
],
"type": "STRING"
}
},
"type": "OBJECT"
},
"Text": {
"properties": {
"text": {
"description": "While simple Markdown is supported (without HTML or image references), utilizing dedicated UI components is generally preferred for a richer and more structured presentation.",
"properties": {
"literalString": {
"type": "STRING"
},
"path": {
"description": "A relative or absolute path in the data model.",
"type": "STRING"
}
},
"type": "OBJECT"
},
"usageHint": {
"description": "A usage hint for the base text style.",
"enum": [
"h1",
"h2",
"h3",
"h4",
"h5",
"caption",
"body"
],
"type": "STRING"
}
},
"required": [
"text"
],
"type": "OBJECT"
}
},
"type": "OBJECT"
},
"id": {
"type": "STRING"
},
"weight": {
"description": "Optional layout weight for use in Row/Column children.",
"type": "INTEGER"
}
},
"required": [
"id",
"component"
],
"type": "OBJECT"
},
"type": "ARRAY"
},
"surfaceId": {
"description": "The unique identifier for the UI surface to create or update. If you are adding a new surface this *must* be a new, unique identified that has never been used for any existing surfaces shown.",
"type": "STRING"
}
},
"required": [
"surfaceId",
"components"
],
"type": "OBJECT"
}
},
{
"description": "Signals the client to begin rendering a surface with a root component.",
"name": "beginRendering",
"parameters": {
"properties": {
"root": {
"description": "The ID of the root widget. This ID must correspond to the ID of one of the widgets in the `components` list.",
"type": "STRING"
},
"surfaceId": {
"description": "The unique identifier for the UI surface to render.",
"type": "STRING"
}
},
"required": [
"surfaceId",
"root"
],
"type": "OBJECT"
}
},
{
"description": "Returns the final output. Call this function when you are done with the current turn of the conversation. Do not call this if you need to use other tools first. You MUST call this tool when you are done.",
"name": "provideFinalOutput",
"parameters": {
"properties": {
"output": {
"properties": {
"response": {
"type": "STRING"
}
},
"type": "OBJECT"
}
},
"type": "OBJECT"
}
}
]
}
]
}
LLM 返回响应，返回交互信息，实际上就是两个 functionCall， 包括 surfaceUpdate 与 beginRendering ，这个需要在 System Prompt 中详细说明，不然 LLM 可能无法生成正确的 UI 组件结构：
{
"candidates": [
{
"content": {
"parts": [
{
"functionCall": {
"args": {
"components": [
{
"component": {
"Text": {
"text": {
"literalString": "I can help you debug your PMS9103M Air Quality Sensor. To get started, please provide the following information:"
},
"usageHint": "h3"
}
},
"id": "intro_text"
},
{
"component": {
"Text": {
"text": {
"literalString": "1. Do you have the sensor's datasheet or protocol documentation available? This will help me understand the message structure."
},
"usageHint": "body"
}
},
"id": "question_1"
},
{
"component": {
"Text": {
"text": {
"literalString": "2. How is the sensor connected to your system (e.g., Serial, USB-to-Serial, etc.)? What are the serial port parameters (baud rate, data bits, parity, stop bits, flow control)?"
},
"usageHint": "body"
}
},
"id": "question_2"
},
{
"component": {
"Text": {
"text": {
"literalString": "3. Can you provide any sample raw data (in hexadecimal format, if possible) that the sensor is sending?"
},
"usageHint": "body"
}
},
"id": "question_3"
},
{
"component": {
"Text": {
"text": {
"literalString": "4. What specific values or measurements do you want to extract from the sensor's data (e.g., PM1.0, PM2.5, PM10 concentrations)?"
},
"usageHint": "body"
}
},
"id": "question_4"
},
{
"component": {
"Column": {
"children": {
"explicitList": [
"intro_text",
"question_1",
"question_2",
"question_3",
"question_4"
]
}
}
},
"id": "main_column"
}
],
"surfaceId": "pms9103m_debug"
},
"name": "surfaceUpdate"
},
"thoughtSignature": "removed"
},
{
"functionCall": {
"args": {
"root": "main_column",
"surfaceId": "pms9103m_debug"
},
"name": "beginRendering"
}
}
],
"role": "model"
},
"finishMessage": "Model generated function call(s).",
"finishReason": "STOP",
"index": 0
}
],
"modelVersion": "gemini-2.5-flash",
"responseId": "B-removed",
"usageMetadata": {
"candidatesTokenCount": 418,
"promptTokenCount": 5379,
"promptTokensDetails": [
{
"modality": "TEXT",
"tokenCount": 5379
}
],
"thoughtsTokenCount": 184,
"totalTokenCount": 5981
}
}
Flutter 更新 UI，然后响应 functionResponse：
{
"contents": [
{
"parts": [
{
"text": "You are a CycBox IoT debugging assistant specializing in Lua script generation. SystemPromptRemoved"
}
]
},
{
"parts": [
{
"text": "Could you help me debug PMS9103M Air Quality Sensor?"
}
],
"role": "user"
},
{
"parts": [
{
"functionCall": {
"args": {
"components": [
{
"component": {
"Text": {
"text": {
"literalString": "I can help you debug your PMS9103M Air Quality Sensor. To get started, please provide the following information:"
},
"usageHint": "h3"
}
},
"id": "intro_text"
},
{
"component": {
"Text": {
"text": {
"literalString": "1. Do you have the sensor's datasheet or protocol documentation available? This will help me understand the message structure."
},
"usageHint": "body"
}
},
"id": "question_1"
},
{
"component": {
"Text": {
"text": {
"literalString": "2. How is the sensor connected to your system (e.g., Serial, USB-to-Serial, etc.)? What are the serial port parameters (baud rate, data bits, parity, stop bits, flow control)?"
},
"usageHint": "body"
}
},
"id": "question_2"
},
{
"component": {
"Text": {
"text": {
"literalString": "3. Can you provide any sample raw data (in hexadecimal format, if possible) that the sensor is sending?"
},
"usageHint": "body"
}
},
"id": "question_3"
},
{
"component": {
"Text": {
"text": {
"literalString": "4. What specific values or measurements do you want to extract from the sensor's data (e.g., PM1.0, PM2.5, PM10 concentrations)?"
},
"usageHint": "body"
}
},
"id": "question_4"
},
{
"component": {
"Column": {
"children": {
"explicitList": [
"intro_text",
"question_1",
"question_2",
"question_3",
"question_4"
]
}
}
},
"id": "main_column"
}
],
"surfaceId": "pms9103m_debug"
},
"name": "surfaceUpdate"
},
"thoughtSignature": "removed"
},
{
"functionCall": {
"args": {
"root": "main_column",
"surfaceId": "pms9103m_debug"
},
"name": "beginRendering"
}
}
],
"role": "model"
},
{
"parts": [
{
"functionResponse": {
"name": "surfaceUpdate",
"response": {
"status": "UI Surface pms9103m_debug updated.",
"surfaceId": "pms9103m_debug"
}
}
},
{
"functionResponse": {
"name": "beginRendering",
"response": {
"status": "Surface pms9103m_debug rendered and waiting for user input."
}
}
}
],
"role": "user"
}
],
"model": "models/gemini-2.5-flash",
"toolConfig": {
"functionCallingConfig": {
"allowedFunctionNames": [
"surfaceUpdate",
"beginRendering",
"provideFinalOutput"
],
"mode": "ANY"
}
},
"tools": [
{
"functionDeclarations": [
{
"description": "Updates a surface with a new set of components.",
"name": "surfaceUpdate",
"parameters": {
"properties": {
"components": {
"description": "A list of component definitions.",
"items": {
"description": "Represents a *single* component in a UI widget tree. This component could be one of many supported types.",
"properties": {
"component": {
"description": "A wrapper object that MUST contain exactly one key, which is the name of the component type (e.g., 'Heading'). The value is an object containing the properties for that specific component.",
"properties": {
"Code": {
"properties": {
"code": {
"description": "The source code to display",
"type": "STRING"
},
"filename": {
"description": "Optional filename or description to display as header",
"type": "STRING"
},
"language": {
"description": "Programming language for syntax highlighting",
"enum": [
"dart",
"lua",
"yaml",
"javascript",
"python",
"json",
"bash",
"c",
"cpp",
"java",
"rust",
"go"
],
"type": "STRING"
}
},
"required": [
"code",
"language"
],
"type": "OBJECT"
},
"Column": {
"properties": {
"alignment": {
"description": "How children are aligned on the cross axis. ",
"enum": [
"start",
"center",
"end",
"stretch",
"baseline"
],
"type": "STRING"
},
"children": {
"description": "Either an explicit list of widget IDs for the children, or a template with a data binding to the list of children.",
"properties": {
"explicitList": {
"items": {
"type": "STRING"
},
"type": "ARRAY"
},
"template": {
"properties": {
"componentId": {
"type": "STRING"
},
"dataBinding": {
"type": "STRING"
}
},
"required": [
"componentId",
"dataBinding"
],
"type": "OBJECT"
}
},
"type": "OBJECT"
},
"distribution": {
"description": "How children are aligned on the main axis. ",
"enum": [
"start",
"center",
"end",
"spaceBetween",
"spaceAround",
"spaceEvenly"
],
"type": "STRING"
}
},
"type": "OBJECT"
},
"Text": {
"properties": {
"text": {
"description": "While simple Markdown is supported (without HTML or image references), utilizing dedicated UI components is generally preferred for a richer and more structured presentation.",
"properties": {
"literalString": {
"type": "STRING"
},
"path": {
"description": "A relative or absolute path in the data model.",
"type": "STRING"
}
},
"type": "OBJECT"
},
"usageHint": {
"description": "A usage hint for the base text style.",
"enum": [
"h1",
"h2",
"h3",
"h4",
"h5",
"caption",
"body"
],
"type": "STRING"
}
},
"required": [
"text"
],
"type": "OBJECT"
}
},
"type": "OBJECT"
},
"id": {
"type": "STRING"
},
"weight": {
"description": "Optional layout weight for use in Row/Column children.",
"type": "INTEGER"
}
},
"required": [
"id",
"component"
],
"type": "OBJECT"
},
"type": "ARRAY"
},
"surfaceId": {
"description": "The unique identifier for the UI surface to create or update. If you are adding a new surface this *must* be a new, unique identified that has never been used for any existing surfaces shown.",
"type": "STRING"
}
},
"required": [
"surfaceId",
"components"
],
"type": "OBJECT"
}
},
{
"description": "Signals the client to begin rendering a surface with a root component.",
"name": "beginRendering",
"parameters": {
"properties": {
"root": {
"description": "The ID of the root widget. This ID must correspond to the ID of one of the widgets in the `components` list.",
"type": "STRING"
},
"surfaceId": {
"description": "The unique identifier for the UI surface to render.",
"type": "STRING"
}
},
"required": [
"surfaceId",
"root"
],
"type": "OBJECT"
}
},
{
"description": "Returns the final output. Call this function when you are done with the current turn of the conversation. Do not call this if you need to use other tools first. You MUST call this tool when you are done.",
"name": "provideFinalOutput",
"parameters": {
"properties": {
"output": {
"properties": {
"response": {
"type": "STRING"
}
},
"type": "OBJECT"
}
},
"type": "OBJECT"
}
}
]
}
]
}
LLM 在接收到响应后会输出 provideFinalOutput，单个交互过程结束：
{
"candidates": [
{
"content": {
"parts": [
{
"functionCall": {
"args": {
"output": {
"response": "I've displayed some questions to help me understand your setup. Please provide the requested information so I can generate the appropriate CycBox configuration and Lua script."
}
},
"name": "provideFinalOutput"
},
"thoughtSignature": ""
}
],
"role": "model"
},
"finishMessage": "Model generated function call(s).",
"finishReason": "STOP",
"index": 0
}
],
"modelVersion": "gemini-2.5-flash",
"responseId": "removed",
"usageMetadata": {
"candidatesTokenCount": 49,
"promptTokenCount": 5865,
"promptTokensDetails": [
{
"modality": "TEXT",
"tokenCount": 5865
}
],
"thoughtsTokenCount": 82,
"totalTokenCount": 5996
}
}
用户再次输入，请求：
{
"contents": [
{
"parts": [
{
"text": "You are a CycBox IoT debugging assistant specializing in Lua script generation. SystemPromptRemoved"
}
]
},
{
"parts": [
{
"text": "Could you help me debug PMS9103M Air Quality Sensor?"
}
],
"role": "user"
},
{
"parts": [
{
"text": "A user interface is shown with the following content:\n{\"surfaceId\":\"pms9103m_debug\",\"rootComponentId\":\"main_column\",\"components\":{\"intro_text\":{\"id\":\"intro_text\",\"component\":{\"Text\":{\"text\":{\"literalString\":\"I can help you debug your PMS9103M Air Quality Sensor. To get started, please provide the following information:\"},\"usageHint\":\"h3\"}}},\"question_1\":{\"id\":\"question_1\",\"component\":{\"Text\":{\"text\":{\"literalString\":\"1. Do you have the sensor's datasheet or protocol documentation available? This will help me understand the message structure.\"},\"usageHint\":\"body\"}}},\"question_2\":{\"id\":\"question_2\",\"component\":{\"Text\":{\"usageHint\":\"body\",\"text\":{\"literalString\":\"2. How is the sensor connected to your system (e.g., Serial, USB-to-Serial, etc.)? What are the serial port parameters (baud rate, data bits, parity, stop bits, flow control)?\"}}}},\"question_3\":{\"id\":\"question_3\",\"component\":{\"Text\":{\"text\":{\"literalString\":\"3. Can you provide any sample raw data (in hexadecimal format, if possible) that the sensor is sending?\"},\"usageHint\":\"body\"}}},\"question_4\":{\"id\":\"question_4\",\"component\":{\"Text\":{\"text\":{\"literalString\":\"4. What specific values or measurements do you want to extract from the sensor's data (e.g., PM1.0, PM2.5, PM10 concentrations)?\"},\"usageHint\":\"body\"}}},\"main_column\":{\"id\":\"main_column\",\"component\":{\"Column\":{\"children\":{\"explicitList\":[\"intro_text\",\"question_1\",\"question_2\",\"question_3\",\"question_4\"]}}}}}}."
}
],
"role": "model"
},
{
"parts": [
{
"text": "I've displayed some questions to help me understand your setup. Please provide the requested information so I can generate the appropriate CycBox configuration and Lua script."
}
],
"role": "model"
},
{
"parts": [
{
"text": "The sensor uses a standard UART (Serial) interface with Baud Rate: 9600 bps, Data Bits: 8, Parity: None, Stop Bits: 1, Flow Control: None\n\nThe data packet is **32 Bytes** long. The format is Big-Endian (High Byte first).\n\n\n| Byte Index | Value/Description | Note |\n| :--- | :--- | :--- |\n| 0 | **0x42** | Start Character 1 (Fixed) |\n| 1 | **0x4D** | Start Character 2 (Fixed) |\n| 2 | Frame Length High | Fixed at 0x00 |\n| 3 | Frame Length Low | Fixed at 0x1C (28 decimal) |\n| 4 | **PM1.0 (Standard) High** | CF=1, Standard Particle |\n| 5 | **PM1.0 (Standard) Low** | Unit: μg/m³ |\n| 6 | **PM2.5 (Standard) High** | CF=1, Standard Particle |\n| 7 | **PM2.5 (Standard) Low** | Unit: μg/m³ |\n| 8 | **PM10 (Standard) High** | CF=1, Standard Particle |\n| 9 | **PM10 (Standard) Low** | Unit: μg/m³ |\n| 10 | **PM1.0 (Atmospheric) High** | **Use this for Air Quality** |\n| 11 | **PM1.0 (Atmospheric) Low** | Unit: μg/m³ |\n| 12 | **PM2.5 (Atmospheric) High** | **Use this for Air Quality** |\n| 13 | **PM2.5 (Atmospheric) Low** | Unit: μg/m³ |\n| 14 | **PM10 (Atmospheric) High** | **Use this for Air Quality** |\n| 15 | **PM10 (Atmospheric) Low** | Unit: μg/m³ |\n| 16-27 | Particle Counts | >0.3um, >0.5um, >1.0um, >2.5um, >5.0um, >10um |\n| 28 | Version No. | |\n| 29 | Error Code | |\n| 30 | **Checksum High** | |\n| 31 | **Checksum Low** | |"
}
],
"role": "user"
}
],
"model": "models/gemini-2.5-flash",
"toolConfig": {
"functionCallingConfig": {
"allowedFunctionNames": [
"surfaceUpdate",
"beginRendering",
"provideFinalOutput"
],
"mode": "ANY"
}
},
"tools": [
{
"functionDeclarations": [
{
"description": "Updates a surface with a new set of components.",
"name": "surfaceUpdate",
"parameters": {
"properties": {
"components": {
"description": "A list of component definitions.",
"items": {
"description": "Represents a *single* component in a UI widget tree. This component could be one of many supported types.",
"properties": {
"component": {
"description": "A wrapper object that MUST contain exactly one key, which is the name of the component type (e.g., 'Heading'). The value is an object containing the properties for that specific component.",
"properties": {
"Code": {
"properties": {
"code": {
"description": "The source code to display",
"type": "STRING"
},
"filename": {
"description": "Optional filename or description to display as header",
"type": "STRING"
},
"language": {
"description": "Programming language for syntax highlighting",
"enum": [
"dart",
"lua",
"yaml",
"javascript",
"python",
"json",
"bash",
"c",
"cpp",
"java",
"rust",
"go"
],
"type": "STRING"
}
},
"required": [
"code",
"language"
],
"type": "OBJECT"
},
"Column": {
"properties": {
"alignment": {
"description": "How children are aligned on the cross axis. ",
"enum": [
"start",
"center",
"end",
"stretch",
"baseline"
],
"type": "STRING"
},
"children": {
"description": "Either an explicit list of widget IDs for the children, or a template with a data binding to the list of children.",
"properties": {
"explicitList": {
"items": {
"type": "STRING"
},
"type": "ARRAY"
},
"template": {
"properties": {
"componentId": {
"type": "STRING"
},
"dataBinding": {
"type": "STRING"
}
},
"required": [
"componentId",
"dataBinding"
],
"type": "OBJECT"
}
},
"type": "OBJECT"
},
"distribution": {
"description": "How children are aligned on the main axis. ",
"enum": [
"start",
"center",
"end",
"spaceBetween",
"spaceAround",
"spaceEvenly"
],
"type": "STRING"
}
},
"type": "OBJECT"
},
"Text": {
"properties": {
"text": {
"description": "While simple Markdown is supported (without HTML or image references), utilizing dedicated UI components is generally preferred for a richer and more structured presentation.",
"properties": {
"literalString": {
"type": "STRING"
},
"path": {
"description": "A relative or absolute path in the data model.",
"type": "STRING"
}
},
"type": "OBJECT"
},
"usageHint": {
"description": "A usage hint for the base text style.",
"enum": [
"h1",
"h2",
"h3",
"h4",
"h5",
"caption",
"body"
],
"type": "STRING"
}
},
"required": [
"text"
],
"type": "OBJECT"
}
},
"type": "OBJECT"
},
"id": {
"type": "STRING"
},
"weight": {
"description": "Optional layout weight for use in Row/Column children.",
"type": "INTEGER"
}
},
"required": [
"id",
"component"
],
"type": "OBJECT"
},
"type": "ARRAY"
},
"surfaceId": {
"description": "The unique identifier for the UI surface to create or update. If you are adding a new surface this *must* be a new, unique identified that has never been used for any existing surfaces shown.",
"type": "STRING"
}
},
"required": [
"surfaceId",
"components"
],
"type": "OBJECT"
}
},
{
"description": "Signals the client to begin rendering a surface with a root component.",
"name": "beginRendering",
"parameters": {
"properties": {
"root": {
"description": "The ID of the root widget. This ID must correspond to the ID of one of the widgets in the `components` list.",
"type": "STRING"
},
"surfaceId": {
"description": "The unique identifier for the UI surface to render.",
"type": "STRING"
}
},
"required": [
"surfaceId",
"root"
],
"type": "OBJECT"
}
},
{
"description": "Returns the final output. Call this function when you are done with the current turn of the conversation. Do not call this if you need to use other tools first. You MUST call this tool when you are done.",
"name": "provideFinalOutput",
"parameters": {
"properties": {
"output": {
"properties": {
"response": {
"type": "STRING"
}
},
"type": "OBJECT"
}
},
"type": "OBJECT"
}
}
]
}
]
}
LLM 响应最终结果，包含我们自定义提供的Code组件：
{
"candidates": [
{
"content": {
"parts": [
{
"functionCall": {
"args": {
"components": [
{
"component": {
"Text": {
"text": {
"literalString": "Based on the information you provided, here is the CycBox configuration (config.yaml) and the Lua script (pms9103m_parser.lua) to parse the PMS9103M sensor data:"
},
"usageHint": "h3"
}
},
"id": "config_intro"
},
{
"component": {
"Code": {
"code": "version: \"1.0\"\ngroups:\n app:\n app_transport: \"serial\"\n app_codec: \"frame_codec\"\n app_transformer: \"disable\"\n app_lua_script_enabled: true\n app_encoding: \"UTF-8\"\n serial:\n serial_requires_codec: true\n serial_port: \"/dev/ttyS0\" # Update if your serial port is different\n serial_baud_rate: 9600\n serial_data_bits: 8\n serial_parity: \"none\"\n serial_stop_bits: \"1\"\n serial_flow_control: \"none\"\n frame_codec:\n frame_codec_prefix: \"42 4d\"\n frame_codec_header_size: 0\n frame_codec_length_mode: \"u16_be\"\n frame_codec_length_meaning: \"payload_checksum\" # Length value 0x1C (28) covers 26 bytes payload + 2 bytes checksum\n frame_codec_checksum_algo: \"sum16_be\" # Assuming 16-bit big-endian sum, adjust if your datasheet specifies a different algorithm\n frame_codec_checksum_scope: \"prefix_header_length_payload\"\n frame_codec_tailer_length: 0\n frame_codec_suffix: \"\"",
"filename": "config.yaml",
"language": "yaml"
}
},
"id": "config_yaml_code"
},
{
"component": {
"Code": {
"code": "-- PMS9103M Air Quality Sensor Protocol Parser\n-- Protocol: 32-byte binary frame with 0x42 0x4D prefix, 2-byte length (0x001C), 26-byte payload, and 2-byte checksum.\n-- Data format: Big-Endian.\n\nfunction on_receive()\n local payload = message:get_payload()\n\n -- Validate payload length. The frame codec strips prefix, length field, and checksum,\n -- so the Lua payload should be 26 bytes (bytes 4-29 from the raw frame).\n if #payload ~= 26 then\n log(\"warn\", string.format(\"PMS9103M: Expected 26 bytes payload, got %d\", #payload))\n return false\n end\n\n -- Parse Atmospheric PM values (1-indexed offsets relative to the Lua payload)\n -- Raw frame bytes 10-11 -> Lua payload bytes 7-8\n local pm1_0_atmospheric = read_u16_be(payload, 7)\n -- Raw frame bytes 12-13 -> Lua payload bytes 9-10\n local pm2_5_atmospheric = read_u16_be(payload, 9)\n -- Raw frame bytes 14-15 -> Lua payload bytes 11-12\n local pm10_atmospheric = read_u16_be(payload, 11)\n\n -- Add values for charting in the UI\n message:add_int_value(\"PM1.0 Atmospheric (μg/m³)\", pm1_0_atmospheric)\n message:add_int_value(\"PM2.5 Atmospheric (μg/m³)\", pm2_5_atmospheric)\n message:add_int_value(\"PM10 Atmospheric (μg/m³)\", pm10_atmospheric)\n\n -- Log the parsed values\n log(\"info\", string.format(\"PMS9103M: PM1.0 Atmos: %d μg/m³, PM2.5 Atmos: %d μg/m³, PM10 Atmos: %d μg/m³\", \n pm1_0_atmospheric, pm2_5_atmospheric, pm10_atmospheric))\n\n return true\nend",
"filename": "pms9103m_parser.lua",
"language": "lua"
}
},
"id": "lua_script_code"
},
{
"component": {
"Column": {
"children": {
"explicitList": [
"config_intro",
"config_yaml_code",
"lua_script_code"
]
}
}
},
"id": "main_column"
}
],
"surfaceId": "pms9103m_debug_config_script"
},
"name": "surfaceUpdate"
},
"thoughtSignature": "removed"
},
{
"functionCall": {
"args": {
"root": "main_column",
"surfaceId": "pms9103m_debug_config_script"
},
"name": "beginRendering"
}
}
],
"role": "model"
},
"finishMessage": "Model generated function call(s).",
"finishReason": "STOP",
"index": 0
}
],
"modelVersion": "gemini-2.5-flash",
"responseId": "removed",
"usageMetadata": {
"candidatesTokenCount": 1143,
"promptTokenCount": 6339,
"promptTokensDetails": [
{
"modality": "TEXT",
"tokenCount": 6339
}
],
"thoughtsTokenCount": 2541,
"totalTokenCount": 10023
}
}