AI and LUI Interfaces

AI and LUI capabilities are exposed through high-level RPC clients and subtitle/ASR stream topics.

Source headers:

• include/booster/robot/ai/client.hpp
• include/booster/robot/ai/api.hpp
• include/booster/robot/ai/const.hpp

Clients

Client	Methods	Description	Minimum firmware
AiClient	StartAiChat, StopAiChat, Speak	AI chat and direct speech.	>= v1.3.1.1
AiClient	StartFaceTracking, StopFaceTracking	Enable or disable face tracking.	>= v1.3.1.1
LuiClient	StartAsr, StopAsr	Enable or disable ASR.	>= v1.5.0.9
LuiClient	StartTts, StopTts, SendTtsText	Configure TTS, stop TTS, and send text.	>= v1.5.0.9
LuiClient	SynthesizeSpeech	Synthesize text-file content read by the application into audio that can be saved as PCM or WAV.	>= v1.8.0.9
LuiClient	RecognizeAudioOnce, StartAudioRecognizer, RecognizeAudioInSession, StopAudioRecognizer	Recognize WAV/MP3 files or base64 PCM data; the application can save the returned text to a file.	>= v1.8.0.9
LuiClient	StartAsr, StopAsr, StartTts, StopTts	Isolate ASR/TTS session ownership and stop operations by client and session.	>= v1.8.0.9

C++ Example

#include <booster/robot/ai/client.hpp>
#include <booster/robot/channel/channel_factory.hpp>

booster::robot::ChannelFactory::Instance()->Init(0, "<network_ip>");

booster::robot::AiClient ai;
ai.Init();
ai.Speak(booster::robot::SpeakParameter("Hello from Booster robot."));

booster::robot::LuiClient lui;
lui.Init();
lui.StartAsr();
lui.StartTts(
    booster::robot::LuiTtsConfig(
        "zh_male_wennuanahu_moon_bigtts"));
lui.SendTtsText(booster::robot::LuiTtsParameter("Hello."));

AI and LUI Topics

Topic	C++ constant	C++ message type	Minimum firmware
rt/ai_subtitle	kTopicAiSubtitle	booster_interface::msg::Subtitle	>= v1.3.1.1
rt/lui_asr_chunk	kTopicLuiAsrChunk	booster_interface::msg::AsrChunk	>= v1.5.0.9

Subscribe with ChannelSubscriber<booster_interface::msg::Subtitle> or ChannelSubscriber<booster_interface::msg::AsrChunk>. See Low-Level Topics for the subscriber template.

Complete Public Interfaces

Public AiClient methods:

Method	Business meaning	Minimum firmware
Init() / Init(robot_name)	Initialize the AI RPC channel for the default or named robot.	>= v1.3.1.1
StartAiChat(param)	Create an AI chat session with TTS, LLM, ASR, and interruption settings.	>= v1.3.1.1
StopAiChat()	Stop the current AI chat; an idle session is normally treated as success.	>= v1.3.1.1
Speak(param)	Synthesize and play a non-empty message in the active chat.	>= v1.3.1.1
StartFaceTracking() / StopFaceTracking()	Request standalone face tracking; requires an NVIDIA perception board and may only be acknowledged by the current RTC service.	>= v1.3.1.1

Public LuiClient methods:

Method	Business meaning	Notes
Init() / Init(robot_name)	Initialize the LUI RPC channel.	Required first
StartAsr() / StopAsr()	Start/stop this client’s microphone ASR; results are published on kTopicLuiAsrChunk. Client and session isolation applies with firmware >= v1.8.0.9.	16 kHz mono 16 bit
StartTts(config) / StopTts()	Create/stop this client’s TTS session. With firmware >= v1.8.0.9, stopping affects only the corresponding session.	Requires audio/network services
SendTtsText(param)	Queue text in the active TTS session; non-empty text and service rate limits apply.	>= v1.5.0.9
SynthesizeSpeech(req, resp, error_message)	Pass text-file content read by the application to the active TTS session and receive base64 PCM and its format; save the PCM directly or wrap it as WAV using the sample rate, channel count, and bit depth. Start the TTS session before calling this method.	>= v1.8.0.9; 1000 Unicode code points
RecognizeAudioOnce(req, resp, error_message)	Recognize one WAV/MP3 file or base64 PCM payload. File paths must be readable by the robot-side LUI service; the application can save response text_ to a text file.	>= v1.8.0.9; 120 s/6 MiB maximum
StartAudioRecognizer() / StopAudioRecognizer()	Create/stop this client’s reusable recognizer session.	>= v1.8.0.9; separate from StartAsr
RecognizeAudioInSession(req, resp, error_message)	Submit a file or PCM recognition request in the reusable session. File paths must be readable by the robot-side LUI service; the application can save response text_ to a text file.	>= v1.8.0.9; call StartAudioRecognizer first
GetCurrentAsrSessionId()	Return the current StartAsr session ID, or empty when inactive.	>= v1.8.0.9; local query
GetCurrentAudioRecognizerSessionId()	Return the reusable recognizer session ID.	>= v1.8.0.9; local query
GetCurrentTtsSessionId()	Return the current standalone TTS session ID.	>= v1.8.0.9; local query
GetClientInstanceId()	Return the LUI client instance ID used to distinguish clients in one process.	>= v1.8.0.9; local query
IsAsrSessionActive(session_id, client_id)	Check whether an ASR session is still owned by a client.	>= v1.8.0.9; static method

AI and LUI enums

AiApiId values:

Enumerator	Value	Meaning
kStartAiChat	2000	Create an AI chat session.
kStopAiChat	2001	Stop the AI chat session.
kSpeak	2002	Speak text in the active chat.
kStartFaceTracking	2003	Request standalone face tracking; the current service may only acknowledge it.
kStopFaceTracking	2004	Request standalone face-tracking stop; the current service may only acknowledge it.

LuiApiId values:

Enumerator	Value	Meaning
kStartAsr / kStopAsr	1000 / 1001	Start / stop standalone ASR.
kStartTts / kStopTts	1050 / 1051	Create / stop the standalone TTS session.
kSendTtsText	1052	Queue text in the TTS session.
kSynthesizeSpeech	1100	Return one synthesized audio payload.
kRecognizeAudioOnce	1101	Recognize one file or PCM payload.
kStartAudioRecognizer / kStopAudioRecognizer	1102 / 1103	Start / stop the reusable recognizer session.
kRecognizeAudioInSession	1104	Recognize one payload in the reusable session.

Public payloads and responses

• TtsConfig contains backend voice ID voice_type_ and ignore_bracket_text_ categories: 1 full-width parentheses, 2 ASCII parentheses, 3 full-width square brackets, 4 ASCII square brackets, and 5 ASCII curly braces.
• LlmConfig contains system_prompt_ (system prompt), welcome_msg_ (greeting), and prompt_name_ (deployed named prompt).
• AsrConfig contains interrupt_speech_duration_ in milliseconds and interrupt_keywords_.
• StartAiChatParameter contains interrupt_mode_, tts_config_, llm_config_, asr_config_, and enable_face_tracking_. Assign uninitialized scalars after default construction.
• SpeakParameter contains non-empty text msg_.
• LuiTtsConfig contains voice ID voice_type_; LuiTtsParameter contains non-empty queued text text_.
• LuiSynthesizeSpeechRequest contains text_, voice_type_ (default default), speed_ (0.5 to 2.0, default 1.0), and playback_. The response contains audio_base64_, sample_rate_hz_, channels_, bits_per_sample_, and format_ (default pcm_s16le).
• LuiRecognizeAudioRequest uses input_type_ file or pcm. File requests use file_path_; PCM requests use audio_base64_, sample_rate_hz_, channels_, bits_per_sample_, and format_. Prefer FromFile and FromPcmBase64 constructors. LuiRecognizeAudioResponse::text_ contains recognized text.
• kLuiMaxSynthesizeTextCodePoints is 1000, kLuiMaxRecognizeAudioDurationMs is 120000, and kLuiMaxRecognizeAudioBytes is 6291456. Limits count Unicode code points, duration, and decoded bytes respectively.

Calling constraints

C++ methods wait synchronously for an RPC response, but success only means that the service accepted the request; check every return value. Python bindings use PascalCase names and commonly raise RuntimeError for high-level failures. rt/lui_asr_chunk broadcasts shared microphone recognition results, and its session_id and client_id fields are currently empty. C++ subscribers can call LuiClient::IsAsrSessionActive with their saved session and client IDs to consume messages only while their local session is active. In Python, use LuiAsrChunkSubscriber.InitChannelForClient after StartAsr succeeds. This session isolation requires firmware >= v1.8.0.9. Use kBoosterRobotUserId to identify robot-generated AI subtitle text.

Complete Payload, Request, and Response Types

All types below are public declarations in api.hpp. They can be used to build high-level requests or to call the same RPC through SendApiRequest. Each FromJson reads the fields listed in the table and ToJson serializes the same fields. Assign every uninitialized field after default construction.

Type	Fields	Description
TtsConfig	voice_type_, ignore_bracket_text_	AI-chat TTS voice and bracket categories to filter; values 1–5 are listed above.
LlmConfig	system_prompt_, welcome_msg_, prompt_name_	LLM system prompt, session greeting, and backend named prompt.
AsrConfig	interrupt_speech_duration_, interrupt_keywords_	Continuous-speech interruption threshold in milliseconds and immediate interruption keywords.
StartAiChatParameter	interrupt_mode_, tts_config_, llm_config_, asr_config_, enable_face_tracking_	Combined AI-chat configuration; interruption and face-tracking intent are booleans.
SpeakParameter	msg_	Non-empty text to synthesize in an active AI chat.

Type	Fields/constructor	Description
LuiAsrStartRequest	session_id_, client_id_; (session_id, client_id)	Low-level request to start an ASR session; both identifiers isolate clients.
LuiAsrStopRequest	session_id_, client_id_; (session_id, client_id)	Stops the specified ASR session and only succeeds for its owning client.
LuiTtsConfig	voice_type_; (voice_type)	Backend voice identifier for a standalone TTS session.
LuiTtsStopRequest	session_id_, client_id_; (session_id, client_id)	Stops the specified standalone TTS session.
LuiTtsParameter	text_; (text)	Non-empty text appended to the current TTS session.

Type	Fields	Constraints/return value
LuiSynthesizeSpeechRequest	text_, voice_type_, speed_, playback_	Text is limited to 1000 Unicode code points; voice defaults to default, speed is 0.5–2.0 (default 1.0), and playback_ controls service-side playback.
LuiSynthesizeSpeechResponse	audio_base64_, sample_rate_hz_, channels_, bits_per_sample_, format_	Base64 audio and format; defaults are 16 kHz, mono, 16 bit, pcm_s16le.
LuiRecognizeAudioRequest	input_type_, file_path_, audio_base64_, sample_rate_hz_, channels_, bits_per_sample_, format_	input_type_ is file or pcm. Use FromFile(path) for WAV/MP3, or FromPcmBase64 for 16 kHz mono 16-bit pcm_s16le raw PCM.
LuiRecognizeAudioResponse	text_	Recognized text returned by the service.

Helpers and limits

• CountUtf8CodePoints(text) counts non-continuation UTF-8 bytes and returns the Unicode code-point count. It is a local pre-check; the service remains authoritative.
• EstimateBase64DecodedBytes(base64_data) estimates decoded byte count from base64 length and trailing padding for comparison with kLuiMaxRecognizeAudioBytes; pass valid base64.
• kLuiMaxSynthesizeTextCodePoints, kLuiMaxRecognizeAudioDurationMs, and kLuiMaxRecognizeAudioBytes are 1000, 120000, and 6291456 respectively.

Generic RPC overloads

In addition to business convenience methods, both clients expose these operations for protocol diagnostics or new IDs not yet wrapped by the SDK. param is the JSON string produced by ToJson().dump() on one of the payload types above.

Interface	Description
AiClient::SendApiRequest(api_id, param)	Send an AI request synchronously and return its status; pass an empty string for no-body operations.
AiClient::SendApiRequestWithResponse(api_id, param, response)	Send an AI request and write the complete Response (status header and body) to the output.
LuiClient::SendApiRequest(api_id, param)	Send a LUI request synchronously and return its status.
LuiClient::SendApiRequestWithResponse(api_id, param, response)	Send a LUI request and return the complete Response.

Call these functions only after the corresponding Init. A success code means that the service accepted the request; it does not mean asynchronous speech or cloud-session work has completed.