Given the ability to control robot dogs/hands, etc.

When we already have GPT Realtime API

Then, magic can happen.

Use GPT stream voice/realtime API to interact with people, via robot dogs or robot hand, etc.

https://platform.openai.com/docs/api-reference/realtime

Related work

Client Side Tool Calling with the OpenAI WebRTC Realtime API

https://www.youtube.com/watch?v=TcOytsfva0o

image.png

ChatGPT + Robot Hand = Magic!

https://www.youtube.com/watch?v=GWHLRgOuJLU

https://github.com/iotdesignshop/dexhand_ros2_meta

image.png