Large Language Models (LLMs) are gaining popularity in the field of robotics.
However, LLM-based robots are limited to simple, repetitive motions due to the
poor integration between language models, robots, and the environment. This
paper proposes a novel approach to enhance the performance of LLM-based
autonomous manipulation through Human-Robot Collaboration (HRC). The approach
involves using a prompted GPT-4 language model to decompose high-level language
commands into sequences of motions that can be executed by the robot. The
system also employs a YOLO-based perception algorithm, providing visual cues to
the LLM, which aids in planning feasible motions within the specific
environment. Additionally, an HRC method is proposed by combining teleoperation
and Dynamic Movement Primitives (DMP), allowing the LLM-based robot to learn
from human guidance. Real-world experiments have been conducted using the
Toyota Human Support Robot for manipulation tasks. The outcomes indicate that
tasks requiring complex trajectory planning and reasoning over environments can
be efficiently accomplished through the incorporation of human demonstrations.