Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.
DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.
The growing complexity of commissioning industrial automation systems has increased the demand for simpler and more intuitive operation, often requiring domain-specific expertise. This paper explores task-oriented natural-language interaction with automation systems, a paradigm that is rapidly gaining traction particularly due to the advent of large language models (LLMs). Leveraging a multi-agent system (MAS), the authors introduce a two-phase approach for controlling an industrial gantry for pick-and-place operations. In phase one, the MAS generates an execution plan based on the user input; in phase two, the plan is executed deterministically on real-world hardware. Our preliminary results demonstrate the advantages of a chat-based human-in-the-loop (HiL) workflow for refining and validating execution plans, and they highlight the promise of natural-language interfaces for operating pick-and-place systems (Demo: Video 1, Video 2).