This is a simple paper that shows the very basic limitations of LLMs in executing planning tasks: the paper uses an LLM that has access to function calls and asks the LLM to plan a travel itinerary under certain constraints. The LLM has access to 6 functions, ranging from CitySearch to AccommodationSearch. It accesses a database that is pre-populated with a fixed number of cities and accommodations at certain prices and with certain features. The LLM is then asked to create an overall plan that satisfies certain sets of constraints. Those plans are compared to ground truths. The image below shows the workflow.