In this paper, we present preliminary work into combining formal models of dialogue and large language models, before going on to discuss how this provides a foundation for similar approaches involving computational models of argument. First, we address the twin issues of how a formal dialogue game can usefully regulate dialogical utterances generated by an LLM during an extended, goal-oriented conversation, and conversely, how LLMs can close the human-level language generation gap associated with formal dialogue games. We then proceed to identify how our solution to these issues can underpin future work towards using computational argumentation to provide reasoning-like capabilities to LLMs, and using LLMs for tasks such as searching and summarisation of analysed argument data.