Geospatial Artificial Intelligence (GeoAI) systems are increasingly used by local governments to manage urban planning activities. However, there is a lack of clear guidelines for responsible artificial intelligence (AI) implementation. We address this gap by applying the task-data-user-technology fit theory. First, we create a conceptual framework that translates ethical AI principles into practical system requirements. Second, we apply this framework through a case study analysis. We examine the geospatial AI system that the city of eThekwini, South Africa, has deployed to monitor their informal settlements. Based on expert interviews, our analysis highlights the ethical trade-offs inherent in the interactions between fits, data, tasks, users, and AI interactions, especially in an inherently localized and multi-stakeholder field such as urban planning. Additionally, we show how these fits are interconnected and cross-dependent. For example, the technical skills and resources of users can influence all other fits. Our study also reveals how task reconfiguration, user adaptability, and data improvement can enhance or hinder alignment with technology. From these insights, we introduce practical and theoretical recommendations for responsible AI development, adoption, and use.