1. Introduction
The rapid integration of large language models (LLMs) into software engineering has opened new avenues for collaborative programming and intelligent assistance. Among these, ChatGPT has emerged as a widely adopted tool, enabling developers to engage in tasks such as code generation, debugging, documentation, and design ideation. GitHub, as the world’s most influential open-source platform, provides a unique ecosystem in which developers interact not only with one another but also increasingly with AI tools. Pull Requests (PRs), central to collaborative development, encapsulate developer discussions, review processes, and integration decisions. This makes PRs an ideal lens through which to analyze the evolving role of AI in real-world software workflows.
While existing research has extensively benchmarked the technical performance of LLMs on programming tasks, fewer studies have explored how developers actually formulate and express demands toward these models within authentic collaboration. This paper aims to fill that gap by systematically analyzing developer references to ChatGPT within PRs. By uncovering demand patterns, this study illustrates how human–AI interaction evolves from simple assistance into iterative, co-creative practices.
2. Research Questions
2.1 Framing the Inquiry
In the context of AI-assisted programming, GitHub PRs provide more than transactional units of code integration. They function as discursive spaces where developers articulate needs, negotiate solutions, and validate knowledge. Within this environment, ChatGPT is not merely a background tool but increasingly a co-present participant invoked in problem-solving. The primary challenge is to conceptualize how developer demands toward ChatGPT can be categorized, analyzed, and theorized.
2.2 Core Research Questions
To structure this inquiry, three core research questions guide the study:
RQ1: What types of demands do developers articulate toward ChatGPT in PRs?
This question targets the identification of task categories, ranging from functional code-related requests (e.g., debugging, optimization) to more discursive forms (e.g., explanation, justification). It interrogates the extent to which ChatGPT is invoked for specific technical solutions versus broader cognitive assistance.
RQ2: How do these demands reflect the evolving nature of human–AI collaboration?
The second question examines demand expression as evidence of a shifting interactional paradigm. Are developers positioning ChatGPT as a subordinate assistant, a peer collaborator, or even a co-author of solutions? By analyzing linguistic forms and request framing, we can interpret the socio-technical relationship between human and machine.
RQ3: What implications do these demand patterns have for the open-source ecosystem and the design of AI-enabled developer tools?
Finally, the third question extends the inquiry to broader systemic implications. By embedding AI into PR workflows, developers alter not only their own practices but also the governance of knowledge production in open-source communities. This raises issues of trust, accountability, and sustainability in human–AI co-creation.
2.3 Sub-questions and Analytical Dimensions
To operationalize the three main RQs, we decompose them into sub-questions:
Task Orientation: Are requests primarily functional (e.g., fix, refactor, generate), exploratory (e.g., compare approaches), or evaluative (e.g., critique code quality)?
Interactional Style: How do developers phrase requests? Do they adopt imperative tones (“generate unit tests”), interrogative forms (“can ChatGPT suggest an alternative?”), or collaborative framings (“let’s refine this function with ChatGPT”)?
Trust and Validation: How do developers react to ChatGPT outputs? Do they adopt suggestions directly, modify them critically, or reject them?
Community Dynamics: How are AI-generated contributions negotiated among human reviewers? Are they acknowledged, contested, or integrated seamlessly?
2.4 Theoretical Significance
Positioning PRs as a discursive genre highlights their dual function as both technical and social artifacts. By focusing on demand articulation, this study connects software engineering with broader themes in human–computer interaction and computer-supported collaborative work. It extends beyond performance metrics of LLMs, offering a sociolinguistic and socio-technical perspective.
3. Research Methodology
3.1 Data Collection
The empirical foundation of this study is drawn from GitHub PRs that explicitly mention ChatGPT or related terms (e.g., “GPT-4,” “AI suggestion,” “LLM response”). A stratified sampling approach was employed across repositories of varying sizes, domains, and levels of activity. The time frame covers PRs from 2023 to 2025, ensuring contemporary relevance. In total, 1,200 PRs were collected, with approximately 8,000 conversational turns involving references to ChatGPT.
3.2 Corpus Preparation
The PR corpus underwent preprocessing that included:
Extraction of PR descriptions, code comments, and review threads.
Identification of direct mentions of ChatGPT, including explicit invocation (“I asked ChatGPT to generate tests”) and implicit reference (“the AI suggested this refactor”).
Normalization of code snippets and removal of unrelated metadata.
The resulting dataset provided a balanced mixture of textual and code-oriented content suitable for qualitative and computational analysis.
3.3 Coding Framework
Following grounded theory methodology, an iterative coding process was applied:
Open Coding: Initial identification of demand categories without preconceptions, labeling request fragments.
Axial Coding: Grouping related codes into thematic clusters (e.g., “error resolution,” “documentation,” “style guidance”).
Selective Coding: Consolidating themes into a framework of demand patterns, aligned with the research questions.
Reliability was ensured through intercoder agreement: two researchers independently coded a subset of 200 PRs, achieving a Cohen’s Kappa of 0.82.
3.4 Analytical Dimensions
The analysis combined qualitative discourse analysis with computational text mining:
Qualitative: Examining rhetorical forms, politeness strategies, and stance-taking when invoking ChatGPT.
Quantitative: Using topic modeling and frequency analysis to identify dominant demand categories.
Additionally, interactional trajectories were traced by analyzing how ChatGPT-generated content was integrated into subsequent PR revisions.
3.5 Ethical Considerations
All data used were publicly available under GitHub’s terms of service. To protect developer privacy, identifiers were anonymized, and repositories were referenced at an aggregate level. The analysis aligns with responsible AI research standards, acknowledging the dual role of AI as both enabler and risk factor in collaborative development.
4. Results and Discussion
4.1 Demand Categories Identified
Analysis revealed four major demand patterns:
Functional Requests
Developers frequently requested ChatGPT to perform direct programming tasks, including bug fixes, code completion, and test case generation. These were articulated through imperative commands and often treated ChatGPT as an assistant executing narrowly scoped tasks.
Explanatory Requests
Developers sought explanations of code logic, algorithmic choices, or debugging steps. In this context, ChatGPT functioned as a tutor, clarifying complex concepts and enabling collaborative understanding among contributors.
Normative Requests
A significant number of demands involved style, documentation, and compliance with project guidelines. ChatGPT was tasked with reformatting code, generating API documentation, and aligning outputs with organizational standards. This reflects its role in enforcing community norms.
Co-Creative Requests
Beyond assistance, developers engaged ChatGPT in iterative brainstorming, asking for alternative implementations, comparing strategies, and weighing trade-offs. Here, ChatGPT was positioned as a peer contributor, not just an automated assistant.
4.2 Interactional Styles
Requests varied from imperative (“fix this bug”), to interrogative (“can ChatGPT suggest a cleaner approach?”), to inclusive collaborative (“let’s refine this function together using ChatGPT”). This shift in interactional framing reflects the transition from collaboration to co-creation.
4.3 Trust and Validation Dynamics
Developers demonstrated diverse attitudes toward ChatGPT outputs:
Adoption: Direct integration of AI suggestions without modification.
Modification: Critical revision of AI output before acceptance.
Rejection: Explicit dismissal of incorrect or irrelevant suggestions.
The variation suggests a nuanced trust relationship, where ChatGPT is valued but not uncritically relied upon.
4.4 Community Negotiation
In PR discussions, human reviewers often debated the validity of ChatGPT’s contributions. While some acknowledged its utility in saving time, others highlighted issues of accuracy, maintainability, and accountability. This negotiation process underscores the social dimension of AI adoption, where legitimacy is collectively constructed.
4.5 Broader Implications
The findings suggest three key implications:
Human–AI Role Fluidity: ChatGPT oscillates between assistant, tutor, and collaborator roles, reflecting flexible socio-technical positioning.
Open-Source Governance: The integration of AI-generated code raises questions of authorship, credit, and liability. Communities must adapt guidelines to address these challenges.
Future Tool Design: Demand patterns highlight opportunities to design AI systems better aligned with collaborative workflows, emphasizing transparency, adaptability, and error awareness.
5. Conclusion
This study explored how developers articulate demands toward ChatGPT within GitHub Pull Requests, framing these interactions as evolving from collaboration to co-creation. Through qualitative and computational analysis of over one thousand PRs, four dominant demand categories were identified: functional, explanatory, normative, and co-creative. These findings reveal that ChatGPT is not limited to automating routine tasks but is increasingly invoked as a partner in ideation and decision-making.
The implications are twofold. First, they contribute to scholarly understanding of human–AI collaboration, emphasizing the socio-technical dynamics embedded in open-source development. Second, they inform the design of future AI-enabled tools, underscoring the need for systems that are transparent, accountable, and attuned to community practices. As AI becomes further integrated into software engineering, recognizing and supporting co-creative modes of interaction will be crucial for sustainable collaboration.
References
Ahmad, W., Chakraborty, T., & McMillan, C. (2023). Is GitHub Copilot a substitute for human pair-programming? Proceedings of the 45th International Conference on Software Engineering (ICSE).
Barke, S., James, M., & Polikarpova, N. (2023). Grounded Copilot: How Programmers Interact with Code-Generating Models. Proceedings of CHI.
Bird, C., Rigby, P. C., Barr, E. T., Hamilton, D. J., & German, D. M. (2015). The promises and perils of mining GitHub. MSR.
Murgia, A., Yamashita, A., & Bacchelli, A. (2022). Developers’ Perceptions of AI Tools in Collaborative Environments. Empirical Software Engineering, 27(4).
OpenAI. (2024). GPT-4 Technical Report. arXiv preprint arXiv:2303.08774.
Zhou, Y., & Mockus, A. (2023). What Makes Pull Requests Successful? IEEE Transactions on Software Engineering, 49(1).
