You are here
ChatGPT in Action
Module 3: ChatGPT in Action
A Practical Guide for ChatGPT-Enhanced Research
by Burak Senel
In our previous posts we invited you on a journey to better understand ChatGPT, from its early beginnings to its current position as a research tool with great potential. We’ve laid bare the workings of this fascinating tool, painting a picture of its visible and invisible aspects to explore its scientific implications.
Now, let’s roll up our sleeves and delve into the practical application of ChatGPT in our research.
Three Facets of ChatGPT in Research
First, some clarification is due. ChatGPT and research can come together in many ways. We identify three categories:
1. Research on ChatGPT: Generative AI tools are ever-evolving. We’re continuously refining, fine-tuning, and reassessing—pushing for increased efficiency, ethical application, and overall excellence. As such, research on ChatGPT is an ongoing endeavor, an unending quest for progress.
2. Research on human interaction with ChatGPT: As these tools become an integral part of our daily lives, the way we converse with them merits exploration. How do we humans communicate with ChatGPT? How does our perception of it influence our interaction? These and other questions lie at the heart of this research category.
3. Research, using ChatGPT: This is where things get exciting. As researchers, it’s up to us to harness the power of tools like ChatGPT responsibly and ethically. With scientific principles as our guide, how can we integrate this tool into our work? This blog post revolves around this intriguing question.
It’s important to note that these categories aren’t independent silos; they may overlap. For instance, a researcher exploring human interaction with ChatGPT could simultaneously be utilizing ChatGPT to enhance her research.
Classifying Research Tasks: Where Does ChatGPT Fit In?
One more thing before we dive deeper—what tasks can ChatGPT handle in a research environment? Broadly speaking, we can categorize tasks into academic (research-related) and non-academic (everything else). We further categorize academic tasks into two categories, high-stakes academic tasks and low-stakes academic tasks. High-stakes academic tasks are critical to the integrity of science and demand cognitive effort. Information synthesis is one example, as it requires fact-checking, considering the context of information (such as the authors of information), where the information appeared, and what others think about the information. High-stakes academic tasks generally require considerable cognitive effort to verify and combine information, as well as moral judgment to evaluate information ethically. Low-stakes academic tasks, on the other hand, are tasks that don’t need much cognitive effort and are generally labor-intensive, such as ensuring data is typed in or formatted correctly.
ChatGPT is a great tool for low-stakes academic tasks. It can be used for cohesion, grammar, making tables, extracting information, and more. For high-stakes academic tasks, guidance is needed from a skilled human researcher due to the science-threatening limitations discussed in our previous post. To counter some of ChatGPT’s limitations in high-stakes academic tasks, researchers can prompt it to:
● Justify its responses for better clarity and evaluation
● Provide sources for fact-checking when possible
● Offer responses only when it’s confident about the veracity of the information
Now, let’s see how a ChatGPT-enhanced human-directed research workflow is structured. We’ll divide our research process into before, during, and after; and we will look at some considerations presented for each.
A Step-by-Step Approach to ChatGPT Use in Research
Before Using ChatGPT
Setting the stage for the use of ChatGPT starts with a well-crafted research design. This blueprint helps align expectations with reality and sets the course for your research project. It’s equally crucial to factor in ethical considerations, ensuring your research respects privacy and uses technology responsibly. Your research credibility hinges on how effectively you manage the quality of data and responses obtained from ChatGPT. Therefore, having a contingency plan becomes essential to keep your research on track despite potential hiccups. Lastly, make sure you are aware of the AI policies of any potential publication outlets so that your research journey is transparent, smooth, and fruitful.
While Using ChatGPT
The next phase is the “doing” stage—engaging in research using ChatGPT. Here, transparency is key. Transparency about the involvement of AI in your research and clarity about the role of ChatGPT in each task will boost your research credibility. It is also important to constantly validate and refine your work based on the quality of AI responses.
After Using ChatGPT
As the research concludes, reflect on your experience. What worked well? What could have been better? Your learnings will influence future research designs. Similarly, reflect on the ethical aspects of AI use in your project. Did you maintain privacy and responsibly use AI? Answering these questions will help you set a higher bar for your next research project. Lastly, share your experience with the larger community to contribute to the evolving body of knowledge about AI in research.
To make all this easier to grasp, let’s consider a practical example.
Applying ChatGPT in Social Science Research: A Case Study
Enter Dr. Ercerman, a computational social scientist made up for illustration purposes, who wants to use ChatGPT to understand public sentiment and common themes around the topic of climate change on social media.
Pre-ChatGPT Use
Dr. Ercerman first outlines her research design, aiming to use ChatGPT for generating arguments on climate change policies. She confirms that the use of publicly available, anonymized data aligns with ethical standards. She then crafts a clear methodology involving well-structured prompts for ChatGPT. With a quality assurance plan in place, she establishes how she will assess ChatGPT’s responses. Considering possible issues with the tool, she prepares a contingency plan. Lastly, she ensures her target journals accept research utilizing generative AI like ChatGPT.
During ChatGPT Use
During the research, Dr. Ercerman establishes an experimental setup for prompt engineering, providing ChatGPT with background information and defining the task, expected output, and constraints. She incorporates external public opinion data into her prompts. She cross-validates some of the sentiments and themes identified by ChatGPT with manual coding, picks the best prompt, and continues to carry out the research, monitoring the tool’s outputs for accuracy and relevance over time. She keeps records of her interactions.
Post-ChatGPT Use
Upon completing her research, Dr. Ercerman documents her interaction process with ChatGPT, noting the version used, the prompts, and the responses. She properly cites the use of ChatGPT in her paper. She decides to share the prompts, model settings, and some of the responses to illustrate good and bad examples. Finally, she reflects on the project, assessing the effectiveness of ChatGPT and considering improvements for future studies.
Links from Table 1 & 3:
Sage’s Chat GPT Policy1
OpenAI’s publication and sharing policy2
APA citation guidelines3
MLA citation guidelines4
About Burak Senel
