Artificial intelligence (AI) and large language models (LLMs) like ChatGPT have transformed the academic landscape, but are there any reasons why not to use ChatGPT for research? We say, YES! When OpenAI released ChatGPT in November 2022, it changed the way we interact with technology, fueling a curiosity within the academic community about how this AI writing tool could be used to aid various research processes. Yet, while appreciating the impressive capabilities that made it one of the most popular LLMs in the world, it is crucial to recognize and consider the limitations of ChatGPT for research use cases.
In this article, we delve into some of the significant risks associated with using ChatGPT for research tasks and review why it is important to tread with caution. We’ll also tell you more about the LLM intelligence powering Paperpal, which makes it a better AI tool for researchers.
| ChatGPT Limitations | Reasons not to use ChatGPT |
| Training dataset | Large proportion of non-academic sources could result in generic, unverified output |
| Prompt reliability | Major variations in the responses generated depending on how you frame the prompt |
| AI hallucination | High chances of meaning changes, fabricated information, and even potential biases creeping in |
| Domain expertise | Not tailored to deliver academic writing, tone, and style, making it an unreliable resource |
General vs. Specialized Training Dataset
OpenAI’s LLM technology has been trained on a vast set of general data, including sources like Wikipedia, open-source blogs, opinion pieces, and a host of other unverified information. It has been designed to be a one-size-fits-all technology for varied use cases, and this reflects in the kind of outputs generated. Unreliable outputs are just one of the reasons not to use ChatGPT for research.
Academia is a fact- and information-driven space that relies on accurate, reliable, and factual content, which ChatGPT cannot guarantee. In contrast, Paperpal’s LLM technology has been designed specifically for the academic use case. Its training dataset has been carefully fine-tuned keeping the researcher’s needs front and center, which is visible in the quality of output it presents.
Prompt Unreliability vs. Intuitive Inputs
When it comes to LLMs, what you get is only as good as what you ask for! Well-designed prompts guide the model to generate accurate and specific responses. By providing clear instructions and context, prompt engineering helps ensure that the generated content aligns with your purpose. This makes the technology more adaptable to your needs. However, it also means that using ChatGPT for research use cases can result in varied, unreliable, and possibly incomplete responses if you don’t structure your prompt correctly.
We know that researchers need fast, precise support, so we thought it best to do some of the heavy lifting to support you better. Don’t worry about framing perfect inputs, Paperpal’s LLM lets you paraphrase academic text to communicate your ideas effectively or decrease word count to meet strict journal requirements with just a click!
500,000 academics trust Paperpal. Start writing for free and see why
AI Hallucinations vs. Need for Accuracy
Journals and other publications have warned of the dangers of misinformation and hallucination within ChatGPT outputs, which is a key reason not to use ChatGPT for research. OpenAI also acknowledges this limitation.1 An article published by the National Institute of Health documented an experiment where ChatGPT was asked to generate a paragraph on the mechanism of homocysteine-induced osteoporosis and validate the explanation with references. None of the provided references were found to be real. The Pubmed IDs shared were those of unrelated papers.2
These are some of the many reasons why not to use ChatGPT for academic writing. “What we have today is essentially a mouth without a brain,” says Yejin Choi, a computer scientist at the University of Washington and the Allen Institute for Artificial Intelligence.3 While LLMs can help researchers save time and boost productivity, it is important to be aware of and know the risks with using ChatGPT for research use cases. If, however, you’re simply looking for great ideas for a dinner party tonight, I think ChatGPT’s come up with some perfect suggestions!
While ChatGPT undoubtedly offers interesting possibilities, it is imperative that researchers approach its use with discernment keeping in mind the ethical considerations that underpin genuine scholarly work. While researchers should definitely embrace new technologies that will reduce time spent on mundane jobs and allow us to focus on more critical thinking tasks, remember to choose reliable AI technology curated for academia that are more suited to aid your publication process. On that note, the Paperpal team has been working on something that will make you sit up and take notice!
No more prompt writing on ChatGPT. Save time with Paperpal’s in-built academic writing prompts.
We’ve got something up our Generative AI sleeve!
In the past year, The Paperpal family has quickly grown to over 300,000 customers and established itself as the trusted partner for more than 400 journals across the world. We’ve grown and evolved from a language checker to now a comprehensive AI writing and publication assistant, offering unlimited translation, paraphrasing, consistency, and journal submission readiness checks with Paperpal Prime.
But we’re nowhere close to done. The Paperpal team is committed to overcome the challenges you face with more generic LLMs and launch the best generative AI features tailored to academic content as early as this September. If you have not explored Paperpal yet, sign up now and transform your academic writing with us!
References:
- Natalie, What is ChatGPT? OpenAI website. Available at https://help.openai.com/en/articles/6783457-what-is-chatgpt
- Alkaissi, H., & McFarlane, S. I. Artificial Hallucinations in ChatGPT: Implications in Scientific Writing. Cureus, February 2023. https://doi.org/10.7759/cureus.35179. Available online at https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9939079/
- Hutson, M. Robo-writers: the rise and risks of language-generating AI. Nature, News, March 2021. https://www.nature.com/articles/d41586-021-00530-0#ref-CR2
Paperpal is a comprehensive AI writing toolkit that helps students and researchers achieve 2x the writing in half the time. It leverages 21+ years of STM experience and insights from millions of research articles to provide in-depth academic writing, language editing, and submission readiness support to help you write better, faster.
Get accurate academic translations, rewriting support, grammar checks, vocabulary suggestions, and generative AI assistance that delivers human precision at machine speed. Try for free or upgrade to Paperpal Prime starting at US$19 a month to access premium features, including consistency, plagiarism, and 30+ submission readiness checks to help you succeed.
Experience the future of academic writing – Sign up to Paperpal and start writing for free!
Paperpal for China
Paperpal for Korea
Paperpal for Japan
Speak to us
