Scholarly record
ARTIFICIAL INTELLIGENCE IN TERMINOLOGY PROCESSING: TESTING THE ROBUSTNESS OF A CUSTOM GPT
Abstract
Following the discontinuance of funding of the Croatian national term base – Struna (http://struna.ihjj.hr/) in 2019. we initiated the development of a new methodology for creating terminological collections independent of field experts' input for the initial terminological data. A possible solution to our problem of finding a compact and robust source for generating information in the early stages of processing terminology (the вЂraw definitions’) in various domains could be found in the publicly available AI language model created by OpenAI called ChatGPT-4. ChatGPT is a large language model whose functions include answering questions, text generation, and completing tasks such as translation and summarisation. A custom GPT is currently being developed that will be used as an assistance module, providing raw information for terminological units to be processed in Struna. Following the initial intensive testing of ChatGPT-4, we have started to develop and train a custom GPT bot (working name: TermAI). The first stage of training consisted of manually providing rules of good practices for terminology management, adapted from the original training of field experts. The second stage consists of feeding the TermAI with modified data exported from Struna. In this paper, we will present the results of the analysis of generated information from the new domain in comparison to the quality of information that was attained in the domain that TermAI was trained on, as well as information obtained from the actual field experts in the novelty domain.
Publication Impact Profile
Publication details
References18
B. Nahod, O umu strucnjaka. Zagreb: Institut za hrvatski jezik i jezikoslovlje, 2016.
“OpenAI GPT-4.” [Online]. Available: https://openai.com/product/gpt-4.
R. Noll, L. S. Frischen, M. Boeker, H. Storf, and J. Schaaf, “Machine translation11th SWS International Scientific Conference on Arts and Humanities ISCAH 2024 of standardised medical terminology using natural language processing: A scoping review,” N. Biotechnol., vol. 77, no. August, pp. 120–129, 2023, doi:
C. A. Gao et al., “Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers,” npj Digit. Med., vol. 6, no. 1, pp. 1–5, 2023, DOI: 10.1038/s41746-023-00819-6.
Y. Ma et al., “AI vs. Human -- Differentiation Analysis of Scientific Content Generation,” no. January, 2023, DOI: 10.48550/arXiv.2301.10416.
Q.-L. Nguyen, M.-H. Nguyen, and V.-P. La, “Academic writing and AI: Day-4 experiment with mindsponge theory,” 2023.
R. V. Deniko, O. G. Shchitova, D. A. Shchitova, and N. T. Lan, “Learning Terminology in the Age of Higher Education Internationalization: Problems and Solutions,” Procedia - Soc. Behav. Sci., vol. 215, no. June, pp. 107–111, 2015, DOI: 10.1016/j.sbspro.2015.11.582.
T. Susnjak, “ChatGPT: The End of Online Exam Integrity?,” pp. 1–21, 2022.
Z. Basic, A. Banovac, I. Kruzic, and I. Jerkovic, “ChatGPT-3.5 as writing assistance in students’ essays,” Humanit. Soc. Sci. Commun., vol. 10, no. 1, pp. 1–6, 2023, DOI: 10.1057/s41599-023-02269-7.
R. Lew, “Dictionaries and lexicography in the AI era,” Humanit. Soc. Sci. Commun., vol. 11, no. 1, 2024, DOI: 10.1057/s41599-024-02889-7.
M. Jakubicek and M. Rundell, “The end of lexicography? Can ChatGPT outperform current tools for post-editing lexicography?,” Proc. Electron. Lexicogr. 21st Century Conf., vol. 2023-June, pp. 518–533, 2023.
G. P. Rees and R. Lew, “The Effectiveness of OpenAI GPT-Generated Definitions Versus Definitions from an English Learners’ Dictionary in a Lexically Orientated Reading Task,” Int. J. Lexicogr., vol. 37, no. 1, 2024, DOI: 10.1093/ijl/ecad030.
G. M. de Schryver, “Generative AI and Lexicography: The Current State of the Art Using ChatGPT,” Int. J. Lexicogr., vol. 36, no. 4, pp. 355–387, 2023, DOI: 10.1093/ijl/ecad021.
B. Nahod, “Od polisemna leksema do homonimna naziva,” in Zbornik Hrvatskog drustva za primjenjenu lingvistiku, 2011, pp. 15–26.
OpenAI, “Introducing GPTs,” 2024. [Online]. Available: https://openai.com/blog/introducing-gpts.
B. Nahod, “Can We Substitute Field Experts with Customized Large Language Model in Processing Specialized Languages – a Case Study,” in XXI EURALEX International Congress, 2024.
M. Mihaljevic, L. Hudecek, and Z. Jozic, Eds., Hrvatsko jezikoslovno nazivlje. Zagreb: Institut za hrvatski jezik i jezikoslovlje.
“mocijski parnjak - Struna,” 2024. [Online]. Available: http://struna.ihjj.hr/naziv/mocijski-parnjak/51161/#naziv.
View or Download full articleAccess options
SWS access login
Login as SWS Scientific CommitteeLogin as SWS Scientific PartnerLogin as SWS AuthorAuthors and approved SWS contributors will read and export their own linked papers after identity matching by SWS profile, email and SGEM GlobalID.
For librarian assistance: [email protected]
Purchase Instant Access
- Article can be downloaded after successful payment.
- Article may be used according to SWS library access terms.
- Article cannot be redistributed.