In preparing the distinguished dissertation, 'Comparative analysis of GLUE (General Language Understanding Evaluation) and SuperGLUE benchmarks', the graduate was supervised by dr hab. inż. Aneta Poniszewska-Marańda, TUL prof., from the Institute of Information Technology in the Faculty of Technical Physics, Information Technology and Applied Mathematics.
'The development of language models, including large language models (LLMs), has added urgency to the quest for effective methods of their evaluation. The study compares GLUE and SuperGLUE benchmarks, provides an assessment of their capability to evaluate natural language processing (NLP) models, and offers a universal method for NLP benchmark comparison', Kacper Królewiak explains.
LLM is an advanced artificial intelligence (AI) tool to analyze and generate texts based on previously learned language patterns. Such models aid in content creation, language translation, answering queries, and many other tasks. The GLUE and SuperGLUE benchmarks, on the other hand, are used to test systems for natural language understanding.
The first and the third prize in the Competition on AI 4 Science were awarded to graduates of Poznań University of Technology and Warsaw University of Technology. The awards presentation gala will be held in October 2024, during which the winners will be given the opportunity to present their work to a greater audience.
The TUL graduate is now pursuing a degree in Information Technology at Warsaw University of Technology, but, in his own words: 'I have been thinking of returning to Lodz University of Technology for my PhD.'
The Competition was organized by the 4 Science Institute sp. z o. o. Among the sponsors were IEEE Xplore, PS IEEE, CERN, Łukasiewicz PIAP, Poznan Supercomputing and Networking Center PSNC, and the National Center for Research and Development. President of the Polish Academy of Sciences held the honorary patronage. Allegro served as the business partner and prize funder, and Billon Group is the Technology Partner.