Scientific workshop on using generative AI for science and higher education studies

The development of generative AI language models represents one of the most significant technological change of the recent years. Generative AI has some attractive features for science and higher education studies, including notably the fact that it can access and summarize a very wide range of sources available on the Internet, thereby retrieving information that is not available on research databases. The interactive features of ChatGPT are potentially relevant as these allow users to interact with the model to get more precise and relevant results. Accordingly, generative AI, and notably ChatGPT, is starting to be tested in scientometrics and bibliometrics for tasks such as benchmark similar institutions, evaluating research quality, and analyzing researchers’ careers.

In face of this rapidly evolving situation in scientometrics, the goal of the workshop is to advance our understanding of the potential of generative AI, including but not limited to ChatGPT, for science of science and higher education studies. The workshop is intended to identify pitfalls and risks and to propose approaches to mitigate them.

We particularly welcome empirical studies comparing systematically the ChatGPT output with more standard approaches in scientometrics, science and higher education studies to perform tasks such as:

- The labeling of research topic based on publication metadata.

- The identification of prominent scholars and the measurement of their performance.

- The evaluation of research quality at the individual and at the institutional level.

- The analysis of scientific careers.

- The identification of similar institutions and their benchmarking.

- The characterization of universities’ missions, strategies and organization.

Contributions will be selected based on an open call for proposals and peer review of submitted short papers.

The workshop will be organized in a set of thematic sessions around the topics introduced above. Each paper will be assigned to a discussant, who will be expected to read the paper in advance and provide a detailed feedback in view also of a scholarly publications. The workshop will also include a final panel session devoted to summarizing the main findings and proposing directions for a sensible use of generative AI in science studies. The publication of a special issue of the leading journal Scientometrics is envisaged.