The Pantagruel project is an ambitious initiative that aims to develop and evaluate multimodal (written, spoken, pictograms) and inclusive linguistic models for French. The project draws on the expertise of researchers from different disciplines, including computer science, signal processing, sociology, and linguistics, to ensure diversity of perspectives, as well as the reliability and relevance of results. The main contributions of the project are the development of freely available self-supervised models for French, including one to three of the modalities for the general and clinical domains. The project will not only produce models but also design test benches to evaluate the generalization of such models, building on the experience gained in the FlauBERT and LeBenchmark projects. Part of the project will be devoted to the biases and stereotypes conveyed in the training corpora and in the downstream models. An ethics committee will help limit the amplification effect of bias within the training corpora, in particular by working on the demographic characteristics of the speakers (for audio or transcribed speech) and of the authors (for part of the written data). We will thus be able to compare the models learned on training corpora with variable proportions for these characteristics, such as gender. This study will quantify to what extent the predictions of the language models are reliable reflections of the upstream corpora and to better control the way in which they can be used as social scientific research tools. The project will develop software components that will facilitate the integration of language models into various applications and allow the development of innovative solutions that exploit the power of multimodal French language models. These tools are particularly intended for non-computer scientists such as those who are members of the consortium (sociologists, linguists, doctors, speech therapists), researchers from other fields, or artists. The Pantagruel project thus has the potential to significantly advance the state of the art in multimodal language models and to have disseminate the use of these models in a wide range of applied fields, ranging from healthcare to the humanities and the social sciences.

Digitisation changed all aspects of the news media landscape, from the way content is created, to how it is distributed and interacted with. Three phenomena shape the face and fate of news media in Europe: decreasing trust and information disorder, digitisation and changing user behaviour, and dominance of global technology and AI. They rock the foundations of the journalist profession. MediaNumeric provides students and young professionals in media and communication studies the theoretical know-how and skills needed to embolden them to take on the opportunities of data-driven journalism and media production. And highlights the potential of using large multimedia databases for data-driven innovations. The Alliance captures the needs of the news and media industry and ties them to the educational offer of Higher Education Institutions (HEIs). It brings together leading actors in academia, industry and audiovisual archives from four EU Member States embedded in the most prominent networks.Three complementary training modules of MediaNumeric cover key subjects critical to data-driven journalism; search and exploration of multimedia data, storytelling, and tracking and debunking disinformation. Third parties are engaged in discussions related to these subjects through a Stakeholder Board, and in effect inform the programme design. The programme is co-developed and tested in three successive training courses (in NL, PL, FR), targeting international groups of students. Training materials are packaged so they can be used as online, open-access learning materials, either as comprehensive, integrated courses or as granular modules. A communication strategy is in place to maximise the uptake of the programme by HEIs across the EU and disseminate outcomes to the other Target Groups; archives, policymakers and the media industry. Sustainability is ensured by putting in place measures that keep the training material updated and support the HEI that use the learning materials.

Large catalogues are moving from the database metadata management age using specific Information Science and Libraries formats, to the Web age using Semantic Web standard languages (RDF / S, OWL). This development, bringing many advantages (better document availability, increased data exchange capabilities, creation of new search / use services for documents), raises important issues about the quality of document databases. This project aims to develop mechanisms to: • describe the quality of an existing document database; • maintain a given level of quality by controlling updates on such databases; • improve the quality of a database; • exploit these databases according to their level of quality (eg the search for documents or combination of bases). Representing data using Semantic Web standards allows for a Knowledge Representation approach to this problem. This approach will allow on one hand to give a logical semantics to the notion of quality and, on the other, to use reasoning mechanisms for dealing with various problems. This approach is rooted on the (i) formalization of knowledge found in document catalogues, (ii) the development of a quality model for the individual entities (named entities) identification problem, (iii) the definition of a trust model suitable for reconciliation and different source information fusion and (iv) the discovery of entity identification characteristics and their manipulation by different techniques (logical, numerical, probabilistic, etc.). A large part of the project is devoted to the evaluation of the proposed approach by experiments conducted on suitable test benchmarks and the development of demonstrators adapted to the two document databases owners involved in the project. The consortium brings together five complementary partners: two major national players of document catalogues and three research groups of computer scientists. The Bibliographic Agency for Higher Education (ABES) and the Institut National de l'Audiovisuel (INA) are managing very large document databases and are heavily involved, both at a national and international level, in the exposure, standardization, interconnection and use of their metadata. The teams of the LIG, LIRMM and LRI involved in this project have a strong expertise in databases, knowledge representation and semantic web. Furthermore, numerous research connections exist between the project partners. The skills of scientific partners and links forged between them as part of joint projects are very important to the success of this multidisciplinary project that involves both Information Science and Libraries as information technology, and should impact not only the field of document databases but also the Web of Data ("Linked Data").

Audiovisual media content created and used in films and videos is key for people to communicate and entertain. It has also become an essential resource of modern history, since a large portion of memories and records of the 20th and 21st centuries are audiovisual. To fully benefit from this asset, fast and effective methods are needed to cope with the rapidly growing audiovisual big data that are collected in digital repositories and used internationally. MeMAD will provide novel methods for an efficient re-use and re-purpose of multilingual audiovisual content which revolutionize video management and digital storytelling in broadcasting and media production. We go far beyond the state-of-the-art automatic video description methods by making the machine learn from the human. The resulting description is thus not only a time-aligned semantic extraction of objects but makes use of the audio and recognizes action sequences. While current methods work mainly for English, MeMAD will handle multilingual source material and produce multilingual descriptions and thus enhance the user experience. Our method interactively integrates the latest research achievements in deep neural network techniques in computer vision with knowledge bases, human and machine translation in a continuously improving machine learning framework. This results in detailed, rich descriptions of the moving images, speech, and audio, which enable people working in the Creative Industries to access and use audiovisual information in more effective ways. Moreover,the intermodal translation from images and sounds into words will attract millions of new users to audiovisual media, including the visually and hearing impaired. Anyone using audiovisual content will also benefit from these verbalisations as they are non-invasive surrogates for visual and auditory information, which can be processed without the need of actually watching or listening, matching the new usage of video consumption on mobile devices.