[Apologies for multiple postings] ------------------------------------------------------------------------- Mowjaz Multi-Topic Labelling Task Website: https://www.just.edu.jo/icics/icics2021/mowjaz/ To be organized at ICICS 2021 (https://www.just.edu.jo/icics/icics2021/) 24 - 26 May Virtual Conference ------------------------------------------------------------------------------- -------------------------- Task Description: -------------------------- Mowjaz (mowjaz.com) is an Arabic topical content aggregation mobile application for news, sport, entertainment and other topics from top publishers that users can follow. Mowjaz search engine and recommendation system is built on top of NLP/NLU machine learning APIs that distinguish it from any other Arabic content applications available, mainly focusing on the users having the best experience and receiving content that is of their interest. Mowjaz is a subsidiary of Mawdoo3.com, the world's biggest Arabic website in terms of number of visitors. One of Mowjaz's top AI powered models is Topic Multi-Labelling, which is the focus of this shared task. This model is basically used to classify articles based on their topics. Additionally, the model predicts multiple topics in one article and is categorized to all possible topics that are present within its content. Mowjaz's topics are classified into ten categories and an article can be classified under as many topics as it covers. This model helps users get and display the most relevant news to their interests. The enhanced user experience that Mowjaz offers makes one news article be classified and shown under all the different topics that it holds. Participating systems are expected to select one or more of the ten topics for each given article. They are evaluated based on their effectiveness and efficiency. Full task description can be found at: https://www.just.edu.jo/icics/icics2021/mowjaz/ -------------------------- Dataset and Evaluation: -------------------------- The dataset consists of 9,590 articles split into training, development and testing sets. The evaluation of the participating systems is done is two separate tracks as follows: - The first track is the effectiveness track, where the participating systems are ranked based on their accuracy. Specifically, the F1 score is used for this track. This track is mandatory for all participating systems. - The second track, which is optional, is the efficiency track. Participating teams are asked to run their systems in a docker container with memory constraints to measure their average testing times. A simple tutorial of this process is provided in the GitHub repository of this task. The score of each system is a weighted average of its F1 score and average testing time (normalized) and the systems are ranked based on these scores. Submissions for this track will be open in the last week of the competition and results will be announced once after the end of the competition. The second track is optional and it promotes the open-source mentality. Thus, teams participating in it are asked to share any resources they use such as external datasets (labeled or unlabeled), pre-trained models, knowledge bases, etc. ------------ Timeline ------------ 18-Feb-2021 Release of task website, dataset & Codalab competition 18-Mar-2021 Start of the evaluation phase for the effectiveness track 12-Apr-2021 Start of the evaluation phase for the efficiency track 19-Apr-2021 Deadline for submitting runs 20-Apr-2021 Results declared 23-Apr-2021 Single-Paragraph descriptions of participating systems due 03-May-2021 Working notes papers due 10-May-2021 Reviews on working notes papers due 17-May-2021 Final version of working notes papers due 24-May-2021 ICICS2021 ---------------- Organizers ---------------- Mahmoud Al-Ayyoub, Department of Computer Science (CS) and Center of Excellence for Innovative Projects (CEIP), Jordan University of Science and Technology, Jordan Haitham Selawi, Mawdoo3 Ltd, Jordan Mohamed Zaghlol, Mawdoo3 Ltd, Jordan Hussein Al-Natsheh, Mawdoo3 Ltd, Jordan Samer Suileman, Center of Excellence for Innovative Projects (CEIP), Jordan University of Science and Technology, Jordan Ali Fadel, Department of Computer Science (CS), Jordan University of Science and Technology, Jordan Riham Badawi, Mawdoo3 Ltd, Jordan Ahmed Morsy, Mawdoo3 Ltd, Jordan Ibraheem Tuffaha, Mawdoo3 Ltd, Jordan Mohannad Aljarrah, Center of Excellence for Innovative Projects (CEIP), Jordan University of Science and Technology, Jordan For regular updates subscribe to our mailing list: mowjaz-task@googlegroups.com For any question about this call, please send an email to Mahmoud Al-Ayyoub : maalshbool@just.edu.jo Regards, Organizers of the Mowjaz Multi-Topic Labelling Task