在过去的十年中，在线教育在为全球学生提供负担得起的高质量教育方面的重要性越来越重要。随着越来越多的学生改用在线学习，这在全球大流行期间得到了进一步放大。大多数在线教育任务，例如课程建议，锻炼建议或自动化评估，都取决于跟踪学生的知识进步。这被称为文献中的\ emph {知识跟踪}问题。解决此问题需要收集学生评估数据，以反映他们的知识演变。在本文中，我们提出了一个新的知识跟踪数据集，名为“知识跟踪数据库”练习（DBE-KT22），该练习是在澳大利亚澳大利亚国立大学教授的课程中从在线学生锻炼系统中收集的。我们讨论了DBE-KT22数据集的特征，并将其与知识追踪文献中的现有数据集进行对比。我们的数据集可通过澳大利亚数据存档平台公开访问。

在这项比赛中，参与者将使用时间序列数据在教育背景下解决机器学习的两个基本因果挑战。首先是确定不同构造之间的因果关系，其中构造被定义为学习的最小要素。第二个挑战是预测学习一个结构对回答其他结构问题的能力的影响。应对这些挑战将使学生的知识获取优化，这可以部署在影响数百万学生的真正的edtech解决方案中。参与者将在理想化的环境中运行这些任务，并具有合成数据和现实情况，并通过一系列A/B测试收集的评估数据。

知识跟踪（KT）是使用学生的历史学习互动数据来对其知识掌握的任务，以便对他们未来的互动绩效进行预测。最近，使用各种深度学习技术来解决KT问题已经取得了显着的进步。但是，基于深度学习的知识追踪（DLKT）方法的成功仍然有些神秘，适当的测量以及对这些DLKT方法的分析仍然是一个挑战。首先，现有作品中的数据预处理程序通常是私人和/或自定义，这限制了实验标准化。此外，现有的DLKT研究通常在评估方案方面有所不同，并且是现实世界中的教育环境。为了解决这些问题，我们介绍了一个综合基于Python的基准平台\ TextSc {Pykt}，以确保通过彻底评估进行跨DLKT方法的有效比较。 \ textsc {pykt}库由不同域的7个流行数据集上的一组标准化的数据预处理程序组成，而10个经常比较了用于透明实验的DLKT模型实现。我们细粒度和严格的经验KT研究的结果产生了一系列观察结果和有效DLKT的建议，例如，错误的评估设置可能会导致标签泄漏，这通常会导致性能膨胀；与Piech等人提出的第一个DLKT模型相比，许多DLKT方法的改进是最小的。 \ cite {piech2015 -Deep}。我们已经开源\ textsc {pykt}，并在\ url {https://pykt.org/}上进行了实验结果。我们欢迎其他研究小组和从业人员的贡献。

Automatic analysis of teacher and student interactions could be very important to improve the quality of teaching and student engagement. However, despite some recent progress in utilizing multimodal data for teaching and learning analytics, a thorough analysis of a rich multimodal dataset coming for a complex real learning environment has yet to be done. To bridge this gap, we present a large-scale MUlti-modal Teaching and Learning Analytics (MUTLA) dataset. This dataset includes time-synchronized multimodal data records of students (learning logs, videos, EEG brainwaves) as they work in various subjects from Squirrel AI Learning System (SAIL) to solve problems of varying difficulty levels. The dataset resources include user records from the learner records store of SAIL, brainwave data collected by EEG headset devices, and video data captured by web cameras while students worked in the SAIL products. Our hope is that by analyzing real-world student learning activities, facial expressions, and brainwave patterns, researchers can better predict engagement, which can then be used to improve adaptive learning selection and student learning outcomes. An additional goal is to provide a dataset gathered from real-world educational activities versus those from controlled lab environments to benefit the educational learning community.

知识追踪是指估计每个学生的知识组成部分/技能掌握水平的问题，从他们过去对教育应用中的问题的回答。一种直接的收益知识追踪方法提供的是能够在未来问题上预测每个学生的表现。但是，大多数现有知识追踪方法的一个关键限制是，他们将学生对问题的回答视为二进制评估，即是正确的还是不正确的。响应正确性分析/预测易于导航，但会丢失重要信息，尤其是对于开放式问题：确切的学生回答可能会提供有关其知识状态的更多信息，而不是仅仅是响应正确性。在本文中，我们首次介绍了对开放式知识追踪的探索，即，在知识跟踪设置中，学生对学生对问题的开放式回答的分析和预测。我们首先制定了一个通用框架，用于开放式知识跟踪，然后通过编程问题详细介绍其在计算机科学教育领域的应用。我们在该域中定义了一系列评估指标，并进行了一系列定量和定性实验，以测试现实世界中学生代码数据集中开放式知识跟踪方法的边界。

Any organization needs to improve their products, services, and processes. In this context, engaging with customers and understanding their journey is essential. Organizations have leveraged various techniques and technologies to support customer engagement, from call centres to chatbots and virtual agents. Recently, these systems have used Machine Learning (ML) and Natural Language Processing (NLP) to analyze large volumes of customer feedback and engagement data. The goal is to understand customers in context and provide meaningful answers across various channels. Despite multiple advances in Conversational Artificial Intelligence (AI) and Recommender Systems (RS), it is still challenging to understand the intent behind customer questions during the customer journey. To address this challenge, in this paper, we study and analyze the recent work in Conversational Recommender Systems (CRS) in general and, more specifically, in chatbot-based CRS. We introduce a pipeline to contextualize the input utterances in conversations. We then take the next step towards leveraging reverse feature engineering to link the contextualized input and learning model to support intent recognition. Since performance evaluation is achieved based on different ML models, we use transformer base models to evaluate the proposed approach using a labelled dialogue dataset (MSDialogue) of question-answering interactions between information seekers and answer providers.

Knowledge tracing (KT) aims to leverage students' learning histories to estimate their mastery levels on a set of pre-defined skills, based on which the corresponding future performance can be accurately predicted. In practice, a student's learning history comprises answers to sets of massed questions, each known as a session, rather than merely being a sequence of independent answers. Theoretically, within and across these sessions, students' learning dynamics can be very different. Therefore, how to effectively model the dynamics of students' knowledge states within and across the sessions is crucial for handling the KT problem. Most existing KT models treat student's learning records as a single continuing sequence, without capturing the sessional shift of students' knowledge state. To address the above issue, we propose a novel hierarchical transformer model, named HiTSKT, comprises an interaction(-level) encoder to capture the knowledge a student acquires within a session, and a session(-level) encoder to summarise acquired knowledge across the past sessions. To predict an interaction in the current session, a knowledge retriever integrates the summarised past-session knowledge with the previous interactions' information into proper knowledge representations. These representations are then used to compute the student's current knowledge state. Additionally, to model the student's long-term forgetting behaviour across the sessions, a power-law-decay attention mechanism is designed and deployed in the session encoder, allowing it to emphasize more on the recent sessions. Extensive experiments on three public datasets demonstrate that HiTSKT achieves new state-of-the-art performance on all the datasets compared with six state-of-the-art KT models.

创新是经济和社会发展的主要驱动力，有关多种创新的信息嵌入了专利和专利申请的半结构化数据中。尽管在专利数据中表达的创新的影响和新颖性很难通过传统手段来衡量，但ML提供了一套有希望的技术来评估新颖性，汇总贡献和嵌入语义。在本文中，我们介绍了Harvard USPTO专利数据集（HUPD），该数据集是2004年至2004年之间提交给美国专利商业办公室（USPTO）的大型，结构化和多用途的英语专利专利申请。 2018年。HUPD拥有超过450万张专利文件，是可比的Coldia的两到三倍。与以前在NLP中提出的专利数据集不同，HUPD包含了专利申请的发明人提交的版本（不是授予专利的最终版本），其中允许我们在第一次使用NLP方法进行申请时研究专利性。它在包含丰富的结构化元数据以及专利申请文本的同时也很新颖：通过提供每个应用程序的元数据及其所有文本字段，数据集使研究人员能够执行一组新的NLP任务，以利用结构性协变量的变异。作为有关HUPD的研究类型的案例研究，我们向NLP社区（即专利决策的二元分类）介绍了一项新任务。我们还显示数据集中提供的结构化元数据使我们能够对此任务进行概念转移的明确研究。最后，我们演示了如何将HUPD用于三个其他任务：专利主题领域的多类分类，语言建模和摘要。

This study evaluated the ability of ChatGPT, a recently developed artificial intelligence (AI) agent, to perform high-level cognitive tasks and produce text that is indistinguishable from human-generated text. This capacity raises concerns about the potential use of ChatGPT as a tool for academic misconduct in online exams. The study found that ChatGPT is capable of exhibiting critical thinking skills and generating highly realistic text with minimal input, making it a potential threat to the integrity of online exams, particularly in tertiary education settings where such exams are becoming more prevalent. Returning to invigilated and oral exams could form part of the solution, while using advanced proctoring techniques and AI-text output detectors may be effective in addressing this issue, they are not likely to be foolproof solutions. Further research is needed to fully understand the implications of large language models like ChatGPT and to devise strategies for combating the risk of cheating using these tools. It is crucial for educators and institutions to be aware of the possibility of ChatGPT being used for cheating and to investigate measures to address it in order to maintain the fairness and validity of online exams for all students.

自从引进原始伯特（即，基础BERT）以来，研究人员通过利用转让学习的好处，开发了各种定制的伯特模型，并通过利用转移学习的好处来提高特定领域和任务的性能。由于数学文本的性质，这通常使用域特定的词汇以及方程和数学符号，我们对数学的新BERT模型的开发对于许多数学下游任务有用。在这个资源论文中，我们介绍了我们的多体制努力（即，美国的两个学习平台和三个学术机构）对此需求：Mathbert，通过在大型数学语料库上预先培训基础伯爵模型来创建的模型预先幼儿园（Pre-K），高中，大学毕业生水平数学内容。此外，我们选择了三个通常用于数学教育的一般NLP任务：知识组件预测，自动分级开放式Q＆A，以及知识追踪，以展示Mathbert对底座的优越性。我们的实验表明，Mathbert以此任务的2-8％达到了1.2-22％，碱基贝尔以前最佳方法。此外，我们建立了一个数学特定的词汇“Mathvocab”，用Mathbert训练。我们发现Mathbert预先接受过的“Mathvocab”优于Mathbert培训的底座伯特词汇（即'Origvocab'）。 Mathbert目前正在参加倾斜平台采用：Stride，Inc，商业教育资源提供商和Accortments.org，是一个免费在线教育平台。我们发布Mathbert以获取公共用途：https://github.com/tbs17/mathbert。

Multiple choice questions (MCQs) are widely used in digital learning systems, as they allow for automating the assessment process. However, due to the increased digital literacy of students and the advent of social media platforms, MCQ tests are widely shared online, and teachers are continuously challenged to create new questions, which is an expensive and time-consuming task. A particularly sensitive aspect of MCQ creation is to devise relevant distractors, i.e., wrong answers that are not easily identifiable as being wrong. This paper studies how a large existing set of manually created answers and distractors for questions over a variety of domains, subjects, and languages can be leveraged to help teachers in creating new MCQs, by the smart reuse of existing distractors. We built several data-driven models based on context-aware question and distractor representations, and compared them with static feature-based models. The proposed models are evaluated with automated metrics and in a realistic user test with teachers. Both automatic and human evaluations indicate that context-aware models consistently outperform a static feature-based approach. For our best-performing context-aware model, on average 3 distractors out of the 10 shown to teachers were rated as high-quality distractors. We create a performance benchmark, and make it public, to enable comparison between different approaches and to introduce a more standardized evaluation of the task. The benchmark contains a test of 298 educational questions covering multiple subjects & languages and a 77k multilingual pool of distractor vocabulary for future research.

培训和评估语言模型越来越多地要求构建元数据 - 多样化的策划数据收集，并具有清晰的出处。自然语言提示最近通过将现有的，有监督的数据集转换为多种新颖的预处理任务，突出了元数据策划的好处，从而改善了零击的概括。尽管将这些以数据为中心的方法转化为生物医学语言建模的通用域文本成功，但由于标记的生物医学数据集在流行的数据中心中的代表性大大不足，因此仍然具有挑战性。为了应对这一挑战，我们介绍了BigBio一个由126个以上的生物医学NLP数据集的社区库，目前涵盖12个任务类别和10多种语言。 BigBio通过对数据集及其元数据进行程序化访问来促进可再现的元数据策划，并与当前的平台兼容，以及时工程和端到端的几个/零射击语言模型评估。我们讨论了我们的任务架构协调，数据审核，贡献指南的过程，并概述了两个说明性用例：生物医学提示和大规模，多任务学习的零射门评估。 BigBio是一项持续的社区努力，可在https://github.com/bigscience-workshop/biomedical上获得。

We present a new NLP task and dataset from the domain of the U.S. civil procedure. Each instance of the dataset consists of a general introduction to the case, a particular question, and a possible solution argument, accompanied by a detailed analysis of why the argument applies in that case. Since the dataset is based on a book aimed at law students, we believe that it represents a truly complex task for benchmarking modern legal language models. Our baseline evaluation shows that fine-tuning a legal transformer provides some advantage over random baseline models, but our analysis reveals that the actual ability to infer legal arguments remains a challenging open research question.

As artificial intelligence (AI) becomes a prominent part of modern life, AI literacy is becoming important for all citizens, not just those in technology careers. Previous research in AI education materials has largely focused on the introduction of terminology as well as AI use cases and ethics, but few allow students to learn by creating their own machine learning models. Therefore, there is a need for enriching AI educational tools with more adaptable and flexible platforms for interested educators with any level of technical experience to utilize within their teaching material. As such, we propose the development of an open-source tool (Build-a-Bot) for students and teachers to not only create their own transformer-based chatbots based on their own course material, but also learn the fundamentals of AI through the model creation process. The primary concern of this paper is the creation of an interface for students to learn the principles of artificial intelligence by using a natural language pipeline to train a customized model to answer questions based on their own school curriculums. The model uses contexts given by their instructor, such as chapters of a textbook, to answer questions and is deployed on an interactive chatbot/voice agent. The pipeline teaches students data collection, data augmentation, intent recognition, and question answering by having them work through each of these processes while creating their AI agent, diverging from previous chatbot work where students and teachers use the bots as black-boxes with no abilities for customization or the bots lack AI capabilities, with the majority of dialogue scripts being rule-based. In addition, our tool is designed to make each step of this pipeline intuitive for students at a middle-school level. Further work primarily lies in providing our tool to schools and seeking student and teacher evaluations.

自然语言处理（NLP）是一个人工智能领域，它应用信息技术来处理人类语言，在一定程度上理解并在各种应用中使用它。在过去的几年中，该领域已经迅速发展，现在采用了深层神经网络的现代变体来从大型文本语料库中提取相关模式。这项工作的主要目的是调查NLP在药理学领域的最新使用。正如我们的工作所表明的那样，NLP是药理学高度相关的信息提取和处理方法。它已被广泛使用，从智能搜索到成千上万的医疗文件到在社交媒体中找到对抗性药物相互作用的痕迹。我们将覆盖范围分为五个类别，以调查现代NLP方法论，常见的任务，相关的文本数据，知识库和有用的编程库。我们将这五个类别分为适当的子类别，描述其主要属性和想法，并以表格形式进行总结。最终的调查介绍了该领域的全面概述，对从业者和感兴趣的观察者有用。

自动简短答案分级是探索如何使用人工智能（AI）的工具来改善教育的重要研究方向。当前的最新方法使用神经语言模型来创建学生响应的矢量表示，然后是分类器以预测分数。但是，这些方法有几个关键的局限性，包括i）他们使用的预培训的语言模型不适合教育主题领域和/或学生生成的文本和ii）它们几乎总是每个问题训练一个模型，而忽略了该模型由于高级语言模型的大小，跨越问题的联系并导致了重要的模型存储问题。在本文中，我们研究了学生对数学问题的回答的自动简短答案分级问题，并为这项任务提出了一个新颖的框架。首先，我们使用Mathbert，这是流行语言模型BERT的一种变体，该模型适合数学内容，并将其微调为学生响应分级的下游任务。其次，我们使用一种文字学习方法，提供评分示例作为语言模型的输入，以提供其他上下文信息并促进对以前看不见的问题的概括。我们在研究学生对开放式数学问题的回答的现实数据集上评估了我们的框架，并表明我们的框架（通常非常明显）优于现有方法，尤其是对于培训期间没有看到的新问题。

如今，由于最近在人工智能（AI）和机器学习（ML）中的近期突破，因此，智能系统和服务越来越受欢迎。然而，机器学习不仅满足软件工程，不仅具有有希望的潜力，而且还具有一些固有的挑战。尽管最近的一些研究努力，但我们仍然没有明确了解开发基于ML的申请和当前行业实践的挑战。此外，目前尚不清楚软件工程研究人员应将其努力集中起来，以更好地支持ML应用程序开发人员。在本文中，我们报告了一个旨在了解ML应用程序开发的挑战和最佳实践的调查。我们合成从80名从业者（以不同的技能，经验和应用领域）获得的结果为17个调查结果;概述ML应用程序开发的挑战和最佳实践。参与基于ML的软件系统发展的从业者可以利用总结最佳实践来提高其系统的质量。我们希望报告的挑战将通知研究界有关需要调查的主题，以改善工程过程和基于ML的申请的质量。

作为网络防御的重要工具，欺骗正在迅速发展，并补充了现有的周边安全措施，以迅速检测出漏洞和数据盗窃。限制欺骗使用的因素之一是手工生成逼真的人工制品的成本。但是，机器学习的最新进展为可扩展的，自动化的现实欺骗创造了机会。本愿景论文描述了开发模型所涉及的机会和挑战，以模仿IT堆栈的许多共同元素以造成欺骗效应。

问答系统被认为是流行且经常有效的信息在网络上寻求信息的手段。在这样的系统中，寻求信息者可以通过自然语言提出问题来获得对他们的查询的简短回应。交互式问题回答是一种最近提出且日益流行的解决方案，它位于问答和对话系统的交集。一方面，用户可以以普通语言提出问题，并找到对她的询问的实际回答；另一方面，如果在初始请求中有多个可能的答复，很少或歧义，则系统可以将问题交通会话延长到对话中。通过允许用户提出更多问题，交互式问题回答使用户能够与系统动态互动并获得更精确的结果。这项调查提供了有关当前文献中普遍存在的交互式提问方法的详细概述。它首先要解释提问系统的基本原理，从而定义新的符号和分类法，以将所有已确定的作品结合在统一框架内。然后，根据提出的方法，评估方法和数据集/应用程序域来介绍和检查有关交互式问题解答系统的审查已发表的工作。我们还描述了围绕社区提出的特定任务和问题的趋势，从而阐明了学者的未来利益。 GitHub页面的综合综合了本文献研究中涵盖的所有主要主题，我们的工作得到了进一步的支持。 https://sisinflab.github.io/interactive-question-answering-systems-survey/

本次调查绘制了用于分析社交媒体数据的生成方法的研究状态的广泛的全景照片（Sota）。它填补了空白，因为现有的调查文章在其范围内或被约会。我们包括两个重要方面，目前正在挖掘和建模社交媒体的重要性：动态和网络。社会动态对于了解影响影响或疾病的传播，友谊的形成，友谊的形成等，另一方面，可以捕获各种复杂关系，提供额外的洞察力和识别否则将不会被注意的重要模式。