可解释人工智能（XAI）：自然语言处理（NLP）版

Original 李春郁国际翻译动态

2024-09-09

Explainable AI (XAI): NLP Edition

可解释人工智能（XAI）：自然语言处理（NLP）版

（图片来自taus官网）

Explaining what Explainable AI (XAI) entails and diving into five major XAI techniques for Natural Language Processing (NLP).

说明何为可解释人工智能（Explainable AI，缩写XAI），并深入研究自然语言处理（NLP）的五种主要可解释人工智能技术。

Background

What is Explainable AI?

什么是可解释人工智能？

As AI is becoming more prominent in high-stakes industries like healthcare, education, construction, environment, autonomous machines, and law enforcement, we are finding an increased need to trust the decision-making process. These predictions often need to be extremely accurate, e.g. critical life or death situations in healthcare. Due to the critical and direct impact AI is having on our day-to-day lives, decision-makers need more insight and visibility into the mechanics of AI systems and the prediction process. Presently, often only technical experts such as data scientists or engineers understand the backend processes and algorithms being used, like the highly complex deep neural networks. The lack of interpretability has shown to be a means of disconnect between technical and non-technical practitioners. In an effort to make these AI systems more transparent, the field of Explainable AI (XAI) came into existence.

随着人工智能在医疗保健、教育、建筑、环境、自动机器和执法等高风险行业中发挥越来越突出的作用，我们发现愈发需要信任决策过程。这些预测通常需要非常准确，例如医疗保健中生死攸关的时刻。由于人工智能对我们的日常生活有着关键且直接的影响，决策者需要对人工智能系统的机制和预测过程具备更深刻的洞察力和透明度。目前，通常只有数据科学家或工程师等技术专家了解所使用的后端流程和算法，例如高度复杂的深度神经网络。缺乏可解释性已成为技术从业人员和非技术从业人员之间的脱节之处。为了使这些人工智能系统更加透明，可解释人工智能（XAI）领域应运而生。

Explainable AI(XAI) is an emerging subset in AI that focuses on the readability of machine learning (ML) models. These tools help you understand and interpret your predictions, reducing the complexity and allowing for non-technically trained practitioners and stakeholders to be more aware of the modeling process. At its core, XAI aims to deconstruct black box decision-making processes in AI. XAI can answer questions like “Why was this prediction made?” or “How much confidence do I have in this prediction?” or “Why did this system fail?”

可解释人工智能（XAI）是人工智能的一个新兴领域，专注于机器学习（ML）模型的可读性。这些工具可帮助您理解及解释您的预测，降低复杂性，并使未经技术培训的从业人员和利益相关者能够更好地了解建模过程。可解释人工智能的核心目标是解构人工智能中的黑匣子决策过程。它可以回答诸如“为什么做出这个预测？”“我对这个预测有多大的信心？” 或“为什么这个系统失败了？”之类的问题。

NLP and XAI

自然语言处理与可解释人工智能

Natural Language Processing(NLP) is a subset of AI (artificial intelligence) and ML (machine learning) which aims to make sense of human language. NLP performs tasks such as topic classification, translation, sentiment analysis, and predictive spelling using text data. NLP has historically been based on interpretable models, referred to as white-box techniques. These techniques include easily interpretable models like decision trees, rule-based modeling, Markov models, logistic regression, and more. As of recent years, however, the level of interpretability has been reduced to black-box techniques, such as deep learning approaches and the use of language embedding features. With reduced interpretability comes reduced trust, especially with human-computer interactions (HCI) such as chatboxes, for example.

自然语言处理（NLP）是AI（人工智能）和ML（机器学习）的子集，旨在理解人类语言。NLP使用文本数据执行主题分类、翻译、情绪分析和预测拼写等任务。NLP历来建立在可解释模型的基础之上，即白盒技术（白盒技术是指对软件、硬件或者系统内部结构进行详细剖析和分析的一种技术手段）。这些技术包括易于解释的模型，如决策树、基于规则的建模、马尔可夫模型、逻辑回归等。然而，近年来，可解释水平已降至黑盒技术状态，例如深度学习方法及语言嵌入功能的运用。随着可解释性降低，信任度也会降低，尤其是在聊天框等人机交互（HCI）方面。

A Survey by IBM - XAI for NLP

IBM进行的调查——

针对自然语言处理（NLP）的

可解释人工智能

A group of researchers at IBM conducted a survey called A Survey of the State of Explainable AI for Natural Language Processing. As one of the few works on the intersection of NLP and XAI, the survey aims to provide an understanding of the current state of XAI and NLP, explain currently available techniques, and bring the research community’s attention to the presently existing gaps. The categories of explanations used consist of whether the explanation is for a single prediction (local) or the model’s prediction process as a whole (global). The main difference between these two categories is that the first explanation is outputted during the prediction process (self-explaining) whereas the second requires post-processing following the model’s prediction process (post-hoc). The authors further introduce additional explainability aspects, including techniques for serving the explanation and presentation type to the end-user.

IBM的一组研究人员进行了一项调查，名为《针对自然语言处理的可解释人工智能状态调查》。作为对NLP和XAI交叉领域的少数研究之一，该调查旨在提供对XAI和NLP现状的分析认知，解释当前可用的技术，并使研究界关注目前存在的差距。所使用的解释类别包括是针对单个预测（局部）还是整个模型的预测过程（全局）。这两个类别之间的主要区别在于，第一种解释在预测过程中即可输出（自我解释），而第二种解释需要在模型的预测过程（事后）之后进行后处理。作者进一步介绍了可解释性的其他方面，包括向最终用户提供解释和演示类型的技术。

5 Major Explainability Techniques

5大可解释性技巧

The researchers in this study presented five major explainability techniques in NLP that characterize raw technical components to present a final explanation to the end-user. These are listed below:

Feature importance: uses the score of the importance of a given feature, which is ultimately used to output the final prediction. Text-based features are more intuitive for humans to interpret, enabling feature importance-based explanations.

Surrogate model: a second model, or proxy model, used to explain model predictions. These proxy models can achieve both local or global explanations. One drawback of this method is that the learned surrogate models and the original model could use different methodologies when making predictions.

Example-driven: uses examples, usually from labeled data, that are semantically similar to the input example to explain the given prediction. An example of an example-driven explanation is using the nearest-neighbor approach, which has been used in areas such as text classification and question answering.

Provenance-based: utilizes illustrations of some or all of the prediction derivation process. This is often an effective explainability technique when the prediction is the outcome of a series of decision rules or reasoning steps.

Declarative induction: involves human-readable transformations, such as rules and programs to deliver explainability.

本研究的研究人员介绍了NLP中的五种主要可解释性技术，这些技术表征了原始技术组件，以便向终端用户提供最终解释。如下所示。

特征重要性：使用给定特征的重要性分数，最后用于输出最终预测。基于文本的特征能更直观地供人类解释，从而实现基于特征重要性的解释。

代理模型：用于解释模型预测的第二个模型或代理模型。这些代理模型可以实现本地或全局解释。这种方法的缺点之一是学习的替代模型和原始模型在进行预测时可能会使用不同的方法。

示例驱动：使用示例（通常来自标记数据），这些示例在语义上与输入示例相似，可对给定的预测进行解释。示例驱动解释的一个例子是使用最近邻方法，该方法已用于文本分类和问答等领域。

基于来源：利用部分或全部预测推导过程的例证。当预测是一系列决策规则或推理步骤的结果时，这通常是一种有效的可解释技术。

陈述式归纳：涉及人类可读的转换，例如提供可解释性的规则和程序。

Visualization Techniques

可视化技术

XAI may be presented to users in different ways, depending on the model complexity and explainability technique used. The visualization ultimately used highly impacts the success of an XAI approach in NLP. Let’s look at the commonly used attention mechanism in NLP, which learns weights (importance scores) of a given set of features. Attention mechanisms are often visualized as raw scores or as a saliency heatmap. An example of a saliency heatmap visualization of an attention score can be seen in Figure 1.

可视化技术

XAI可能会以不同的方式呈现给用户，具体取决于所使用的模型复杂性和可解释性技术。最终使用的可视化极大地影响XAI方法能否在NLP中取得成功。让我们来看NLP中常用的注意力机制，它会对给定特征集的权重（重要性分数）进行学习。注意力机制通常可视化为原始分数或显著性热图。注意力分数的显著性热图可视化示例如图1所示。

Figure 1- Weights assigned at each step of translation

图1-翻译各步骤中分配的权重

Salience-based visualizations focus on making more important attributes or factors more visible to the end-user. Saliency is often used in XAI to depict the importance scores for different elements in an AI system. Examples of saliency-based visualizations include highlighting important words in a text and heatmaps.

Other visualization techniques of XAI for NLP include raw declarative representations and natural language explanations. Raw declarative representations assume that end-users are more advanced and can understand learned declarative representations, such as logic rules, trees, and programs. Natural language explanation is any human-comprehensible natural language, generated from sophisticated deep learning models. For example, these can be generated using a simple template-based approach or a more complex deep generative model. At its core, it turns rules and programs into human-readable language.

基于显著性的可视化侧重于使最终用户更容易看到更重要的属性或因素。显著性通常在XAI中用于描述AI系统中不同元素的重要性分数。基于显著性的可视化示例包括突出显示文本与热图中的重点单词。

XAI用于NLP的其他可视化技术包括原始陈述式表述和自然语言解释。原始陈述式表述会假设最终用户更高级，还能理解习得陈述式表示，例如逻辑规则、树和程序。自然语言解释是由复杂的深度学习模型生成的人类可理解的任何自然语言。例如，这些可以使用简单的基于模板的方法或更复杂的深度生成模型来生成。从本质上讲，它将规则和程序转化为人类可读的语言。

Conclusion

结论

The survey presented displays the connection between XAI and NLP, specifically how XAI can be applied to an NLP-based system. The field of XAI is meant to add explainability as a much-desired feature to ML models, adding to the model’s overall prediction quality and interpretability. Explainability can be categorized into different sectors of the NLP model, as well as being depicted by different visualization techniques seen above. Because of the large-scale presence of NLP around us, including chat boxes, predictive typing, auto-correct, and machine translation, it is important for any end-user, especially in NLP-based organizations, to understand the behind-the-scenes grunt work of the model. XAI allows for the end-user to gain trust in the NLP application being used and therefore allowing for a positive feedback loop, to ultimately make the algorithm even better. As XAI is still a growing field, there is plenty of room for innovation on improving the explainability of NLP systems.

该调查展示了XAI和NLP之间的联系，特别是如何将XAI应用于基于NLP的系统。XAI领域的目的是将可解释性作为亟需的一种特征添加到ML模型中，从而增加模型的整体预测质量和可解释性。可解释性可以分为NLP模型的不同部分，也可以通过上述不同的可视化技术进行描述。由于我们周围有大量的NLP，包括聊天框、预测输入（predictive typing）、自动更正和机器翻译，因此任何最终用户都必须了解该模型繁重的幕后工作，尤其是基于NLP的组织。XAI使得最终用户能信任所使用的NLP应用程序，从而允许正反馈循环，最终使算法变得更好。由于XAI仍然是一个不断发展的领域，因此在提高NLP系统的可解释性方面还有很大的创新空间。

原文网址：

https://www.taus.net/resources/blog/explainable-ai-xai-nlp-edition

utm_content=287277342&utm_medium=

social&utm_source=linkedin&hss_channel=lcp-894145