Character-level Intra Attention Network

Implementation of the paper Character-level Intra Attention Network (CIAN) in proceddings of the RepEval Workshop in The 2017 Conference on Empirical Methods on Natural Language Processing.

Architecture of the model:

Requirements

Code is written in python 2.7 and requires Keras 2.

Data

Dataset could be downloaded at MultiNLI and SNLI.

How to run

First to do a modification with Keras, see the following section.

Dataset should be be put in folder ./data

To run the model, use

python ./model.py

The result and log file will be saved in ./log folder.

Modification with keras

In #Python_Path\Lib\site-packages\keras\preprocessing\text.py, line 39,

CHANGE

    text = text.translate(maketrans(filters, split * len(filters)))

TO

    try:
        text = unicode(text, "utf-8")
    except TypeError:
        pass
    translate_table = {ord(c): ord(t) for c, t in zip(filters, split * len(filters))}
    text = text.translate(translate_table)

Result

Visualization of Attention

PairID 192997e, label Entailment

PairID 254941e, label Entailment

Reference

[1] Character-Aware Neural Language Models

[2] Intra Attention Mechanism