Testing the performance of CNN and pretrained BERT embeddings on the GLUE Tasks
The BERT model used is the BERT-Base, Uncased: 12-layer, 768-hidden, 12-heads, 110M parameters
and BERT-Large, Uncased: 24-layer, 1024-hidden, 16-heads, 340M parameters
Bert Base available at https://storage.googleapis.com/bert_models/2018_10_18/uncased_L-12_H-768_A-12.zip
Bert Large available at https://storage.googleapis.com/bert_models/2018_10_18/uncased_L-24_H-1024_A-16.zip
the tokenization.py file is from the google-research/bert repo https://github.com/google-research/bert
execute python ./utils/download_glue_data.py --data_dir ./data/glue_data to download the tasks.
execute python ./utils/preprocess_tasks.py --data_dir ./data/glue_data to process the tsv files so pandas can read them.
the keras template is based on https://github.com/Ahmkel/Keras-Project-Template
The keras implementation of BERT is based on https://github.com/CyberZHG/keras-bert