[Corpora-List] BERT + NN finetune

Viktor Schlegel viktor.schlegel at manchester.ac.uk
Sat Aug 29 11:08:12 CEST 2020


Hello,

I believe this should get you started for training BERT-related models on task of sentence pair classification:

https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/trainer/01_text_classification.ipynb (very high-level) [https://colab.research.google.com/img/colab_favicon_256px.png]<https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/trainer/01_text_classification.ipynb> Show notebooks in Drive<https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/trainer/01_text_classification.ipynb> This site may not work in your browser. Please use a supported browser. More info colab.research.google.com https://github.com/huggingface/transformers/blob/master/examples/text-classification/run_xnli.py (bit more detailed) [https://repository-images.githubusercontent.com/155220641/a16c4880-a501-11ea-9e8f-646cf611702e]<https://github.com/huggingface/transformers/blob/master/examples/text-classification/run_xnli.py> huggingface/transformers<https://github.com/huggingface/transformers/blob/master/examples/text-classification/run_xnli.py> 🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0. - huggingface/transformers github.com here more implementations of other (non-BERT based) models for sentence pair classification:

https://github.com/coetaur0/ESIM/blob/master/esim/model.py [https://avatars0.githubusercontent.com/u/17785983?s=400&v=4]<https://github.com/coetaur0/ESIM/blob/master/esim/model.py> coetaur0/ESIM<https://github.com/coetaur0/ESIM/blob/master/esim/model.py> Implementation of the ESIM model for natural language inference with PyTorch - coetaur0/ESIM github.com https://github.com/allenai/allennlp-models/blob/master/allennlp_models/pair_classification/models/decomposable_attention.py https://github.com/allenai/allennlp-models/blob/master/allennlp_models/pair_classification/models/bimpm.py https://github.com/allenai/allennlp-models/blob/master/allennlp_models/pair_classification/models/esim.py

[https://avatars1.githubusercontent.com/u/5667695?s=400&v=4]<https://github.com/allenai/allennlp-models/blob/master/allennlp_models/pair_classification/models/decomposable_attention.py> allenai/allennlp-models<https://github.com/allenai/allennlp-models/blob/master/allennlp_models/pair_classification/models/decomposable_attention.py> Officially supported AllenNLP models. Contribute to allenai/allennlp-models development by creating an account on GitHub. github.com the last three you can use in the Allennlp framework (https://demo.allennlp.org/textual-entailment and then navigate to usage) [https://demo.allennlp.org/allennlpdemo-social-card.png]<https://demo.allennlp.org/textual-entailment> AllenNLP - Demo<https://demo.allennlp.org/textual-entailment> A collection of interactive demos of over 20 popular NLP models. demo.allennlp.org hope that helps,

Viktor ________________________________ From: corpora-bounces at uib.no <corpora-bounces at uib.no> on behalf of s.z. aftabi <s.z.aftabi at gmail.com> Sent: Saturday, August 29, 2020 9:49 AM To: corpora at uib.no <corpora at uib.no> Subject: [Corpora-List] BERT + NN finetune

Dear all,

I'm already working on a two sentence classification task. I aim at using BERT as an embedding layer with a neural network on top (NN with more than one layer) then train the NN model and also finetune BERT, both with respect to the loss of classification. I appreciate your helping me find good reference or implementation about how to do that?

Best regards, S.Zahra -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/html Size: 17021 bytes Desc: not available URL: <https://mailman.uib.no/public/corpora/attachments/20200829/84d0a92c/attachment.txt>



More information about the Corpora mailing list