BENDeep(Deep Learning For Bengali NLP)

BENDeep is a pytorch based deep learning solution for Bengali NLP Task like bengali translation, bengali sentiment analysis and so on.

https://github.com/sagorbrur/bendeep

Installation

Dependency

Pretrained Model

API

Sentiment Analysis

Analyzing Sentiment

from bendeep import sentiment
model_path = "senti_trained.pt"
vocab_path = "vocab.txt"
text = "রোহিঙ্গা মুসলমানদের দুর্ভোগের অন্ত নেই।জলে কুমির ডাংগায় বাঘ।আজকে দুটি ঘটনা আমাকে ভীষণ ব্যতিত করেছে।নিরবে কিছুক্ষন অশ্রু বিসর্জন দিয়ে মনটাকে হাল্কা করার ব্যর্থ প্রয়াস চালিয়েছি।"
sentiment.analyze(model_path, vocab_path, text)

Training Sentiment Model

Example:

,review,sentiment
0,তোমাকে খুব সুন্দর লাগছে।,1
1,আজকের আবহাওয়া খুব খারাপ।,0

reviewsentiment0তোমাকে খুব সুন্দর লাগছে।11আজকের আবহাওয়া খুব খারাপ।0

from bendeep import sentiment
data_path = "sentiment_data.csv"
sentiment.train(data_path)
# you can also pass these parameter
# sentiment.train(data_path, batch_size = 64, epochs=100, model_name="trained.pt")

after successfully training it will complete training and save model as trained.pt also save vocab file as vocab.txt

Machine Translation

Translate Bengali to English

from bendeep import translation
from bendeep.translation import EncoderRNN
from bendeep.translation import AttnDecoderRNN
data_path = "data/translation/eng-ben.txt"
encoder = "models/translation/encoder.pt"
decoder = "models/translation/decoder.pt"
input_sentence = "আমার শীত করছে।"
translation.bn2en(data_path, encoder, decoder, input_sentence)
# outupt
# > আমার শীত করছে ।
# = i feel cold .

Training Translation Model

Example:

I eat rice. আমি ভাত খাই।
He goes to school. সে বিদ্যালয়ে যায়।
from bendeep import translation
from bendeep.translation import EncoderRNN
from bendeep.translation import AttnDecoderRNN
data_path = "data/translation/eng-ben.txt"
translation.training(data_path, iteration=75000)

after successfully training it will complete training and save encoder and decoder model as encoder.pt, decoder.pt. Also display some random evaluation results.