Skip to main content

Posts

Showing posts from November 15, 2024

LLM KRL English to Tamil Version 1 - Bert model used

  In Kaggle Notebook Published Here   import pandas as pd import torch from sklearn.model_selection import train_test_split, KFold from sklearn.metrics import accuracy_score, f1_score, confusion_matrix, mean_squared_error, roc_auc_score, precision_recall_curve, auc, roc_curve from transformers import AutoTokenizer, AutoModelForSequenceClassification, Trainer, TrainingArguments, EarlyStoppingCallback import numpy as np import matplotlib.pyplot as plt import random # Expand dataset with 100 samples by duplicating and adding slight variations sample_data = {     "text_english": [         "I am very happy to meet you", "I am disappointed with your work",         "You have done an excellent job, well done", "This is not good, I expected better",         "Thank you very much for your support", "I don't like your attitude",         "I'm grateful for your guidance", "Your work lacks quality", ...