Posts

Showing posts from November, 2024

LLM KRL English to Tamil Version 1 - Bert model used

  In Kaggle Notebook Published Here   import pandas as pd import torch from sklearn.model_selection import train_test_split, KFold from sklearn.metrics import accuracy_score, f1_score, confusion_matrix, mean_squared_error, roc_auc_score, precision_recall_curve, auc, roc_curve from transformers import AutoTokenizer, AutoModelForSequenceClassification, Trainer, TrainingArguments, EarlyStoppingCallback import numpy as np import matplotlib.pyplot as plt import random # Expand dataset with 100 samples by duplicating and adding slight variations sample_data = {     "text_english": [         "I am very happy to meet you", "I am disappointed with your work",         "You have done an excellent job, well done", "This is not good, I expected better",         "Thank you very much for your support", "I don't like your attitude",         "I'm grateful for your guidance", "Your work lacks quality",    

LLM KRL English to Tamil Version 1 - Bert model used

  In Kaggle Notebook Published Here   import pandas as pd import torch from sklearn.model_selection import train_test_split, KFold from sklearn.metrics import accuracy_score, f1_score, confusion_matrix, mean_squared_error, roc_auc_score, precision_recall_curve, auc, roc_curve from transformers import AutoTokenizer, AutoModelForSequenceClassification, Trainer, TrainingArguments, EarlyStoppingCallback import numpy as np import matplotlib.pyplot as plt import random # Expand dataset with 100 samples by duplicating and adding slight variations sample_data = {     "text_english": [         "I am very happy to meet you", "I am disappointed with your work",         "You have done an excellent job, well done", "This is not good, I expected better",         "Thank you very much for your support", "I don't like your attitude",         "I'm grateful for your guidance", "Your work lacks quality",    

LLM KRL Model Sample usecase Google Gemma English to Tamil translation with LLM Respect Language models LLM_KRL models

Image
​​To develop a multi-level contextual classification model for English-to-Tamil sentiment analysis, incorporating Google's Gemma models can enhance performance, especially for Tamil language processing.​​ Here's a structured approach: 1. Data Preparation: Dataset Creation: ​​Compile a dataset containing English sentences, their Tamil translations, sentiment labels (positive/negative), and, for positive sentiments, an additional label indicating respect.​​ Example Data Structure: ​​| English Text                              | Tamil Translation                              | Sentiment | Respect (if Positive) | |-------------------------------------------|------------------------------------------------|-----------|-----------------------| | I am very happy to meet you               | உங்களை சந்திப்பதில் மிகவும் மகிழ்ச்சி           | Positive  | Respect               | | I am disappointed with your work          | உங்கள் வேலைக்கு நான் வருத்தப்படுகிறேன்          | Nega