Skip to main content

Posts

Showing posts with the label llm_krl

GenAI LLM for Faster Translation Product Feature - Architecute and Framework

  As part of LLM KRL ( LLM Know Respect Language ) model this is the product feature choices fastest way to translate and comprehend !! Enabling Human Communication Multi Cultural and Multi Language support, still native people feel free to talk/ consult and communicate with respectful and fastest word exchanges !! Designing an LLM (Large Language Model) and GenAI (Generative AI) architecture for faster translations involves a combination of strategies to optimize model selection, training, inference, and deployment. Below is a step-by-step architecture design that includes the latest techniques for scalable and efficient translation. 1. High-Level Architecture Key Components: Data Preprocessing : Text cleaning, tokenization, and language alignment. Model Backbone : Efficient transformer-based models (e.g., MarianMT, BLOOM, or distilled versions of GPT). Training Optimization : Mixed precision, parameter-efficient fine-tuning (PEFT), and low-rank adaptation (LoRA). Inference ...

LLM KRL English to Tamil Version 1 - Bert model used

  In Kaggle Notebook Published Here   import pandas as pd import torch from sklearn.model_selection import train_test_split, KFold from sklearn.metrics import accuracy_score, f1_score, confusion_matrix, mean_squared_error, roc_auc_score, precision_recall_curve, auc, roc_curve from transformers import AutoTokenizer, AutoModelForSequenceClassification, Trainer, TrainingArguments, EarlyStoppingCallback import numpy as np import matplotlib.pyplot as plt import random # Expand dataset with 100 samples by duplicating and adding slight variations sample_data = {     "text_english": [         "I am very happy to meet you", "I am disappointed with your work",         "You have done an excellent job, well done", "This is not good, I expected better",         "Thank you very much for your support", "I don't like your attitude",         "I'm grateful for your guidance", "Your work lacks quality", ...

LLM KRL Model Sample usecase Google Gemma English to Tamil translation with LLM Respect Language models LLM_KRL models

​​To develop a multi-level contextual classification model for English-to-Tamil sentiment analysis, incorporating Google's Gemma models can enhance performance, especially for Tamil language processing.​​ Here's a structured approach: 1. Data Preparation: Dataset Creation: ​​Compile a dataset containing English sentences, their Tamil translations, sentiment labels (positive/negative), and, for positive sentiments, an additional label indicating respect.​​ Example Data Structure: ​​| English Text                              | Tamil Translation                              | Sentiment | Respect (if Positive) | |-------------------------------------------|------------------------------------------------|-----------|-----------------------| | I am very happy to meet you               | உங்கள...