What is the best part of becoming an engineer? You can build things. Just like a superpower. One rainy afternoon, I had a random idea of creating an emotional text input with a smiley face, which changed the way the text was expressed. The more positive the words are, the happier the smile is. Here are some interesting concepts to learn here, so let me guide you through how the project works!
Prerequisites
To follow, you need the following package:
- CustomTkinter
- Opencv-Python
- torch
- transformer
use Ultraviolet raysyou can add dependencies using the following command:
uv add customtkinter opencv-Python torch transformersnotes: When using Ultraviolet rays With Torch, you need to specify the index of the package. For example, if you want to use CUDA, you need the following
pyproject.toml:[[tool.uv.index]] name = "pytorch-cu118" url = " explicit = true [tool.uv.sources] torch = [{ index = "pytorch-cu118" }] torchvision = [{ index = "pytorch-cu118" }]
UI layout skeleton
For these types of projects, I always like to start with a quick layout of UI components. In this case, the layout will be very simple, with a text box with a line at the top, filling the width and filling the rest of the space under it. This will be our smiley face 🙂
use customtkinterwe can write the layout as follows:
import customtkinter
class App(customtkinter.CTk):
def __init__(self) -> None:
super().__init__()
self.title("Sentiment Analysis")
self.geometry("800x600")
self.grid_columnconfigure(0, weight=1)
self.grid_rowconfigure(0, weight=0)
self.grid_rowconfigure(1, weight=1)
self.sentiment_text_var = customtkinter.StringVar(master=self, value="Love")
self.textbox = customtkinter.CTkEntry(
master=self,
corner_radius=10,
font=("Consolas", 50),
justify="center",
placeholder_text="Enter text here...",
placeholder_text_color="gray",
textvariable=self.sentiment_text_var,
)
self.textbox.grid(row=0, column=0, padx=20, pady=20, sticky="nsew")
self.textbox.focus()
self.image_display = CTkImageDisplay(self)
self.image_display.grid(row=1, column=0, padx=20, pady=20, sticky="nsew")Unfortunately, an out-of-the-box solution for drawing OpenCV frames on UI elements, so I built it myself CTkImageDisplay. If you want to know more about how it works, check out my previous posts. In short, I use CTKLabel Component and threads that will update the image from the GUI thread using a synchronous queue.
Program Smiley
For our smiley faces, we can use different discrete images for emotional ranges, for example, save three images Negative,,,,, Neutral and Positive. However, to visualize more fine-grained emotions, we need more images, and it quickly becomes unfeasible and we won’t be able to transition between these images.
A better approach is to generate an image of a smiling face on the program at runtime. For simplicity, we will only change the background color of the smiley face and the curve of its mouth.
First, we need to generate a canvas image on which we can draw a smiley face.
def create_sentiment_image(positivity: float, image_size: tuple[int, int]) -> np.ndarray:
"""
Generates a sentiment image based on the positivity score.
This draws a smiley with its expression based on the positivity score.
Args:
positivity: A float representing the positivity score in the range [-1, 1].
image_size: A tuple representing the size of the image (width, height).
Returns:
A string representing the path to the generated sentiment image.
"""
width, height = image_size
frame = np.zeros((height, width, 4), dtype=np.uint8)
# TODO: draw smiley
return frameOur image should be transparent outside the smiley face, so we need 4 color channels and the last one is the Alpha channel. Since the OpenCV image is represented as numpy Array and Unsigned 8-bit integerwe use np.uint8 Data type. Remember, the array is stored Y-first,so height of image_size First pass to the array to create
We can define some variables for the size and color of the smile that will help when painting.
color_outline = (80,) * 3 + (255,) # gray
thickness_outline = min(image_size) // 30
center = (width // 2, height // 2)
radius = min(image_size) // 2 - thickness_outlineThe background color of the smiling face should be the red of negative emotions, and the positive emotions should be green. To achieve this with uniform brightness throughout the transition, we can use the HSV color space and simply insert the tones between 0% and 30%.
color_bgr = color_hsv_to_bgr(
hue=(positivity + 1) / 6, # positivity [-1,1] -> hue [0,1/3]
saturation=0.5,
value=1,
)
color_bgra = color_bgr + (255,)We need to make sure that the color is completely opaque by adding 100% alpha value to the fourth channel. Now, we can draw a circle of smiling faces with the boundaries.
cv2.circle(frame, center, radius, color_bgra, -1) # Fill
cv2.circle(frame, center, radius, color_outline, thickness_outline) # Border
So far, everything is fine and now we can add eyes. We calculate the offset from the center to the left and right to place the two eyes symmetrically.
# calculate the position of the eyes
eye_radius = radius // 5
eye_offset_x = radius // 3
eye_offset_y = radius // 4
eye_left = (center[0] - eye_offset_x, center[1] - eye_offset_y)
eye_right = (center[0] + eye_offset_x, center[1] - eye_offset_y)
cv2.circle(frame, eye_left, eye_radius, color_outline, -1)
cv2.circle(frame, eye_right, eye_radius, color_outline, -1)
Now reach the challenging part, mouth. The shape of the mouth will be a properly scaled parabola. We can simply multiply by the standard parabola y=x² Have positive scores.
Finally, the cv2.polylinesxy coordinate pair is required. use np.linspace We generate 100 points on the X-axis polyval Function to calculate the y value of the polygon.
# mouth parameters
mouth_wdith = radius // 2
mouth_height = radius // 3
mouth_offset_y = radius // 3
mouth_center_y = center[1] + mouth_offset_y + positivity * mouth_height // 2
mouth_left = (center[0] - mouth_wdith, center[1] + mouth_offset_y)
mouth_right = (center[0] + mouth_wdith, center[1] + mouth_offset_y)
# calculate points of polynomial for the mouth
ply_points_t = np.linspace(-1, 1, 100)
ply_points_y = np.polyval([positivity, 0, 0], ply_points_t) # y=positivity*x²
ply_points = np.array(
[
(
mouth_left[0] + i * (mouth_right[0] - mouth_left[0]) / 100,
mouth_center_y - ply_points_y[i] * mouth_height,
)
for i in range(len(ply_points_y))
],
dtype=np.int32,
)
# draw the mouth
cv2.polylines(
frame,
[ply_points],
isClosed=False,
color=color_outline,
thickness=int(thickness_outline * 1.5),
)etvoilà, we have a program-based smile!
To test this function, I use pytest This can save smiles with different emotional scores:
from pathlib import Path
import cv2
import numpy as np
import pytest
from sentiment_analysis.utils import create_sentiment_image
IMAGE_SIZE = (512, 512)
@pytest.mark.parametrize(
"positivity",
np.linspace(-1, 1, 5),
)
def test_sentiments(visual_output_path: Path, positivity: float) -> None:
"""
Test the smiley face generation.
"""
image = create_sentiment_image(positivity, IMAGE_SIZE)
assert image.shape == (IMAGE_SIZE[1], IMAGE_SIZE[0], 4)
# assert center pixel is opaque
assert image[IMAGE_SIZE[1] // 2, IMAGE_SIZE[0] // 2, 3] == 255
# save the image for visual inspection
positivity_num_0_100 = int((positivity + 1) * 50)
image_fn = f"smiley_{positivity_num_0_100}.png"
cv2.imwrite(str(visual_output_path / image_fn), image)
Sentiment Analysis
To determine the happiness or sadness of our smiling faces, we first need to analyze the text input and calculate a mood. This task is called Sentiment Analysis. We will use a pre-trained transformer model to predict the classification score of a class Negative,,,,, Neutral and Positive. We can then fuse the confidence scores of these categories to calculate the final emotional score between -1 and +1.
Using pipelines from the Transformers library, we can define processing pipelines based on HuggingFace’s pre-trained model. use top_k Parameters, we can specify how many classification results should be returned. Since we want all three classes, we set it to 3.
from transformers import pipeline
model_name = "cardiffnlp/twitter-roberta-base-sentiment"
sentiment_pipeline = pipeline(
task="sentiment-analysis",
model=model_name,
top_k=3,
)To run sentiment analysis, we can call the pipeline using string parameters. This will return a list of results with a single element, so we need to untie the first element.
results = self.sentiment_pipeline(text)
# [
# [
# {"label": "LABEL_2", "score": 0.5925878286361694},
# {"label": "LABEL_1", "score": 0.3553399443626404},
# {"label": "LABEL_0", "score": 0.05207228660583496},
# ]
# ]
for label_score_dict in results[0]:
label: str = label_score_dict["label"]
score: float = label_score_dict["score"]We can define a tag map that tells us how each confidence score affects the final emotion. We can then aggregate the enthusiasm on all confidence scores.
label_mapping = {"LABEL_0": -1, "LABEL_1": 0, "LABEL_2": 1}
positivity = 0.0
for label_score_dict in results[0]:
label: str = label_score_dict["label"]
score: float = label_score_dict["score"]
if label in label_mapping:
positivity += label_mapping[label] * scoreTo test our pipeline we can wrap it in the classroom and use pytest. Sentences that we verify positive emotions have scores greater than zero, and vice versa should have scores below zero.
import pytest
from sentiment_analysis.sentiment_pipeline import SentimentAnalysisPipeline
@pytest.fixture
def sentiment_pipeline() -> SentimentAnalysisPipeline:
"""
Fixture to create a SentimentAnalysisPipeline instance.
"""
return SentimentAnalysisPipeline(
model_name="cardiffnlp/twitter-roberta-base-sentiment",
label_mapping={"LABEL_0": -1.0, "LABEL_1": 0.0, "LABEL_2": 1.0},
)
@pytest.mark.parametrize(
"text_input",
[
"I love this!",
"This is awesome!",
"I am so happy!",
"This is the best day ever!",
"I am thrilled with the results!",
],
)
def test_sentiment_analysis_pipeline_positive(
sentiment_pipeline: SentimentAnalysisPipeline, text_input: str
) -> None:
"""
Test the sentiment analysis pipeline with a positive input.
"""
assert (
sentiment_pipeline.run(text_input) > 0.0
), "Expected positive sentiment score."
@pytest.mark.parametrize(
"text_input",
[
"I hate this!",
"This is terrible!",
"I am so sad!",
"This is the worst day ever!",
"I am disappointed with the results!",
],
)
def test_sentiment_analysis_pipeline_negative(
sentiment_pipeline: SentimentAnalysisPipeline, text_input: str
) -> None:
"""
Test the sentiment analysis pipeline with a negative input.
"""
assert (
sentiment_pipeline.run(text_input) Integration
Now, the last part of the lost is just connecting the text box to our emotional pipeline and updating the displayed image with the corresponding smiley face. We can add one trace To a text variable, which will run the emotional pipeline in a new thread managed by the thread pool to prevent the UI from freezing while the pipeline is running.
class App(customtkinter.CTk):
def __init__(self, sentiment_analysis_pipeline: SentimentAnalysisPipeline) -> None:
super().__init__()
self.sentiment_analysis_pipeline = sentiment_analysis_pipeline
...
self.sentiment_image = None
self.sentiment_text_var = customtkinter.StringVar(master=self, value="Love")
self.sentiment_text_var.trace_add("write", lambda *_: self.on_sentiment_text_changed())
...
self.update_sentiment_pool = ThreadPool(processes=1)
self.on_sentiment_text_changed()
def on_sentiment_text_changed(self) -> None:
"""
Callback function to handle text changes in the textbox.
"""
new_text = self.sentiment_text_var.get()
self.update_sentiment_pool.apply_async(
self._update_sentiment,
(new_text,),
)
def _update_sentiment(self, new_text: str) -> None:
"""
Update the sentiment image based on the new text input.
This function is run in a separate process to avoid blocking the main thread.
Args:
new_text: The new text input from the user.
"""
positivity = self.sentiment_analysis_pipeline.run(new_text)
self.sentiment_image = create_sentiment_image(
positivity,
self.image_display.display_size,
)
self.image_display.update_frame(self.sentiment_image)
def main() -> None:
# Initialize the sentiment analysis pipeline
sentiment_analysis = SentimentAnalysisPipeline(
model_name="cardiffnlp/twitter-roberta-base-sentiment",
label_mapping={"LABEL_0": -1, "LABEL_1": 0, "LABEL_2": 1},
)
app = App(sentiment_analysis)
app.mainloop()
Finally, visualize the smiley face in the application and change dynamically with the views entered in the text!
For full implementation and more details, check out the project repository on GitHub:
All visualizations in this post were created by the author.
