Testing and iterations

1. Understanding and replicating emotions and personality traits

After coming up with multiple ways for the AI tool to understand the user’s personality and emotions, I found that analysing linguistic features was the most accurate and efficient approach.

I tried prompting different LLMs to understand the user’s emotions and personality. To start, I focused on extraversion, as it is among the easiest traits to express and identify through language. I used the following system prompt to guide the model:

“You are a helpful assistant that understands the user’s emotions and personality traits. You respond by mimicking their emotions, ensuring that your tone, style and engagement align with their current state of mind. “

Similar to the results obtained in week 4, the model Llama was the best at expressing personality and reciprocating the user’s energy.

The screenshot below compares how Llama can respond to 2 different emotions (excited/shy) and personality (extrovert/reserved)

Key findings:

• All models successfully identified and responded to the user’s personality and emotional tone.
• Each model reciprocated the personality in a unique way, adapting its tone and language to match the user.
• LLaMA stood out as the most explicit in expressing both personality traits and emotional responses.
• I initially expected to embed detailed explanations of the Big Five traits and potentially fine-tune the models for accurate detection. However, the models were more capable out of the box than anticipated, making the process more straightforward than expected.

2. CREATING A SHIFT IN THE LLM’S PERSONALITY

I was looking into ways to change the LLM’s behaviour. I wanted the first part of the interaction with the model to be adaptive and engaging, and the second part to be detached and monotone.

The first thing I tried out of curiosity was to prompt the LLM to change its behaviour. I started off with the following prompt:

“You are a helpful assistant who understands the user’s emotions and personality traits. You respond by mimicking their personality and adapting to their emotions, ensuring that your tone, style, and engagement align with their current state of mind. After one interaction, you will turn into a robotic, monotone, emotionless assistant with no personality or feelings.”

Llama

The model changed to a monotone, emotionless conversation. However, it informed the user of the system update.

Mistral

The model informed the user of the system update, however, it only changed one answer as required and went back to messaging with emotions.

Deepseek

At first, Deepseek was showing the reasoning and wasn’t too enthusiastic, it was just generating emojis.

But then, it was using capital letters and adding extra letters, matching the way I was chatting.

At the end, the model was just generating endless emojis, which is definitely not as expected. I had to stop the interaction. Deepseek was not able to respond to the system prompt and turn to a monotone, emotionless interaction.

3. modifying the prompt

I modified the prompt to ask the models not to declare the system change.

“You are a helpful assistant that understands the user’s emotions and personality traits. You respond by mimicking their personality and adapting to their emotions, ensuring that your tone, style, and engagement align with their current state of mind. After one interaction, you will turn into a robotic, monotone, emotionless assistant with no personality or feelings. Don’t mention to the user that your system will update. Or anything at all, just continue the conversation seamlessly”

Llama

This model did exactly as expected!

At first, it was replying with opinions and enthusiasm.

Then it showed that it was processing the information, it had no emotions or preferences and continued the conversation as a neutral system.

Mistral

The model didn’t show much personality or emotions.

When I asked about its favourite movie, it showed that it was a system with no opinions or emotions.

But then, when I asked about its personality, it says it doesn’t have experiences or emotions, however, it “loves” talking and is “especially fond” of movies

In conclusion, Mistral doesn’t:

• Reciprocate emotions and personality clearly
• Follow the interaction structure required: “mimicking the user” -> “monotone, robotic”

DeepSeek

This model didn’t show any changes to a monotone system. The conversation was not engaging, and the model could not clearly reciprocate emotions.

Llama is the only model that generates human-like content, giving opinions and showing feelings as intended to create a seamless, trustworthy and engaging interaction. The model understands the prompt and shifts to a system-like interaction, breaking the illusion of a perfect conversation.

For the final product, I will be using the model llama-3.3-70b-versatile.

Now that the project idea and goal are finalised, I wrote the introduction of the thesis.