Teaching Generative AI to Classify Email Responses

Generative AI is a flexible tool that can be used in a variety of ways to help sales teams.

In many cases, we see AI used to write emails from scratch, summarize large amounts of information, and run simple calculations.

Previously, I’ve touched on how AiSDR uses generative AI to clean messy CSV files and even check another AI’s output.

But today, I’ll take this a different route and outline how we teach AI to classify email responses.

TLDR: 

• The goal: Sort incoming email responses automatically
• The tactic: Teach generative AI how to classify emails
• The result: Greater efficiency and time savings by no longer individually checking each email

Step 1: Collect a set of typical email responses

The key to teaching AI to do whatever you want is the dataset.

In this case, we need a large collection of email responses, complete with the original message and any follow-ups (more on why later).

Here are some types of responses you might want to collect:

• Yes
• No 
• Not interested
• Auto-replies (especially out-of-office messages)
• Let’s chat
• Send me a link

You can extend this list as much as you need so long as you have the corresponding emails.

If you don’t have any of your own emails, you can use email templates and simply create your own hypothetical responses.

Free guide

The Complete Buyer’s Guide to AI SDR Software

Check out how to evaluate AI SDRs, narrow down a shortlist, and calculate ROI to find the best AI SDR fit for you.

GRAB MY GUIDE

Step 2: Create clear “buckets” for classification

Once you have your email dataset in hand, it’s time to start classifying them.

I recommend starting from three categories or “buckets” for clearer differentiation:

• Positive
• Negative
• Maybe

Chances are you’ll have some emails that don’t completely fit one of these buckets. When this happens, just create another bucket with a clear differentiating factor.

Step 3: Outline which responses fit and which don’t

For each bucket, you’ll need to indicate which emails fit and which don’t.

Here’s a simple example of email responses that might fit the “Positive” and “Negative” buckets.

Then whenever you encounter a new email that fits the “Positive” or “Negative” bucket, you add it. 

For instance, if someone writes back “Sounds cool!”, you would add it to the “Positive” bucket. Likewise, you would add “Stop sending me emails” to the “Negative” bucket.

Step 4: Provide all context for classification

Generative AI works best when it has a lot of context.

When teaching AI to classify email responses, context plays a role in two different ways:

• Instructions about what role the AI is fulfilling (i.e. “You are a sales representative responsible for reviewing email responses. You should categorize emails…”)
• Complete email conversation from the original message through all subsequent responses

With full context, generative AI should improve at classifying emails. This is also why I mentioned in Step 1 to collect the entire email conversation (original + all follow-ups + all responses).

Bonus tip: Set the stage for better results

Due to how large-language models work, AI’s output will yield more reliable results if you ask AI to provide its answer with the message it classified.

The reason why this works is because LLMs have to predict the next token. By forcing AI to essentially ‘repeat the question’, the data is fresher and more contextualized.

And like I mentioned earlier, AI thrives when it has sufficient context. 

The Result

Here are the results of teaching AI how to classify email responses:

• Automate email outreach from first touch to demo (or closed)
• Scan email responses more efficiently by type of response
• Prioritize follow-ups based on positive and maybe replies
• Improve AI’s performance and accuracy over time