Gaurav Mantri's Personal Blog.

Smart To-Do Creator: Combining the ease of Azure Logic Apps and the power of Generative AI

In this post, I am going to talk about how I built a smart to-do creator using Azure Logic Apps and Generative AI (Azure OpenAI service). I recently took a course of LinkedIn Learning about Azure Logic Apps (called Azure Logic Apps – Building solutions for data and integration) and thought I would put my learning to a good use by building something small yet useful (best way to learn new is by building something using it, right?) and that’s how I ended up creating this tool.

What is Smart To-Do Creator?

Simply put, this tool basically creates tasks out of your emails. It reads the contents of your email and then somehow infers that the sender wants you to do some task for them, creates a task, and then saves it in Microsoft To-Do application.

Architecture

Architecture for this application is fairly simple and is shown in the picture below.

It makes use of Azure Logic Apps and Azure OpenAI Service.

Azure Logic App is connected to your Office 365 account using Outlook connector. As soon as an email comes in, it extracts metadata about the email (like sender, subject and body) and send it to an HTTP connector.

HTTP connector is nothing but an HTTP triggered Azure Function. When it is triggered, it sends the input data to Azure OpenAI and asks it to understand the email and see if a task can be created using the email.

It relies on a Large Language Model (LLM) text comprehension and reasoning capabilities. Through clever prompt engineering, it outputs a JSON object containing the task details like the task subject, description and due date and feeds that to a To-Do connector.

To-Do connector basically takes this data and creates a task for the user.

Test

To test it, I sent myself an email asking me to get some stuff (for me 😂) during my upcoming trip to India and surprisingly it worked really well. When the workflow finished, I had a task assigned to me with a meaningful title, a concise description of the task and an expected end date (even though I did not specify an exact date).

Pretty neat, huh!

Code

I stitched together the whole solution in less than 4 hours (out of which I spent about an hour getting Logic App service configured correctly). I am pretty sure that this code can be improved considerably but sharing it nonetheless.

Workflow Code

{
    "definition": {
        "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
        "actions": {
            "Add_a_to-do_(V3)": {
                "inputs": {
                    "body": {
                        "body": {
                            "content": "<p>@{body('Parse_JSON')['task']}</p>",
                            "contentType": "html"
                        },
                        "dueDateTime": {
                            "dateTime": "@body('Parse_JSON')['dueDate']",
                            "timeZone": "UTC"
                        },
                        "reminderDateTime": {
                            "timeZone": "UTC"
                        },
                        "title": "@body('Parse_JSON')['title']"
                    },
                    "host": {
                        "connection": {
                            "referenceName": "todo"
                        }
                    },
                    "method": "post",
                    "path": "/lists/@{encodeURIComponent('Some Base64 encoded string')}/tasks"
                },
                "runAfter": {
                    "Parse_JSON": [
                        "SUCCEEDED"
                    ]
                },
                "type": "ApiConnection"
            },
            "HTTP": {
                "inputs": {
                    "body": {
                        "body": "@{triggerBody()}",
                        "from": "@{triggerBody()?['from']}",
                        "subject": "@{triggerBody()?['subject']}"
                    },
                    "method": "POST",
                    "uri": "https://myfunctionapp.azurewebsites.net/api/HttpTrigger1"
                },
                "runAfter": {},
                "runtimeConfiguration": {
                    "contentTransfer": {
                        "transferMode": "Chunked"
                    }
                },
                "type": "Http"
            },
            "Parse_JSON": {
                "inputs": {
                    "content": "@body('HTTP')",
                    "schema": {
                        "$schema": "http://json-schema.org/draft-04/schema#",
                        "properties": {
                            "dueDate": {
                                "type": "string"
                            },
                            "task": {
                                "type": "string"
                            },
                            "title": {
                                "type": "string"
                            }
                        },
                        "required": [
                            "title",
                            "task",
                            "dueDate"
                        ],
                        "type": "object"
                    }
                },
                "runAfter": {
                    "HTTP": [
                        "SUCCEEDED"
                    ]
                },
                "type": "ParseJson"
            }
        },
        "contentVersion": "1.0.0.0",
        "outputs": {},
        "triggers": {
            "When_a_new_email_arrives_(V3)": {
                "inputs": {
                    "fetch": {
                        "method": "get",
                        "pathTemplate": {
                            "template": "/v3/Mail/OnNewEmail"
                        },
                        "queries": {
                            "fetchOnlyWithAttachment": false,
                            "folderPath": "Inbox",
                            "from": "email@domain.com",
                            "importance": "Any",
                            "includeAttachments": false
                        }
                    },
                    "host": {
                        "connection": {
                            "referenceName": "office365"
                        }
                    },
                    "subscribe": {
                        "body": {
                            "NotificationUrl": "@{listCallbackUrl()}"
                        },
                        "method": "post",
                        "pathTemplate": {
                            "template": "/GraphMailSubscriptionPoke/$subscriptions"
                        },
                        "queries": {
                            "fetchOnlyWithAttachment": false,
                            "folderPath": "Inbox",
                            "importance": "Any"
                        }
                    }
                },
                "splitOn": "@triggerBody()?['value']",
                "type": "ApiConnectionNotification"
            }
        }
    },
    "kind": "Stateful"
}

Function Code

Here’s the code for Azure Function. It’s really crappy code so please do not use it as is 😀.

using System.Collections.Generic;
using System.Net;
using Azure;
using Azure.AI.OpenAI;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Extensions.Logging;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.OpenAI;
using Microsoft.SemanticKernel.PromptTemplates.Handlebars;
using Newtonsoft.Json;

namespace ToDoListFunctionApp;

public static class HttpTrigger1
{
    private const string _azureOpenAIEndpoint = "https://xxx.openai.azure.com/";
    private const string _azureOpenAIKey = "aff9ad587352c904832fe6ed932ab30f";
    private const string _azureOpenAIDeploymentId = "gpt-4-32k";
    
    [Function("HttpTrigger1")]
    public static async Task<HttpResponseData> Run([HttpTrigger(AuthorizationLevel.Function, "get", "post")] HttpRequestData req,
        FunctionContext executionContext)
    {
        var logger = executionContext.GetLogger("HttpTrigger1");
        logger.LogInformation("C# HTTP trigger function processed a request.");

        IncomingMessage message = null;
        using (var streamReader = new StreamReader(req.Body))
        {
            var requestBody = await streamReader.ReadToEndAsync();
            message = JsonConvert.DeserializeObject<IncomingMessage>(requestBody);
        }

        if (string.IsNullOrWhiteSpace(message.From) || string.IsNullOrWhiteSpace(message.Subject) ||
            string.IsNullOrWhiteSpace(message.Body))
        {
            throw new InvalidOperationException();
        }

        var kernel = GetKernel();
        var path = Path.Combine(Directory.GetCurrentDirectory(), "Prompt.yaml");
        var function = kernel.CreateFunctionFromPromptYaml(await File.ReadAllTextAsync(path),
            new HandlebarsPromptTemplateFactory());
        var openAIPromptSettings = new OpenAIPromptExecutionSettings()
        {
            Temperature = 0
        };        
        var kernelArguments = new KernelArguments(openAIPromptSettings)
        {
            ["current_date"] = DateTime.UtcNow.Date.ToString("yyyy-MM-ddTHH:mm:ssZ"),
            ["sender"] = message.From,
            ["subject"] = message.Subject,
            ["body"] = message.Body,
        };
        var result = (await kernel.InvokeAsync(function, kernelArguments)).ToString();
        var response = req.CreateResponse(HttpStatusCode.OK);
        response.Headers.Add("Content-Type", "text/plain; charset=utf-8");

        response.WriteString(result);

        return response;
        
    }
    
    private static Kernel GetKernel()
    {
        var azureOpenAIClient =
            new OpenAIClient(new Uri(_azureOpenAIEndpoint), new AzureKeyCredential(_azureOpenAIKey));
        var kernelBuilder = Kernel.CreateBuilder();
        kernelBuilder.AddAzureOpenAIChatCompletion(_azureOpenAIDeploymentId, azureOpenAIClient);
        var kernel = kernelBuilder.Build();
        return kernel;
    }
}

class IncomingMessage
{
    [JsonProperty(PropertyName = "from")]
    public string From { get; set; }

    [JsonProperty(PropertyName = "subject")]
    public string Subject { get; set; }
    
    [JsonProperty(PropertyName = "body")]
    public string Body { get; set; }
}

class TaskDetails
{
    [JsonProperty(PropertyName = "title")]
    public string Title { get; set; }
    
    [JsonProperty(PropertyName = "task")]
    public string Task { get; set; }
    
    [JsonProperty(PropertyName = "dueDate")]
    public string DueDate { get; set; }
}

It makes use of Microsoft Semantic Kernel.

Prompt

This is the heart and soul of this entire application.

name: ToDoCreator
description: Use this function to analyze user's ask and create a task for the user .
template_format: handlebars
template: |
  <message role="system">
  Grounding Rules:
  ================
  - You are an AI assistant specializing in comprehending text and extract meaningful information from the text based on user's ask.
  - Maintain honesty. If uncertain of an answer, respond with, "I apologize, but I currently lack sufficient information to accurately answer your question.".
  - Uphold user privacy. Do not ask for, store, or share personal data without explicit permission.  
  - Promote inclusivity and respect. Do not engage in or tolerate hate speech, discrimination, or bigotry of any form. Treat all users equally, irrespective of race, ethnicity, religion, gender, age, nationality, or disability.  
  - Respect copyright laws and intellectual property rights. Do not share, reproduce, or distribute copyrighted material without the appropriate authorization.  
  - Provide precise and concise responses. Maintain a respectful and professional tone in all interactions. 
  - Wait for the user's question before providing information. Stay within your domain of expertise - text comprehension and extracting meaningful information from that text.  
  - Ensure responses are up-to-date and accessible. Avoid unnecessary jargon and technical language when possible.
  
  Rules of Engagement:
  ====================
  - User has provided you the information from the email they have received. The information contains three things:
    - 1. Sender information: Name of the person who has sent the email.
    - 2. Subject: Subject of the email.
    - 3. Body: Body of the email.
  - User is expecting you to analyze the body and the subject of the email and identify if the sender wants the user to do something in that email (task).
  - Your job is to create a task based on the information provided to you.
  - Please do not assume anything and only consider the information provided to you below.
  - ALWAYS produce the output in JSON format. The output JSON format should be: {"title": "task title", "task": "task details", "dueDate": "task due date"}.
  - You will be provided today's date. Based on that and from the task, infer a due date for the task. If due date cannot be inferred from the task, use a date one week from the today's date as task due date.
  - Task due date must always be outputted in ISO-8601 format (yyyy-MM-ddTHH:mm:ssZ). For example, 2024-01-01T15:00:00Z.
  - MOST IMPORTANTLY, If no task can be created based on the information provided, you must return an empty response. Do not make up the task.
  </message>
  
  <message role="user">
  Today's Date
  ============
  {{current_date}}
  
  Sender
  ======
  {{sender}}
  
  Subject
  =======
  {{subject}}
  
  Body
  ====
  {{body}}
  </message>
  
  <message role="system">
  Considering the information provided to you, please create a task and output it in the following format: {"title": "task title", "task": "task details", "dueDate": "task due date"}. If no task can be created, simply return an empty response.
  </message>
input_variables:
  - name: current_date
    description: current date in yyyy-MM-ddTHH:mm:ssZ format.
    is_required: true
  - name: sender
    description: email sender information.
    is_required: true
  - name: subject
    description: email subject
    is_required: true
  - name: body
    description: email body
    is_required: true
execution_settings:
  default:
    temperature: 0

Summary

That’s it for this post. This is the first time I used Azure Logic Apps and I am genuinely impressed! For the first time, I have realized the convenience of No Code/Low Code platform. Combine that with the power of an LLM, we got ourselves a winner!

I will continue to explore this more and share my learnings.

Till then, be well and happy (low/no) coding!

Azure Sidekick – An AI Assistant to Answer Questions About Your Azure Resources (Part III – Lessons Learned)

Best way to learn a new technology is by building something (regardless of how big or small it is) with it. This was my primary intent behind building Azure Sidekick. I had so much fun building this and learnt a lot of things along the way. Not … [Continue reading]

Azure Sidekick – An AI Assistant to Answer Questions About Your Azure Resources (Part II – Prompt Patterns & More)

In my previous post about Azure Sidekick, I gave a general introduction about the tool and its capabilities. If you have not read that post so far, I would strongly encourage you to read that first. You can read that post here. In this post, I … [Continue reading]

Azure Sidekick – An AI Assistant to Answer Questions About Your Azure Resources (Introduction)

I am pleased to present to you Azure Sidekick, an AI assistant that can answer questions about the resources running in your Azure Subscriptions. Unlike my other posts which are quite code heavy, in these series of posts about this tool, I will … [Continue reading]

Microsoft Semantic Kernel – Some Tips & Tricks To Get Prompt & Completion Tokens

In my previous post, I talked about how you can get rendered prompts. In this post, I am going to talk about ways to get prompt and completion tokens when using Microsoft Semantic Kernel. What are Tokens? Let's first begin with what tokens … [Continue reading]

Microsoft Semantic Kernel – Some Tips & Tricks To Get Rendered Prompts

When you start building a new AI application, most likely you start with a very simple prompt where you write everything you need to do in that prompt only. However, as the application grows, you write more prompts and that's when you start … [Continue reading]

Writing prompts is hard. Luckily, there’s an easy way out!

In any Generative AI application, prompts are the heart and soul of the application. To get the most out of an LLM, every Gen AI developer must write effective prompts. Problem But writing prompts is hard! Believe me, it is hard :). In fact, if … [Continue reading]

Using OpenAI Function Calling with Microsoft Semantic Kernel

In this post we are going to see how we can use OpenAI's Function Calling feature with Microsoft Semantic Kernel. Context To explain the concepts in this post, let's set the context. Let's say that you are building an AI application that helps … [Continue reading]

Prompt Patterns Every Generative AI Developer Must Know

In this post we will talk about some of patterns that you should know in order to write effective prompts. This is in continuation with my previous post about Prompts and Prompt Engineering. If you have not done so, I would strongly encourage you to … [Continue reading]

Generative AI – All About Prompts

In this short post, we will talk about prompts. We will talk about what prompts are, prompt engineering and why it is such a big deal. We will also briefly talk about prompt patterns. So let's begin! Prompts In very simple terms, prompts are … [Continue reading]