Mocking Mirror

🤡 Mirror, mirror on the screen, who knows nothing about AI at all?

Overview

This repository contains the public code for the EDJEovation Days 2024 hackathon project to create an AI solution that can comment on video of a user and provide them with feedback on their appearance, with a sarcastic twist. The project is called the Mocking Mirror.

This version of the project has been modified for public use and maintenance by myself. The original project was created by a team of Leading EDJE developers.

The solution for this hackathon includes:

🌐 A React web client application for user interactivity and presentation.

💜 An ASP.NET Core Web API that processes an image and returns text and audio responses.

✨ Azure AI services for generative image-to-text and text-to-speech synthesis.

🛠️ .NET Aspire for orchestration of the local development environment.

Table of Contents

• Overview
• Table of Contents
• Get Started
  • Prerequisites
  • .NET Aspire
  • Set up Azure AI services
  • HTTPS
  • Clone the repo
  • Set user secrets
  • Run the app
• Acknowledgements

Get Started

Prerequisites

To run this project, you will need to have the following software installed on your machine:

• Git
• .NET 9 SDK
• Node.js
• An OCI compliant container runtime, e.g.:
  • Docker Desktop
• An IDE or text editor of your choice
  • Visual Studio
  • Visual Studio Code

WARNING: Additional changes may be required if using an alternative to Docker Desktop.

.NET Aspire

This project uses .NET Aspire to orchestrate the local development environment. This simplifies the process of running the frontend and backend services together.

For more information on or troubleshooting .NET Aspire, see the Aspire documentation.

Set up Azure AI services

This project uses multiple backing Azure AI services. To run the application, you will need to set up the following services:

• Azure AI Services
  • Azure OpenAI Service
  • Azure AI Speech

You will also need to deploy a vision-enabled model to your OpenAI service. Additionally, you may need to adjust the content filters to allow for the generation of sarcastic responses. For more information, see the Azure OpenAI documentation.

Infrastructure as code (IaC) is provided in the Infra directory to help you deploy the necessary resources. The IaC is written in Bicep and can be deployed using the Azure CLI or Azure PowerShell with the following steps:

1. Update the parameters in the main.bicepparam file.

2. Run the following respective Azure CLI or Azure PowerShell command with your resource group name (use -WhatIf flag to preview changes):

New-AzResourceGroupDeployment -Name MockingMirrorInit -ResourceGroupName '<rg-name>' -Mode Complete -TemplateFile ./infra/main.bicep -TemplateParameterFile ./infra/main.bicepparam

az deployment group create --name MockingMirrorInit --resource-group '<rg-name>' --mode Complete --template-file ./infra/main.bicep --parameters ./infra/main.bicepparam

WARNING: Azure AI services may require a subscription and may incur costs. Be sure to review the pricing and terms of service before using these services.

HTTPS

This project uses HTTPS for local development. This requires the use of a trusted certificate. Before running the application, the following command will check for an existing certificate or generate a new one.

dotnet dev-certs https --trust

Clone the repo

To clone the repository, run the following command in your terminal:

git clone https://github.com/victorfrye/mockingmirror.git

Set user secrets

To run the application, you will need to set the user secrets for the WebApi project. An example file is provided sharing the flattened document structure and variables expected. To set the user secrets, run the following PowerShell script with your modified file command in the root of the project or use your IDE:

Get-Content ./path/to/your/usersecrets.json | dotnet user-secrets set --project ./src/WebApi/WebApi.csproj

WARNING: The example file should be copied outside the repository and modified. Modifying the file directly or a copy inside the repository may accidentally push secrets to the origin repository.

Run the app

To run the application, simply start the AppHost project with the following command in the root of the project or your IDE:

dotnet run --project ./src/AppHost/AppHost.csproj

Acknowledgements

This project was created by a team of Leading EDJE developers for the EDJEovation Days 2024 hackathon. It won second place. The original project was created by:

• Victor Frye
• Matt Eland
• Terry Welsh
• Jon Trotter
• Bob Fornal
• Ben Schwemlein
• Julie Yakunich
• Dave Michels

The project was modified for public use and maintenance by myself, Victor Frye. However, it would not have been a reality if not for each of these individuals and the support of Leading EDJE.