-
With the flexible Azure platform and a wide portfolio of AI productivity tools, developers can build the next generation of smart applications where their data lives, in the intelligent cloud, on-premises, and on the intelligent edge.
-
Teams can achieve more with the comprehensive set of flexible and trusted AI services - from pre-built APIs, such as Cognitive Services and Conversational AI with Bot tools, to building custom models with Azure Machine Learning for any scenario.
-
Visual Studio 2017 or 2019 Preview 1 with Xamarin workloads installed.
-
Android Emulator with Hyper-V Windows Feature installed as per this document.
-
Android SDK 28 installed (Android Pie 9).
-
Active Azure subscription.
-
Working on local machine is preferred in order to use camera through emulator. However, if you are working on a VM, the emulator will not be able to access your camera. Follow these steps to enable local camera access for your VM.
-
Install Azure CLI version 2.0.45.
AI Visual Provision mobile app uses Azure Cognitive Services (Computer Vision and Custom Vision) to deploy whiteboard drawings to an Azure architecture. It uses Cognitive Services to detect the Azure services among Azure Service logos and handwriting using the phone camera. The captured image is analyzed and the identified services are deployed to Azure taking away all the pain and complexity from the process. You can see how to create Azure Cognitive Services and configure them in this mobile app.
-
Open your Visual Studio 2019 Preview in Administrator mode and click on Clone or checkout code.
-
Copy and paste the URL: https://github.com/Microsoft/AIVisualProvision in Code repository location textbox and click Clone button.
-
While the cloning is still in progress, the code is already available for use. It has brought up the folder view. Before loading the solution in Visual Studio 2019 Preview, navigate to AIVisualProvision/Source/VisualProvision.iOS/ and delete the folder Assets.xcassets as per workaround mentioned here (which is supposed to be fixed in coming update). Double click on the solution VisualProvision.sln to load it in Solution Explorer.
-
Now you can see that 4 projects are successfully loaded under the solution.
-
Navigate to AppSettings.cs file under VisualProvision project. You need to provide the Client ID and tenantId if you are running the app in Emulator in the Debug mode so that you don't have to enter it all the time you debug. Also, the CustomVisionPredictionUrl, CustomVisionPredictionKey, ComputerVisionEndpoint and ComputerVisionKey has to be entered.
-
In order to get the Client ID and tenantId, type az login in the command prompt and press Enter. Authorize your login in the browser.
Type az ad sp create-for-rbac -n “MySampleApp” -p P2SSWORD in the command prompt to get the Service Principal Client ID and the Service Principal Client Secret.
Copy and note down the appId which is the Client ID
Copy and note down the tenantId
P2SSWORD is the Client Secret or Password, which will be required to run the app.
-
Switch to Visual Studio 2019 and paste the Client ID and TenantId in the AppSettings.cs file.
-
You need a Computer Vision service in order to use the handwriting recognition features of the app. To create your own Computer Vision instance navigate to this page. Click Get API Key button under Computer Vision.
You can refer to this link for more details on Computer Vision service.
-
Click Sign In and login to your Azure account.
-
In the Azure portal, you will be prompted to create a new Computer Vision service. Provide Name, Subscription, Location, Pricing tier and Resource group. Click Create.
-
Once you have provisioned the service, note down the service endpoint URL and endpoint Key as per the Images below:
-
Switch to Visual Studio 2019 and paste the copied URL under ComputerVisionEndpoint and Key under ComputerVisionKey in the AppSettings.cs file.
-
If you wish to use only the handwriting recognition service in your app, skip to the next section to Build and Run the app. But, if you wish to add in-app logo recognition to your app, continue with next step.
-
The in-app logo recognition is accomplished by using Azure Custom Vision. In order to use the service in the app, you need to create a new Custom Vision project and train it with the Images provided in the repo under the documents/training_dataset folder. Go to Azure Custom Vision page and click Get started, and Sign In using your Azure credentials.
-
Agree to terms by clicking Yes and I agree respectively.
-
You will be taken to the landing page of your Azure Custom Vision account. Click New Project and provide a Name. Leave rest options as it is and click Create project.
-
Click on Add Images. Upload an image of Azure App Service logo and tag it as APP_SERVICE. If you don't have the magnets or Images handy, refer to included 2 PDFs (sheet1, sheet2) with all the magnet logos. Make sure you cut only the App Service image from the pdf. Select Done once the Images have been uploaded.
-
In order to Train you need to have at least 2 tags and 5 Images for every tag. Return to the previous step of this section and repeat the step to add at least 5 Images for APP_SERVICE and SQL_DATABASE tags. To train the classifier, select the Train button.
You should use the following set of tags, as they are the expected tags in the application. Tags are located at VisualProvision\Services\Recognition\RecognitionService.cs file.
-
The classifier uses all of the current Images to create a model that identifies the visual qualities of each tag.
-
Optionally, to verify the accuracy click on Quick Test and provide any similar Image URL of either App Service or SQL Database.
-
Under Performance tab, click on Prediction URL and note down the service endpoint URL and endpoint Key as per the Images below:
-
Switch to Visual Studio 2019 and paste the copied URL under CustomVisionPredictionUrl appended by /image, and Key under CustomVisionPredictionKey in the AppSettings.cs file. Click Save.
-
Right click on the solution and select Rebuild Solution.
-
On a whiteboard, design a simple web app architecture consisting of below resources:
- App service to host the app
- SQL database for the data
- Key Vault to store certificates and sensitive data
-
Go to Tools > Android > Android Device Manager. Right click on your Android emulator and select Edit.
-
Under Property hw.camera.back and hw.camera.front, select Value webcam0 so that computer/laptop camera can be accessed by the emulator. Click Save and close the window.
-
In the Android Device Manager, click Start. Android Emulator will show up.
-
Click Run to deploy your app to Android Emulator (launches in Debug mode). Wait for a while until AI Visual Provision app shows up on the emulator.
-
Optionally, if you are having trouble with emulator you can navigate to AIVisualProvision\Source\VisualProvision.Android\bin\Debug folder and install com.microsoft.aiprovision-Signed.apk on your Android mobile device and continue with below steps.
-
Click Allow in order to access camera and photos.
Note: If you get the error: "Guest isn't online after 7 seconds, retrying..", open the Xamarin Android SDK Manager in Visual Studio by going to Tools > Android > SDK Manager. Update the Android SDK Tools and Android Emulator to the latest Version.
-
Enter the password as P2ssword which was noted down earlier. Client ID and Tenant ID will be auto-populated. Click Login.
-
Select your Azure Subscription and click Continue.
-
Take a picture of whiteboard which has your architecture diagram consisting of Azure Service logos or handwritten App Service and SQL Database. Whereas the Key Vault is mentioned in handwriting. For example, refer below architecture.
-
You can see that Azure resources have been identified by the app. Click Next.
-
Select Region and Resource group. Click Deploy.
-
You can see the progress of your deployment right from your emulator or mobile device.
-
Switch to your Azure portal and navigate to the newly created resource group to see the Azure resources created through your emulator or mobile device.
-
For some reason if you are unable to complete this lab, you can download the mobile apps from the App Center through the following links to try it out:
Today you have more power at your fingertips than entire generations that came before you. Powerful mobile applications can be built using the power of Azure, AI and .NET