In this AI Builder three-part blog series, first blog was about how to use AI Builder to extract Form document data and second blog highlighted the use of AI Builder Text classification model.

Now in the third and final blog we will see how to build an AI Object Detector using Power Platform.

Object detection refers to the capability of computer and software systems to detect images, locate objects in an image and identify each object. Object detection can be used to expedite or automate business processes in multiple industries.

AI Object detection is a new feature in Power Platform which can be accomplished using AI Builder. AI Builder allows you to add artificial intelligence to your apps even if you have no coding or data science skills.

Let’s see how an AI object detection model is built using AI builder. On the other hand, Object detection App is built using PowerApps and data is transferred from PowerApps to Dynamics 365 CRM using Microsoft Flow in Power Platform.

1. Build Object Detection Model using AI Builder

a. To build a model using AI Builder, sign in to PowerApps and, in the left navigation pane, select AI Builder > Build. Then select the Object Detection model type.

b. Name your model and Create

c. Select object names – Select the object names by selecting an entity from Common Data Service. Here, we have selected a custom entity with name ‘Electronic Item’ and Field ‘Name’. This entity has records with name of electronic items like Monitor, Keyboard, Mobile and Mouse.

d. Add images – Click on ‘+Add Images’ button to upload images of objects that you want to detect and identify. Upload 15+ images of each object.

e. Tag images– Teach the model to identify objects by tagging and labeling them. For each image, draw a rectangle around the object which you want to identify and tag it with correct name. Go through all the images to tag the objects. Once all the images have been tagged, click on Done tagging.

f. Train Model – Click on Next >Train. It will take several minutes to train the model. Use Quick Test to test the model’s accuracy and then Publish the model to consume it in the app.

Once the model is published its status changes to Live from Draft.

2. Build Object Detector App using PowerApps

a. Click on Create app button shown in previous step. You can create a Canvas App or Model-driven App or Portal from blank depending upon your choice. In our example, we are creating a Mobile Canvas App.

b. Insert Object Detector by navigating to Insert > AI Builder > Object detector. Drag the Object detector to your screen and select the existing object detector AI model.

The object detector control looks like below:

c. Insert a Gallery Control with Title and subtitle layout.

Set the properties of Gallery and its Title and SubTitle as follows:-

Gallery Items  = ObjectDetector1.VisionObjects

Title Text = ThisItem.displayName

Subtitle Text = ThisItem.count

d. Insert some label controls for storing the values of Count for each individual item i.e. Monitor, Keyboard, Mobile and Mouse Counts and set their text properties.

For Example: –

lblKeyboard.Text =First(ObjectDetector1.VisionObjects).count

lblMobile.Text= First(Filter(ObjectDetector1.VisionObjects,StartsWith(displayName,”Mobile”))).count

lblMonitor.Text= First(Filter(ObjectDetector1.VisionObjects,StartsWith(displayName,”Monitor”))).count

lblMouse.Text= Last(ObjectDetector1.VisionObjects).count

You can make your own formula by utilizing the Gallery or Object Detector controls

e. Insert a Create button. Click on Create button, navigate to Action > Flows > Select the existing flow or Create a new Flow as described in 3^rd point. It will start processing and add the selected flow to PowerApp.

Modify the function of Onselect as shown below by passing the parameters to the flow. Here, CreateElectronicEntry is the name of flow.

OnSelect =CreateElectronicEntry.Run(Concat(Gallery1.AllItems,displayName & “-” & count & “,”),lblMobile.Text,lblKeyboard.Text,lblMonitor.Text,lblMouse.Text)

f. Save and Publish the App.

3. Pass data from PowerApps to Dynamics 365 CRM using Microsoft Flow

You can pass data to Dynamics 365 CRM using the Patch statement on click button in PowerApps itself but here we are showing it using a flow because Microsoft flow has many capabilities and you can perform so many tasks by connecting your app to different connectors depending on your business requirements.

a. Create a new instant Flow with trigger of PowerApps.

b. Add New Step >> Search for an Action ‘Common Data Service’ >> Create a New record.

c. Select your ‘Environment’ and ‘Entity Name’. We have created another custom entity ‘Electronic Entry’ which has the fields Monitor, Keyboard, Mobile and Mouse which shows the count of these devices.

d. Map the fields by clicking on ‘Ask in PowerApps’. It will automatically map relevant PowerApps fields with the CRM fields.

e. Save the flow.

4. Run the App

You can capture an image from your mobile camera or select an existing image from Gallery. Click on the Detect button. The objects will be identified if it matches with trained objects and the list would show count of each object detected in the image.

When Create button is clicked it triggers the flow which creates a record in CRM as shown below: –

Conclusion

In this way, you can build AI Object detection App which is an easy to use tool, just point and click, to tailor AI in your business as per your requirements. Object Detection lets you to count, locate and identify selected objects within any image. You can use this model in PowerApps to extract information from pictures you take with the camera.

Please visit the Source and support them

The post Power Platform: How to Build AI Object Detector using Power Platform appeared first on Microsoft Dynamics 365 Blog.

In our earlier article, we explored the reference app released by Microsoft for Document Automation using the Form Processing model of AI Builder.

The app that is a ready-to-use sample app demonstrates the process of reading the form submitted through email and extracting the data submitted in the form and presenting it to the users in a Power Apps Canvas App for further manual verification and processing by the users.

In this part, we will use the flows to not only extract the data submitted using the form but also go ahead and automate the process for verifying the submissions if the confidence score returned is on the higher side.

Note: At the time of writing the blog this feature is not available in all regions. We have used an environment in the US region (.crm) for the purpose of this article.

Pre-requisite

The form processing model has already been trained on the sample forms that need to be processed. You can follow the steps explained in the earlier article to train the model.

For this example, we are using the model that has been trained to extract Name, Age, and Address from the forms submitted.

AI builder form processing in action

Let us walk through a scenario where we need to verify the document submitted to SharePoint against a predefined and trained AI model in AI builder. If the AI model is unable to confidently read the data extracted from the form, we will send an approval to the manager to review the document manually.

Start with a new Cloud flow and we will watch the ‘When a file is created’ trigger of the SharePoint connector.

Connect to the SharePoint site and set the document library to watch.

The next step is to read the file content. If the trigger does not return the actual file content use the Get file content action from the SharePoint connector and pass the Identifier that you receive in the earlier trigger as a parameter to identify the file to be read.

Now that we have the content, we pass it to the AI builder model to process the file. Look for the AI builder connector and choose the ‘Process and save information from forms’ action as shown below:

This action requires the file content, file type and model to be used for processing.

Choose your model from those listed in the dropdown. This will list all published models in the current environment.

In this case, we have hardcoded the form type to be PDF.

For the Form – we need to provide the actual file content. We get this as the output of the Get file content action. Choose file content from there as shown in the screenshot below:

This action returns the result of the fields trained by the selected model. Our model was trained to identify Name, Age, and Address from the form. It, therefore, returns the following for each of the fields:

The Name value has the actual name included in the form and the Name confidence score provides the accuracy level of the extraction. You see that this one is rated as 0.36 which means the accuracy level is low and the model wasn’t really sure of the data extracted.

We will read the confidence score of each of the fields and if any of these is less than 0.75, we will send out an approval request with the link to the file.

For the approval request, we provide the following;

Type – If you would like any one user to approve or do you want approval from all users that this request is being sent to.

Title – Title for the approval message.

Assigned to – The users to which the approval request needs to be sent. You can provide more than one user here and the message will be sent out to all the users listed here.

Item Link – Use the dynamic properties to select the link to the item from the SharePoint connector.

The approval request will be received on Flow mobile app.

The approvals can be viewed on the device as follows:

And can also be seen in the action items section of the flow portal.

This way you can quickly design a flow and automate the document verification process using the form processing model of AI builder.

Modifying the process in the Microsoft Document Automation app

If we go back to the Microsoft Document Automation app, we can modify the flows in this app to a similar logic as explained above.

Once the app has been installed, you will find the Document automation base kit solution listed in your solutions. This solution has all the assets shipped as part of the Document Automation app. We will specifically focus on the Power Automate Flows used to automate the processes.

1. Document Automation Email Importer

This flow triggers when a new mail is received and it reads the attachment files and creates a new record in the custom Document Processing table and uploads the file there.

2. Document Automation Processor

This flow triggers when the record is created in the Document Processing table after successful execution of the flow listed above.

This flow picks up the file that has been uploaded, reads the AI model to use that is stored in the configuration record, and then calls the Predict custom action of the AI model table and passes the file and model to it for processing.

We could edit this flow to process the file saved in the Document Automation Processing table using the AI builder action and logic similar to the one explained above and if the validation fails, update the Document Automation Processing record status to Manual Review or mark it Validated if the confidence score is above 0.75 for each of the extracted fields.

3. Document Automation Validator

This flow updates the status of the record in the Document Processing Table, we can skip this flow as we are handling the same in the above flow itself.

Please Follow the Source

The post Document Automation with Forms Processing model using AI Builder Connector in Power Automate Flows appeared first on Microsoft Dynamics 365 Blog.

AI Builder is a low code artificial intelligence platform that supports the Power Platform. it consumption on data that already exists in the Common Data Service (CDS), AI Builder is the platform for providing a low-code user experience for every developer to create and customize their PowerApps and Flows.

AI model type	Category	Start here
Prediction	Prediction	Overview of prediction model
Form processing	Vision	Overview of form classification model
Object detection	Vision	Overview of object detection model
Category classification	Language	Overview of category classification model
Entity extraction	Language	Overview of entity extraction model

Official Documentation: https://docs.microsoft.com/en-us/ai-builder/

Digitize your Forms to extract information, store or process them​

Scenario

• Will train our model to read Internet Bills
• Use the trained Model in power Apps
• Extract the Total Amount Collected information
• Will automate extracted Total Amount and other values to Microsoft dynamics 365 or Note or your Local database (Just for the sack of example)
• Will Store extracted data in the Legacy System using UI FLOW (RPA)

Prebuilt AI models

AI model type	Category	Start here
Business card reader	Vision	Overview of business card reader
Text recognition	Vision	Overview of text recognition
Category classification	Language	Overview of category classification
Key phrase extraction	Language	Overview of key phrase extraction
Language detection	Language	Overview of language detection
Sentiment analysis	Language	Overview of sentiment analysis

The post Using AI BUILDER in Power Apps and power Automate , UI FLOW Robotic Process Automation of Microsoft power Platform appeared first on Microsoft Dynamics 365 Blog.

In the last post, I have explained how to use text recognizer fo the aim of detecting the text in a picture using the AI builder in Power Apps.

In this post and video, you will how to call it in Power Automate and extract the text from an image that is located in Onedrive and the result will be written back in Azure Blob Storage.

To start open the Power Apps and click on the AI Builder and Build and choose the text recognition.

then use the option the use in flow (Power Automate)

Next, in the Power Automate, click on the flow and create a new flow with a trigger of the When a file is created.

Next, the Predict module with the Model: TerxtRecognition model and the image from the previous trigger.

Next, use the Create blob for the output in Microsoft Azure.

for the name of the file, I will put the current time plus the file type that is a text file

Next, for the Blob content, use the function concat and from the Dynamic content select the PredictionOutput.

All done!

just put an image file like this on in the folder in one drive and run the flow to see the result in the blob storage!

See the video from

The post Power Automate for OCR appeared first on RADACAD.

Follow Source

The post Power Automate for OCR appeared first on Microsoft Dynamics 365 Blog.

Thanks to AI Builder in Microsoft Power Apps and Flow, now we have access to sentiment analysis,

Keyword extraction and Language detection inside Microsoft Flow is now available.

In this post, I am going to show how to do sentiment analysis inside Power Automate.

First What is sentiment Analysis and what sort of analysis we have inside AI Builder for text analytics.

Text Analytics

in Power Automate and Power Apps three analytics is available as sentiment analytics, keyword extraction, and Language detection as below

As you can see in the below picture in AI builder under Build we have these three modules.

In this example, I am going to show sentiment analytics. The first login to account with AI builder account, then click on the Build and then click on the sentiment analysis

Then the new windows show the main purpose of the sentiment analysis. As you can see in the below picture, the only option to use is on Microsoft Power Automate (flow).

Next, you able to see some hints about using it in the Microsoft power Automate

Note: For long Text, analyze each sentence, not the whole document

Note: Use the same language for a document not multiple languages in a document

Then click on the Use in a Flow,

IN the Power Automate ( make sure you are in the same environment), then click on the solution and then click on the New then Flow.

In the new flow, click on the When a new email arrives

then choose the folder as inbox and the subject as Hotel

Next search for the predict for CDS, and for the model choose the SentimentAnalysis model, Then for the text choose the Body Preview,

Next for the language choose the language you will have as it is en for me

In the next step,  we are going to store the result in one drive for example, that you can provide a specific name for the file using the Concat function. In this example, I use the Utcnow to get the current time and date and the file type that is Text file as .txt.

for the file content, I am going to use the sentence to put in the file, before creating it, just click on the run you had from the previous example

as you can see we have a sentence-level sentiment analysis and a global level. I am going to Concat both of them.

then after the run code you should see below text file in the onedrive.

The post Sentiment Analysis Application: Microsoft Power Automate, Power Apps and AI Builder appeared first on RADACAD.

Follow Source

The post Sentiment Analysis Application: Microsoft Power Automate, Power Apps and AI Builder appeared first on Microsoft Dynamics 365 Blog.

In the previous posts ( Post 1, Post 2, and Post 3) the process of analyzing the forms using the Form processing feature in AI Builder has been explained.
from last month some changes happen in the Prediction module in Microsoft Power Automate that will impact the form processing experience using flow.
In this post, I will show you how to work with the new one. The previous of you created still working.
the changes happen in the Predict module.

What we Have

in the below flow we have a module Predict that has a Request Payload, and we need to specify the type of the image and the file as a string, also later we need to parse the result by another component name Parse JSON.

What to Change

Now everything gets a bit easier, we have a Predict module that in the separate box it gets the document type and document itself, next in the create type we just need to use the Concat function to get the result and values.

I have used the below code for the File Content:

concat(‘ Session Title: ‘,outputs(‘Predict’)?[‘body/responsev2/predictionOutput/labels/Session_85648318f538ad191ff5651a30c24966/value’],’ ,Speaker Name: ‘,outputs(‘Predict’)[‘body/responsev2/predictionOutput/labels/Speaker_64a30215a5867855cdcb495ee6eef3ec/value’],’ ,Comments: ‘,outputs(‘Predict’)?[‘body/responsev2/predictionOutput/labels/What_0027580d34b848d69176d3f4a73de5c5450/value’])

also, watch below video

The post Recent Update in Predict Module in Power Automate Predict- Part 4 appeared first on RADACAD.

Follow Source

The post Recent Update in Predict Module in Power Automate Predict- Part 4 appeared first on Microsoft Dynamics 365 Blog.

There are many usages of  AI Builder for Object detection, form processing, Binary Prediction, Text classification, business card Reader. However, besides them, there is another one that is available now in Power Automate. In this post, I am going to show you how to use this feature, for creating a process for automatically apply sentiment analysis and create an output file to store back to Onedrive and then clean the result using Power Query.

Power Automate

navigate to Power Automate 

login to the account you have AI Builder environment, Then under the solution, create a new flow with the trigger of “When a file is Created” in Onedrive

In the next step, you need to select the Predict to component from the Next Step action, in the model section please choose the SentimentAnalysisModel 

Then in the Request Payload section, you need to first click in the box, then write the below code

{“text”:””,”language”:”en”} (make sure the it is “)

new window chooses the Expression and clicks on the ” ” in front of the text and from the Dynamic Content, choose the File content.

now choose a new action, and select the Create file to send back the result to the Onedrive.

for the Folder path just choose the location you want to save, for the file name you can use the Expression box to create a simple name and type like

then in the File content select the ResponsePayload from Dynamic Content

Now just save the flow and test it you should upload a text file that has some comments like Hotel comment in the text format, then you need to run the flow.

below text can be a sample from hotel review

“The suite was awesome. We did not have much interaction with the staff. We did sleep on the queen size sofa bed in the living room instead of the queen size bed in the actual bedroom due to the temperature. It was very hot and humid. The air conditioner in the living room does not cool off the bedroom. This suite is very close to shopping and dining. After each day of adventures we would walk to dinner at different joints. I will stay here again when I return.”

Now a JSON file should created and you can use Power BI to analyze it in an easy way.

In the next post, I will explain how to analyze the result with Power Query.

see the full video from here

The post Automation of Sentiment Analysis using AI Builder appeared first on RADACAD.

Follow Source

The post Automation of Sentiment Analysis using AI Builder appeared first on Microsoft Dynamics 365 Blog.

AI Builder brings a huge change to creating smart self-service Applications. In this post we are going to see how to create a smart application that able to read the Business Card content and extract them, then import them using Power Automate to the Outlook contact list.

Creat Business Card Reader using Power Apps and AI Builder

First login in the environment that supports the AI Builder.

Now in the new  Canvas app, create a new Phone layout

In the new page, click on the Insert option, then click on the AI Builder –> Business Card Reader

In the next step, click on the Insert Lable, and for the label write the below formula, BusinessCardReader1.FullName

Copy and paste two times for the email and Phone number

BusinessCardReader1.Email

and

BusinessCardReader1.BusinessPhone

Next, Save the App and Navigate to the Power Automate (Microsoft Flow)

Create Flow in Power Automate

Now navigate to the same Environment in Power Automate with the same account, First, create a new Flow, and the trigger should be Power Apps that do not get any input from the Power Automate,

Next, click on the Next step and add the “Create contact(V2)”

we need to set up some part of this component.

First the Folder ID, for me is Contacts

Next, the Given Name which is mandatory, click on the box, choose to Ask in Power Apps, then another component fir the GivenName will show choose that one and remove the Ask in pOwer APPs, do the same for the Home Phone and Email address

Then save the Flow and back to the Power Apps

Connect the Power Apps and Power Automate

Back to Power Apps to connect the Apps with Power Automate. Click on the Insert, then click on the button, and change the name of the Button to the Add

Next, click on the Button, then Action, then Power Automate, and in the list at the right sides of the page choose the Flow you just created, then at the formula bar you should it.

The list of required parameters has been listed also there.

Next, add the parameters as below

GettheContact_powerapps.Run(BusinessCardReader1.FullName,BusinessCardReader1.BusinessPhone,BusinessCardReader1.Email)

As you can see all the elements we already add in the Flow to be added in the Contact have been requested here as input parameters.

Run the application using the Run button at the top right

you should see the details and also you should see the contacts have been added to the contact section in outlook

see the video from here

https://youtu.be/lcQoQOhw_uA 

The post Business Card Reader Automation with AI Builder, Power Automate and Power Apps appeared first on RADACAD.

Follow Source

The post Business Card Reader Automation with AI Builder, Power Automate and Power Apps appeared first on Microsoft Dynamics 365 Blog.

Text classification is one of the important tasks for the aim of classifying the texts based on the allocated tags.
In the previous blog, the process of how to create Text classification in the Power Apps using AI builder has been explained,

In this Blog Post, you will see how to use the created Text classification model in the Power Automate (Microsoft Flow).

to strat

1- Login to Power Automated Account and to the same environment you have your own Text classify model created in the previous steps in the same environment.

2- In the next step, you need to login to the Power Automate,  just make sure to log in to the Power Automate in the same environment. Same as Other AI Builder solution in Power Automate, you need to create a new solution and other that create the Microsoft flow.

then click on the new Solution, “AICourse”. Then create a new Flow.

In the new page, create the Microsoft Flow, by trigger by ” Manually trigger a flow”, and add Input for the Text

in the new page call the text as “Mytext”. Next, we need to call the AI Builder using Predict for Common Data Services (CDS). So click on the “New Step” and search for the Predict.

In the list of the model, you can see the “textclassification”, we created alongside with other models. For the Request Payload

write the

{“text”:” “}

and

for the formula for the white space click on the “Mytext” which is the output of the previous step.

Next, change the name of the Flow and save the flow to click on the text at the top right of the window .

For the first time, you need to trigger manually.

Next, run the flow to get the JSON schema

When the  run finished, click on the output and copy the respond Payload

{“predictionOutput”:{“results”:[{“type”:”Care”,”score”:0.603922}]},”operationStatus”:”Success”,”error”:null}

Click on the Edit, and add a new Step for the Pars JSON.  then Click on the “Geberate from Sample” to extract the JSON code. The yu should see something like below

Just save and run the code. This time after a click on the Test Flow, I choose the Using data from Previous runs.

Now check the output, as you can see in the JSON code the output id Type is Care and the Score is 0.60.

The output is on the JSON format, you can then use it to automatically send the result to other platforms.

https://docs.microsoft.com/en-nz/ai-builder/text-classification-model-in-flow

The post Text Classification using AI Builder inside Power Automate (Microsoft Flow) appeared first on RADACAD.

Follow Source

The post Text Classification using AI Builder inside Power Automate (Microsoft Flow) appeared first on Microsoft Dynamics 365 Blog.

Another exiting new update in AI Builder is Form Processing. With for Processing you able to train the model with forms you have and AI builder able to detect each session of the form and the related values. This feature is so exiting form as for 5 years I am one of the organizer of SQL Saturday Auckland and always I have challenge with evaluation. In some strange way people prefer to evaluate the sessions via paper rather online link. So I need to create an application with cognitive service that able to detect the elements in a form. Such as title of the session, speaker name and the comments from the attendees. In this post and the next one, I will explain the process from creating the form processing model to Flow application that helps me.

in this post and next One, I will explain the process.

First: Set up the Form Processing in AI Builder

Before start Make sure

! Make sure you have forms and at least 5 forms with less than 4 MB size and if it is handwritten form, the quality of form in jpg/png or PDF make sure to be good.

! You already access to AI Builder environment see Post. 

Now, you can start logging into the Power Apps make sure to access the environment you can see the AI Builder. You should see the AI Builder icon in left side of the window.

Next, under the AI Builder, choose the Build Option, then from AI model, choose the Form Processing.

In the new page, Form Processing put a name for the Model Name then select the create

The first step is Analyse Documents, you need to import at least 5 samples of the filled form you have. Just make sure the quality be good and the size of all file does not exceed 4MB.

As I mentioned before, I want to use this form processing for the aim of session evaluation, so I import 6 different session evaluation like below :

After importing the forms, you need to analyse the documents, so AI Builder can detect each section of the forms and related values.

Analysing the form take a couple of minutes, after that, you need to check the identified fields and choose the one that is matter to you

As you can see in the below picture, in this example three fields have been identified, and you able to see the confidence of each part by hovering your mouse over each field. Choose the fields you are happy about. The selected fields will be shown in the right panel.

After selecting the fields, you are in the second step of building form processing, to navigate to the third step, click on the Next.

Now, you imported your sample forms, at this stage you can train your model, you can see a summary of the model as the number of forms and number of selected fields. Then click on the Train option to navigate to the last step.

Training will take about a couple of minutes, depends on the number of documents for training, after training complete, Go to details page.

In the details page, you can see the number of documents you uploaded for training, the number of detected fields in each form, You can also test the model and then publish it.

for testing the mode, you need to import a sample of the form that you have, then wait to see the result in the Quick Test page.

In the sample, all trained fields should be identified. Next Back to the details page and Publish the model so you can use it in other applications

I want to create a workflow so I can upload pictures in OneDrive, then using this model and store back to OneDrive the result as a text file so I can use it Power BI report. As a result, after creating a model, I navigate to the Microsoft Flow. and I log in in the same environment I created the form processing model.

To make sure I can see the model I just created in Microsoft Flow, click on the model and you should see the Form Processing model is available with Publish Status as Live.

In the next post, I will explain how I create a Workflow that able to read scanned file from the OneDrive folder then apply the created model and store back in a text file in the OneDrive again.

The post Form Processing Work flow- Create Form Processing Model Part 1 appeared first on RADACAD.

Follow Source

The post Form Processing Work flow- Create Form Processing Model Part 1 appeared first on Microsoft Dynamics 365 Blog.