AI-Based document processing in SharePoint Online using Project Cortex / SharePoint Syntex

Overview of SharePoint Syntex in Microsoft 365

No comments

 1,156 total views,  4 views today

If you have been following the Microsoft Ignite 2020, the project Cortex and SharePoint Syntex is not a new name – it was first announced in November 2019 to apply advanced artificial intelligence (AI) to empower people with knowledge and expertise in the apps they use every day or we can say in other words, Microsoft is investing in next-generation knowledge management to bring data, information, and records together across Microsoft 365 family-like SharePoint, Teams, Microsoft 365 Exchange so it can be accessed, shared, and most effectively used. In this article, we will understand the overview of SharePoint Syntex in Microsoft 365 in other words what is project cortex and SharePoint Syntex in Microsoft 365.

Key-Highlights: What you will learn from this tutorial?

  • Traditional way processing the document in SharePoint
  • What is Project Cortex and Microsoft SharePoint Syntex?
  • How does AI-based document processing work in SharePoint Syntex?
  • Understand Content in SharePoint Syntex
  • Process Content in SharePoint Syntex
  • Content Compliance in SharePoint Syntex
  • SharePoint Syntex pricing/licensing
  • General availability of SharePoint Syntex

Traditional way processing the document in SharePoint

As we know SharePoint is one of the famous document management systems available in the digital document management world.

So let’s understand – how does SharePoint work or manage the document?

In SharePoint, inside the site, we have a document library that contains a collection of list columns or site columns or content type (with site columns) configured. So, when the user uploads a document to the document library user manually selects the content type then updates the content-type metadata value. For example, if we imagine a review document we can think of the document review date, reviewer name, expiry date, etc as document metadata, or even we can apply the taxonomy as metadata for the categorization, and while we upload this kind of review document we need to update this metadata manually. Then once the search crawler crawls the document, these documents will be available in the search along with their metadata, then finally user can get these document from the search result and can filter or refine those documents from it and get it for further processing. Having said this, when we say documents will be available in the search – it means the document will be visible to the right authorized person, so nowhere security is compromised. This is the typical way the documents get processed in SharePoint – and all these things are done manually.

Document Processing Flowchart in SharePoint
Traditional way document Processing Flowchart in SharePoint

Using the above flow we as SharePoint users every day need to perform ‘n’ number of tasks that need more reading, acting, organizing, and managing accordingly – all these things are done manually.

Now, the question is how we could automate the above document management process in SharePoint Online. This could be automated thru Artificial Intelligence (AI) technique – these days the automatic business process has been a trend across all sectors, where all industries run by Artificial Intelligent (AI), for example, if you log into any Banking, Finance, or Retail Marketing web site, you could notice a virtual assistant chat app which will behave like a human being sitting on the other side and providing the hassle-free 24/7service to the customer which is not true – here the Artificial Intelligence (AI) technique is used where the system behaving like a human being.

When other industries already have adopted the use of the Artificial Intelligence (AI) technique, Microsoft has started thinking of how they could do something for their Office 365 customer who uses primarily SharePoint as a document management system or repository using which document classification can be done automatically and based on the classification of the document – the document will be available to their respective system like SharePoint, Teams, Microsoft Exchange – not only this, the document could be uploaded anywhere in Microsoft 365, however, the document will be available to their respective media or channel. This automation could be done thru Project Cortex and SharePoint Syntex which uses advanced artificial intelligence and machine learning (ML) services to automatically recognize the content, extract important information and apply metadata tags – here in SharePoint Syntex we train the system rather than the users. In other words, it is a trainable AI for finding important metadata within organizations. The basic idea is to employ human experts to first train Syntex and then use that model more generally. In the below section, we will learn about what is Project Cortex and SharePoint Syntex in detail.

What is Project Cortex and Microsoft SharePoint Syntex?

Project Cortex is a Microsoft initiative focused on accessing, managing, and extracting the knowledge you have in your organization. Project Cortex extracts this knowledge from the data you store in your systems (list/document library) and then categorize it in meaningful ways based on content models that we set up and configure. The purpose of Project Cortex is to apply AI Artificial Intelligence — to help transform content into knowledge that connects the content and people of an organization with the power of apps and Microsoft 365. Project Cortex has already released its first product: SharePoint Syntex.

Microsoft SharePoint Syntex is the first product has released from the Project Cortex initiative – it is yet to roll out across the tenant globally. SharePoint Syntex is currently available for Microsoft 365 commercial customers and can be turned on in your tenant now.

SharePoint Syntex enables you to build content models that review/understand documents or process forms. Models are built via AI-based machine-learning algorithms and can be taught how to interpret documents. Document understanding models help Syntex deliver knowledge to the right users at the right time without compromising the access rights in the actual content. It uses AI to help with the categorization, process, and compliance control of the content, both internally created from Microsoft 365 and external sources.

AI-Based document processing in SharePoint Online using Project Cortex / SharePoint Syntex
AI-Based document processing in SharePoint Online using Project Cortex / SharePoint Syntex

How does AI-based document processing work in SharePoint Syntex?

Before we realize the actual benefits of SharePoint Syntex, we must configure it in the Microsoft 365 admin center, for setting up the SharePoint Syntex I will write a separate article. In general, the key to the SharePoint Syntex configuration is the content center where the data model and AI content type is used. When we upload any document to the SharePoint document library, the AI engine automatically scans the document and store in the data model, then it classifies, applies appropriate metadata, and accordingly categorize them. This is the whole idea behind this AI-based automation.

In a more technical way, we can say – SharePoint Syntex uses a SharePoint Content Center site to create, manage, and deploy understanding models. These models teach Microsoft’s AI how to review your content and make connections to build knowledge. Here, users can train Syntex to read content the way they do, and advanced AI – artificial intelligence in the product captures and tags both structured and unstructured content, with the objective of meeting the product’s tagline – the purpose is to transform the content into knowledge.

There are three main pillars in SharePoint Syntex:

  • Understand content
  • Process Content
  • Content Compliance

Understand Content in SharePoint Syntex

  • Understand what is inside of the document, apply AI models to identify the type of information
  • We can easily create no-code AI models that teach the cloud to classify and extract information and automatically apply metadata for discovery, search, and classification.

Process Content in SharePoint Syntex

  • Understand the classification of the document, categorizing in the right way
  • Use power automate and workflows for processing the document
  • Apply the corporate taxonomy to your content, optimizing your knowledge graphs
    Automatic classification and routing of content using AI engine

Content Compliance in SharePoint Syntex

  • Security and governance, controlling the lifecycle of the content and the rights to access it
  • It is fully integrated with Microsoft Information Protection

SharePoint Syntex pricing/licensing

Every user using the services of SharePoint Syntex needs to be licensed. At launch, Microsoft has priced SharePoint Syntex at an additional cost of $5 per user per month which is an Add-on to Microsoft 365 F1/F3/E3/A3/E5/A5 or Office 365 F3/E1/A1/E3/A3/E5/A5. This per-user offer enables customers to get access to Advanced AI and machine teaching to amplify human expertise, automate content processing, and transform content into knowledge; and Content centers, digital image processing, form processing, document understanding, advanced taxonomy services, content connectors.

So I think it’s worth taking the time to explore and understand the functionality with a small number of licenses before committing to anything larger scale.

General availability of SharePoint Syntex

Microsoft announced that SharePoint Syntex is available from 1 October 2020. However, it won’t be available at a time in all tenants globally – service will be deployed in various phases across the tenants.

Key-takeaway from this lesson

As we have seen we need to create an AI-based data model while setting up the Sharepoint Syntex configuring it in Microsoft 365 admin center, we need to train the model we create which is a straightforward 4-step process:

  1. Add sample test files (minimum of 5), we could add more also – try to upload the real document from your business which will help the system to classify it in a better-structured way.
  2. Then classify the files we uploaded & run training (label your positive and negative sample files & train the model on keywords and phrases that are important to you in each file – here mostly we can use most relevant business-related metadata which describes the file better way)
  3. Then, create and train extractors
  4. Finally, apply the new model to the document library where you want to have this SharePoint Syntex enabled – note while configuring the SharePoint Syntex in Microsoft 365 admin center we have the option like in which are the sites we can have the SharePoint Syntex service on.

Summary: What we had here?

Thus, in this article, we have learned the below with respect to Microsoft SharePoint Syntex and Project Cortex:

  • Overview of SharePoint Syntex in Microsoft 365.
  • How does the SharePoint document gets processed in a traditional way?
  • What is Project Cortex and Microsoft SharePoint Syntex?
  • How does AI-based document processing work in SharePoint Syntex?
  • Understand Content in SharePoint Syntex
  • Process Content in SharePoint Syntex
  • Content Compliance in SharePoint Syntex
  • SharePoint Syntex pricing/licensing
  • General availability of SharePoint Syntex

See Also,

You may also like the below SharePoint Syntex Article:

How to enable SharePoint Syntex in Office 365 tenant?