Reduce false positives with SITs and advanced classifiers in Microsoft Purview.
Your DLP policies fire on content that isn’t sensitive. Each false match costs someone time to check, and some block work that should go ahead. Enough of them and the real risks get lost in the noise. This guide follows Microsoft’s model for cutting those false matches by picking the right detection method for each kind of data.
It’s written for the administrators who build sensitive information types and run DLP policies in Microsoft Purview. Microsoft’s model has four steps, one per detection method, and Microsoft estimates two to three weeks for each. You may need only some of them: the chooser in Step 1 shows which.
Microsoft’s four detection methods on one page.
Microsoft’s blueprint gives each detection method a column: what to do, what you get and how long it takes. The efforts are the blueprint’s estimates, with no stated basis, so treat them as a starting point, not a commitment. Your tenant’s size and data will move them.1
- System classification in the blueprint means sensitive information types (SITs).1, 9
- Read the four columns as options, not a rollout you must finish: each method suits a different kind of data. The chooser in Step 1 shows which apply to you.1
Microsoft’s steps at a glance
| Step | What it does | Outcome | Microsoft’s planning guide |
|---|---|---|---|
| Step 1 | Configure pattern-based sensitive data detection | Well known patterns of data identified | 3 weeks |
| Step 2 | Implement Exact Data Match for precise matching | Match for specific known data ensured | 2 weeks |
| Step 3 | Use trainable classifiers for complex content | Data along certain topics or document types with no strict patterns identified | 2 weeks |
| Step 4 | Apply document fingerprinting for templates | Well-known document assets strictly matched | 2 weeks |
Detection is licensed feature by feature.
Microsoft doesn’t license this model as one product. Each feature below has its own terms, as the service description and the feature’s Learn article state them. Pay-as-you-go features are marked.8
Licences by feature 4, 5, 6, 8, 10
| Feature | Licence | Notes |
|---|---|---|
| DLP for Exchange Online, SharePoint and OneDrive | Microsoft 365 E5, A5, G5, E3, A3, G3; Microsoft 365 Business Premium; Office 365 E5, A5, G5, E3, A3, G3; Microsoft Purview Suite | The policies your SITs and classifiers feed. |
| Automatic sensitivity labelling | Microsoft 365 E5 or Microsoft 365 Information Protection and Governance | Step 2 uses an auto-labelling policy in simulation to measure EDM accuracy. Manual labelling is also in Microsoft 365 E3 and Business Premium. |
| Content Explorer and Activity Explorer | E5, A5 or G5 | E3, A3 and G3 tenants still collect the data, but can’t open these views. |
| Auto-apply retention labels with a trainable classifier | Microsoft 365 E5, A5, G5; Microsoft Purview Suite; Microsoft 365 E5, A5, F5, G5 Information Protection and Governance | |
| Credential scanning SITs | E5 | |
| Create or change document fingerprints | Not E3 | E3 customers can’t create new fingerprints or change existing ones. |
| Test a SIT | At least one Exchange Online licence in the tenant | Without it, Test is greyed out. |
| On-demand classification | Pay-as-you-go billing, per-user licensing, or both | Each scan can add cost. Check the estimate before you start. |
| EDM SITs and trainable classifiers in DLP | Not named per plan in the service description | The nearest rows: advanced classifiers in Outlook DLP policy tips need Microsoft 365 E5, A5 or G5, Microsoft Purview Suite, or E5 Information Protection and Governance. Confirm with your Microsoft account team before you plan Step 2 or Step 3. |
Two things this guide uses are in preview: the faster, automated way to create a custom trainable classifier, and the Business context classifier Microsoft uses as its Step 3 example.11, 12
Choose your methods, then tune the pattern-based SITs.
- Outcome
- Well known patterns of data identified
- Microsoft’s planning guide
- 3 weeks
A sensitive information type (SIT) finds data by its pattern, such as a card number or a tax file number. It’s quick to deploy and comes built in. This step starts by mapping your data and choosing a method for each kind, then tightens the SITs you keep.1, 9
-
Map your sensitive data and choose a method for each kind
Microsoft Purview portal › Solutions › Data Lifecycle Management › Explorers › Content Explorer (classic)
Which method for which data 1
Your data Preferred method Alternative Step Common types with a standard pattern, such as card numbers Built-in SITs, or custom SITs Manual labelling Step 1 Personal data about known customers or patients, held in a business system EDM, from an extract of that system Custom SITs with an ID plus common personal data SITs Step 2 Personal data about employees or contractors EDM, from an HR system extract Custom SITs with an employee ID plus named entities Step 2 Contracts, legal documents and other business documents with varied wording Custom trainable classifiers, with OCR for scans Built-in trainable classifiers, with OCR Step 3 Forms with personal data Form fingerprinting plus standard or custom SITs Custom SITs, with OCR for scanned forms Step 4 Standard templates such as contracts, invoices and RFPs Document fingerprinting Step 4 - List the kinds of sensitive data your organisation holds and where each lives. Microsoft puts this first.9
- Open Content Explorer (classic) to see which SITs and labels already match content, and where.13
- Match each kind of data to a row of the table above. Skip any step that no row sends you to.
Why A pattern can’t tell your customer’s number from a stranger’s number in the same format. EDM can, because it flags only data that matches a row you uploaded.1, 14
-
Start from a built-in SIT and copy it to change it
Microsoft Purview portal › Information Protection › Classifiers › Sensitive info types
- Find the built-in SIT closest to your data. Use it as it is if it fits.9
- If it doesn’t fit, select it, choose Copy, then Yes.15
- Edit or remove patterns in the copy, select Next, then Save.15
Check The copy appears in the Sensitive info types list beside the original, which stays unchanged.15
Watch out
Nine built-in SITs, such as the EU and Canada driver’s licence types, can’t be copied in the portal. A SIT with 10 text processors has to be copied with PowerShell.15
-
Create a custom SIT with a tight primary element
Microsoft Purview portal › Information Protection › Classifiers › Sensitive info types › Create sensitive info type
- Enter a Name and Description, then select Next and Create pattern.15
- Choose the primary element: the regular expression, keyword list, keyword dictionary or function that identifies the data.9, 15
- For a regular expression, put
\bat each end so it can’t match inside a longer number, or use lookarounds. A loose\d{6}also matches the first six digits of longer numbers;\b(\d{6})\bmatches six digits standing alone.9, 14 - Attach a validator, such as a checksum or date check, where one exists. It adds logic a regular expression can’t express.9, 16
- For a keyword list, choose word match in space-delimited languages such as English. Use string match for Chinese, Japanese, Thai and Korean.9
- Keep keyword dictionaries short. They’re always case-insensitive, and all dictionaries in the tenant share 1 MB of compressed text.9, 17
- Select Create, Next, review the recommended confidence level, then Save.15
If you’d rather build and test the pattern first, a tool in the related links below writes it with the single capturing group.
-
Add supporting elements, proximity, confidence and exclusions
- Add supporting elements: keywords that should appear near the match. A date alone flags every email header and timestamp; a date near “DOB” or “DateOfBirth” is far more likely to be a date of birth.9
- Set Character proximity to 300 for any new custom SIT. Don’t choose unlimited proximity (anywhere in the document).9, 15
- Give patterns with more supporting elements or tighter expressions a higher confidence level. Set confidence to low where the SIT has few or no supporting elements.9
- Add exclusions for text, prefixes or suffixes you know aren’t sensitive, such as your own phone number ranges.7, 9
How the proximity window decides a match. Microsoft’s example uses 250 characters; the Step 1 article recommends 300 for new custom SITs. Diagram · Microsoft Learn, Learn about sensitive information types -
Test the SIT on a matching and a non-matching file
Microsoft Purview portal › Information Protection › Classifiers › Sensitive info types
- Prepare two files: one with content the SIT should match, and one with content it shouldn’t.3
- Select the SIT, choose Test, and upload one file. Then repeat with the other.3
- Test again in each workload where you’ll use the SIT. The same SIT can give different results in email, files and on devices.6
Check The matching file returns matches at the confidence you expect, and the non-matching file returns none.3
-
Run an on-demand scan against sample locations
Microsoft Purview portal › Information Protection › Classifiers › On-demand classification
- Select New scan.5
- Set the scope to a few sample locations, choose your custom SITs (up to 50 at a time), and set the date range and file extensions. By default the scan covers items created or modified in the past year.5
- Select View estimation, check it, then Start classification.5
Why A test covers one file. An on-demand scan tests the SIT against real, targeted data, including existing files that haven’t been recrawled.9, 15
Watch out
On-demand classification uses pay-as-you-go billing, per-user licensing or both. In a tenant that uses EDM, a scan also rescans files already classified, which adds to the cost. Look at the estimate before you start.5
-
Combine SITs in the DLP rule and set confidence and instance counts
- In the rule’s Content contains condition, add a SIT for each identifier a record can carry, such as passport and driver’s licence numbers. Join them with Any of these (OR), rather than relying on one SIT.9, 18
- To demand context, put a second group of SITs in the rule and join the groups with AND. Use a group joined with NOT for exclusions.18
- Set each SIT’s confidence level: low is 65, medium 75, high 85. High confidence gives the fewest false positives but may miss more.6
- Set the instance count. Microsoft suggests high-confidence patterns with low counts, say five to 10, and low-confidence patterns with higher counts, say 20 or more. For data where one occurrence matters, use a count of one.1, 6
Two groups of SITs in one rule, each matched on any of its SITs, joined by AND, so the rule needs an identifier and a diagnosis code together. Screenshot · Microsoft Learn, Data Loss Prevention policy reference Each SIT in a condition has its own confidence level and instance count range. Screenshot · Microsoft Learn, Automatically apply a sensitivity label to Microsoft 365 data Video · Microsoft Learn · 2 min 31 s
How a SIT reaches low or high confidence. A 16-digit number that passes the Luhn check is the primary element, and a nearby keyword or expiry date lifts it to high. Ends with the counts Microsoft recommends for each level. From Learn about sensitive information types, Microsoft Learn. Video by Microsoft, with Microsoft's captions in the player. Check In simulation, the policy’s matches drop on content you know isn’t sensitive and stay on content you know is.2
-
Mark each match right or wrong, and tune from the results
Microsoft Purview portal › Information Protection › Classifiers › Sensitive info types
- Open the SIT, then its Matched items tab, and open a document.7
- On the Contextual Summary tab, select Match or Not a Match. Select Withdraw feedback to undo.7
- For email, open the alert under Data loss prevention > Alerts, select View details, then the Events tab, Actions and Not a match.7
- Where false matches pile up, raise the confidence or instance count, add conditions to a copy of the built-in SIT, or exclude known prefixes and suffixes.7
While a policy is new, user overrides give you a second feedback channel. An override can ask for a business justification, and a user can report the match as a false positive.18
The user override settings in a DLP rule. Override justifications and false positive reports help you tune the policy. Screenshot · Microsoft Learn, Data Loss Prevention policy reference Watch out
Feedback works only on items created or changed after the feedback experience was turned on for your tenant. The contextual summary shows a limited number of matches.7
Done when Each kind of sensitive data has a method, and each SIT you keep passes its matching and non-matching test files at the confidence and instance count its DLP rule uses.
Match the records you already hold, and nothing else.
- Outcome
- Match for specific known data ensured
- Microsoft’s planning guide
- 2 weeks
Exact data match (EDM) flags only data that matches a row of a reference table you upload, such as customer or patient records. Only hashed values are uploaded, and matching compares those one-way hashes. Skip this step if no row of the chooser sent you here.14, 19
-
Create the EDM classifier from a sample file
Microsoft Purview portal › Information Protection › Classifiers › EDM classifiers
- Prepare a sample file with your table’s column headings and 10 to 20 rows of representative data. Use values that aren’t sensitive: the sample isn’t hashed.4
- Check that the New EDM experience toggle is on, then select Create EDM classifier and name it.20
- Select Upload a file containing sample data. Purview reads it and proposes the schema.14, 20
Records you protect are hashed and uploaded; Purview flags content only when it matches a row. The names are Microsoft’s fictitious examples. Diagram · Microsoft Learn, Learn about exact data match based sensitive information types Watch out
The new experience pairs each schema with one SIT, and a tenant can have at most 10 schemas, so 10 EDM SITs. Use the classic experience if you need more.4
-
Choose unique primary elements mapped to a tight SIT
- Select one to 10 primary elements: columns whose values make each row unique, such as a customer ID. Don’t use columns like a first name or date of birth.20
- Under Match Validation, choose Full match.20
- Map each primary element to a SIT that matches its format closely, adds no surrounding characters, and doesn’t exclude any valid value.14, 20
Why A loose SIT on the primary element makes EDM check far too much text. The classifier then throttles or times out, and real matches are missed.4, 14
Watch out
Avoid these loose SITs as primary elements: numbers under five digits, an email SIT that doesn’t exclude your own domain, and a date of birth that allows recent or future dates. All Full Names, GUID patterns and street addresses are loose too.14
-
Set delimiters, match modes and detection rules
- Turn on Ignore delimiters and punctuation for columns such as phone numbers, so 02 9999 0000 and 0299990000 match the same row.14, 20
- Choose each column’s match mode: a SIT, Single token or Multi-token. Don’t use multi-token for a column that only ever holds one token. A multi-token field can hold up to five tokens.14, 20
- Add supporting columns as corroborating evidence. Require more of them for high-confidence rules.14
- Set proximity (300 by default) and choose Narrow Mode where you want fewer false positives, or Wide Mode.20
Watch out
If you ignore delimiters on a primary element, its mapped SIT must match the value both with and without them.20
-
Hash and upload the reference table
- Export the reference data from its source system as a .csv, .tsv or pipe-delimited file.21
- Have a Groups Administrator or User Administrator create the
EDM_DataUploaderssecurity group and add the uploading account to it.21 - Download the EDM Upload Agent (the Commercial + GCC version for most tenants). Install the same version on both computers: Microsoft’s best practice is to hash on one and upload from another.21
- On the computer that connects to Microsoft 365, open Command Prompt as an admin in the agent’s folder. Run
EdmUploadAgent.exe /Authorizeand sign in with theEDM_DataUploadersaccount.21 - Save the schema with
EdmUploadAgent.exe /SaveSchema /DataStoreName <schema name> /OutputDir <folder>.21 - On the secure computer, check the file with
EdmUploadAgent.exe /ValidateData /DataFile <data file> /Schema <schema file>.21 - Then hash it with
EdmUploadAgent.exe /CreateHash /DataFile <data file> /HashLocation <folder> /Schema <schema file> /AllowedBadLinesPercentage <value>. This writes anEdmHashfile and anEdmSaltfile.21 - Copy both files securely to the connected computer, then run
EdmUploadAgent.exe /UploadHash /DataStoreName <data store name> /HashFile <hash file path>.21 - Check the upload with
EdmUploadAgent.exe /GetDataStore, and list a store’s uploads withEdmUploadAgent.exe /GetSession /DataStoreName <data store name>.21 - Refresh the table as the source changes. It can be refreshed daily and hold up to 100 million rows.19
Check
/GetDataStorelists your data store and when it was last updated.21 -
Test the EDM SIT, then measure it in simulation
Microsoft Purview portal › Information Protection › Classifiers › EDM classifiers
- Wait an hour after the table finishes uploading and indexing, then test the EDM SIT with sample content.22
- Include samples written with and without the delimiters you chose to ignore.22
- Wait 24 hours after testing before you use the EDM SIT in a DLP policy.20
- To measure accuracy on real content, create an auto-labelling policy with the EDM SIT and run it in simulation. Review Items to review, then tune.22, 23
In DLP rules, Microsoft suggests using the EDM SIT for higher confidence and the related built-in SIT for lower confidence.19
Check Rows you hold match. Lookalike values that aren’t in your table don’t.14
Done when Each EDM SIT matches rows from your reference table in simulation, ignores lookalikes, and has a refresh schedule for its table.
Recognise documents by what they are, not by a pattern.
- Outcome
- Data along certain topics or document types with no strict patterns identified
- Microsoft’s planning guide
- 2 weeks
A trainable classifier learns a kind of document, such as a contract, from hundreds of examples. Use one when the content is specific to your organisation and SITs or Microsoft’s pretrained classifiers can’t detect it reliably.24, 25
-
Try the pretrained classifiers first
Microsoft Purview portal › Data loss prevention › Data classification › Classifiers › Trainable classifiers
- Find the pretrained classifier that fits your use case. Pretrained classifiers show as Ready to use.12, 24
- Test it on your own sample data to check it does what you need.24
- If none is accurate enough, create a custom classifier.24
Watch out
Microsoft’s built-in classifiers don’t detect every document of their kind, and your organisation stays responsible for every decision based on them.12
-
Seed a custom classifier with positive and negative samples (preview)
Microsoft Purview portal › Data loss prevention › Data classification › Classifiers › Trainable classifiers › Create trainable classifier
- Collect positive samples: documents a related pretrained classifier matched correctly, with its false positives removed and documents you found by hand added.24
- Collect negative samples: unrelated documents, plus documents the original classifier matched wrongly.24
- Put positives and negatives in separate SharePoint folders, not a Teams or OneDrive folder. Use text-based files only, and allow at least an hour for a new site to be indexed.11
- Enter the positive folder URL, then the negative folder URL, and select Create trainable classifier.11
You need at least 50 positive samples (up to 500) and at least 150 negative samples (up to 1,500). Microsoft suggests at least 200 items in your test set. Only the creator can train a custom classifier and review its predictions by default.11
Microsoft’s process for a custom classifier. The faster, automated creation flow is in preview. Diagram · Microsoft Learn, Learn about trainable classifiers -
Review the test results before you publish
-
Combine the classifier with a SIT in the rule
- Add the classifier and a SIT to the rule’s Content contains condition and choose All of these (AND), so both must match. Microsoft’s example pairs the pretrained Business context classifier (preview) with a custom SIT for contract sums over 100,000 dollars.12, 24
- For important documents, you can combine several classifiers in one rule, such as Contracts and a classifier for one project’s documents.1
- Or put the classifier and the SIT in separate groups and join the groups with AND.18
Check In simulation, the rule matches contracts above the threshold and leaves ordinary business documents alone.2
Done when Each classifier in use is pretrained and tested on your data, or custom, published and tested on at least 200 items, and paired with a SIT where context matters.
Fingerprint the templates your business fills in.
- Outcome
- Well-known document assets strictly matched
- Microsoft’s planning guide
- 2 weeks
Document fingerprinting turns a standard template, such as a contract, invoice, RFP or form, into a SIT that matches documents filled in from it. It suits organisations with standard templates that carry sensitive data.10, 26
-
Choose a few high-value templates
- Pick a small number of templates that matter most, and plan to fingerprint them at full or high match rates.26
- Check each template is between 256 and 204,800 characters of text, and not password protected or image only.10
A blank template becomes a fingerprint; a document filled in from it matches. Contoso and the inventor are Microsoft’s fictitious examples. Diagram · Microsoft Learn, Document fingerprinting Watch out
A tenant holds about 100 fingerprints. Files over 4 MB, password-protected files, image-only files and documents missing text from the original form aren’t detected.10
-
Create the fingerprint SIT
Microsoft Purview portal › Information Protection › Classifiers › Sensitive info types › Create Fingerprint based SIT
- Name the SIT and upload the template.10
- Set a partial match percentage for each confidence level, from 30% to 90%. Microsoft’s example uses 50% for low, 70% for medium and 90% for high.10, 26
- Select Next, Create, then Done. You can also create fingerprints in Security & Compliance PowerShell.10
Watch out
Text in embedded documents isn’t used, and you can’t create a second fingerprint from a template that already has one.10
-
Adjust match thresholds from real results
- If you see many false positives, raise the minimum match percentage. If you see many false negatives, lower it.26
- Set high confidence to Exact only for highly structured, critical documents. Even a small change then stops a match.10, 26
- For other templates, use a high rate rather than a full one. Saving as PDF, adding a header or footer, or upgrading the file format can change a document enough to miss a 100% match.26
Done when Each fingerprinted template matches its filled-in copies, including a PDF, at the confidence its rule uses, and ignores unrelated documents.
What detection doesn’t tell you.
- A match isn’t proof of coverage. Built-in classifiers don’t detect every item of their kind, and Content Explorer (classic) counts lag by days. A quiet policy may mean the content wasn’t scanned, not that it isn’t there.12, 13
- Existing content. A new custom SIT finds existing SharePoint and OneDrive content only after it’s recrawled. Run an on-demand scan, or wait for files to change.5, 15
- EDM in Office apps. A sensitivity label that auto-labels in Office apps can use an EDM SIT only alongside at least one SIT that doesn’t use EDM. With EDM SITs alone, auto-labelling is turned off for that label.23
- Encrypted items. Encrypted files aren’t inspected for SITs or trainable classifiers.18
- Workloads differ. The same SIT can return different results in Exchange, SharePoint and on devices. Endpoint DLP checks files against every SIT in the tenant.6
- Portal routes. The Classifiers pages sit under both Information Protection and Data loss prevention. If a path here doesn’t match your portal, try the other.5, 10
- SIT limits. A tenant can create 500 custom SITs in the portal. Each SIT can hold 20 distinct regular expressions of up to 1,024 characters.17
- Support. Microsoft support can’t help write custom classifications or regular expressions.15
- Upkeep. Remove custom SITs you no longer use, and redesign any that match most of your files.6
Fewer false alerts, and tests that show why.
- A method chosen for each kind of sensitive data, with the steps you skipped and the reason.
- SITs, EDM classifiers, trainable classifiers and fingerprints, each with its test files and simulation results.
- DLP rules whose confidence levels and instance counts you can explain from those results. Treat them as tuned detection, not proof that every sensitive item is found.12
Where each step comes from.
- 1 Microsoft Learn Reduce false positives by using SITs and advanced classifiers
- 2 Microsoft Learn Create and deploy data loss prevention policies
- 3 Microsoft Learn Test sensitive information types
- 4 Microsoft Learn Get started with exact data match sensitive information types in Microsoft Purview
- 5 Microsoft Learn Create and analyze an on-demand classification scan
- 6 Microsoft Learn Learn about sensitive information types
- 7 Microsoft Learn Increase classifier accuracy
- 8 Microsoft Learn Microsoft Purview service description
- 9 Microsoft Learn Step 1: Configure pattern-based sensitive data detection
- 10 Microsoft Learn Document fingerprinting
- 11 Microsoft Learn Get started with trainable classifiers
- 12 Microsoft Learn Trainable classifiers definitions
- 13 Microsoft Learn Get started with Content Explorer (classic)
- 14 Microsoft Learn Step 2: Implement Exact Data Match for precise matching
- 15 Microsoft Learn Create custom sensitive information types
- 16 Microsoft Learn Sensitive information type regex validators and additional checks
- 17 Microsoft Learn Sensitive information type limits
- 18 Microsoft Learn Data Loss Prevention policy reference
- 19 Microsoft Learn Learn about exact data match based sensitive information types
- 20 Microsoft Learn Create an EDM SIT schema and rule package (New experience)
- 21 Microsoft Learn Hash and upload the sensitive information source table for exact data match sensitive information types
- 22 Microsoft Learn Test an exact data match sensitive information type
- 23 Microsoft Learn Automatically apply a sensitivity label to Microsoft 365 data
- 24 Microsoft Learn Step 3: Use trainable classifiers for complex content
- 25 Microsoft Learn Learn about trainable classifiers
- 26 Microsoft Learn Step 4: Apply document fingerprinting for templates
Have your detection checked.
LensIQ reviews your SITs, classifiers and DLP rules against this model, and shows you where false positives come from and what to change.