Quick Summary
To find duplicate files in SharePoint, start with built-in views and search, then use a PowerShell script with PnP to group files by name and size. Confirm true duplicates by comparing file hashes before deleting anything.
- Sort libraries by file name and size, since duplicates often sit next to each other.
- Check version history first, because a duplicate may be a legitimate older version.
- Run the PnP PowerShell script in read-only mode and export groups with more than one file to CSV.
- Use file hashes such as SHA-256 to confirm files are byte-for-byte identical.
- Back up flagged files and confirm owners before any deletion, with two-person approval for large batches.
Table of Contents
- Why Duplicate Files Pile Up in SharePoint
- Check Your Libraries First: Built-in Ways to Spot Duplicates
- PowerShell Script to Find Duplicate Files in SharePoint
- SharePoint Duplicate File Removal Tools: Native vs. Third-Party
- How Versioning Affects Duplicate Detection
- Removing Duplicates Safely: Risk Controls Before You Delete
- SharePoint Storage Management Best Practices to Prevent Duplicates
- Scheduling Your First Duplicate Cleanup Review with MTRC
- Frequently Asked Questions
Why Duplicate Files Pile Up in SharePoint
Last Updated: October 30, 2026
Duplicate files are one of the most common SharePoint problems in mid-market organizations. A team copies a file to "make a quick change," someone uploads a new version under a slightly different name, and a sync glitch creates a second copy nobody notices. Within a year, a single library can hold dozens of near-identical documents.
Duplicates cost more than storage. They confuse people about which file is current, slow down search, and create compliance risk when outdated contracts or policies stay in circulation.
Several everyday habits drive the problem:
- Copy-and-paste workflows that duplicate files instead of moving them
- Downloading and re-uploading a file after editing it offline
- Sync conflicts from OneDrive or the SharePoint sync client
- Migrations that bring over old file shares without cleanup
- Multiple sites with no shared naming standard
Knowing the cause helps you pick the right fix. A one-time cleanup works for migration leftovers, but a habit like re-uploading needs a change in process.
Check Your Libraries First: Built-in Ways to Spot Duplicates
Before you install anything, use the tools SharePoint already gives you. Start by sorting each library by file name and size. Duplicates often sit next to each other, and identical sizes are a strong hint.
Next, check these built-in options:
- Sort and filter views by name, modified date, or file size
- Search for file name fragments like "copy," "final," or "v2"
- Check version history to see whether a "duplicate" is really an older version
- Review recycle bin contents to avoid restoring files you already deleted
Be careful with version history. A file that looks like a duplicate may be a legitimate older version that your team needs for audits. Treat versions and copies as different problems.
PowerShell Script to Find Duplicate Files in SharePoint
When libraries hold thousands of files, manual checks break down. A PowerShell script using PnP PowerShell can list every file in a library and group them by name and size. This gives you a report you can review before touching anything.
The basic approach works in four steps:
- Connect to the site with PnP PowerShell
- Pull all file items from the target library
- Group items by file name and size
- Export any group with more than one file to a CSV report
Run the script in read-only mode first. Review the CSV with the document owners before you delete anything.
Comparing File Hashes to Confirm True Duplicates
Matching names and sizes is a good first filter, but it can miss real duplicates or flag false ones. Two files can share a name and size yet differ inside. A file hash, such as a SHA-256 checksum, compares the actual content.
Download the candidate files, compute the hash for each, and group by hash value. Matching hashes mean the files are byte-for-byte identical. This is a stronger test, but it takes more time and bandwidth, so reserve it for suspected duplicates.
Automating Duplicate Cleanup on a Schedule
Once your review process works, you can schedule the script to run weekly or monthly. Use a service account with the least permissions needed, and have the script write its report to a review location instead of deleting files automatically.
Automation should flag duplicates, not remove them. A person should approve every deletion.
SharePoint Duplicate File Removal Tools: Native vs. Third-Party
You have two broad paths: native SharePoint features and scripts, or third-party tools. Each has trade-offs.
| Option | Best For | Main Trade-off |
|---|---|---|
| Native views and search | Small libraries, quick checks | Manual and slow at scale |
| PowerShell with PnP | Custom reports, repeatable checks | Needs scripting skills to maintain |
| Third-party tools | Large estates, guided cleanup | Added licensing cost and vendor review |
Native options cost nothing extra and keep data inside your tenant. Third-party tools often add dashboards and safer deletion workflows, but they require a security review and a budget line.
Whichever path you choose, confirm the tool respects SharePoint permissions and keeps an audit trail.
How Versioning Affects Duplicate Detection
Versioning changes what counts as a duplicate. When versioning is on, SharePoint stores each saved change as a version of the same file. A matching name and size across versions does not mean a duplicate exists.
Your detection logic should exclude version history from the comparison, or it will flag the same document many times. Conversely, a file with many versions can hide a real duplicate elsewhere in the library.
Check your version limits too. Heavy versioning inflates storage use, and that can make duplicate cleanup look more urgent than it is. Review retention settings before you decide what to remove.
Removing Duplicates Safely: Risk Controls Before You Delete
Deleting the wrong file can break a workflow, a link, or an audit trail. Put these controls in place before any cleanup:
- Back up or export the library or flagged files first
- Confirm the owner of each duplicate before removal
- Check links and workflows that may point to the file
- Delete in batches and monitor for complaints
- Keep deleted items in the recycle bin long enough to restore if needed
Use a two-person approval for large deletions. One person runs the report and another reviews the list.
A quick check helps: before deleting, search the file name across the site and confirm no Power Automate flow or page references it.
SharePoint Storage Management Best Practices to Prevent Duplicates
Cleanup fixes the past. Prevention keeps the problem from returning. These habits help most:
- Set a naming standard with version numbers or dates in the file name
- Use check-out and check-in for documents that change often
- Edit in the browser instead of downloading and re-uploading
- Set retention and version limits that match your policy
- Assign a library owner responsible for periodic reviews
- Move files, don't copy them, when reorganizing content
Good governance makes prevention stick. Document these rules and train the team on them, so duplicates stop being a recurring surprise.
Scheduling Your First Duplicate Cleanup Review with MTRC
Start small: pick one busy library, run a read-only report, and review the results with its owners. Set a monthly review on the calendar so cleanup becomes routine instead of a one-time project.
If your sprawl spans many sites, or you want a repeatable process your team can run without outside help, MTRC Enterprises can help you design the script, the review workflow, and the training so your staff can own it. Our engagements use fixed-price scopes with clear deliverables, and we focus on hands-on setup and knowledge transfer. For current pricing, contact MTRC Enterprises for a quote. You can also learn how to organize documents in SharePoint Online to reduce duplicates from the start.
Frequently Asked Questions
Can I use PowerShell to find duplicate files in SharePoint?
Yes. PowerShell with the PnP.PowerShell module can connect to a SharePoint Online site, enumerate files in a document library, and group them by name, size or file hash. The script returns a list of candidate duplicates that you review before taking any action. Run it first against a test site or a copy of the library, confirm the account has only the permissions it needs, and export results to CSV so the team can verify each match before anything is deleted.
Does SharePoint automatically detect duplicate files?
SharePoint does not automatically remove or merge duplicate files. Files can have the same name in different folders, or identical content under different names, and SharePoint treats each as a separate item. Duplicates typically appear through sync conflicts, repeated uploads, or users copying files between sites. Detection requires a view-based search, a script, or a third-party tool that compares names, sizes or hashes.
How do duplicate files affect SharePoint storage limits?
Duplicate files count against your tenant storage quota just like any other file, and version history can multiply the impact because each version consumes space. Organizations with large libraries often find that a meaningful share of storage is redundant copies. Reviewing storage reports in the SharePoint admin center and cleaning up duplicates before a quota warning appears gives you more control over the timing and the decisions.
What is the best way to clean up duplicate files in Microsoft 365?
Start with an inventory: identify duplicates by name, size and hash within each library, then confirm with the file owners which copy is authoritative. Move suspected duplicates to a review folder rather than deleting them, keep a rollback path such as the recycle bin and version history, and remove files in small batches. Document the process so IT and operations teams can repeat it on a schedule.
