By 1,667 total views, 3 views today
SharePoint full crawl never ends: Revealed the technical reasons Why SharePoint Search Crawler never ends- in this article, we will learn how to fix the SharePoint full crawl never ends, issue. SharePoint Full or Incremental Crawl Stuck? And also we will learn about how to forcibly stop the SharePoint search crawler using the PowerShell script. Many times, what happens is either an incremental crawl or a full crawl – if it is running, it will never stop – it gets stuck – this behavior is seen in SharePoint 2013 and as well as SharePoint 2016.
For this issue, if we try to stop the search crawl manually through the search crawl menu – it will not stop. In order to stop the search crawl, we need to execute the PowerShell script. This will stop search crawlers either incremental or full but it will end up with other issues like automatically it will again start the full crawl which will never stop and will go on infinite.
Key Highlights: SharePoint crawling never stops fix
The traditional fix for the SharePoint full crawl never ends – SharePoint 2016/2013
If we google around for the SharePoint full crawl never ends, everywhere we will get the below troubleshooting tips:
- Restart the search service and timer services in the application server (generally, the search service application runs on the application server).
- Clear config cache in SharePoint from both the application and frond end server
- Perform IIS Reset
- Server restarts
If the above solutions do not work, perform the reset search index from the central administration site. If this solution doesn’t bring the full or incremental back to normal, all experts from the community or forum suggest that we have no luck if the search service is corrupted or damaged, we must create a new search service application and delete the faulty one. Creating the new search service application and bringing back the old configuration will take cumbersome efforts.
Here is a new un-revealed solution that is not available anywhere on google where you don’t need to delete your search service application, we can fix the faulty search service application. Before that let’s understand why SharePoint’s full crawl never ends.
Why SharePoint’s full crawl never ends (SharePoint full crawl never ends)?
First, let’s understand the SharePoint Search database. When we create a search service application in the SharePoint application server, in the SQL backend, the below database will be created automatically:
- Crawl database
- Link database
- Analytics reporting database
- Search administration database
Crawl database: SharePoint full crawl never ends
This database stores tracking information and historical information about crawled items such as documents and URLs. It also stores information such as the last crawl time, the last crawl ID, and the type of update (add, update, delete) during the last crawl.
Link database:
This database stores unprocessed information that is extracted by the content processing component and information about search clicks. The analytics processing component analyzes this information.
Analytics reporting database:
This database stores the results of usage analysis.
Search administration database
This database stores search configuration data.
All the databases will be created in the below format:
- <Search_Service Application_Name>_Crawl_GUID
- <Search_Service Application_Name>_Link_GUID
- <Search_Service Application_Name>_Analytics_GUID
- <Search_Service Application_Name>_SearchAdministration_GUID
The SharePoint’s full crawl never ends issue lies in the Crawl database. If we open this database from the SQL server, we can see the below table:
- Table Name: MSSCrawlURL
In this table along with many columns, we have the column “ErrorCount” whose type is “smallint”, and the “smallint” datatype can hold a maximum value of 32767 (smallint range is “-32,768” to “32767”).
The problem is here in this “ErrorCount” column, while full crawl runs and that generates plenty of crawl errors – all details get logged to this MSSCrawlURL table. When the search service application is no more able to add any records in this table while running the full or incremental crawl as the “ErrorCount” value reached the max (32767), in this scenario, the crawler goes on the infinite loop which never ends.
By now, we have known the technical reason for the SharePoint’s crawl never ends issue, now in the below section we will learn how to fix this issue.
How to fix SharePoint’s crawl never ends issue (SharePoint full crawl never ends)?
Login to the SQL server and open the crawl database, update the “ErrorCount” column value from 32767 to 1 wherever it reached the max. If you are an expert in SQL database administration you can handle it otherwise take help from the SQL database administration team. Then using the below PowerShell scripts, stop the crawl (Full and Incremental whichever running), and restart the search and timer service in the application server (detailed steps mentioned in the below sections).
Then don’t do anything, leave it as its – from the next craw schedule the SharePoint search crawl will be running as usual without any issues.
Few Suggestions (Disclaimer):
- You can take this decision at the final when you have decided there is no other go than to delete the old SSA (search service application) and re-creating the new search service application.
- You can execute this step at your own risk, the above solution worked for us, so shared it.
SharePoint Incremental Crawl Stuck? Force stop search crawl using PowerShell
Using the below PowerShell script, we can force stop search crawl (incremental/full):
####The below script is used to stop the SharePoint Search content source crawler forcibly######.
Add-PSSnapin Microsoft.SharePoint.Powershell -ErrorAction SilentlyContinue
cls
$PSshell = Get-PSSnapin "Microsoft.SharePoint.PowerShell" -ErrorVariable err -ErrorAction SilentlyContinue
if($PSshell -eq $null)
{
Add-PSSnapin "Microsoft.SharePoint.PowerShell"
}
$fileName = "StopSearchCrawlerUsingPowerShell"
#'yyyyMMddhhmm yyyyMMdd
$enddate = (Get-Date).tostring("yyyyMMddhhmmss")
#$filename = $enddate + '_VMReport.doc'
$logFileName = $fileName +"_"+ $enddate+"_Log.txt"
$invocation = (Get-Variable MyInvocation).Value
$directoryPath = Split-Path $invocation.MyCommand.Path
$directoryPathForLog=$directoryPath+"\"+"LogFiles"
if(!(Test-Path -path $directoryPathForLog))
{
New-Item -ItemType directory -Path $directoryPathForLog
#Write-Host "Please Provide Proper Log Path" -ForegroundColor Red
}
#$logPath = $directoryPath + "\" + $logFileName
$logPath = $directoryPathForLog + "\" + $logFileName
$isLogFileCreated = $False
function Write-Log([string]$logMsg)
{
if(!$isLogFileCreated){
Write-Host "Creating Log File..."
if(!(Test-Path -path $directoryPath))
{
Write-Host "Please Provide Proper Log Path" -ForegroundColor Red
}
else
{
$script:isLogFileCreated = $True
Write-Host "Log File ($logFileName) Created..."
[string]$logMessage = [System.String]::Format("[$(Get-Date)] - {0}", $logMsg)
Add-Content -Path $logPath -Value $logMessage
}
}
else
{
[string]$logMessage = [System.String]::Format("[$(Get-Date)] - {0}", $logMsg)
Add-Content -Path $logPath -Value $logMessage
}
}
#########The below section code will continue to stop the search crawler for a particular content source until it stops #######
#Here we need to pass the content source name for which we want to stop the crawling.
$contentSourceName= "Your content source Name" #The default content source is "Local SharePoint sites"
#Get the search service application from your application or search server
$SSA = Get-SPEnterpriseSearchServiceApplication -Identity "" #If you have only one SSA in your server, no need to pass the -Identity parameter, automatically connect to default SSA.
#Get the content source
$contentSource = $SSA | Get-SPEnterpriseSearchCrawlContentSource -Identity $contentSourceName
#Check whether the crawler is not running and in "Idle" state
if($contentSource.CrawlState -ne "Idle")
{
Write-Host "Stopping the crawl for the $contentSourceName...."
$contentSource.StopCrawl()
#The below code will continue to execute till its stopped
while($contentSource.CrawlState -ne "Idle")
{
Write-Host "Waiting to $contentSourceName crawl to be stopped......." -f Yellow
sleep 10
}
write-host "The $contentSourceName crawl has been stopped Successfully!" -f Green
}
else
{
Write-Host "The search crawl is not running! for the $contentSourceName" -f Red
}
#########The content sourcer stop crawler code ends here #######
Notes:
- The above script will continue to execute the code until the particular search crawl doesn’t stop.
- Sometimes the code execution takes a longer time, for this while the code is running try to restart the SharePoint Search and Timer Job service from the server services console (service.msc).
- The code must be run inside the SharePoint server.
SharePoint Incremental Crawl Stuck? Disable all continuous crawl in SharePoint using PowerShell
Using the below PowerShell script, we can disable the continuous crawl:
#########The below code will disable all continuous content source crawl #####################
$SSA = Get-SPEnterpriseSearchServiceApplication -Identity "" #If you have only one SSA in your server, no need to pass the -Identity parameter, automatically connect to default SSA.
$contentSourceColl = $SSA | Get-SPEnterpriseSearchCrawlContentSource | Where{$_.Type -eq "SharePoint"}
foreach ($oneContentSource in $contentSourceColl)
{
$oneContentSource.EnableContinuousCrawls = $false
$oneContentSource.Update()
}
#########The below code will disable all continuous content source crawl - ends here ###########
SharePoint Incremental Crawl Stuck? Stop search crawl for All Content sources using the PowerShell Script
Using the below PowerShell script we can stop the search crawl for all running content sources:
###############The below code will stop all content sources crawl forcibly ######################################
#Get the search service application from your application or search server
$SSA = Get-SPEnterpriseSearchServiceApplication -Identity "" #If you have only one SSA in your server, no need to pass the -Identity parameter, automatically connect to default SSA.
#Get the content source collections
$contentSourceCollections = $SSA | Get-SPEnterpriseSearchCrawlContentSource
foreach ($oneContentSource in $contentSourceCollections)
{
#Check whether the crawler is not running and in "Idle" state for one content source
if($oneContentSource.CrawlState -ne "Idle")
{
Write-Host "Stopping the crawl ......"
$oneContentSource.StopCrawl()
#The below code will continue to execute till its stopped
while($oneContentSource.CrawlState -ne "Idle")
{
Write-Host "Waiting to crawl to be stopped......." -f Yellow
sleep 10
}
write-host "The crawl has been stopped Successfully!" -f Green
}
else
{
Write-Host "The search crawl is not running!" -f Red
}
}
###############The below code will stop all content sources crawl forcibly - code ends here ######################
Summary: SharePoint full crawl never ends
Thus, in this article, we have learned about how to fix SharePoint’s search full crawl never ends issue and how to stop the SharePoint search crawl (Incremental/Full) by force using the PowerShell script and also learned the hidden technical reasons for the search full crawl never end issue.
See Also: SharePoint PowerShell Tutorial
You may also like the following SharePoint PowerShell tutorials:
- Re-Provisioning SharePoint Search Service Application using PowerShell
- [Fixed]: Start SharePoint Central Administration service using PowerShell
- [Fixed]: Restore the timer service in SharePoint Server using PowerShell script
- [Fixed]: Remove-MsolServicePrincipalCredential : Access Denied. You do not have permissions to call this cmdlet.
- [Fixed]: The term ‘Get-MsolServicePrincipal’ is not recognized as the name of a cmdlet, function, script file,
- [Verified]: Find all InfoPath forms in SharePoint using PowerShell
- [Verified]: Remove recycle bin items in SharePoint using PowerShell
- [Verified]: Cancel SharePoint workflows using PowerShell
- SharePoint Online: Remove custom app using PnP PowerShell
- Office 365: How to create content type in SharePoint Online using PowerShell?
- Office 365: How to create document library in SharePoint Online using PowerShell?
- Export SharePoint user information list to CSV(Excel) file using PowerShell
- How to fix “The term ‘Get-MsolUser’ is not recognized as the name of a cmdlet”
- How to fix the “The term ‘Get-SPWeb’ is not recognized as the name of a cmdlet, function” PowerShell error
- How to hide quick launch menu in SharePoint online using PnP PowerShell
- Edit user Permission is greyed Out SharePoint Online
- Get workflow inventory from SharePoint online using PowerShell CSOM
- Create a modern team site using PnP PowerShell in SharePoint
- In 2 steps convert a classic SharePoint page to modern using PnP
- SharePoint Online: Delete All Files from document library for the given date – PowerShell CSOM
- Create SharePoint online list using PnP provisioning template
- SharePoint Automation: PowerShell script to get remote server information
- Office 365: Retrieve hub sites and associated sites using PnP Powershell
- SharePoint Online Automation – O365 – Upload files to document library using PowerShell CSOM
- SharePoint Online Automation – O365 – Create multiple items in a list using PowerShell CSOM
- SharePoint Online Automation – O365 – Update document library metadata using PowerShell CSOM
- SharePoint PowerShell
- Search Service database details in MSDN
About Post Author
