Error during Content DB Mount and Dismount - SharePoint

o    Stated tried moving the central admin to other server and they wanted to run PSCONFIG, so they dismounted all the DB's.

o    After PSCONFIG was successfully, trying to mount the db.’s, where they are getting below error. Error while mounting the DB:

Mount-Spcontentdatabase: An item with the same key has already been added

o    We checked with Get-spcontentdatabase, list of attached DBS, we found there was no content db. present in that list 'Test_DB001' which is trying to attach.

o    We collected the web application id by below command

get-spwebapplication | select name, id

o    We ran the below query on SQL end to find the orphan GUID

select * from dbo.Objects where id = GUID '

o    Found around 85 orphan databases.

o    We ensured with you if they have good known backup of the content database, you confirmed they have.

o    We ran the below command to check if the Orphan GUID getting deleted

$wa = Get-SPWebApplication ABC

$wa.ContentDatabases.Delete(GUID)

o    We checked with SQL query and compared, found above Orphan guid were deleted.

$guids ="…"

ForEach($guid in $guids){

$wa.ContentDatabases.Delete($guid)

}

o    Ran the above script successfully and took results of the SQL Query (select * from dbo.Objects where id = GUID)

o    We tried to mount the content database 'Test_DB001'  and successfully mounted the DB.

SharePoint: All about one-time timer jobs

Issue:

--------------

They are trying to remove the Faulty search service application "Search service Application AB"

Used below command to delete the SSA

$app=Get-SPServiceApplication | ? {$_.displayname –eq “search service application_AB"}
$app.delete()

$spapp = Get-SPServiceApplication -Name "Search Service Application_AB"

Remove-SPServiceApplication $spapp -RemoveData

Ran PowerShell script to check the history and entries:https://blogs.technet.microsoft.com/spjr/2017/08/16/sharepoint-all-about-one-time-timer-jobs/

Issue during search index component movement + SharePoint

Issue : Command Set-SPEnterpriseSearchTopology -Identity $Clone is stuck and endless running.

Analysis : We see the topology is changed successfully from OWS Timer category of ULS log . So we performed some basic checking, it looks like everything is running fine and is healthy.  It sounds like that the timer job doesn’t signal the PowerShell process when activating is finished. So looks like the communication issue between timer job and powershell process. So we decide to turn all timer service off from all the other servers other than the admin component server and the new component server. And it is working well. Later, we start all the timer service again. Running the script again, all are working fine. So the only difference compared with fail scenario is that the restarting of the timer service. Hence we have a solution below.

Solution :

1. Restart all timer service for all servers.

2. Then using below command to verify its status

 

$farm  = Get-SPFarm

$disabledTimers = $farm.TimerService.Instances | where {$_.Status -ne "Online"}

if ($disabledTimers -ne $null)

{

    foreach ($timer in $disabledTimers)

    {

        Write-Host "Timer service instance on server " $timer.Server.Name " is not Online. Current status:" $timer.Status

        Write-Host "Attempting to set the status of the service instance to online"

        $timer.Status = [Microsoft.SharePoint.Administration.SPObjectStatus]::Online

        $timer.Update()

    }

}

else

{

   Write-Host "All Timer Service Instances in the farm are online! No problems found"

}

SharePoint External Zone Site Down

Symptom:

• After adding the new server to the farm and pointing the request using WFE, its failing Server application on _trust URL

Following error is logged:

Application error when access /_trust, Error=An XML error has occurred.

at Microsoft.SharePoint.SPSaml11BaseTokenHandler.ReadAssertion(XmlReader reader)

at Microsoft.IdentityModel.Tokens.Saml11.Saml11SecurityTokenHandler.ReadToken(XmlReader reader)

Cause:

• There’s a Ping Federate setting “Include Raw Key in KeyValue”. If that is enabled, it will add a second RSA key to the SAML assertion causing the issue.

Resolution:

• You need to change this setting as per the following article:

Reference : https://joshroark.com/sharepoint-saml-auth-login-error-there-are-multiple-keys-on-the-token/

• After you changed the setting the issue got resolved.

Unable to retrieve topology component health states + SharePiont Search Issue

While configuring the SharePoint 2013 Search Service Application using Central Administration. I got the above error:- "Unable to retrieve topology component health states"

  

But If we create the SharePoint 2013 Search Service Application using Powershell script which prompt lots of other related error like:-

1. SharePoint 2013 Search Topology Activation Error: "Unable to retrieve topology component health states. This may be because of the admin component is not up and running".
2. Exception calling "Activate" with "0" argument(s): "Topology does not contain any components of type Microsoft.Office.Server.Search.Administration.Topology.AdminComponent.
3. Crawling Stuck with Starting or Stopping Mode.

    Corrective Action should be considered:-

           1.     Checked User have Farm Administrator Permission or not.

2.       Checked User have Managed Account or not.

3.       Checked the User have Local Administrator Privileges or not.

4.       Checked User have owner permission in all Database as created during search topology.

5.       Checked extra space in Environment variable i.e. PSModulePath – No space exists.

6.       Checked the Managed Metadata Service and user permission

7.       Checked the Health and Usage Services and user permission

8.       Re-configured the Health and Usage Services.

9.       Re-start the Application Server.

10.   Window Service in Application Server or  WFE is running fine with same account & Re-started the service in all server also.

      Window Services :- SharePoint  Search Host Controller & SharePoint Server Search 15

11.   Same user having Farm Administrator level permission and having managed account also.

12.   All DB are also up and online.

But we need correctly check the Noderunner.exe.config - Might be someone has changed the value which may create issue - This was the exactly the case for one of project:-

Resolution :-

1.       noderunner.exe.config file located at “C:\Program Files\Microsoft Office Servers\15.0\Search\Runtime\1.0” had a wrong value i.e. 100" />  It should be .

Location of noderunner.exe.config

Noderunner.exe.config file (Should be)

In the nodeRunnerSettings node edit the attribute “memoryLimitMegabytes”. The initial value is set to “0”, what means that there is no limit.

Unable to reorder content type + SharePoint

Have user copy the list template from another site and want to reorder the content type, when click on the column order at the content type you will see the error :

Object reference not set to an instance of an object.   at Microsoft.SharePoint.ApplicationPages.ChangeFieldOrderPage.OnLoad(EventArgs e)

   at System.Web.UI.Control.LoadRecursive()

   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

Medium        Application error when access /_layouts/15/ChangeFieldOrder.aspx, Error=Object reference not set to an instance of an object.   at Microsoft.SharePoint.ApplicationPages.ChangeFieldOrderPage.OnLoad(EventArgs e)     at System.Web.UI.Control.LoadRecursive()     at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)        

Unexpected        System.NullReferenceException: Object reference not set to an instance of an object.    at Microsoft.SharePoint.ApplicationPages.ChangeFieldOrderPage.OnLoad(EventArgs e)     at System.Web.UI.Control.LoadRecursive()     at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)        

Getting Error Message for Exception System.Web.HttpUnhandledException (0x80004005): Exception of type 'System.Web.HttpUnhandledException' was thrown. ---> System.NullReferenceException: Object reference not set to an instance of an object.     at Microsoft.SharePoint.ApplicationPages.ChangeFieldOrderPage.OnLoad(EventArgs e)     at System.Web.UI.Control.LoadRecursive()     at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)     at System.Web.UI.Page.HandleError(Exception e)     at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)     at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)     at System.We...        

...b.UI.Page.ProcessRequest()     at System.Web.UI.Page.ProcessRequest(HttpContext context)     at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()     at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)     at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Work around:-

you will not be able to manage the order of your columns on the list settings page again until you switch “Allow management of content types” back to No. When the management of content types if enabled, you can then reorder the columns for each specific content type. You may or may not want this to be your standard setup, so my best practice here would be to turn this feature back off when you are finished managing the content type.

Detail step:-

1. change the list setting at Advanced settings and change the Allow management of content types to no. > click ok

2. under the column, you will see Column ordering. Click on the column ordering and change the order.> click ok when you done

3. change the list setting at Advanced settings and change the Allow management of content types to yes > click ok

Guessing of the root cause:-

I think because the content type is attach to the template and no content type created for the sites. once you click on ordering, then will go to search the content type gallery at site level. once cannot found it and error message will show at above. Iguess is consider bug :)

Orphans Object Clean Up in SharePoint

Orphaned Objects are items that do not have any parent or child relationship and remain in the content database uncleaned. When  you perform the Test-SPContentDatabase to validate a content database you might receive errors which provides information about orphans in the content database

These orphaned objects need to be removed on the below situations:

1. Before applying service packs or cumulative updates to the SharePoint servers

2. Before performing a migration from a older version of SharePoint to a newer version.

3. Before reapplying templates or performing any updates to the database.(certain powershell commands would not work when running update scripts on the database.  ex. Applying modification to a Custom Master Page that is already deployed to the sites). The execution of powershell scripts will fail when iterating the tables for fetching or applying update to the data in SharePoint content databases

There are 2 types of Orphan Objects

1. Config Orphans (Orphan Site exist in Configuration Database but does not have any child item in Content Database). Central Admin ->View All Site Collections shows the site info, but the actual site cannot be accessed via browser. 

2. Content Orphans (Orphans include Lists without Parent sites, Documents with no parent document library, Web Parts that does not have an assembly, Web Pages with no Parent Site E.t.c..). Site Collections that are available in content database but not in config db (Will not show in Central Admin -> View All Site Collections page)

  

A.    Config Orphan Object Removal:

Cleaning the config orphan is the easiest one.

Simply detach the content database that has the orphan site from your farm and attach it back. This will refresh and remove the stale entries.

B.     Content Orphan Object Removal

There are 2 ways to remove content orphans from the content database

1. Use SharePoint Health Analyzer

2. Use STSADM or PowerShell Commands

1.      SharePoint Health Analyzer 

(Note : Not recommended in a Production farm as it sometimes might cause SQL Blocking if there are very large content databases)

Go to Central Admin - > Monitoring > Health Analyzer Section > Review Problems and Solutions > Content Database Contain Orphaned Items - Click on the Ribbon Command "Repair Automatically".

2.      Using STSADM Commands

I have provided two scripts to first log the orphans in text file.(STSADM Method)

When you execute the first script, this will list out the orphaned objects in text file

The second one will delete the orphans.

This method will delete only when you have orphan sites in one content database.

Modify the site collection name and database name related to the STSADM command line. 

To Check Orphans (Store the below content in a .Bat file)

echo on

rem ***  set local environment variables

setlocal

set LogFile="D:\LogOrphans.txt"

c:

cd "\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\BIN" >> %LogFile%

Echo "List Orphans ...." >> %LogFile%

stsadm.exe -o databaserepair -url https://contoso.com/sites/sitecollection -databasename Content10_ContentDB  >> %LogFile%

Echo "************** E N D*********" >> %LogFile%

pause

PowerShell Command to check orphans:

$contentdb = Get-SPContentDatabase | Where-Object {$_.Name –match "Content_Contentdb"}

test-spcontentdatabase $contentdb -showrowcounts | Format-Table -Wrap | Out-File -FilePath c:\testcontent.txt

To Delete Orphans(Store the below content in a .Bat file):

echo on

rem ***  set local environment variables

setlocal

set LogFile="D:\LogOrphans.txt"

c:

cd "\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\BIN" >> %LogFile%

Echo "List Orphans ...." >> %LogFile%

stsadm.exe -o databaserepair -url https://contoso.com/sites/sitecollection -databasename Content10_ContentDB -deletecorruption  >> %LogFile%

Echo "************** E N D*********" >> %LogFile%

pause

The Ouput in log File below:

"List Orphans ...." 

 

 

 

 

" />

"************** E N D*********" 

Note:

In some scenarios certain orphan object might not get removed, but still show that it is removed and will remain in the database. 

To validate this, run the below command to list the objects in the content database and cross verify with the log files

stsadm -o enumallwebs -databasename Content10_ContentDB 

Output of the above Stsadm command

 

   

     

   

 

 

   

     

   

 

If the property InSiteMap=False then the site collection is an orphan.

Final Validation:

If you still have the orphaned objects listed, then try the below options and then validate again with the log file

1. Disconnect the database and reconnect the database so that the references will be removed.

2. Try removing the corrupted list or item using the sharepoint designer tool.

3. Try deleting the site using below Stsadm commands

stsadm -o deletesite -force -url http://contoso.com/sites/contososubsite11 -databaseserver ds -databasename contentdb1

(or)

stsadm -o deletesite -force -siteid e2a114b8-80c9-41f6-87bf-3feddf2ad9b6 -databaseserver ds -databasename contentdb1

4.  Try running the below Powershell commands

(Get-SPContentDatabase ).Repair($true)

Ex:

(Get-SPContentDatabase Collab_ContentDB).Repair($true)

SharePoint Admin Topics

SharePoint  Admin Daily / Weekly / Monthly Activities

§  Managing and checking the overall server health and functionality

§  Monitoring SharePoint disk space usage through the built-in SharePoint reports for each site collection

§  Managing SharePoint permissions

§  Analyzing and reporting upon SharePoint usage and activity

§  Moving/copying sites

§  Supporting network load balancing needs and ensuring its correct operation (NLB)

§  Regular review of the events and messages reported in Event Viewer and Performance Monitor

§  Regular review, clean-up, management and configuration of SharePoint accounts and sites. This portion of the role will work closely with an Active Directory administrator if they are separated.

§  Regularly analyzing SharePoint content and storage

§  Monitoring SharePoint trends (e.g. site usage and growth, disk space usage and growth)

§  Setting up alerts and enforcing policies

§  Regularly auditing your SharePoint environment

§  Identifying and reporting governance violations

§  Checking for operating system, SQL Server and SharePoint patches and cumulative updates.

One-time / First-time Farm Activities

1. Determine general patch approach for applying patches

2. Windows O/S (Windows Update)

3. SQL Server (Service Packs)

4. Document the overall system architecture

5. Document the disaster recovery plan for data center, server, or document recovery

6. Determine search architecture (What is indexed?, how often?, etc.)

7. Determine design continuity (What templates are used for each area)

8. Determine best practices for creation of Sites, Site Collections, and content databases

9. Determine strategy for business intelligence in SharePoint (upon dashboards, where is

the data stored?, how is it presented?, etc.)

10. Create SharePoint Service Level Agreement

11. Create best practices document for taxonomies

12. Create best practices document on security configuration

13. Create SharePoint roles document

14. Determine who can access the site (internal only? customers?)

15. Determine how partners and users will navigate the sit

16. Determine the process to request a site

17. Determine the process to request access to sites

18. Create best practices document to explain how and when to publish a document to a

global audience (should they be PDFs?)

19. Create feedback and/or suggestion mechanism for site enhancements

20. Determine naming conventions

21. Develop process for FAQ (users and help desk)

22. Develop training process

Daily Activities

1. Review Windows Event Logs for high-priority issues

2. Review the SharePoint Logs for high-priority issues

3. Monitoring Third party tool, Services, web sites, applications

4. Tickets ,XIS Reports, Calls, Think about new PowerShell Script (Such as Automation , Monitoring tools)

5. Workflow task, web site permission issue , make daily To-do list , IIS Logs , UlS Logs, Think about innovation ideas.

6. Search about SharePoint Blogs , Article, Forums and utilizes resolved the issues. 

Weekly Activities

1. Review Search Usage reports

2. Attempt searches upon typical end-user searches – verify they work properly

3. Verify Alerts are functioning properly

4. Enhance search through audit of Search patterns (implementation of noise words, best bets,

keywords, thesaurus entry, etc.)

5. Review and maintain current BI cubes and web parts (if applicable)

6. Audit server utilization (disk IO, disk space, CPU, etc.) and update baseline

7. Review past week questions and issues and update FAQ

8. Ensure off-site backup procedure working properly

Monthly Activities

1. Review overall server architecture based on current use

2. Audit individual server design (each server)

3. Review released patch list for Windows Operating System, SQL Server and SharePoint

4. Apply patches per the pre-defined patch approach

5. Review patches or maintenance to third-party controls (web parts, iFilters, etc.)

4. Review overall server architecture diagrams and documentation for updates and revisions

5. Review search architecture for new areas to index, crawl or exclude

6. Report on SharePoint uptime and SLA compliance

7. Review security hierarchy

8. Review new functions or sites deployed. Determine if training needs to be updated

9. Verify backups are valid and contain data

Quarterly Activities

1. Review company disaster recovery plan

Annual Activities

1. Exercise a company disaster recovery plan

2. Plan for upgrade: SQL, Windows, SharePoint 2016, PU, CU, Production end user cleanup task (site, dbs, uls, old user, ups cleanup task)