Dennis Gorelik (![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png)
dennisgorelik) wrote2016-08-10 11:11 pm
dennisgorelik) wrote2016-08-10 11:11 pm
Entry tags:
Remote PowerShell
We tried to run PowerShell remotely (in order to automate build deployment).
We managed to make it work on developers' machines, but on production server it just refuses to work:
----------------------------------------------------------------
C:\Windows\system32>powershell
Windows PowerShell
PS C:\Windows\system32> Enable-PSRemoting -SkipNetworkProfileCheck -Force
PS C:\Windows\system32> Enter-PSSession -ComputerName localhost
----------------------------------------------------------------
Note "Unable to check the status of the firewall." part of the message.
Why would Enable-PSRemoting command try to check the status of the firewall?
Another Windows WTF reported by yatur
Update
Finally we were able to fix this remote powershell issue.
The problem was in Group Policy for IpV4Filter on our production machine.
IpV4Filter was limited to a single IP address (main address of that production machine).
I have no idea why it was setup that way.
This is how I fixed WinRM localhost access problem:
In the end, PowerShell and Microsoft server tools leave a negative impression due to bugs and pathetic diagnostic.
Consider another PowerShell surprise:
"ls" and "dir" commands produce empty output in case when folder is empty. No headers, no message that says there are no files. Just nothing. WTF?
We managed to make it work on developers' machines, but on production server it just refuses to work:
----------------------------------------------------------------
C:\Windows\system32>powershell
Windows PowerShell
PS C:\Windows\system32> Enable-PSRemoting -SkipNetworkProfileCheck -Force
WinRM is already set up to receive requests on this computer. Set-WSManQuickConfig : <f:WSManFault xmlns:f="http://schemas.microsoft.com/wbem/wsman/1/wsmanfault" Code="2" Machine="localhost"><f:Message><f:ProviderFault provider="Config provider" path="%systemroot%\system32\WsmSvc.dll"><f:WSManFault xmlns:f="http://schemas.microsoft.com/wbem/wsman/1/wsmanfault" Code="2" Machine="sv7731"><f:Message>Unable to check the status of the firewall. </f:Message></f:WSManFault></f:ProviderFault></f:Message></f:WSManFault> At line:65 char:17 + Set-WSManQuickConfig -force -SkipNetworkProfileCheck + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidOperation: (:) [Set-WSManQuickConfig], InvalidOperationException + FullyQualifiedErrorId : WsManError,Microsoft.WSMan.Management.SetWSManQuickConfigCommand
PS C:\Windows\system32> Enter-PSSession -ComputerName localhost
Enter-PSSession : Connecting to remote server localhost failed with the following error message : Access is denied. For more information, see the about_Remote_Troubleshooting Help topic. At line:1 char:1 + Enter-PSSession -ComputerName localhost + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidArgument: (localhost:String) [Enter-PSSes sion], PSRemotingTransportException + FullyQualifiedErrorId : CreateRemoteRunspaceFailed
----------------------------------------------------------------
Note "Unable to check the status of the firewall." part of the message.
Why would Enable-PSRemoting command try to check the status of the firewall?
Another Windows WTF reported by yatur
Update
Finally we were able to fix this remote powershell issue.
The problem was in Group Policy for IpV4Filter on our production machine.
IpV4Filter was limited to a single IP address (main address of that production machine).
I have no idea why it was setup that way.
This is how I fixed WinRM localhost access problem:
Run gpedit.msc Local Computer Policy Computer Configuration Administrative Templates Windows Components Windows Remote Management (WinRM) WinRM Service Allow remote server management through WinRM In "IPv4 filter:" change "208.43.198.72" to "*": IPv4 filter: *
In the end, PowerShell and Microsoft server tools leave a negative impression due to bugs and pathetic diagnostic.
Consider another PowerShell surprise:
"ls" and "dir" commands produce empty output in case when folder is empty. No headers, no message that says there are no files. Just nothing. WTF?
no subject
Set-Item -Force WSMan:\localhost\Client\AllowUnencrypted true
Set-Item -Force WSMan:\localhost\Client\TrustedHosts *
In the firewall on the server, check the ports 5985 (WinRm HTTP) and maybe also 5986 (WinRM HTTPS).
no subject
2) Creating "Windows Remote Management (HTTP-In)" firewall rule helped to fix "Enable-PSRemoting" command:
---
PS C:\Windows\system32> Enable-PSRemoting -Force
WinRM is already set up to receive requests on this computer.
WinRM is already set up for remote management on this computer.
---
It is still a mystery why Enable-PSRemoting command cares about "Windows Remote Management (HTTP-In)" firewall rule.
And if Enable-PSRemoting checks for that firewall rule, why Enable-PSRemoting cannot create that rule in case if that rule is missing.
Or, at least, why Enable-PSRemoting cannot explicitly complain about missing "Windows Remote Management (HTTP-In)" firewall rule.
Enable-PSRemoting is able to enable disabled "Windows Remote Management (HTTP-In)" firewall rule (but not create it).
3) I successfully ran your commands:
------------------------------------
PS C:\Windows\system32> Set-Item -Force WSMan:\localhost\Client\AllowUnencrypted true
PS C:\Windows\system32> Set-Item -Force WSMan:\localhost\Client\TrustedHosts *
------------------------------------
That did not help running "Enter-PSSession" command:
============================
PS C:\Windows\system32> Enter-PSSession -ComputerName localhost
Enter-PSSession : Connecting to remote server localhost failed with the following error message : The client cannot connect to the destination specified in the request. Verify that the service on the destination is running and is accepting requests. Consult the logs and documentation for the WS-Management service running on the destination, most commonly IIS or WinRM. If the destination is the WinRM service, run the following command on the destination to analyze and configure the WinRM service: "winrm quickconfig". For more information, see the about_Remote_Troubleshooting Help topic. At line:1 char:1 + Enter-PSSession -ComputerName localhost + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidArgument: (localhost:String) [Enter-PSSes sion], PSRemotingTransportException + FullyQualifiedErrorId : CreateRemoteRunspaceFailed============================no subject
3) If it's not domain-joined, you have to use the option -Credential. Just give it the user name (like "~\administrator") and it will prompt you in the GUI for the password. Or generate the credential object from the name and password:
function mkcr { <# .SYNOPSIS Make a credentials object out of the user name and password. #> param( ## User name. [string] $user, ## Password in plain text. [string] $passwd ) $secpwd = ConvertTo-SecureString -Force -AsPlainText $passwd New-Object System.Management.Automation.PSCredential @($user, $secpwd) }then use -Credential (mkcr ".\administrator" "password")
For some weird reason the GUI form doesn't allow the syntax with "." for the host part in the username but does allow "~". If you don't use the host part on a non-domain-joined server, your connection will be very slow.
Setting up an HTTPS connection is a whole separate world of pain :-)
no subject
I entered password, but it did not change anything - it still returned the same generic error:
"Enter-PSSession : Connecting to remote server localhost failed with the following error message : The client cannot connect to the destination specified in the request. ....."
I actually do NOT need -Credential parameter in order to Enter-PSSession on another Windows computer (which is also "not domain-joined").
Should I try reinstalling PowerShell?
no subject
no subject
They do not cover such obvious problem as an inability to run "Enter-PSSession -ComputerName localhost".
My guess is that some components are broken, but WinRS is unwilling to diagnose that problem.
So the best course of action is to either reinstall PowerShell or just move to another server with clean Windows install.
Windows remoting sucks.
no subject
Just in case, did you reboot the machine? If not, you can try restarting the WinRM service, i.e. "restart-service WinRM". Or rebooting.
Have you tried running "winrm quickconfig"? If not, try it. You can also test the connection by "winrm identify" (run it with /? to see the help).
no subject
But restarted WinRM:Restarting WinRM did not help.
Do you think full reboot still may help?
Usually we reboot our server once per month (when we install windows updates).
2) Do you know why Windows tools are so poor at diagnosing problems? Is it intentional?
no subject
BTW, try this:
PS C:\> dir WSMan:\localhost\Listener\ Type Keys Name ---- ---- ---- Container {Transport=HTTP, Address=*} Listener_1084132640Does your result look anything like this?
Look in WSMan:\localhost\Service\ , does it look reasonable? In particular, what does WSMan:\localhost\Service\Auth\ contain for the authentication "negotiate"? What about WSMan:\localhost\Client?
Here is an example of what works for me:
(this machine also has CredSSP enabled, you don't really need that for the basic connections).
2) Looks like just nobody cares, it's not considered an important property and requires extra effort. The developers tend to create the one-off solutions for the problems that reach them, and these get occasionally published, so you can find a whole lot of the workaround recipes on the internet. Everything is built as workarounds on top of workarounds. So try going through the support channel, maybe you'll get a solution.
no subject
I noticed couple of differences:
1) CredSSP is false for me and true for you. But you expected this.
2) IPv4Filter for you is "*", but for my server is "GPO 208.43.198.72"
Could that be the culprit?
On my local machine (where remote PowerShell works), I, like you, have
"IPv4Filter *"
How do I change IPv4Filter?
PS C:\> dir WSMan:\localhost\Listener\ WSManConfig: Microsoft.WSMan.Management\WSMan::localhost\Listener Type Keys Name ---- ---- ---- Container {Transport=HTTPS, Address=*} Listener_1305953032 Container {Transport=HTTP, Address=*} Listener_1084132640 PS C:\> dir WSMan:\localhost\Client WSManConfig: Microsoft.WSMan.Management\WSMan::localhost\Client Type Name SourceOfValue Value ---- ---- ------------- ----- System.String NetworkDelayms 5000 System.String URLPrefix wsman System.String AllowUnencrypted true Container Auth Container DefaultPorts System.String TrustedHosts * PS C:\> dir WSMan:\localhost\Client\Auth\ WSManConfig: Microsoft.WSMan.Management\WSMan::localhost\Client\Auth Type Name SourceOfValue Value ---- ---- ------------- ----- System.String Basic true System.String Digest true System.String Kerberos true System.String Negotiate true System.String Certificate true System.String CredSSP false PS C:\> dir WSMan:\localhost\Service\Auth\ WSManConfig: Microsoft.WSMan.Management\WSMan::localhost\Service\Auth Type Name SourceOfValue Value ---- ---- ------------- ----- System.String Basic false System.String Kerberos true System.String Negotiate true System.String Certificate false System.String CredSSP false System.String CbtHardeningLevel Relaxed PS C:\> dir WSMan:\localhost\Service\ WSManConfig: Microsoft.WSMan.Management\WSMan::localhost\Service Type Name SourceOfValue Value ---- ---- ------------- ----- System.String RootSDDL O:NSG:BAD:P(A;;GA;;;BA)(A;;GR;;;IU)S:P(AU;FA;GA;;;WD)(AU;SA;GXGW;;;WD) System.String MaxConcurrentOperations 4294967295 System.String MaxConcurrentOperationsPerUser 1500 System.String EnumerationTimeoutms 240000 System.String MaxConnections 300 System.String MaxPacketRetrievalTimeSeconds 120 System.String AllowUnencrypted false Container Auth Container DefaultPorts System.String IPv4Filter GPO 208.43.198.72 System.String IPv6Filter GPO System.String EnableCompatibilityHttpList... false System.String EnableCompatibilityHttpsLis... false System.String CertificateThumbprint System.String AllowRemoteAccess GPO trueno subject
no subject
PS C:\> Set-Item -Force WSMan:\localhost\Service\IPv4Filter * Set-Item : The config setting IPv4Filter cannot be changed because is controlled by policies. The policy would need to be set to "Not Configured" in order to change the config setting. At line:1 char:1 + Set-Item -Force WSMan:\localhost\Service\IPv4Filter * + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [Set-Item], InvalidOperationException + FullyQualifiedErrorId : System.InvalidOperationException,Microsoft.PowerShell.Commands.SetItemCommandSo yes - some group policy is responsible for that.
I just need to find out how to find and change that Group Policy...
no subject
Finally fixed - IpV4Filter Global Policy configuration
I updated this post.
---
Run gpedit.msc
Local Computer Policy
Computer Configuration
Administrative Templates
Windows Components
Windows Remote Management (WinRM)
WinRM Service
Allow remote server management through WinRM
In "IPv4 filter:" change "208.43.198.72" to "*":
IPv4 filter: *
---
Re: Finally fixed - IpV4Filter Global Policy configuration
https://blogs.msdn.microsoft.com/wmi/2010/03/16/collecting-winrm-traces/
Re: Finally fixed - IpV4Filter Global Policy configuration
Then I sent remote powershell command from my dev machine to production:
Invoke-Command -ComputerName 208.43.198.72 -Script { cd \; ls }"Operational" section got multiple events ("None", "WSMan API call", "Response handling", "Request handling").
"Analytic" section got lots of events ("Response handling, "None", "WSMan API call").
"Debug" section got no events at all.
When I re-executed powershell commands, "Operational" section got more entries, but "Analytic" section got no new entries.
"Debug" section stays empty.
I did not find any single error message in these logs.
These logs could have helped with diagnostic, but they are clearly not reliable.
Were you able to find something useful in these Windows Remote Management logs?
Re: Finally fixed - IpV4Filter Global Policy configuration
"Operational" should be like the normal syslog level. "Analytical" and "Debug" get enabled by increasing the level of collected events, I didn't look too deeply into that yet. I guess the script had enabled only up to the analytical level.
If I remember right, the level numbering goes approximately like this:
0-5 - operational ("Error" to "Verbose")
up to 16 - analytical
up to 255 - debug
Re: Finally fixed - IpV4Filter Global Policy configuration
Re: Finally fixed - IpV4Filter Global Policy configuration
winrm identify
IdentifyResponse ProtocolVersion = http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd ProductVendor = Microsoft Corporation ProductVersion = OS: 6.3.9600 SP: 0.0 Stack: 3.0 SecurityProfiles SecurityProfileName = http://schemas.dmtf.org/wbem/wsman/1/wsman/secprofile/http/spnego-kerberos, http://schemas.dmtf.org/wbem/wsman/1/wsman/secprofile/https/spnego-kerberosDoes it tell you anything?
Re: winrm identify
Re: winrm identify
PS C:\> winrm identify -r:localhost WSManFault Message = The client cannot connect to the destination specified in the request. Verify that the service on the destination is running and is accepting requests. Consult the logs and documentation for the WS-Management service running on the destination, most commonly IIS or WinRM. If the destination is the WinRM service, run the following command on the destination to analyze and configure th e WinRM service: "winrm quickconfig". Error number: -2144108526 0x80338012 The client cannot connect to the destination specified in the request. Verify that the service on the destination is running and is accepting requests. Consult the logs and documentation for the WS-Ma nagement service running on the destination, most commonly IIS or WinRM. If the destination is the WinRM service, run the following command on the destination to analyze and configure the WinRM servic e: "winrm quickconfig".So, pretty much the same error message, but, for some reason, repeated twice.
no subject