This section includes:
The goal of this page is to help with Cloud Connector Edition (CCE) deployment and troubleshooting issues.
We recommend running the SBC and SbcComms latest firmware before deploying CCE.
Confirm CCE state
Confirm CCE Status
Once CCE deployment is complete, confirm the following:
Log into the SBC Edge WebUI.
Access Tasks > Application Solution Module > Operational Status.
If there is only the AD VM, and you are using a proxy on the ASM host, exclude the Management Network and Local Network from the proxy usage. To bypass the proxy, specify the WinHTTP Proxy settings set with your Proxy Server and a Bypass list including the 192.168.213.* network used by your Cloud Connector Managements services and the Local subnet used by your VM.
> netsh winhttp set proxy "10.10.10.175:8080" bypass-list="192.168.213.*;192.168.214.*". > netsh winhttp show proxy Current WinHTTP proxy settings: Proxy Server(s) : 10.34.207.20:8080 Bypass List : 192.168.213.*;192.168.214.*
Ensure four VMs are displayed. If not, restart the deployment.
Ensure the latest Software Installed Version (CCE) is installed.
Ensure the Auto Maintenance Status is set as None.
- If the status is Update OS or Bits, you should wait until the end of the update.
- If the status is Auto-Recover, check: https://blogs.technet.microsoft.com/sfbhybridvoice/2017/03/10/understanding-cloud-connector-edition-auto-recovery.
Ensure the proper version is displayed as the Software Running Version.
- If the Software Running version is None or Unknown, log into the ASM using Remote Desktop.
Connect the SBC Edge VM, open Services and ensure the following six Skype for Business Servers are Running. See below.
c. If the Edge Servers do not start, see below for identifying solutions.
Confirm Hyper-V State
On HyperV manager, check the Networking for each VM and confirm that the Corpnet Adapter is always attached to the SfB CCE Corpnet Switch. If a VM has two IPv4 addresses on the Corpnet Adapter, and one is beginning with 169, there is most likely an IP address conflict with local equipment.
On HyperV manager, check the Networking for the Edge VM and confirm that the Internet Adapter is attached to the SfB CCE Internet Switch. If the Edge VM has two IPv4 addresses on the Internet Adapter, and one is beginning with 169, there is most likely an IP address conflict with DMZ equipment.
Confirm Office 365 Tenant
On the Office 365 Tenant, confirm that OsUpdate is not currently running:
PS> Get-CsHybridPSTNAppliance -Identity 091287a9-1c8f-416e-b8ba-b250fba56181 Identity : 091287a9-1c8f-416e-b8ba-b250fba56181 . RegistrationStatus : RegisteredAndDeployed . DeploymentStatus : {"Status":"Deployed","Version":"1.4.1","Error":null,"StartTime":"2016-12-19T21:10:50.9209Z"} OsUpdateStatus : {"Status":"None","Error":null,"StartTime":null}
If OsUpdateStatus displays the status below, wait for the update to complete. To set a time window for the update, see: Managing Your Office 365 Tenant.
{"Status":"Updating","Error":null,"StartTime":"2016-10-31T11:47:04.1526Z"},- If the OsUpdateStatus displays the error below, you can ignore it since it only appears once, and will disappear the next time Windows Update is running. This can also be fixed by removing the Windows Update on the ASM host.
"{"Status":"Error","Error":"2017-03-21T12:55:18.9300569Z: System.Management.Automation.RuntimeException: Cannot compare \"HKEY_LOCAL_MACHINE\\Software\\Policies\\Microsoft\\Windows\\WindowsUpdate\\AU\" because it is not IComparable.\r\n" - While Running Publish for HA, if you receive the error "Appliance working on a task", retry again later.
At this point, the CCE is running.
Confirm SBC state
On SBC Edge WebUI, access Settings > Signaling Groups. On the CCE Signaling Group, ensure the "Service Status" is Up.
Click on Counters and ensure Options are flowing in both directions and that the number of 2xx answers is matching.
If there are no Options flowing, log into the ASM using Remote Desktop.
Execute the following command on a shell of the ASM Host:
ipconfig /all
Confirm :
- "Ethernet adapter vEthernet (SfB CCE Corpnet Switch)" adapter MAC start by 00:90
- "Ethernet adapter vEthernet (SfB CCE Internet Switch)" adapter MAC start by 00:10
- If unable to confirm, and the node has come directly form Manufacturing, a re-initialization will solve the issue.
At this point, both the CCE and SBC are up and running.
Confirm environmental state
Confirm the public SBC Edge FQDN is properly mapped to the External IP used in Configure CCE (via SBC Edge WebUI).
Any changes to the Configure CCE tab requires the CCE to be redeployed.
From a PC with open internet access, run Nslookup on the public CCE FQDN. If the result is "Error: Requested name myccesite1.mydomain.com could not be resolved", the Public DNS server needs to be corrected.
nslookup av.contoso.com
Ensure the IP address returned is the Edge Server External Public IP. If the IP is not the correct one, you need to fix your Public DNS server.
- Ensure the Edge VM has proper DNS resolution.
- Log into the ASM using Remote Desktop.
- Connect to the the Edge VM, and open the Powershell command prompt.
Run the command below for all the domains assigned on your O365 tenant. If the command returns as Unknown, modify the firewall to open the port from "Internet facing DMZ" to the Internet.
nslookup -q=cname lyncdiscover.domain.com. 8.8.8.8 nslookup -q=cname sip.domain.com. 8.8.8.8 nslookup -type=SRV _sip._tls.domain.com. 8.8.8.8 nslookup -type=SRV _sipfederationtls._tcp.domain.com. 8.8.8.8
Confirm the user state
For details, see Managing Your Office 365 Tenant.
- Access the Office 365 portal, access Admin Center> Skype for Business> Voice> Voice User.
- Confirm the user has a phone number and a license assigned.
Confirm that PSTN connectivity is "On-Premises".
- Log into your Office 365 Tenant using PowerShell
On Office 365 Tenant, use "Get-CsOnlineUser" to confirm that the user is activated for EnterpriseVoice, HostedVoiceMail and HybridVoice.
PS> $user = Get-CsOnlineUser site2user1 PS> $user.EnterpriseVoiceEnabled True PS> $user.HostedVoiceMail True PS> $user.VoicePolicy HybridVoice
On Office 365 Tenant, use "Get-CsUserPstnSettings" to confirm that the user is assigned to the proper HybridPstnSiteName.
PS> Get-CsUserPstnSettings -Identity Site2User1 RunspaceId : aa696d9a-67b1-4580-9a60-179cb90fd854 UserPrincipalName : site2user1@SonusMS01.com SipAddress : sip:site2user1@SonusMS01.com AllowInternationalCalls : True HybridPstnSiteName : AEPSITE2 HybridPstnSiteFqdn : AEPSITE2.SONUSMS01.COM
Connect a Skype for Business client using the account below:
Confirm the call establishment
- On the SBC Edge WebUI, access Diagnostics> Test a Call. For details, refer to Testing a Call
- Enter the phone number of your Skype Client and select your Call Routing Table "From Sip Trunk". The call should reach your Skype Client.
- Access Settings > Signaling Groups.
On the CCE SG, click on Counters. For details, refer to Viewing SIP Counters and Channels.
Ensure there is Incoming Invite and that the number of 1xx answer is matching. If you get a "Additional Information: Failed Reason Q.850 Cause Code: 1.", ensure that:
- You are calling a number that exists in O365.
- You are using the proper SG.
- Your Calling Number Translation are sending E164 to CCE.
- Click on Reset Counters.
Make a call from the Skype client to a PSTN number that does not exist on your Office365 Tenant.
From the CCE Signaling Group, click on Counters.
Ensure there is an Incoming Invite and that the number of 1xx answer matches.
If the PSTN provider is slow to route the call, the Skype client may cancel the call too fast. Access Tasks / Cloud Connector Edition / Setup / Confgure CCE / Raw Ini config, and add the EnableFastFailoverTimer=false parameter into the section [TrunkConfiguration].
- Click on Reset Counters.
- Make a call from the PSTN to the Skype Client number on your Office365 Tenant.
- From the CCE Signaling Group, click on Counters.
Ensure there is an Outgoing Invite, and that the number of 1xx answer matches.
Click on Reset Counters.
Confirm the call status
- If the call is not presenting a calling number:
- When using a SIP Trunk, you may have to Disable the "Send Assert Header" if your Sip Trunk provider does not support multiple PAI.
- When using a SIP Trunk, you may have to Disable the "Send Assert Header" if your Sip Trunk provider does not support multiple PAI.
- If the call is having bad audio quality:
- Use the Call Quality Dashboard to ensure the firewall is properly configured.
If the Forward/Transfer or Conference are failing to escalate from Online call to CCE call:
Verify the Mediation server have been added to the teantusing the command:
Get-CsOnlineUser -Filter "HostingProvider -like '*.<YourDomain.com>'" | Get-CsHybridMediationServer
- If the Mediation Server is missing, follow "Configure Online Hybrid Mediation Server settings" on https://technet.microsoft.com/en-us/library/mt740651.aspx.
- Ensure the SBC and CCE are perfectly in sync with the Internet time.
- If the PSTN provider is slow to route the call, the Skype client may cancel the call too fast. Access Tasks / Cloud Connector Edition / Setup / Confgure CCE / Raw Ini config, and add the EnableFastFailoverTimer=false parameter into the section [TrunkConfiguration].
Confirm Auto-Update
- If you receive the following message: Not enough memory in the system to start the virtual machine 1.4.2, close all the open software running on the Host.
- If the CCE update failed to download, and you are using a proxy on the host, specify the proxy settings per machine rather than user. You can proxy settings per machine with a registry change, or with the Group Policy setting as follows:
- If you receive the following message: Not enough memory in the system to start the virtual machine 1.4.2, close all the open software running on the Host.
Registry: HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\CurrentVersion\Internet Settings] ProxySettingsPerUser dword:00000000.
Group Policy: Computer>Administrative Templates>Windows Components> Internet Explorer: Make Proxy settings per machine (rather than per user).
Collect Log on CCE
- Log into SBC EdgeWebUI.
- Access Diagnostics> Cloud Connector Edition Logging> Logging Configuration.
Select Enable CCE Logging.
- Place a call.
Click Download Log and specify a time window.
- StartTime. The time you run Start-CCLogging. Refer to the time in the upper right corner of the WebUI.
EndTime. The current time. Refer to the time in the upper right corner of the WebUI.
- Open the Log file using the latest version of Snooper.
Update and Confirm Window's Service Version
Cloud Link is affected by an issue on Windows Server 2012R2 and Cloud Connector Edition; this issue creates a memory leak on the CCE Virtual Machine. As a result, the Mediation and Edge servers run out of memory after a few weeks. As a fix to this issue, ensure the Window's service version is 6.3.9600.17552 and the Windows Update KB2919355 is installed on the CCE VM, as follows:
- Through Windows, log into the CCE's VM.
- Locate C:\Windows\System32\profsvc.dll.
Right-click on the filename profsvc.dll and select Properties.
- Select the Details tab.
If the file version is 6.3.9600.17930, no action is required.
- Access the cmd prompt and execute the systeminfo.ext command.
Verify prerequisite KB2919355 is already installed on the VM.
Access the Control Panel and select System and Security > Windows Update.
- Click Automatic Updates.
Following Windows checking for updates, click on the “<number> important updates are available" link.
- Navigate to the Windows Update Options and turn on Check for Updates Automatically.
Select Update for Windows Server 2012 R2 (KB3078676) and click Install.
After installiationiscomplete, the option to restart the PC to complete the update is displayed. You must turn off Windows Update before restarting.
DO NOT RESTART THE PC PRIOR TO TURNING OFF WINDOWS UPDATE.
Turn off windows update before you restart to prevent the VM from automatically doing other updates. See Change Settings.Through System and Security > Windows Update, click Change Settings and change Important updates to “Never check for updates (not recommended) and click OK.
Restart the virtual machine.
- Check windows update history to verify KB3078676 was installed successfully as follows:
Access the Control Panel and select System and Security > Windows Update.
Click View update history.
Verify File version of profsvc.dll has been updated to 6.3.9600.17930.
- Locate C:\Windows\System32\profsvc.dll.
- Right-click on the filename profsvc.dll and select Properties.
- Click Details.
Verify product version is 6.3.9600.17930.