Citrix Troubleshooting Tools

Citrix Troubleshooting Tools 

highlight some of the tools available to help us troubleshoot our XenDesktop & XenApp 7.x environments, making life a little easier. Some of which we used as well. I won’t go into any troubleshooting methodologies or anything, I’ll leave that up to you. Just make sure that you have a clear definition on the issue and try to rule out any miscommunications between you and the rest of the team. Also make sure that you understand and know about the underlying architecture, what each component does (or is supposed to do), which services and agents are involved and how they all interact.

XDDBDiag

The XenDesktop Database Diagnostic tool. It was first designed with XenDesktop version 5 but can be used with all new 7.x editions as well. This command line support tool performs a consistency data check on the data and connectivity verification in a XenDesktop database. A great tool to do some pro-active administrating as well. Diagnostic output can be saved in the form of a comma-separated value (.csv) files located in a compressed file (.zip) named computername_XDDBDiag_Output.zip to the same directory in which the program is located. It provides following information:

• Site Information
• Virtual Desktop Agent Information
• Current Connections / Connection Log
• Hypervisor Connections
• Policy Information
• Desktop Group
• Controller Information
• SQL Information and more.

It will automatically search for new updates when launched but this is something that can easily be turned off as well. Check out this article for some more details on how to use it. http://support.citrix.com/article/CTX128075

Legacy alert: Check the following registry key to find some specific connection details: HKLM\SOFTWARE\Citrix\Ica|Session\[n]\Connection Useful when troubleshooting session sharing, for example when users are consuming multiple sessions when they are not supposed to.

XDPing tool

Another command-line tool originating back to XenDesktop version 5 and commonly used to trace and track down connectivity issues. As of version 2.2 the XDPing tool also supports all current XenDesktop 7.x editions. It automates the process of checking for the causes of common configuration issues in a XenDesktop environment. The tool can be used to verify configuration settings on both the XenDesktop Broker and VDA machines, both from the console and remotely. Read through this CTX article to see which command-line options you have when executing the tool, it also includes a short video on how to use it:

http://support.citrix.com/article/CTX123278

It can also monitor and check certain XenDesktop services information and query the local event log to check for known events that are related to XenDesktop. All in all a great tool to handle some of those more common pro-active admin tasks as well.

To give you an idea on how it looks:

https://www.youtube.com/watch?v=NNKFpYlj0Zc

Jack Cobben also wrote an article on it a few years back, you’ll find it here:

http://www.jackcobben.nl/?p=2669

Old school

It’s great that we have such an extensive toolset at our disposal but let’s not forget about the basics. A simple NetStat and/or firewall/port check, a Ping to check network connectivity perhaps, Tracert, Telnet etc… Let’s not forget about those guys shall we?! Are all our services up and running, no errors or warnings, no time or sync issues, a quick and dirty manual event log check, and when you do, remember to check all components that might be involved, your Controller, StoreFront, WebInterface, VDA etc. you get my point right? Of course this kind of functionality is build into most of the tools as well, but sometimes all you need is a simple DOS Prompt and you’re good to go. No separate install or configuration steps needed.

Services and logs overview

With both XenDesktop & XenApp, it is important to understand what is taking place under the covers, which processes and services are involved when enumerating applications, connecting and disconnecting users, provisioning new machines etc. something I’m not going to discuss at this time. Having said that, with XenDesktop we can enable something called service logging, again, assuming you know what you are doing.

Service logging can be enabled either from the command-line or through Citrix Scout (installed by default on XenDesktop and XenApp 7.5, and up, Delivery Controllers) using a graphical user interface, using Scout will also automate the log folder creation process. Just keep in mind that Scout still lacks the ability to log data for certain services like the Broker Agent, this is the VDA as of version 7.x (an important one), for example. When you enable service logging using the command-line all service will be available. Other services include, the ADIdentity, Broker, Configuration, Host, Machine Identity and Machine Creation services. This is what a manual command-line may look like:

Citrix.MachineCreation.SdkWcfEndpoint.exe-LogFile C:\XDLogs\Name.log

Next to the Delivery Controller (Broker Service log) and the VDA (VDA Broker Agent log) the PortICA service a.k.a. the ICA service (PortICA logging) is probably one of the most important ones to keep an eye on and log information on when troubleshooting connectivity issues. It handles just about everything except for direct communication with the Delivery Controller. Before I continue, all this functionality is build into Citrix scout as well, just so you know.

How to enable VDA Broker Agent logging (not possible using Scout):

http://support.citrix.com/article/CTX117452

This next article sums up the services and agents available for logging, do note that it only applies to XenDesktop version 5.x. As a result some service names have changed with the introduction of XenDesktop 7.x. Like the VDA Broker Agent mentioned in the previous CTX article for example. The rest is still valid though.

http://support.citrix.com/article/CTX127492

PortICA logging, in addition to enabling the VDA Broker Agent log, can be a useful tool when troubleshooting issues with the Virtual Desktop Agent (VDA). PortICA logging is not enabled by default on the VDA. Check out:

http://support.citrix.com/article/ctx118837

HDX Monitor

We all know HDX right? This tool will provide you with detailed diagnostics information on all HDX technologies known today. HDX Monitor version 3.x has been upgraded to also support version XenDesktop version 7.x as well as XenApp 6.5. This CTX article will tell you all you need to know:

http://support.citrix.com/article/CTX135817

Here’s a Citrix Blog on HDX:

http://blogs.citrix.com/2013/06/12/hdx-monitor-3-2-published/

Citrix Scout

As of XenDesktop and XenApp 7.5 Citrix Scout is installed by default. It is a powerful tool that combines, or aggregates, a bunch of the individual tools discussed throughout this article. For example, CDF Control is build into Scout as well and so is XDPing, it also allows you to enable service logging for most of the primary services. But it can do more than that though, it collects data (referred to as Data points) related to the BIOS, the OS installed, Memory information, all sorts of drivers, it also reads certain registry keys, system and application event logs, Site and Farm information, WinRM settings and lot lot more!

Data needs to be collected before it can be automatically uploaded to Citrix. First you need to select a certain number of machines from which data will be collected, reviewed and eventually uploaded (this is also where Citrix Insight Services, the taas.citrix.com website, comes into play. I mention it near the bottum). You can select up to 10 machines in total. The data will first be analyzed and if any possible corrective actions are found they will be shown on screen giving you the option to execute them. Next you will be asked to choose a location where you would like to store the data in a .zip format. Once the data is saved you will be presented with a dialogue explaining the upload & Analysis process. Click Continue on this to proceed. Than the Upload to TaaS.citrix.com dialog appears, enter your My Citrix Username, password etc. and click Upload.

The status report is sent directly to Citrix Technical Support. An MD5 checksum test is performed to ensure your upload was successful.

This is the CTX article for Scout, it’s full of information. It also lists all so-called Data points mentioned earlier. http://support.citrix.com/article/CTX130147

CDF Control

CDF stand for Citrix Diagnostic Facility, it has been around for over 7 years and it’s still one of the most used diagnostic toolkits used today! While build into Citrix Scout it’s also available as a stand-alone download and fully supports all new XenDesktop Editions. It is an event tracing controller/consumer, geared towards capturing Citrix Diagnostic Facility (CDF) trace messages that are output from the various Citrix tracing providers. Note that you will need to have local admin permissions to be able to start tracing. Here’s where to get it and how to use it: http://support.citrix.com/article/CTX111961

Consider it to be a good practice to try and collect some CDF traces prior to opening up a support case with Citrix, since this is probably one of the first things they’ll have you do, unless you are in doubt on where to start of course.

Policies: Check Active Directory and IMA based policies by using policy modeling and or group policy results. Depending on your configuration run these tools either from Studio, an active Domain Controller or both.

Citrix print detective

This tool has been around for a few years now and as of version 1.2.1.5 it also supports XenDesktop 7. Print Detective is an information gathering utility that can be used for troubleshooting problems related to print drivers. It enumerates all printer drivers from the specified Windows machine, including driver specific information. It can also be used to delete specified print drivers. It allows for log file capabilities and provides a command-line interface as well. It supports all of Microsoft’s desktop and server platforms, although I’m not sure on Windows Server 2012, I think it should work though. How to use print detective? Go to:

http://support.citrix.com/article/CTX116474

Citrix Director

Known by most, but for those of you who are still unfamiliar with Director, have a look at this Citrix E-Docs page , it includes an introduction video:

http://support.citrix.com/proddocs/topic/xenapp-xendesktop-75/cds-monitor-director-wrapper.html

Director has two separate views depending on your permissions, a Helpdesk view and an Administrative view. It goes without saying that the Administrative view hands us some additional features to work with. It offers historical views with data storred ranging from a single hour, 24 hours or 7 days in total. On the Administrative dashboard it displays the average time it takes for users to logon (which can also be vieuwed on a per user basis), how many sessions are active, if there are any failed machines and or connections, if your Delivery Controllers are up and running and more.

On a more granular level you can get specific ‘per user’ information: which processes are currently loaded, how HDX is doing, applications launched, the name of the VDA, how much memory and CPU resources are in use, which policies are applied etc. You can also send remote messages and power off or shut down a machine remotely. Shadowing user sessions is now also done directly from Director. By using different kinds of filters you can change the type of information displayed including detailed overviews on the current active or disconnected sessions. As mentioned, logon performance can be viewed overall or on a per user basis, and as you can see in the screenshot it shows very detailed information with regards to what is going on during the logon process, very usefull.

 PowerShell

PowerShell is another great example of a toolkit that can be used proactive as well as reactive when it comes to checking up on certain crucial infrastructural services. For example, there are a whole bunch of PowerShell ‘Get’ commands available that will quickly and easily show us how our current infrastructure is doing. I think every company should make use of this and make it part of a daily infrastructure check. Here are some of the most commonly used Get commands:

Get-BrokerServiceStatus (Broker), Get-SFServiceStatus (StoreFront), Get-HypServiceStatus (Host service), Get-ConfigServiceStatus (Configuration), Get-AcctServiceStatus (AD Identity).

Of course there is a whole lot more that you can do using PowerShell, everything you do using XenDesktop Studio for example can be done using PowerShell scripts as well. In fact, everything you configure using Studio is actually PowerShell being executed in the background. It is very tightly integrated, which is a good thing! Studio sort of records all of your manual configuration steps and turns them into PowerShell scripts for you to copy and perhaps reuse.

Here is some additional info from the Citrix E-Docs website:

http://support.citrix.com/proddocs/topic/xenapp-xendesktop-75/cds-sdk-wrapper-rho.html

Citrix (XenDesktop) Studio

Studio is the main console used for managing users, applications, delivery groups, machine catalogs and reporting, plus it can also be used to do some basic troubleshooting. It doesn’t really have any diagnostic capabilities but it can quickly show you how many machines are actively being used, the number of users connected and applications in use. Of course it can also be used to disconnect sessions and reboot virtual as well as physical machines. It also has an alert and notifications section worth having a look at. Together with Director, Studio functions as your fist line of defense against unwanted errors and warnings.

Citrix Insight Services 

Insight Services is the glue binding it al together, here you can upload your log and trace files and link them to any support cases you might have registered earlier. Once you upload a file Insight Services will automatically analyze your log files and scan them for hundreds of know issues. From their website: Citrix Insight Services reads log files from XenDesktop, XenServer, XenApp, NetScaler, PVS, ByteMobile, XenMobile, CloudBridge, CPBM and XD/XA Connector, and we’ll be adding products to it over time.We’re adding new plug-ins that capture known issues for these products all the time. So Citrix Insight Services will keep getting better.

When Citrix Insight Services discovers any known issues in your environment, it suggests hot fixes, patches and updates with red/yellow/green prioritization. It will also analyze your configuration and give you best-practice advice, with links to relevant articles or white papers. Citrix Insight Services isn’t just for troubleshooting. It’s also a great way to give your system a quick Health Check, so you can spot any issues before they become real problems. Check it out here:

https://taas.citrix.com/AutoSupport/howitworks/

A global overview (no details)

To finalize as far as troubleshooting tools go, here’s an overview on all tooling Citrix has available. It only lists the tools and their accompanying CTX articles, it doesn’t say anything on the supported editions of XenDesktop & XenApp or in some cases Provisioning Services and WebInterface or StoreFront.

http://support.citrix.com/article/CTX126294

Conclusion

I think it’s safe to state that Citrix offers a great set of tools to troubleshoot our current as well as our more legacy environments. Their tool set has improved and matured over time and it’s good to see that Citrix keeps investing time and effort into developing new tools and initiatives like Citrix Insight Services, which isn’t new, I know, but still! And what about the PowerShell integration?! About my comments on Citrix support earlier, I can only say that I’m not here to bring them down or anything, I just like to keep them on their toes 🙂 I hope this article has been as informative as it was fun to write. Thanks again for reading!