Thursday, 29 September 2016

Monitor Squid Proxy with OMS, Part One

With the release of View Designer in OMS, we now have the capability to use the power of log search to create our own bespoke solutions.  This capability is very powerful and allows for the creation of single pane of glass dashboards across data sets for any technology for which we are collecting data. Goodtimes 

For this demo, I will create a custom Squid Proxy solution, the same process can be used for any application or technology you can gather log data for.
As far as I can tell (and my google-fu is strong) there are not many OMS blogs cover Squid proxy performance logs.
I want to do this to show how easy it is to make this happen.  Demonstrating OMS’s flexibility.  I am going detail how we can utilise this data to create a custom Squid solution in OMS using the newly released View Designer. For the purpose of the demo, I will

  •         Configure OMS to collect the Squid access.log
  •         Configure a custom solution to visualise the data
  •         Add a few saved queries to a List of Queries blade.
Please note that this process can be used to create a custom solution for any technology.  If the data is collected by OMS, it can be utilised in any solution.
 You can collect this data from a Syslog server however, for this example I will be reading straight from the Squid log location.

Note: Any KPIs and logs used in this example are for demonstration only.  Any KPIs or logs can be obtained from Linux and log data from Squid:

There are a few good blogs out there that detail different ways to collect and store Linux data in OMS. For this demo, I will demonstrate how to manually import a squid log for collection in OMS.

Before we start a few assumptions

  • ·      OMS agent has been deployed the to target Linux computer node running the Squid proxy service.
  • ·      The /var/log/squid/access.log file has the permission of 744 set (Otherwise you get an access denied, check the OMS log on the Linux server) Please note I would not recommend the permissions in a production environment, but it’s up to you!
  • ·      You are collecting Linux performance data.

When we are finished the dashboard will look something like this:

Not very in depth however, it is a good base to build on.  Also, I am still learning how to tweak the Squid log data myself.

Let’s get some data! (Never thought I would be getting excited over that)

Open the settings interface:

Go to Preview features> Enable ‘Custom Logs’:

Go to Custom Logs > 'Add+':

From the Squid proxy copy the access.log file to your computer.  From the dialogue select as below:

Select “New line” as the delimiter.

Now enter the location on the Linux server of the Squid access.log file and click the + button.

Give your new custom log a name and add any info you want.   Make a note of the name you give it.

Note: It can take up to an hour for the data to start being collected, so go and have a coffee or juggle some other work for a bit.

To test that the log data is being collected go back to the Log search. In the search window type:
Type=Squid_CL (Or whatever you called your custom log)

This will return (if the data is being ingested into OMS) the Rawdata

So in this first part, we have successfully connected to the Squid access.log on the Linux server and are importing the data into OMS.  

In the next part we are going to making the data useful and setup the required queries needed to make it all look good and more importantly relevant.

Join me in part two for some more OMS magic....

It is amazing what you find that you have left behind on the Internet...