Forum Discussion

rb1980_75708's avatar
rb1980_75708
Icon for Nimbostratus rankNimbostratus
Mar 14, 2008

WA access logs

I am setting up a new deployment of some WA's and was surprised at the lack of any options to manage the access logs. In poking around, I see there's a cron that runs hourly to logrotate at 10MB size. My WA's will be in a pretty high-traffic environment and this seems too low.

 

I'd be curious to hear what others are doing to manage large amounts of access logs. I'm estimating my logs to be about 600-700MB/day per box.
  • OK! So after about 2 dozen emails back and forth with support I finally have a workable solution. Thanks to all who contributed.

     

    Let me recap:

     

    My goal was to be able to control my own access log rotation however I wanted to, without being locked into F5's defaults.

     

    After applying the Hotfix-BIG-IP-9.4.3-HF2 which fixed the 'bug' with the wrong path, the WA access logs were now included in the hourly rotation schedule and configured to rotate at 10MB. This was not ideal in my setup so I pressed on for a better solution.

     

    Turns out it was not that hard, just mainly my lack of understanding of how the WA manages the config files.

     

    So here is the solution I came up with that I am happy with.

     

    NOTE: This configuration is not offically sanctioned by F5 and may be wiped out by future upgrades. Use at your own risk.

     

     

    1. Edit /usr/share/defaults/config/templates/warotate.tmpl and remove the /var/log/wa/access/*.log filespec.

     

     

    2. Run: bigpipe logrotate wa include \"blah\" (this is a hack to get it to regenerate the wa conf files)

     

     

    3. Regenerate the configs with: bigpipe save all

     

     

    4. Create a standard logrotate conf file here: /var/run/config/logrotate.d/wa_access with your custom settings. Here is mine:

     

     

     

    /var/log/wa/access/*.log {

     

    compress

     

    ifempty

     

    rotate 7

     

    daily

     

    olddir /var/log/wa/access/archive

     

    sharedscripts

     

    postrotate

     

    /bin/kill -HUP `/sbin/pidof -s pvac 2> /dev/null` 2> /dev/null || true

     

    cd /var/log/wa/access/archive;for i in `ls -1 *.1.gz`;do /usr/bin/scp $i user@host:/destdir/`date -r $i +%Y%m%d`-$i;done

     

    endscript

     

    }

     

     

     

     

     

    5. You need to mkdir /var/log/wa/access/archive

     

     

    6. Check your config by running: logrotate -d /etc/logrotate.conf

     

     

    Note: Because I opted for a daily rotation in this case, the default logrotate script that gets called in /etc/cron.daily will pick up the wa_access file from /var/run/config/logrotate.d. If you wanted to run hourly rotation, you'd be better off following the method in SOL8320 and adding it to the wa include section.

     

     

    I just put together this configuration today and haven't yet let it run on it's own, but it looks good on the debug check. I will come back and modify the post if it needs any tweaks. The only thing I am not sure about is whether or not I will leave it in cron.daily, since this gets run at 04:02. Because I'm anal, I might change it to run in a seperate crontab at midnight or just change the time in /etc/crontab.
  • RB1980,

     

     

    Thanks for sticking with the logrotate stuff and posting the result here. I understand your need to customize your logging and am sorry that you had to spend time to get it the way that you want it. In addition to spending a few minutes to speak to you about your needs, I also wanted to let you know that I am tracking this thread and will look into how we can incorporate changes into WebAccelerator to allow you to more easily modify the logrotation to the way you would like it. I am the Product Manager for Acceleration at F5 which includes the WebAccelerator product. While DevCentral is not an official way to ensure that your problems with F5 solutions get addressed, I want to let you know that I am committed to making F5 products as easy to use as well as flexible.

     

     

    In short, I am sorry for the bug that you ran into with logrotate and I will work with the WebAccelerator developers to provide a long term solution to your issue. If we have not spoken by the time that you get this email, feel free to drop me a note at j.hicks at f5 dot com .

     

     

    Thanks,

     

     

    Joe
  • update: as expected after installing the latest round of hotfixes, these changes get wiped out. Also, I think they got wiped out when I did a reboot a few days ago. Any suggestions from the F5 folks on how to make this "permanent"?

     

    I suppose I could run a script on startup that copies stuff from a "safe" location back into place.

     

    I'm not really versed in the ways of F5 yet...
  • Have you considered changing the rotation to nightly instead of hourly by moving the cron job from cron.hourly to cron.daily?
  • You need to re-read my previous post: it IS rotating nightly, as called by the logrotate script in cron.daily. But thanks.