NimbostratusHi @bjones2480,
Review the timers or the intervals that you set in this Active Health monitor if it's in the default values ( Interval 5 seconds & Timeout on 16 Seconds ) Bigip will mark this Pool member as an offline "Red Rhombus" after 16 seconds then it will not send any further requests.
But within 16 second it will use this pool member in load balancing decisions.
you can lower the interval and timeout values, so Bigip will mark it offline quickly and prevent any requests from passing to this pool member.
NimbostratusMohamed,
Thank you for this information and we are in fact using the defaults.
If I understand you right. 5 seconds is the duration between the health checks and 16 seconds is the time the server has to respond before being marked as failing the health check?
However what we are seeing is the LTM sending requests to these servers failing the health check. Is there an instance or a config that would allow this to happen in BigIP?
Hi,
Do you mean Bigip sends requests to a down pool member even after the monitor marks it down ??!!
I doubt that happens, Bigip will send requests till the timeout value reached ~"16" seconds but after that the assigned health monitor will mark this pool member as down and stop sending traffic to it.
Take a packet capture for a minute to validate that
or reset pool member stats after being down and see packet in/out to confirm this.
NimbostratusMohamed,
I verified this functionality after noticing proxy errors from one of my systems intermittently. When I investigated the load balancing pool members I noticed one was failing a health check. As long as that server remained red in the load balancer (failing health check) I would get those intermittent proxy errors. As soon as I forced that failing pool member offline the intermittent errors would disappear.
I reached out to the team that manages our load balancing and they told me the following:
"Existing connections are the connections which were going to the server when it was up, before failing healthcheck. Those connections would continue to connect to failed server until they are terminated by the client itself or they are forced to terminate manually by us. Changing the load balancing method will not help as existing connections will still continue to connect to failed server"
Is this the timing issue you were speaking of? What existing connections are they talking about if the system is configured for round robin load balancing, wouldn't each request be a new connection?
"I will have to agree with you load balancers not smart enough, unfortunately that’s what we have noticed over the last few years and the only way to get the fresh connections initiate is to either force offline the server or kill the connections that’s connected to the failed server." This comment seems to fly in the face of everything I have read from BigIP and what you are telling me. So I am trying to understand if I am not conveying my question properly to them or not. While both of these comments make me question the purpose of a health check for a load balancer in our org if it isn't affecting how the system balances the load.
I am just trying to educate myself enough to take this concern to them and speak to it coherently. So I appreciate you taking the time to answer these questions.
