Ah, thanks. The removal was intentional, but missed the documentation update amidst the other changes. Is this a feature you need resurrected for some reason?

If you are running a reverse-proxy the round-robin is better served with either cache_peer LB options or an internal service name with short DNS TTL.

Intercepting or explicit/forward proxies are where the session stickyness problems are encounered and this option causes more harm than good in most current Internet traffic.

Yes, I use DNS with a 3-5 minute TTL to allow adding/removing origin web servers, and remapping sites to servers (e.g. providing more servers for busier sites), without having to reconfigure each Squid instance for each change. Especially as reconfiguring Squid sometimes means it stops serving requests for a noticeable amount of time while it inspects the cache.

If the latter wasn't an issue (i.e. there's some determinism to when it inspects the cache), I could look at some sort of scripted adjusting of cache_peer/cache_peer_domain/cache_peer_access rules, but that's a whole new learning/testing curve.

(Using DNS also means internal clients that don't go through the proxies use the same web servers as external clients, so it's a nice way to have a single configuration for both.)

Did you know that cache_peer can be configured with a DNS name (eg 'cache_peer server-a.local parent 80 0 originserver') which is looked up on each use of the peer now? so your IP sets can be per-cache_peer rather than per-domain.

And that an external_acl_type helper can send a tag= naming which cache_peer to use based on wherever you store that domain-to-peer/server mapping? (the 'OK' says allow the traffic to http_access via external ACL type, the "tag=serverA" says which peer is providing it to cache_peer_access via the note ACL type).

The end product only needs to be reconfigured if you are adding a new cache_peer POP/shard server to the collection. domains can be added/removed/shuffled between peers in (almost) realtime with no reconfig of Squid-3.

I've just tried putting a multiple-IP hostname as cache_peer in my 3.5.20 instance, and it still appears to only use whichever address is the first entry in the ipcache for that name, and ignoring the other two addresses. Unless I've misunderstood your point.

I'm running into this as well.  I can't see the potential downside for session issues.  The problem I was trying to solve is ClamAV database mirrors.  I would like squid to cycle through the various IPs when a non-200 result code is returned.  As it stands, it keeps trying the same failing mirror (or sometimes switching one):

1522273416.085   6308 127.0.0.1 TCP_MISS/404 543 GET http://db.us.clamav.net/daily-24428.cdiff - HIER_DIRECT/200.236.31.1 text/html
1522273422.129   5809 127.0.0.1 TCP_MISS/404 543 GET http://db.us.clamav.net/daily-24428.cdiff - HIER_DIRECT/200.236.31.1 text/html
1522273428.127   5816 127.0.0.1 TCP_MISS/404 543 GET http://db.us.clamav.net/daily-24428.cdiff - HIER_DIRECT/200.236.31.1 text/html
1522273469.728    173 127.0.0.1 TCP_MISS/403 483 GET http://db.us.clamav.net/daily-24428.cdiff - HIER_DIRECT/198.148.78.4 text/html
1522273469.997     85 127.0.0.1 TCP_MISS/403 483 GET http://db.us.clamav.net/daily-24428.cdiff - HIER_DIRECT/198.148.78.4 text/html
1522273470.089     87 127.0.0.1 TCP_MISS/403 483 GET http://db.us.clamav.net/daily-24428.cdiff - HIER_DIRECT/198.148.78.4 text/html
1522273475.564     85 127.0.0.1 TCP_MISS/403 483 GET http://db.us.clamav.net/daily-24428.cdiff - HIER_DIRECT/198.148.78.4 text/html
1522273475.832     86 127.0.0.1 TCP_MISS/403 483 GET http://db.us.clamav.net/daily-24428.cdiff - HIER_DIRECT/198.148.78.4 text/html
1522273475.923     85 127.0.0.1 TCP_MISS/403 483 GET http://db.us.clamav.net/daily-24428.cdiff - HIER_DIRECT/198.148.78.4 text/html

Does that seem like a reasonable option?  This is with 3.5.20-10.el7

Sorry, that should be "I *can* see"

We are also seeing this issue when proxying out to a DNS entry which returns a number of A records.

With IaaS, this pattern is very common in order to provide resiliency across data centres so I'm surprised this is not being flagged up as a bug more frequently.

The first item returned in the DNS response receives all of the traffic until TTL expiry. With AWS this results in traffic flip/flopping every minute or so between backend nodes.

Could you please confirm whether this is by design or a defect? I feel like the ability to round-robin backends is fundamental so I must be missing something.

I've tried the recommendation of using cache_peers but the result is the same.

An example of the debug log when using a plain http_access approach is as follows. Cache_peer does basically the same:

2018/08/09 14:18:16.268 kid1| 44,2| peer_select.cc(281) peerSelectDnsPaths:   always_direct = DENIED
2018/08/09 14:18:16.268 kid1| 44,2| peer_select.cc(282) peerSelectDnsPaths:    never_direct = DENIED
2018/08/09 14:18:16.268 kid1| 44,2| peer_select.cc(286) peerSelectDnsPaths:          DIRECT = local=0.0.0.0 remote=x.y.z.202:443 flags=1
2018/08/09 14:18:16.268 kid1| 44,2| peer_select.cc(286) peerSelectDnsPaths:          DIRECT = local=0.0.0.0 remote=x.y.z.127:443 flags=1
2018/08/09 14:18:16.268 kid1| 44,2| peer_select.cc(295) peerSelectDnsPaths:        timedout = 0
2018/08/09 14:18:16.268 kid1| 28,4| FilledChecklist.cc(66) ~ACLFilledChecklist: ACLFilledChecklist destroyed 0x7ffea56b6f40
2018/08/09 14:18:16.268 kid1| 28,4| Checklist.cc(197) ~ACLChecklist: ACLChecklist::~ACLChecklist: destroyed 0x7ffea56b6f40
2018/08/09 14:18:16.268 kid1| 17,3| FwdState.cc(1307) GetMarkingsToServer: from 0.0.0.0 netfilter mark 0
2018/08/09 14:18:16.268 kid1| 26,3| tunnel.cc(1181) tunnelPeerSelectComplete: paths=2, p[0]={local=0.0.0.0 remote=x.y.z.202:443 flags=1}, serverDest[0]={local=0.0.0.0 remote=x.y.z.202:443 flags=1}

The debug message suggests it is just choosing the first item, is this correct?

This is with Squid 3.5.27 on Centos 6.9.

Thanks very much,

Adam.

AFAICT from looking at the code, Squid v3.5 and later versions accept but ignore the balance_on_multiple_ip directive. This silent ignorance is a bug.

My interpretation of comment #1 is that removing balance_on_multiple_ip support was an intentional change. However, comment #1 does not have enough info for me to quickly pinpoint or justify that change. Hopefully, Amos will supply the missing pieces.

The purpose of this balance feature was to make Squid attempt each IP consecutively in the event of connection failures - instead of repeatedly re-trying the same broken IP. 3.2 moved to that behaviour by default (including more IPs than just the DNS current ones) so the option was no longer relevant nor possible (no wrap if all IPs have failed).

By the time this rotation case was brought up Squid were already assembling these lists of IPs based on 'happy eyeballs' lookup behaviour which adds even more complication. Squid these days is not guaranteed to know the full set of IPs it will be rotating through. So we cannot reliably re-instate the rotate behaviour for people (ab)using it to force client-driven load balancing on DIRECT traffic.

People needing load balancing of peers should use the cache_peer LB algorithms instead, or trust that the DNS authority is using appropriate load balancing for their sites need.

Amos, Thanks for the reply.

This had been my expectation after digging through the code, but the cache-peer round-robin functionality doesn't seem to be actually iterating through the list either. When I try it. I get a very similar output to the log pasted below:

2018/08/09 14:18:16.268 kid1| 26,3| tunnel.cc(1181) tunnelPeerSelectComplete: paths=2, p[0]={local=0.0.0.0 remote=x.y.z.202:443 flags=1}, serverDest[0]={local=0.0.0.0 remote=x.y.z.202:443 flags=1}

The paths are duplicated in the cache_peer scenario, so 2 A records become 4 (should be fine), but the code always chooses the first item. Could you point me towards the code that iterates the counter? I can see code that does that (ipcacheCycleAddr), but I can't work out the logic of how it is instantiated against a cache_peer as it is only executed from ipcacheMarkBadAddr, which seems to relate to bad nodes rather than standard happy path. 

Thanks,

Adam.

The peer round-robin operates on multiple cache_peer links. To rotate individual IPs of a single server each of those IPs needs its own cache_peer line with the specific raw-IP rather than the shared server name. We do not yet have a simpler way to round-robin peers, sorry.

(the ipcacheCycleAddr is to "cycle" IPs detected as broken back into use / non-broken state - only happens if no IPs at all are working to catch any broken ones that are now fixed).

I have opened https://github.com/squid-cache/squid/pull/273 to track the missing documentation.

Amos,

 Thanks for the quick reply. Now I understand. I appreciate that aspect may have hijacked the original issue posted by Sven so the original ticket is good to be closed once the documentation is updated. 
I suspect you'll get requests for this functionality in the future though so it might be worth logging it somewhere as a feature request. I'll find a way to work around it within our infrastructure.

Thanks again,

Adam

Documentation updates applied to Suqid-5.

Applied to Squid-4