Using elgg with a load balancer... any problems? sessions?

We're reaching the top end of what a single web server can handle (We already moved all the images over to a pool of load-balanced lighttpd image servers) and now we're starting to think about adding more web servers and a load balancer for them. We're probably going to go down the route of an HAProxy software load balancer. 

Question is will running through a load balancer cause any problems for elgg?

The first thing which springs to mind is sessions. I know some session info is stored in the database but I have a feeling elgg is relying on Apache for some of it so maybe if the same client hits a different server their session will get lost? Not had time to really dig into it and confirm yet. Can anyone fill in the details?

The other issue will be IP address tracking. My custom security plugin does it but I can fix that to take the IP from the special headers added by HAProxy... I'm assuming elgg itself doesn't use client IP addresses anywhere.

Any other issues I should consider?

Thanks very much for your help!

Chris