Here is a fantastic overview of a real, working, high-performance system which carries a big load
Web conferencing services - inside and out
This week's featured article offers four case studies in Java-based Web conferencing services development. Like all profiles in this new series, this week's profile concludes with a snapshot of the development platform, system hardware, and lessons learned in the process of bringing these Java-based services to life. Ed Hardebeck, vice president of engineering for Glance, submitted the following specs for this desktop sharing tool:
Java, C++, Assembler when it needs to be done this nanosecond
At two co-location sites, one east coast and one west coast, we run clusters of inexpensive Intel-based servers.
Web servers are IIS on Windows using Windows Load Balancing (Web service is not a bottleneck for us).
Database is replicated between east and west coasts.
Web services and database use proprietary monitoring and fail-over architectures. Either co-lo could die completely and we'd still be live.
Communication servers that route and manage our streaming media use our proprietary traffic-based load balancing.
Because connection speed is so important in a Web conferencing application, we have redundant optimized routing to all major Internet backbones.
What we know now
Release early, rev often.
You can't have too much bandwidth or too little latency.
Connections are more important than memory; memory is more important than CPU.
Multi-threading is a convenient, elegant, and slow way to write a server. Locking and context-switching are not your friends.
Monitor everything (we use Hobbit as well as many custom scripts), and have extensive logging and analysis to diagnose bottlenecks and problems.
For any kind of online service, everything reflects on you, even things you can't control. And the hardest thing to control is the last mile, and the last five feet. The customer's experience depends so much on their Internet connection and their local firewall, proxy server, wireless connection or $50 router (especially when streaming data up!). You can't predict what speed of machine with what video card or driver they'll have, what operating system of what version, what personal firewall, what browser or what version of Java. You just have to always have another technology trick up your sleeve, and write code that better and better adapts to whatever environment it finds.