Yeah, about those server problems...

Long time Engadget readers know it's been a while since we've seen a service outage during Steve Jobs keynotes, which basically come close to making asplode the internets. Today, unfortunately, we did see some issues -- we're hosted by AOL, which obviously has more bandwidth than God, and yet still two data centers went out. Of course, there are far more data centers hosting us than just two, so a lot of readers saw no issues whatsoever. We are already in the midst of a post-mortem, but it's pretty clear you all must really love what we've got going on here, because this was easily our most trafficked live coverage to date.

The upshot, of course, is that we're fully back online right now and anticipate zero further issues throughout the day. So head on over and hit up our hands-ons and check out all the news of all the new Apple gear! Don't worry, we can handle it.

Apple MacBook Air first hands-on
Apple TV Take 2 hands-on

The MacBook Air
The MacBook Air SuperDrive
Meet the new MacBook family
Apple introduces Time Capsule, a NAS companion to Time Machine
Apple unveils the Apple TV, take 2
iPhone firmware 1.1.3 update announced: it's just like we heard
Apple introduces iTunes movie rentals, HD rentals
Live from Macworld 2008: Steve Jobs keynote

[AlexP @ Jan 15th 2008 2:36PM]

I figured, I asked some people to check the website and they said they were having trouble connecting as well. Then I figured it wasn't limited to Engadget because Autoblog was having the same issue.

[JJV @ Jan 15th 2008 4:11PM]

I wonder how many people swarmed this site....

Any numbers Ryan?
It would be very Interesting to see and compare it with the previous keynotes!

[James @ Jan 15th 2008 6:27PM]

I could really care less about Apple or Steve'o, I just missed my other gadget news.

[Ben @ Jan 15th 2008 2:40PM]

I had to resort to m.engadget.com and another coverage site (which I had already opened).

Still, congrats for the great coverage Ryan!

[yyy @ Jan 15th 2008 2:42PM]

Maybe you should add an auto-refresh feature so that users won't have to manually refresh the page. That way, it is less likely that users would refresh the page too often and the site won't crash.

[SLAYTANIC @ Jan 15th 2008 2:49PM]

page refresh = ad impression, so don't get your hopes up.

[Eugene @ Jan 15th 2008 2:41PM]

Good to hear you guys are back up and running, I missed you for a while there *tear*

Any chance you can give us an idea of just what sort of traffic you guys were getting? Enquiring minds just love statistics like that, gives us warm fuzzies you see...

[Student Driver @ Jan 15th 2008 2:42PM]

No, what's clear is that many of us were hopeful SJ would have something cool and relevant for us. I, for one, don't really love any of what's going on.

However, you coverage was great; the keynote wasn't.

[AlexP @ Jan 15th 2008 3:15PM]

I gotta agree, his 4 latest keynotes (not only Macworld) have all been disappointments.

[Andrew @ Jan 15th 2008 2:43PM]

I think we should sue you for knowing that there would be high traffic during those times and now appropriately preparing for it, thus leaving many readers with a lack of service!

;)

[Ryan Block @ Jan 15th 2008 3:30PM]

It's one thing to prepare for a keynote, it's another thing when traffic is literally 10x what was expected. We'll let you guys know what's up.

[Andrew @ Jan 15th 2008 3:36PM]

Oh come on Ryan, I thought you of all people would have picked up on the XBL joke. ;)

[robwariii @ Jan 15th 2008 2:45PM]

your coverage is always the best. but i am a little let down by the macbook air. you can get a regular macbook cheaper and it has all the stuff like optical drive, ports, and more powerful

[Spiritsnare @ Jan 15th 2008 2:45PM]

Speaking of server issues and Apple, I love how Apple's servers are getting so slammed right now that they can only deliver the downloadable Macbook Air presentation to me at 30~50kb/s.

And this is at a work connection.

[darkstar @ Jan 15th 2008 2:48PM]

aol has more bandwidth than god??
oh man, you are going straight to hell for being blasphemous! assuming you're religion. lol. i could care less about god

[booticon @ Jan 15th 2008 2:50PM]

It's just a figure of speech. Chill.

[Wolfticket @ Jan 15th 2008 2:50PM]

I bet god has usb 3.0 as well.

[Richard @ Jan 15th 2008 2:50PM]

I don't know who dropped the ball on this one, but it certainly left a smear on the site. A lot of us count on you guys to deliver, and most of those people went elsewhere to read the live keynote blogs.

I'm sure Ryan wasn't personally responsible, nor were any of the blogging staff, but this is a high-traffic, very popular website. You'd think they'd be ready for this kind of spike in traffic?

I kept coming back over and over throughout the keynote to see if things improved, which they clearly did not.

Ah well, I really hope the infrastructure staff learns a lesson or 5 from this.

[Ari @ Jan 15th 2008 4:04PM]

unfortunately i agree. i actually headed over to gizmodo for live coverage because they were fine. doesnt change the future- engadget is still better, but i was disappointed.

[Andrew @ Jan 15th 2008 4:27PM]

Agreed. Go take some lessons from Anand on how to create a server array that doesn't buckle that easily under load.

[Ryan Block @ Jan 16th 2008 3:17AM]

You know Andrew, it's pretty easy for people to Monday morning quarterback. Ever had to set up a custom multi-city, multi-data center, edge-cached content network? It's a lot harder than you might think, and it's got a LOT of moving parts. We haven't had an outage in a really long time and unlike Xbox, we were only down a couple of hours while the rest of the internet exploded. So cut us some slack as we pick up the pieces, yeah?

[sitruc @ Jan 16th 2008 10:43AM]

Definitely one of those comment sections you should have just stayed away from, Ryan.

[Eric M. @ Jan 15th 2008 2:56PM]

Its all good guys, next time I'll let you use my Comcast High Speed for extra bandwidth. Umm just don't have any bittorrents running because you know how Comcast loves to chop speeds in half for that.

[varun @ Jan 15th 2008 2:59PM]

Still not seeing the stylesheet even after purging the cache.

I can haz stylesheet?

[Carbonize @ Jan 15th 2008 3:00PM]

Now you just need to admit that your blog script is having problems so you can start to fix them.

[Brandon @ Jan 15th 2008 3:01PM]

Any chance we might be able to get some stats on data flow & pageviews?

I remember a couple years back you'd chime in from time to time during the keynote with page-view stats.

[zack @ Jan 15th 2008 3:04PM]

I know macrumors.com has some funky ajax stuff that manages to update the page without reloading everything each time (ie all those images mainly). Any chance you could implement something similar?

Thanks for the keynote coverage though!

[frank @ Jan 15th 2008 3:09PM]

Why doesn't Apple just use Silverlight and stream the keynote like what Microsoft did for CES. I guess they know they probably couldn't keep their servers from crashing either.

[Chad @ Jan 15th 2008 8:32PM]

Or use a flash video service. Whoever has that working properly next year is going to win. Fill the rest of the page with ads for all I care.

[Udayan Tripathi @ Jan 15th 2008 3:12PM]

meh no biggie. it was a huge event.

[jason w @ Jan 15th 2008 3:29PM]

I want a free arcade game (still havent seen the one MS promised me) for compensation....... :)

[Paul @ Jan 15th 2008 3:32PM]

The difference is you don't pay for access to the blog, so it can go down whenever it pleases and you shouldn't say crap.

XBox live you are paying a monthly fee to access so it should stay up and when it doesn't people have a right to complain, but thats about it since they agreed to the xbox live TOS.

[Peter @ Jan 16th 2008 9:24AM]

Eh, well you certainly weren't the only one. Techcrunch barely stood up, Crunchgear's coverage failed completely, even Twitter was down. The only site I saw with reasonable robustness was macrumorslive.com.

[andy @ Jan 15th 2008 3:36PM]

Having xbox live flashbacks over here.

[bdkennedy1 @ Jan 15th 2008 3:42PM]

Well, I'm glad the site is back up now, but the point is that this happens every year. Then Engadget has a whole year to do something about it, and when the time comes, nothing. It's redundant to make a big hoopla about live coverage of the keynote, and then no one can get to it.

[JJV @ Jan 15th 2008 3:56PM]

Well, last year i had no problems reading about the keynote. But as Ryan Said, 10X the amount of people is a lot. We crashed the servers, I even went on to the apple site and it started crashing a bit as floods of people surfed onto it.

It happens a lot especially if you are refreshing constantly on a page, And technically it is AOL's fault for not having reliable servers. But oh well, no use crying over spilt milk.

[Vince @ Jan 15th 2008 3:39PM]

I truly hope that you are joking, or you really have some issues.

[wesg @ Jan 15th 2008 3:40PM]

How about offering a page of the keynote with no bells and whistles? ie. minima graphics and comments on a separate page?

[JJV @ Jan 15th 2008 3:53PM]

yeah, i was thinking that they should have a keynote page with no pictures and just text about it which would reduce bandwidth problems.

But i bet millions (literally) were reading/watching that keynote.

Its kinda funny how MacWorld practically overshadows everything else. The Jobsian Effect, i guess.

[Vince @ Jan 15th 2008 3:44PM]

People need to chill. You're not paying for anything.

I think Engadget had the best coverage, but I was going to CNET to get frequent updates, but then getting more pictures and details whenever Engadget came up.

I lived, and I'm not whining about it.

[sitruc @ Jan 15th 2008 4:26PM]

You're missing the point. Engadget said they made improvements and they knew what to expect. My time is money. Therefore I pay for visiting this site. I want my money back. I don't care about games and gimmicks that I won't use. They didn't even say anything about the server problems until they claimed they were resolved. I had problems after this article was posted. I've spoken to some of my friends in Texas and they definitely aren't cool with this.

[Vince @ Jan 15th 2008 8:29PM]

I'm definitely not missing any point. I think that you are taking this way too seriously. Engadget is a free blog. If you don't like it, don't visit it. Nobody is forcing you to spend your time here. You don't "pay money" to come here, you decide to come here in your extra time. There are plenty of other sites that you could have gone to to get live coverage, and you wouldn't have missed a beat.

What are you going to do? Sue them? And what is "I've spoken to some of my friends in Texas and they definitely aren't cool with this" supposed to mean?

[sitruc @ Jan 16th 2008 1:26AM]

I think you're taking my post too seriously...:(

Go back a few days and look at some of those xbox live outage threads.

HINT: I was kidding.

[haydenseq @ Jan 15th 2008 4:28PM]

Gizmodo had no problems this year. I guess it pays to have two, nearly identical, gadget blogs. You can never be too redundant when needing the newest stuff yesterday.

[theCardinal @ Jan 15th 2008 5:32PM]

I bet you sympathize with the xbox live guys now. :D

[Scott @ Jan 15th 2008 7:12PM]

Sorry Ryan, but I was trying to use Engadget, but I was using mostly Gizmodo. But you guys still Rock!!

[webon @ Jan 15th 2008 7:51PM]

Icon CHANGE!!

[jamesFF @ Jan 15th 2008 8:20PM]

Why is every article on this page crossed out?

[Chad @ Jan 15th 2008 8:30PM]

Part of the downfall: reloading the entire page! Yes, the images should be cached, but come on! After waiting for 20 minutes to connect (North East area), I went over to mac rumors. They had a frame in the middle of the page that automatically refreshed with updates and a column of photos on the right that refreshed automatically too... No scrolling on my behalf at all! It was great. You'll have to do much better than the scroll-a-thon next year to keep my impressions.