In 2004 there was a dust storm that started on Tuesday night and never really stopped until Friday. Sustained winds weren't too hellacious, but the cumulative effects were that it never got truly hot during the day and night time was FREEZING! The weather was also severe enough that registered art no-shows and late arrivals were a significant problem. It was my first year so I had nothing to compare against, but the veterans with me mentioned several times that it seemed like there was a lot less art on the playa (there was a big furor over it over on tribe.net at the time). I noticed that most of the big art was either being worked on all week or didn't get finished until the weekend (that year was David Best's biggest temple, and took them until Friday or Saturday during the event to finish.
In 2007 the worst of it was both before and after the event, though there was a terrible storm (and the first condition alpha to happen during the event) on Friday. The storm only lasted a few hours, but at one point rangers with bullhorns actually went up the streets (we were camped at 9:30) to tell people to take cover. There was a storm on Thursday afternoon that I slept through, but was so severe as to bend the steel framework of my mutant vehicle. We got it fixed, but it took 2 days. There was a condition alpha storm at some point before the event too, it lasted more than 6 hours and set my campmates and I back a full day on trying to unload trucks or get anything set up. After the event, we thought we were safe as we struck camp on Tuesday, the weather had been clear all morning and through early afternoon and we thought whatever insane weather system we were dealing with had passed. Then in what seemed like minutes things got dusty fast, and soon the winds were so severe they were knocking over full packed bins and we couldn't see our hands in front of our faces (condition alpha). For a minute we thought we'd try and brave through it hoping that we'd get enough visibility to be able to finish taking apart the frame of our dome, but soon realized that a) there was some big shit flying around, and we didn't want to get clobbered, and b) if we tried it, we'd probably lose the nuts/bolts for the dome in the wind. Pandorra and I dropped to our knees and crawled in the direction of our van (fortunately it was where we expected it). I ducked back outside for a minute when the bins started getting blown around, only to be pelted by the stinging small black rocks as I tried to secure what I could. Later there was a little hail, followed by rain. That didn't let up until sometime just before sunup on Wednesday.