ProSoundWeb Community

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1] 2  All   Go Down

Author Topic: Nightly downtime  (Read 9394 times)

Clayton Luckie

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 989
Nightly downtime
« on: May 28, 2008, 02:35:58 AM »

I've noticed that the nightly downtime has recently started happening at Midnight Central US time.  I'm sure that it needs to happen, so thats fine.  And since this is a worldwide forum, its hard to pick a good time to do it, but midnight seems a bit early for many US users (who make up the majority of users).  Especially for the West Coast readers, shutting down at 10pm seems early.

Does the nightly maintenance have to be operated manually, by a person who is awake?  If so, then I certainly understand having it at a reasonable hour.

Just my opinions... worth their weight in dirt  Very Happy

cl
Logged

Evan Kirkendall

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 6824
    • http://www.evankirkendall.com
Re: Nightly downtime
« Reply #1 on: May 28, 2008, 12:56:37 PM »

Yeah... I was wondering about that too. It used to be midnight west coast time, which was great for me, as I'd be in bed by 2am eastern time. What's the deal with the change?



Evan
Logged
Not all change is good change.

Ernie Black

  • SR Forums
  • Full Member
  • *
  • Offline Offline
  • Posts: 190
    • http://www.standingstone.com
Re: Nightly downtime
« Reply #2 on: June 03, 2008, 08:22:04 PM »

The forum doesn't shut down nightly or on any regular schedule. There may at times be some downtime due to systems maintenance at Rackspace (our colo), or there may be the odd mysql fallover which needs to be restarted (though these are very few and far between indeed).
Logged
Ernie Black, Code Warrior
ProSoundWeb Forums Webmaster

Clayton Luckie

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 989
Re: Nightly downtime
« Reply #3 on: June 06, 2008, 11:31:07 PM »

From my experience, the forum seems unaccessible from 12:00am midnight until about 1:30am Central time.  I haven't been able to access it during that time in about a month.  Before that, the downtime seemed to happen later, maybe 1:30am or 2:00am central.

I have not checked www.prosoundweb.com at that time, but will check next time I'm up at that time.  

Just wanted you to know...  thanks, Ernie.

cl
Logged

Ernie Black

  • SR Forums
  • Full Member
  • *
  • Offline Offline
  • Posts: 190
    • http://www.standingstone.com
Re: Nightly downtime
« Reply #4 on: June 07, 2008, 01:54:57 PM »

Nothing on the PSW side of things, Clayton. The board does not shut down at any time, with the occasional exception of a power outage or other technical factor at the data center. I have no idea why you can't access it at those times.
Logged
Ernie Black, Code Warrior
ProSoundWeb Forums Webmaster

Philip Roberts

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 265
Re: Nightly downtime
« Reply #5 on: June 07, 2008, 10:43:20 PM »

Ernie,
I can corroborate, Clayton's report of the forums being down every night for a while after midnight CDT over the last couple of weeks. Before that is was midnight PDT. What ever it is also affects the two REP forums I read.

I recall some one speculating about a slow indexing process that starts a midnight. It might have been Dave Stevens but I don't see it a scan of his posts from the last year.

Is there anything we/I can do as a user to determine if this a real or phantom problem ?

Thanks

Philip

Ernie Black wrote on Sat, 07 June 2008 13:54

Nothing on the PSW side of things, Clayton. The board does not shut down at any time, with the occasional exception of a power outage or other technical factor at the data center. I have no idea why you can't access it at those times.

Logged
Philip Roberts
Director of Media Engineering
Pioneer Memorial Seventh Day Adventist Church
Berrien Springs MI

Evan Kirkendall

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 6824
    • http://www.evankirkendall.com
Re: Nightly downtime
« Reply #6 on: June 07, 2008, 10:53:23 PM »

Ernie,
Try accessing the forums around 1am EST. You'll get a blank screen. It happens every night. It's my normal browsing time after gigs, and I always have to deal with it..




Evan
Logged
Not all change is good change.

John Schmidt

  • Full Member
  • ***
  • Offline Offline
  • Posts: 157
Re: Nightly downtime
« Reply #7 on: June 08, 2008, 02:23:45 AM »

Evan Kirkendall wrote on Sat, 07 June 2008 19:53


Try accessing the forums around 1am EST. You'll get a blank screen. It happens every night. It's my normal browsing time after gigs, and I always have to deal with it..

OK - I myself happened to be online tonight a little after 10 pm Pacific, (1 am Eastern), and sure enough, there was the blank page. It finally came back up about 11:10 Pacific.

More info - this isn't a "page not found" error like the site is down - it's just a blank white page, and the loading status says "Done" at the bottom, even with a refresh. But here's where it gets weird. Viewing the source, instead of getting a pagefull of HTML, below is the sole HTML for the page at that time:


<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=windows-1252"></HEAD>
<BODY></BODY></HTML>


Nothing else at all. I tried this at several intervals with the same result. Now that it's back online, all the normal HTML code you'd expect to see is also back.

Something is apparently substituting the normal content with the above code during these downtimes - what the heck could be causing that, especially every night?


Edit: OK - I'm actually motivated to find out what's going on because for the last couple months I haven't been able to access any messages on another forum because they show as blank, though their main pages are OK. The blank pages there display the exact same source code - it's apparently plugged in when either code or settings are incorrect on the server. So I spent a while searching for info and came up with some links, (mostly tech forum threads), that might shed some illumination on the situation. Unfortunately, I'm not code savvy enough to get much further.

The odd thing here is that it seems to be a temporary disruption at this forum, while at the other site it's constant, but dynamic, because many other users can see them fine. And yes, I've tried it on numerous different computers, (like 30), so I know it's not just an individual browser issue, rather, something in the code that interacts differently depending on the client {settings} it sees. And while many of the docs refer to IIS, it appears it may involve php and sql here.

Links:

http://forums.iis.net/p/1146080/1855373.aspx#1855373

 http://blogs.msdn.com/david.wang/archive/2006/05/08/HOWTO-Us eful-ASP-page-to-return-configurable-HTTP-Status-codes.aspx

https://forums.iis.net/p/1148194/1864452.aspx#1864452

http://forums.iis.net/t/1093499.aspx?PageIndex=1

http://forums.iis.net/t/1146911.aspx?PageIndex=1

http://support.microsoft.com/default.aspx/kb/328498

'Hope this helps...

John
Logged

Ernie Black

  • SR Forums
  • Full Member
  • *
  • Offline Offline
  • Posts: 190
    • http://www.standingstone.com
Re: Nightly downtime
« Reply #8 on: June 09, 2008, 09:31:20 AM »

Clayton, John, Philip and Evan: Thanks for your input on this. I was unaware of the issue, as I'm usually asleep by 1 am EST, and we at PSW do not shut down the forums as I had said. However, last night I stayed up to 1:30 to check the status of things and run some tests in the background, and discovered what is going on. It turns out that our host, Rackspace, runs their managed database backup at that time. When the MySQL backup agent is running on the server backing up the database, the table currently being backed up is locked. This causes errors such as those you are reporting. I'm surprised that it takes so long to back up the data. They will reschedule their backups to better fit our needs. However, this being a global community and so many of our users being sound engineers working late at night, I'm not sure what the best time slot would be. I'll give that some consideration and will let you know what we decide.

Funny that no one had ever brought this to my attention before, and I never knew about it.

John, thanks for your research. IIS doesn't apply as we're running Apache on Linux.
Logged
Ernie Black, Code Warrior
ProSoundWeb Forums Webmaster

Mac Kerr

  • SR Forums
  • Hero Member
  • *
  • Offline Offline
  • Posts: 10223
thanks Ernie
« Reply #9 on: June 09, 2008, 11:25:41 AM »

Last night a bunch of us were in the Sunday Chat at 1amEDT when the backup started, so it was well documented. I was hoping to see it come back up, but at 1:45am I went to bed. Do you think there is anything that can be done to either make the back up faster, or make it not lock out the users? As you say, in an international community it is always cocktail time somewhere.

Mac
Logged
Pages: [1] 2  All   Go Up
 

Site Hosted By Ashdown Technologies, Inc.

Page created in 0.038 seconds with 21 queries.