Forum outage 2020-01-14(ish)

Talk about whatever you feel like.
User avatar
Martin Blank
Knower of Things
Knower of Things
Posts: 12685
Joined: Fri Feb 07, 2003 4:11 am
Real Name: Jarrod Frates
Gender: Male
Location: Dallas, TX

Forum outage 2020-01-14(ish)

Post by Martin Blank » Thu Jan 21, 2021 1:50 am

Not quite sure what happened, but the Apache2 service on the forums stopped. I got logged in and restarted it, and now, of course, it's up (at least while I'm writing this).
If I show up at your door, chances are you did something to bring me there.

User avatar
Doc Giggles
Redshirt
Posts: 807
Joined: Fri Feb 14, 2003 8:13 am

Re: Forum outage 2020-01-14(ish)

Post by Doc Giggles » Thu Jan 21, 2021 2:25 am

I was wondering what had happened. Glad you got it taken care of.
Disciple of Gnarr
“Let there be truth, happiness, and waffles”
-Nomad, Dark Lord & Avatar of Gnarr

nosystemd
Redshirt
Posts: 94
Joined: Thu Jan 24, 2019 2:27 am

Re: Forum outage 2020-01-14(ish)

Post by nosystemd » Thu Jan 21, 2021 3:08 am

Thanks for fixing this.

User avatar
Deacon
Shining Adonis
Posts: 44205
Joined: Wed Jul 30, 2003 3:00 pm
Gender: Male
Location: Lakehills, TX

Re: Forum outage 2020-01-14(ish)

Post by Deacon » Thu Jan 21, 2021 4:31 am

Thank you!
The follies which a man regrets the most in his life are those which he didn't commit when he had the opportunity. - Helen Rowland, A Guide to Men, 1922

User avatar
Jamie Bond
Agent 0.07
Agent 0.07
Posts: 4736
Joined: Fri Feb 14, 2003 11:05 pm
Gender: Male
Location: BC, Canada

Re: Forum outage 2020-01-14(ish)

Post by Jamie Bond » Thu Jan 21, 2021 6:00 am

I appreciate your time on this :)
"Go get the Happy!"

User avatar
BtEO
Redshirt
Posts: 4798
Joined: Tue Feb 18, 2003 2:28 pm
Location: England

Re: Forum outage 2020-01-14(ish)

Post by BtEO » Thu Jan 21, 2021 11:24 am

Thanks.

User avatar
Rorschach
The Immoral Immortal
Posts: 17734
Joined: Tue Feb 18, 2003 7:35 am
Gender: Male
Location: Glasgow, Scotland

Re: Forum outage 2020-01-14(ish)

Post by Rorschach » Thu Jan 21, 2021 12:41 pm

Yay! We're back on!

Thank you.
To Let

User avatar
FirebirdNC
Mad Hatteras
Posts: 1964
Joined: Tue Feb 13, 2007 3:00 pm
Real Name: Jennifer
Gender: Female
Location: Hatteras,NC

Re: Forum outage 2020-01-14(ish)

Post by FirebirdNC » Thu Jan 21, 2021 9:07 pm

I am as always both pleased and regretful to be here hanging out with my fellow nerdlings.
~Insert clever bon mot here~

User avatar
Martin Blank
Knower of Things
Knower of Things
Posts: 12685
Joined: Fri Feb 07, 2003 4:11 am
Real Name: Jarrod Frates
Gender: Male
Location: Dallas, TX

Re: Forum outage 2020-01-14(ish)

Post by Martin Blank » Fri Jan 22, 2021 5:10 pm

I think I may have figured out what is causing it, but not why it's happening.

I use Let's Encrypt for all HTTPS certificates. The certbot process that it currently uses requires that it stop the HTTP service, does its thing, then restarts the service. What seems to be happening is that the service is getting stopped, but not necessarily restarted. The major reasons for this are the graceful stop of the HTTP service, and also that at least this morning's outage was roughly coincident with a failed cert update attempt.

There are a couple of possibilities for this, one of them being that the legacy "service apache2 stop" and "service apache2 start" commands are for some reason not properly executing (the modern commands are "systemctl stop apache2" and "systemctl start apache2").

Another is that certbot may have changed somehow in the last few weeks in a breaking way. This wouldn't be the first time certbot made a breaking change, especially regarding HTTP renewals. It does give me the impetus to make a move to the more stable DNS update method. I kind of know how that is supposed to work, but not entirely.

In the meantime, I have disabled the certbot service until I can get whatever it is figured out. Hopefully, that will at least get the server stable.
If I show up at your door, chances are you did something to bring me there.

User avatar
Martin Blank
Knower of Things
Knower of Things
Posts: 12685
Joined: Fri Feb 07, 2003 4:11 am
Real Name: Jarrod Frates
Gender: Male
Location: Dallas, TX

Re: Forum outage 2020-01-14(ish)

Post by Martin Blank » Sun Jan 24, 2021 6:05 pm

Well, that didn't go as planned. Trying to make some things work, I broke some networking things, and then the server was completely unavailable. The provider's link to the VMWare remote console was broken, and it took them two days to fix. Frustrating as hell. Things are mostly working but not entirely. Trying to fix all the issues.
If I show up at your door, chances are you did something to bring me there.

User avatar
Deacon
Shining Adonis
Posts: 44205
Joined: Wed Jul 30, 2003 3:00 pm
Gender: Male
Location: Lakehills, TX

Re: Forum outage 2020-01-14(ish)

Post by Deacon » Mon Jan 25, 2021 1:39 am

Martin Blank wrote: Sun Jan 24, 2021 6:05 pmThings are mostly working but not entirely. Trying to fix all the issues.
As frustrating as it is, it’s working now! Thank you :)
The follies which a man regrets the most in his life are those which he didn't commit when he had the opportunity. - Helen Rowland, A Guide to Men, 1922

Who is online

Users browsing this forum: Common Crawl (Research) and 1 guest