Anope IRC Services

Please login or register.

Login with username, password and session length
Advanced search  

News:

Pages: [1]   Go Down

Author Topic: Services crash on netsplit  (Read 13903 times)

0 Members and 1 Guest are viewing this topic.

Aristeo

  • Anope User
  • Offline Offline
  • Posts: 10
Services crash on netsplit
« on: March 24, 2008, 02:08:40 PM »

Hey,

About 40% of the time we have a netsplit (server/server split), services die with the following output (or something similar):

Quote
<Global> Backtrace: report the following lines
<Global> Backtrace: Anope version 1.7.21 (1341) build #1, compiled Jan 25 2008 20:03:27  QM
<Global> Backtrace(0): /home/ecnet/services/services(do_backtrace+0x70) [0x136700c0]
<Global> Backtrace(1): /home/ecnet/services/services(sighandler+0x13d) [0x136702bd]
<Global> Backtrace(2): /lib/libc.so.6 [0xa0e20aa8]
<Global> Backtrace(3): /home/ecnet/services/services(chan_set_modes+0x3ad) [0x13651bdd]
<Global> Backtrace(4): /home/ecnet/services/services(do_sjoin+0x40d) [0x13652a0d]
<Global> Backtrace(5): /home/ecnet/services//modules/runtime/unreal32.so.pAimAB(anope_event_sjoin+0x3b) [0xa0deb01b]
<Global> Backtrace(6): /home/ecnet/services/services(process+0x304) [0x13687694]
<Global> Backtrace(7): /home/ecnet/services/services(main+0x4d3) [0x1366ff93]
<Global> Backtrace(8): /lib/libc.so.6(__libc_start_main+0xe2) [0xa0e0cff2]
<Global> Backtrace(9): /home/ecnet/services/services [0x1364ae61]
<Global> Backtrace: complete
<Global> Services terminating: Segmentation fault


And two globals like these go out to ircops

Quote

[euphoria.irc.ecnet.org]: *** Global -- from services.ecnet.org: PANIC! buffer = :laser.irc.ecnet.org SJOIN !17vl9} #translator +ntr :Things
[euphoria.irc.ecnet.org]: *** Global -- from pseudo-servers.irc.ecnet.org: Received SQUIT services.ecnet.org from services.ecnet.org[209.9.228.99] (Services terminating: Segmentation fault)


Anyone have an idea as to what's going on?
Logged

Jan Milants

  • Team
  • *
  • Offline Offline
  • Gender: Male
  • Posts: 1372
(No subject)
« Reply #1 on: March 24, 2008, 02:13:07 PM »

which version of unreal are you running?
any third party modules(both anope and ircd)?

[Edited on 24-3-2008 by Viper]
Logged
If you like me donate coins to 1FBmZVT4J8WAUMHKqpWhgNVj3XXnRN1cCk :)

Aristeo

  • Anope User
  • Offline Offline
  • Posts: 10
(No subject)
« Reply #2 on: March 24, 2008, 02:34:40 PM »

We're running 3.2.5 on a server, 3.2.6 on a server, and 3.2.7 on two servers.

All of our servers are running these modules:

Quote

*** cloak - $Id: cloak.c,v 1.1.2.3 2005/02/04 00:26:37 syzop Exp $ (Official cloaking module (md5))
*** commands - $Id: l_commands.c,v 1.1.2.63.2.11 2007/01/22 12:46:54 syzop Exp $ (Wrapper library for m_ commands)
*** antirandom - v0.1.1 (Randomness detector) [3RD]
*** m_privdeaf - v0.0.6 (private messages deaf (+D)) [PERM] [3RD]


A couple of our servers are running one or more of these modules:

Quote

*** hideserver - $Id: hideserver.c,v 4.8 2004/04/05 07:25:04 angrywolf Exp $ (Hide servers from /map & /links) [3RD]
*** cgiirc - $Id: cgiirc.c,v 1.17 2004/07/12 21:25:03 angrywolf Exp $ (CGI:IRC support) [3RD]
*** ircops - v3.6 (/IRCOPS command that lists IRC Operators) [3RD]


And Anope is running the following modules:

Quote

Module: enc_old [$Id$] [Encryption]
Module: unreal32 [$Id: unreal32.c 1291 2007-08-30 20:59:14Z drstein $] [Protocol]
Logged

Jan Milants

  • Team
  • *
  • Offline Offline
  • Gender: Male
  • Posts: 1372
(No subject)
« Reply #3 on: March 24, 2008, 02:42:01 PM »

mm none of those ircd modules should cause a crash...
can't predict the effect of running different versions of unrealircd on multiple servers though. The documentations says it s possible for a few days during an upgrade round, but it says nothing about doing it on long term... and i know for sure it s not recommended

[Edited on 24-3-2008 by Viper]
Logged
If you like me donate coins to 1FBmZVT4J8WAUMHKqpWhgNVj3XXnRN1cCk :)

Jobe

  • Contributor
  • *
  • Offline Offline
  • Gender: Male
  • Posts: 1023
    • Anope IRC Services
(No subject)
« Reply #4 on: March 24, 2008, 02:43:58 PM »

Well firstly, in the Unreal 3.2 series, CGI:IRC support is built in so consequently, no UnrealIRCd 3.2.* server needs the cgiirc module.

[Edited on 24-3-2008 by Jobe1986]
Logged
Your IP: ()
My IRC Status:

Come along and visit http://www.anopequotes.org/

Aristeo

  • Anope User
  • Offline Offline
  • Posts: 10
(No subject)
« Reply #5 on: March 26, 2008, 03:24:40 AM »

So does anyone have any suggestions on what we should do?
Logged

Jan Milants

  • Team
  • *
  • Offline Offline
  • Gender: Male
  • Posts: 1372
(No subject)
« Reply #6 on: March 26, 2008, 09:39:59 AM »

You could start with running the same version of an ircd on all servers as the unreal team advises and then see if it still happens..
Logged
If you like me donate coins to 1FBmZVT4J8WAUMHKqpWhgNVj3XXnRN1cCk :)

Aristeo

  • Anope User
  • Offline Offline
  • Posts: 10
(No subject)
« Reply #7 on: March 27, 2008, 08:20:35 PM »

Quote
Originally posted by Viper
You could start with running the same version of an ircd on all servers as the unreal team advises and then see if it still happens..


Well, let me ask this in a different way:

What does the error mean? What condition causes Anope to throw this error?
Logged

reznic

  • Anope User
  • Offline Offline
  • Posts: 5
(No subject)
« Reply #8 on: March 29, 2008, 06:16:43 PM »

we get the same thing when new servers link and this just started after upgrading to anope-1.7.21, version 1.7.20 did not have this problem.  all the servers are running the same ircd version of unreal3.2.7

ReZNiC
Logged

katsklaw

  • Guest
(No subject)
« Reply #9 on: March 30, 2008, 01:41:20 PM »

I'm gunna guess bad ulines on 1 or more of the servers.
Logged

reznic

  • Anope User
  • Offline Offline
  • Posts: 5
(No subject)
« Reply #10 on: April 10, 2008, 01:37:25 AM »

Well we can rule bad ulines out since each server and each set of services and the new servers have the same ulines of all hubs and services, etc.  Now to clearify, the attempt to connect NEW servers, not just linking servers that had already been linked before, but brand new never linked servers.  And before you ask, the C & H lines were set accordingly in the servers and hubs.  Once putting version 1.7.20 back up there was no problems.  The new servers connected just fine.  Without any crash/segmentation fault of either services.  BTW, the second set of services are ran with -readonly as a backup, shame it doesn't work the way its read to believe.  But that is another problem and not specific to this thread.

ReZNiC
Logged

katsklaw

  • Guest
(No subject)
« Reply #11 on: April 10, 2008, 02:08:07 AM »

well hundreds of others user unreal/Anope without this issue so it has to be system/config specific.

Secondly. hideserver unreal module may have adverse effects on Anope. Try without any additional 3rd party ircd modules.

Thirdly, that's not a complete list of Anope modules. Anope loads more than 2 modules in it's default configuration.
Logged

reznic

  • Anope User
  • Offline Offline
  • Posts: 5
(No subject)
« Reply #12 on: April 10, 2008, 04:22:30 AM »

I'm guessing then we are without any help on this severe problem that anope-1.7.21 has that anope-1.7.20 doesn't have.  Guessing again that we'll have to wait until anope coders figure out they have a boo boo and get it fixed with another update.  Until then, guess we all have to hang with the security problem of anope-1.7.20.


ReZNiC
Logged

Jan Milants

  • Team
  • *
  • Offline Offline
  • Gender: Male
  • Posts: 1372
(No subject)
« Reply #13 on: April 10, 2008, 03:07:35 PM »

if the programmers aren't told they there is a problem (and very exact), they can't fix it... as for anope-1.7.20.... any of your users could crash it, hence why an upgrade to .21 is strongly recommended

.21 was mainly a quick fix release of .20 and contains very little other changes (of which most only affect inspircd anyways).. just check the changelog

and
Quote
guess we all

i m afraid you 2 are the only ones with a problem out of 1000s :)
... maybe you too are running different versions of unrealircd on your net?

[Edited on 10-4-2008 by Viper]
Logged
If you like me donate coins to 1FBmZVT4J8WAUMHKqpWhgNVj3XXnRN1cCk :)

katsklaw

  • Guest
(No subject)
« Reply #14 on: April 10, 2008, 05:03:48 PM »

Quote
Originally posted by reznic
I'm guessing then we are without any help on this severe problem that anope-1.7.21 has that anope-1.7.20 doesn't have.  Guessing again that we'll have to wait until anope coders figure out they have a boo boo and get it fixed with another update.  Until then, guess we all have to hang with the security problem of anope-1.7.20.


ReZNiC


What you are failing to see is that YOU are the only one with the problem! You don't have access to see the bug lists and other reports that I do .. and I'm telling you NO ONE ELSE  is seeing this issue but YOU!!

Thousands of other users use 1.7.21 WITHOUT this problem. This logical statement says that it's NOT in Anope 1.7.21!! You are simply experiencing this problem with 1.7.21.

Anope is just ONE out of hundreds of variables that are in play here.

Additionally, you ARE getting help, you just fail to see that too. I and others have asked you to try different things to help find the problem, your response was to blame Anope instead of helping yourself fix your problem.

EVERYONE that replied to you in this thread use 1.7.21 with NO problems!!!

2 people out of 5000 doesn't make the problem ours. It just means that the 2 of you have something in common other than using Anope-1.7.21 and Unreal.
Logged

Jobe

  • Contributor
  • *
  • Offline Offline
  • Gender: Male
  • Posts: 1023
    • Anope IRC Services
(No subject)
« Reply #15 on: April 10, 2008, 06:09:24 PM »

I can also guarantee Anope 1.7.21 doesn't crash on net splits on OTHER networks, which goes as far as proving it's something localised to YOU!!
Logged
Your IP: ()
My IRC Status:

Come along and visit http://www.anopequotes.org/

reznic

  • Anope User
  • Offline Offline
  • Posts: 5
(No subject)
« Reply #16 on: April 10, 2008, 07:15:38 PM »

Its very apparent that the message content wasn't read completely.  In our senario, it isn't the netsplit item discussed, but the linking of a new (never linked) server that causes the same type of crash with the segmentation fault under version .21.  This is a panic crash segmentation fault.
Anope doesn't crash on a netsplit or a known server link, just on a brand new server link.  This senario may not have been seen on others using Anope if they have not had a new server link.

ReZ
Logged

katsklaw

  • Guest
(No subject)
« Reply #17 on: April 10, 2008, 08:04:51 PM »

Quote
Originally posted by reznic
Its very apparent that the message content wasn't read completely.  In our senario, it isn't the netsplit item discussed, but the linking of a new (never linked) server that causes the same type of crash with the segmentation fault under version .21.  This is a panic crash segmentation fault.
Anope doesn't crash on a netsplit or a known server link, just on a brand new server link.  This senario may not have been seen on others using Anope if they have not had a new server link.

ReZ


You are correct, you did not read the message completely. The topic at hand is "Services crash on netsplit". If you are talking about something other than Services crash on netsplit please create another thread and don't hijack someone's thread.

Additionally, panic crashes can be cause by malformed data that is sent to services from a trusted source such as the ircd. It's not possible to foresee all possible scenarios and the most logical solution is to stop the malformed data from being sent to Services. Since thousands of other users also use Unreal and Anope-1.7.21 without issues, it's logical to assume that the malformed data is coming from a 3rd party module or there is something in your specific system configuration that is causing the crash. The crashing program is NOT the only reason  that cause said crash, it IS however the only thing that we can control.

I can promise you that out of the thousands of nets using Anope1-7.21 that at least 1 other network has added a new server since it's release.

You have already been asked to unload all 3rd party modules and try again, you have since ignored this request.

We offer help only to those wiling to do as we ask, since you are obviously not interested in following simple instructions, please stop posting your accusations before you get banned.

[Edited on 10-4-2008 by katsklaw]
Logged

Jan Milants

  • Team
  • *
  • Offline Offline
  • Gender: Male
  • Posts: 1372
(No subject)
« Reply #18 on: April 10, 2008, 08:19:23 PM »

Quote
Originally posted by reznic This is a panic crash segmentation fault.


you still haven't posted a debug log nor a backtrace though...
http://anope.info/wiki/Other:GDB
we need the backtrace and the last portion of the debug log

but like katsklaw said, i m guessing it s probably malformed data being send to anope..

[Edited on 10-4-2008 by Viper]
Logged
If you like me donate coins to 1FBmZVT4J8WAUMHKqpWhgNVj3XXnRN1cCk :)

reznic

  • Anope User
  • Offline Offline
  • Posts: 5
(No subject)
« Reply #19 on: April 11, 2008, 12:35:31 AM »

Nevermind

Sorry I even offered the info.  Oh, btw, a backtrace was never asked for but yet I do have that.  I'll see where the error is and fix it.

Good Day,
ReZNiC
Logged

Aristeo

  • Anope User
  • Offline Offline
  • Posts: 10
(No subject)
« Reply #20 on: April 28, 2008, 05:50:41 AM »

Thank you for trying to help with the problem we have been having with Anope Services. Unfortunately, after trying everything you all suggested and a couple of additional suspected problem-causers, Anope Services still continues to crash occasionally on a netsplit.

To make the problem less dramatic for now, we plan on setting up a cron job that restarts services whenever they crash. Because this issue doesn't seem to have a resolution, and since, to be honest, a few of our users haven't been satisfied with Anope Services, we have started development on a new IRC services package that will hopefully serve to resolve our technical difficulties and better suit our users' tastes.

I'll still keep an eye on this thread in case someone finds something that may be causing the problem. Thanks again for the help!
Logged

Jobe

  • Contributor
  • *
  • Offline Offline
  • Gender: Male
  • Posts: 1023
    • Anope IRC Services
(No subject)
« Reply #21 on: April 28, 2008, 03:10:11 PM »

Out of curiosity, have you tried running Anope on a different box to see if it still crashes then?

If it doesn't, you could consider moving Anope to that box and still keep using Anope too then.
Logged
Your IP: ()
My IRC Status:

Come along and visit http://www.anopequotes.org/
Pages: [1]   Go Up