Has The Messenger Team Gone Out Of Their Minds?! Options

[TheBlasphemer]

As you might know, Messenger will freeze if someone sends you an extremely big message.
Up till now, I figured it'd just be some kind of bug, and I felt that it was not my duty to fix this.
However, recently someone at the Messenger Plus! forum wrote a little script that would "fix" the freezing, and that got me curious as to what might be causing it.
After about an hour of reversing, I found which piece was locking up, and to my suprise it wasn't a bug, and it hadn't actually frozen, it was just terribly busy!
It's related to the fact that messenger checks each and every message for security (like .pif and such), and part of that is done on the client side. This is done through so called "regular expressions", which is a pretty nice method of matching certain parts of a string against something specific. The only downside to regular expressions is that if used improperly, they can get rather slow.
Messenger checks your message against more than 97 (I got tired of counting) of these regexps, most of which are written incredibly inefficient. An example is:
".*groupicture\.php.*"
The .* before and after the regexp basically mean "match everything". These are very powerful, but one of the slowest bits of Regular expressions! and most importantly, as regular expressions only try to match part of a string, they are completely unneeded in this context, as it basically means "find grouppicture.php with anything in front or after it", while "find grouppicture.php" alone would cut it :/
Matching 97 of such regular expressions on a large string takes *ages*, and that's what's freezing your messenger... If you remove that checking, messenger will have no problem whatsoever with handling bigger messages :S

So yeah, I guess this means that I'm going to wrote a feature in SP to disable these security checks, to speed up your messenger. I feel it's a good thing to build in some security, but 97 regular expressions that could've been replaced by simple string searches is just outrageous...

EDIT: I've now completed a feature that replaces the simple regular expressions (of form .*constant.*) with a string-search. This means that messenger would no longer freeze as long as it did before, while still keeping the security at the same level as before Expect it in the next build!

[jerone]

so i'm wondering; are you ganna delete this security check or are you ganna replace it with normal searches in SP ?

[TheBlasphemer]

so i'm wondering; are you ganna delete this security check or are you ganna replace it with normal searches in SP ?

Not quite sure yet.
First thing I did was inform some people at MS about this, so they get a chance to fix it too.
And now I'm researching what the best option would be.
The best solution would be to replace those silly regexps with a simple string-search, but I'm not sure if that's feasible/possible.
So I'm going to try and just make it faster, but if that doesn't work, I'll just remove the security checks completely, and warn about that when enabling the feature.

[Annuix]

At least people are getting known about this matter before hand and I'm sure Microsoft have a good reason to check it against 97 different expressions, otherwise they wouldn't need to do it.
I'm also really suprised that WLM was actually extremely busy, not freezing because it does the same thing on other known, poweful pc's (a pal's Intel Extreme Core, blah blah) and seriously should of done these security checks in a matter of a few ticks.

Anyway, this would be a great feature to implement if MS do not sort it out themselves, as it's not the large messages i'm worrying about as most people can't send really large messages, it's those dreaded multi convo's

[TheBlasphemer]

At least people are getting known about this matter before hand and I'm sure Microsoft have a good reason to check it against 97 different expressions, otherwise they wouldn't need to do it.

They have a good reason to check against 97 known virus-URLs. However, they certainly do not have a good reason to use regular expressions for this, as a simple string-search (which would be at least 100x faster) would be more than sufficient for these cases.

I'm also really suprised that WLM was actually extremely busy, not freezing because it does the same thing on other known, poweful pc's (a pal's Intel Extreme Core, blah blah) and seriously should of done these security checks in a matter of a few ticks.

That's the whole problem. A simple string search, or even 97 of them, would take a few ticks at most. However, a regular expression, especially on a long string, takes several ms, or sometimes even a few second, each. Multiply that by 97, and you see why it appears to be freezing.

[NF2K]

Heh, yeah I posted this to bugs some time ago, and figured it would have been some algorithm (or RegEx for that matter) checking through the contents of the messages for some reason or other. I realised it wasn't a full crash as I left it for some time to see if it would recover, which it did, as you say. The longer the message; the longer the lock-up.

I actually got as bored as to write an autotalker using the old SP2 which would send an amount of random characters the user would specify. This would allow bypass of the 5,000 character limit. An important side-note in case you intend to re-implement it in any future versions of StuffPlug.

Additionally, if this RegEx checking is to be removed by StuffPlug, what security risks does this imply exactly? Thanks.

[TheBlasphemer]

Additionally, if this RegEx checking is to be removed by StuffPlug, what security risks does this imply exactly?

If I would completely remove the checking, this would mean that URLs or texts that the messenger servers tell you to block, would no longer be blocked. Basically this would mean that you can receive URLs pointing to virii and such.

[Keikonium]

I assume these regex check against sending files right?
So if you remove them, would I beable to send .exe files?
Would the other person need SP3 also to accept the file?

[doggie]

Sorta means your bypassing security to increase stability?

[Aardvark]

I'd rather stability over security any day! That's why I got rid of AV software completely, but then when it infected and broke all of my installers I kinda had to bring good ol' Norton back to fix things, anyway that's off topic.

Sure this may fix the issue with sending singular large messages, but would it also work for multiple large chunks of text? Thus meaning it is now impossible to get lagged by consistent large blocks of writing?

[TheBlasphemer]

I'd rather stability over security any day! That's why I got rid of AV software completely, but then when it infected and broke all of my installers I kinda had to bring good ol' Norton back to fix things, anyway that's off topic.

Sure this may fix the issue with sending singular large messages, but would it also work for multiple large chunks of text? Thus meaning it is now impossible to get lagged by consistent large blocks of writing?

As I'm not removing the security checks, merely changing them to faster checks, you may still encounter lag when you receive subsequent large chunks of text. However, I think it'd be fast enough not to really notice that

[warmth]

very nice feature TB... I hope to try it soon...

[Annuix]

Ahh, I understand now...it's suddenly all clear.

The removal of a more instense security check for a faster, more reliable one which would therefore improve the speed and stability of WLM. The security checks are still there as they have been replaced...not removed. Okay, got it.

Nice to see that Stuffplug is actually being made to improve the stablity of messenger instead of adding features to it, it's seems like a nice project to take on. I like the idea of a multi-purpose addon.

I was going to ask about how hard it would be...it seems like a pretty complex task. But as I've just seen your edit there is nothing to worry about.

Finally no more freezing (major). Yay

This post has been edited by Annuix: Apr 26 2007, 10:53 PM

[Salem]

Excellent new TB. Looking forward to the next update.

[Matti]

No way... Who on earth would write something like that in an application like Messenger? Maybe Microsoft should fire half of their employees and get some real developers on the projects!

Good work, TB. I'm amazed how you manage to fix all stuff Microsoft did wrong. You're going to enable that feature by default, I hope?

[J-Thread]

Maybe Microsoft should fire half of their employees and get some real developers on the projects!

They already started with the latter...

[vikke]

TheGuruSupremacy, the creator of the Messenger Plus! Live script found that the Messenger team now removed their checks on client side. So this feature is no longer needed.

[deAd]

TheGuruSupremacy, the creator of the Messenger Plus! Live script found that the Messenger team now removed their checks on client side. So this feature is no longer needed.

Microsoft probably won't release another build for a while, they wouldn't release a special build just to fix one bug like this so it'll be needed until Messenger is updated.

[Aardvark]

Does this mean it will be fixed in the next messenger build, or is it already fixed on the server? I ask this because the other day I was doing some lag tests and for some strange reason it wasn't lagging as much as it should.

[vikke]

I don't really see how TheGuruSupremacy could see this without an WLM update from Microsoft (maybe they contacted him?).

However, I think you should keep this feature for at least this WLM release.