IPB

Welcome Guest ( Log In | Register )

3 Pages V  1 2 3 >  
Reply to this topicStart new topic
> Has The Messenger Team Gone Out Of Their Minds?!
TheBlasphemer
post Apr 25 2007, 07:24 PM
Post #1


Administrator
***

Group: Root Admin
Posts: 270
Joined: 27-May 06
Member No.: 1



As you might know, Messenger will freeze if someone sends you an extremely big message.
Up till now, I figured it'd just be some kind of bug, and I felt that it was not my duty to fix this.
However, recently someone at the Messenger Plus! forum wrote a little script that would "fix" the freezing, and that got me curious as to what might be causing it.
After about an hour of reversing, I found which piece was locking up, and to my suprise it wasn't a bug, and it hadn't actually frozen, it was just terribly busy!
It's related to the fact that messenger checks each and every message for security (like .pif and such), and part of that is done on the client side. This is done through so called "regular expressions", which is a pretty nice method of matching certain parts of a string against something specific. The only downside to regular expressions is that if used improperly, they can get rather slow.
Messenger checks your message against more than 97 (I got tired of counting) of these regexps, most of which are written incredibly inefficient. An example is:
".*groupicture\.php.*"
The .* before and after the regexp basically mean "match everything". These are very powerful, but one of the slowest bits of Regular expressions! and most importantly, as regular expressions only try to match part of a string, they are completely unneeded in this context, as it basically means "find grouppicture.php with anything in front or after it", while "find grouppicture.php" alone would cut it :/
Matching 97 of such regular expressions on a large string takes *ages*, and that's what's freezing your messenger... If you remove that checking, messenger will have no problem whatsoever with handling bigger messages :S

So yeah, I guess this means that I'm going to wrote a feature in SP to disable these security checks, to speed up your messenger. I feel it's a good thing to build in some security, but 97 regular expressions that could've been replaced by simple string searches is just outrageous...

EDIT: I've now completed a feature that replaces the simple regular expressions (of form .*constant.*) with a string-search. This means that messenger would no longer freeze as long as it did before, while still keeping the security at the same level as before smile.gif Expect it in the next build! smile.gif
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
jerone
post Apr 25 2007, 08:22 PM
Post #2


Newbie
*

Group: New Members
Posts: 4
Joined: 14-April 07
From: Haarlem, Netherlands
Member No.: 2,050



so i'm wondering; are you ganna delete this security check or are you ganna replace it with normal searches in SP ?


--------------------
IPB Image
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
TheBlasphemer
post Apr 25 2007, 08:32 PM
Post #3


Administrator
***

Group: Root Admin
Posts: 270
Joined: 27-May 06
Member No.: 1



QUOTE(jerone @ Apr 25 2007, 08:22 PM) *

so i'm wondering; are you ganna delete this security check or are you ganna replace it with normal searches in SP ?

Not quite sure yet.
First thing I did was inform some people at MS about this, so they get a chance to fix it too.
And now I'm researching what the best option would be.
The best solution would be to replace those silly regexps with a simple string-search, but I'm not sure if that's feasible/possible.
So I'm going to try and just make it faster, but if that doesn't work, I'll just remove the security checks completely, and warn about that when enabling the feature.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Annuix
post Apr 25 2007, 09:46 PM
Post #4


Advanced Member
***

Group: Members
Posts: 182
Joined: 20-November 06
From: United Kingdom (Scotland)
Member No.: 250



At least people are getting known about this matter before hand and I'm sure Microsoft have a good reason to check it against 97 different expressions, otherwise they wouldn't need to do it.
I'm also really suprised that WLM was actually extremely busy, not freezing because it does the same thing on other known, poweful pc's (a pal's Intel Extreme Core, blah blah) and seriously should of done these security checks in a matter of a few ticks.

Anyway, this would be a great feature to implement if MS do not sort it out themselves, as it's not the large messages i'm worrying about as most people can't send really large messages, it's those dreaded multi convo's


--------------------
IPB Image
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
TheBlasphemer
post Apr 25 2007, 10:05 PM
Post #5


Administrator
***

Group: Root Admin
Posts: 270
Joined: 27-May 06
Member No.: 1



QUOTE(Annuix @ Apr 25 2007, 09:46 PM) *

At least people are getting known about this matter before hand and I'm sure Microsoft have a good reason to check it against 97 different expressions, otherwise they wouldn't need to do it.

They have a good reason to check against 97 known virus-URLs. However, they certainly do not have a good reason to use regular expressions for this, as a simple string-search (which would be at least 100x faster) would be more than sufficient for these cases.
QUOTE

I'm also really suprised that WLM was actually extremely busy, not freezing because it does the same thing on other known, poweful pc's (a pal's Intel Extreme Core, blah blah) and seriously should of done these security checks in a matter of a few ticks.

That's the whole problem. A simple string search, or even 97 of them, would take a few ticks at most. However, a regular expression, especially on a long string, takes several ms, or sometimes even a few second, each. Multiply that by 97, and you see why it appears to be freezing.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
NF2K
post Apr 25 2007, 10:11 PM
Post #6


Newbie
*

Group: New Members
Posts: 7
Joined: 25-January 07
Member No.: 1,498



Heh, yeah I posted this to bugs some time ago, and figured it would have been some algorithm (or RegEx for that matter) checking through the contents of the messages for some reason or other. I realised it wasn't a full crash as I left it for some time to see if it would recover, which it did, as you say. The longer the message; the longer the lock-up.

I actually got as bored as to write an autotalker using the old SP2 which would send an amount of random characters the user would specify. This would allow bypass of the 5,000 character limit. An important side-note in case you intend to re-implement it in any future versions of StuffPlug.

Additionally, if this RegEx checking is to be removed by StuffPlug, what security risks does this imply exactly? Thanks.


--------------------
IPB Image
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
TheBlasphemer
post Apr 25 2007, 10:20 PM
Post #7


Administrator
***

Group: Root Admin
Posts: 270
Joined: 27-May 06
Member No.: 1



QUOTE(NF2K @ Apr 25 2007, 10:11 PM) *

Additionally, if this RegEx checking is to be removed by StuffPlug, what security risks does this imply exactly?

If I would completely remove the checking, this would mean that URLs or texts that the messenger servers tell you to block, would no longer be blocked. Basically this would mean that you can receive URLs pointing to virii and such.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Keikonium
post Apr 26 2007, 01:54 AM
Post #8


Advanced Member
***

Group: Moderator
Posts: 147
Joined: 3-January 07
Member No.: 1,078



I assume these regex check against sending files right?
So if you remove them, would I beable to send .exe files?
Would the other person need SP3 also to accept the file?
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
doggie
post Apr 26 2007, 01:58 AM
Post #9


Newbie
*

Group: New Members
Posts: 4
Joined: 25-April 07
Member No.: 2,101



Sorta means your bypassing security to increase stability?tongue.gif
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Aardvark
post Apr 26 2007, 05:12 AM
Post #10


Advanced Member
***

Group: Members
Posts: 734
Joined: 16-November 06
From: Sydney, Australia
Member No.: 33



I'd rather stability over security any day! That's why I got rid of AV software completely, but then when it infected and broke all of my installers I kinda had to bring good ol' Norton back to fix things, anyway that's off topic.

Sure this may fix the issue with sending singular large messages, but would it also work for multiple large chunks of text? Thus meaning it is now impossible to get lagged by consistent large blocks of writing?


--------------------
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
TheBlasphemer
post Apr 26 2007, 06:06 AM
Post #11


Administrator
***

Group: Root Admin
Posts: 270
Joined: 27-May 06
Member No.: 1



QUOTE(Aardvark @ Apr 26 2007, 05:12 AM) *

I'd rather stability over security any day! That's why I got rid of AV software completely, but then when it infected and broke all of my installers I kinda had to bring good ol' Norton back to fix things, anyway that's off topic.

Sure this may fix the issue with sending singular large messages, but would it also work for multiple large chunks of text? Thus meaning it is now impossible to get lagged by consistent large blocks of writing?

As I'm not removing the security checks, merely changing them to faster checks, you may still encounter lag when you receive subsequent large chunks of text. However, I think it'd be fast enough not to really notice that wink.gif
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
warmth
post Apr 26 2007, 08:36 PM
Post #12


Advanced Member
***

Group: Members
Posts: 172
Joined: 1-January 07
From: Venezuela
Member No.: 1,006



very nice feature TB... I hope to try it soon...


--------------------
IPB Image
IPB ImageIPB Image
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Annuix
post Apr 26 2007, 10:43 PM
Post #13


Advanced Member
***

Group: Members
Posts: 182
Joined: 20-November 06
From: United Kingdom (Scotland)
Member No.: 250



Ahh, I understand now...it's suddenly all clear.

The removal of a more instense security check for a faster, more reliable one which would therefore improve the speed and stability of WLM. The security checks are still there as they have been replaced...not removed. Okay, got it.

Nice to see that Stuffplug is actually being made to improve the stablity of messenger instead of adding features to it, it's seems like a nice project to take on. I like the idea of a multi-purpose addon.

I was going to ask about how hard it would be...it seems like a pretty complex task. But as I've just seen your edit there is nothing to worry about.

Finally no more freezing (major). Yay

This post has been edited by Annuix: Apr 26 2007, 10:53 PM


--------------------
IPB Image
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Salem
post Apr 27 2007, 03:40 PM
Post #14


Newbie
*

Group: New Members
Posts: 3
Joined: 20-November 06
From: Cambridge, United Kingdom
Member No.: 224



Excellent new TB. Looking forward to the next update.


--------------------
IPB Image
IPB ImageOfficial Windows Live Messenger 9.0 (2009) Beta Tester | IPB ImageMessenger Plus! Live 4.51 Beta Tester | IPB ImageMicrosoft Windows Live Butterfly | Web2Live User
Salem[at]MsgPlusLive[dot]co[dot]uk
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Matti
post Apr 27 2007, 04:24 PM
Post #15


Advanced Member
***

Group: Translator
Posts: 51
Joined: 15-November 06
Member No.: 13



No way... Who on earth would write something like that in an application like Messenger? Maybe Microsoft should fire half of their employees and get some real developers on the projects! tongue.gif

Good work, TB. I'm amazed how you manage to fix all stuff Microsoft did wrong. You're going to enable that feature by default, I hope? smile.gif
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
J-Thread
post Apr 29 2007, 09:55 AM
Post #16


Advanced Member
***

Group: Moderator
Posts: 54
Joined: 28-May 06
Member No.: 5



QUOTE(Matti @ Apr 27 2007, 06:24 PM) *

Maybe Microsoft should fire half of their employees and get some real developers on the projects! tongue.gif

They already started with the latter... rolleyes.gif
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
vikke
post Apr 29 2007, 07:17 PM
Post #17


Advanced Member
***

Group: Beta testers
Posts: 73
Joined: 16-November 06
Member No.: 69



TheGuruSupremacy, the creator of the Messenger Plus! Live script found that the Messenger team now removed their checks on client side. So this feature is no longer needed. sad.gif


--------------------
I'm back in Sweden! :-)
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
deAd
post Apr 30 2007, 12:50 AM
Post #18


Member
**

Group: Beta testers
Posts: 19
Joined: 2-January 07
From: USA
Member No.: 1,067



QUOTE(vikke @ Apr 29 2007, 03:17 PM) *

TheGuruSupremacy, the creator of the Messenger Plus! Live script found that the Messenger team now removed their checks on client side. So this feature is no longer needed. sad.gif
Microsoft probably won't release another build for a while, they wouldn't release a special build just to fix one bug like this wink.gif so it'll be needed until Messenger is updated.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Aardvark
post Apr 30 2007, 06:05 AM
Post #19


Advanced Member
***

Group: Members
Posts: 734
Joined: 16-November 06
From: Sydney, Australia
Member No.: 33



Does this mean it will be fixed in the next messenger build, or is it already fixed on the server? I ask this because the other day I was doing some lag tests and for some strange reason it wasn't lagging as much as it should.


--------------------
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
vikke
post Apr 30 2007, 08:01 AM
Post #20


Advanced Member
***

Group: Beta testers
Posts: 73
Joined: 16-November 06
Member No.: 69



I don't really see how TheGuruSupremacy could see this without an WLM update from Microsoft (maybe they contacted him?).

However, I think you should keep this feature for at least this WLM release. wink.gif


--------------------
I'm back in Sweden! :-)
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

3 Pages V  1 2 3 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



Lo-Fi Version Time is now: 30th July 2014 - 02:57 AM