MWP 2010 Things I don't understand

[eyeofskye]

When I got the offer to 'upgrade' to MWP2010 I did it without thinking, believing it to be just an upgrade of the program, but it seems it is not so simple.

1. I got an e-mail on 10th Aug asking me to renew my Firetrust subscription for use with v6 which I did (£19 or so). Then I get the offer of 2010 on the 19th Aug which I paid for. Now I discover that the $10 for that includes the subscription to FirstAlert. Have I paid for it twice then?

2. If I continue to use v6 as well as 2010 do I have to pay 2 subscriptions a year to use FirstAlert with both?

3. In MWP2010 I enabled FirstAlert immediately but, from day one, I am having to identify nearly all the spam manually. When I installed v6 it successfully identified nearly all the spam out of the box but now I might as well do without Mailwasher altogether. How can I find out if 2010 is actually using FirstAlert at all?

4. Is there any guidance on how to use the Spam ratings weighting? Now that I have the facility I haven't a clue how to use it effectively.

Brian

[anniebrion]

How can I find out if 2010 is actually using FirstAlert at all?

Display the sidebar and you can see the total given by FirstAlert. You can enable this in the Inbox and/or Recycle Bin.

20-08-2010 3-59-49pm.png (18.82 KiB) Viewed 12402 times

20-08-2010 3-48-43pm.png (12.44 KiB) Viewed 12402 times

[eyeofskye]

Thanks for that Annie. Unfortunately though I have had quite a lot of mail, and most of it spam, none of the icons have numbers on them so I guess there is no filtering going on at all. Can you speculate as to why this might be? I have not changed anything since the default installation except to specifically enable FirstAlert.

[anniebrion]

It can take a couple of hundred emails to train MWP2010, manually clicking the Thumbs up/down icons will help train it.

[rusticdog]

1. I got an e-mail on 10th Aug asking me to renew my Firetrust subscription for use with v6 which I did (£19 or so). Then I get the offer of 2010 on the 19th Aug which I paid for. Now I discover that the $10 for that includes the subscription to FirstAlert. Have I paid for it twice then?

In short yes, you've paid us too much. Typically what I recommend in this situation is refunding the FirstAlert subscription payment for MW6, or alternatively as this service is 3x the price of the MailWasher 2010 annual renewal, we can just transfer that and add an extra 3 years to your existing MailWasher 2010 subscription.

Or we could of course funnel out these excess payments to the 'rusticdog needs a new fridge that doesn't pool water in the bottom causing the dog roll to get soggy fund', so far we have no takers though.....and to be honest I don't think the dog notices, he's getting so old now he's even stopped chewing and sort of just sucks everything straight down, a lot like a wet and dry vacuum cleaner....but I digress....

2. If I continue to use v6 as well as 2010 do I have to pay 2 subscriptions a year to use FirstAlert with both?

FirstAlert is optional in MailWasher 6, so personally I'd say not to bother given the price, but technically I suppose if you wanted both it would need to be that way. Then again, I often forget to actually disable FirstAlert accounts when they are refunded or transferred as above, so if I was to forget and when you let me know about what you want to do regarding post 1 above you might find that the old FirstAlert account still works at least until it expires around August 2011.

*sigh*....there goes my bonus if The Big Cheese reads this....oh wait, he doesn't pay bonuses. Carry on then

3. In MWP2010 I enabled FirstAlert immediately but, from day one, I am having to identify nearly all the spam manually. When I installed v6 it successfully identified nearly all the spam out of the box but now I might as well do without Mailwasher altogether. How can I find out if 2010 is actually using FirstAlert at all?

If you go Help >> Send Support Logs >> enter chris as your Ticket Number >> Send Logs >> I can take a look to be sure it's working and not being blocked by the Firewall etc..
Typically I would expect to be seeing FirstAlert working well, it's much better now than it ever used to be. The SideBar Annie posted is a great way to see what different tools in MailWasher hit that message.

4. Is there any guidance on how to use the Spam ratings weighting? Now that I have the facility I haven't a clue how to use it effectively.
Brian

Out of the box I think we have it pretty good, so it shouldn't need adjustment. That said it just comes back to how much authority do you want to give that Tool...

For example, I never see FirstAlert tag legitimate emails, so I could quite reasonably give it the highest authority possible and move the Slider to -200

However, I do see false hits on DNSBLs, so I leave that to it's default.

I also get a lot of spam where the sending address is forged to be one of my own, my personal solution is that I don't add my own addresses to my Friends List, however if you do then you won't want to give your Friends List full authority either. The other reason I don't give my Friends List full authority is that my email address gets 'out and about' and I do get quite a few emails from people I've dealt with in the past who seem to have picked up viruses, so I get quite a few nasty or junk emails from legitimate senders.

I don't use the Blacklist, but the same more or less applies above...

The one Tool missing from the Spam Ratings is the Learning/Bayesian. That is already on a scale where the Learning result is based on factors like the number of words in the email, how many emails you have trained. So it will calculate it's own weighted number already.


I'd be interested to hear what you find works for you, I find this part of MailWasher and how folk might use it quite interesting.

[eyeofskye]

Yes I'm doing that but some of the spam I get is run of the mill stuff that FirstAlert should be recognising. After a couple of days of operation neither FirstAlert nor the DNS blacklists have identified any of the spam e-mails. v6 identified almost all my spam on the first day of installation without any training. I think perhaps I have a bug unless there is some secret enabling button that I have not found yet.



Brian

[rusticdog]

Clearly I'm too slow typing

Thanks for that Annie. Unfortunately though I have had quite a lot of mail, and most of it spam, none of the icons have numbers on them so I guess there is no filtering going on at all. Can you speculate as to why this might be? I have not changed anything since the default installation except to specifically enable FirstAlert.

I assume the spinners to the left of the emails aren't still going round right ?

What you can do is click on View >> Bayesian Debug >> next time you get an email that you think ought to have some kind of result click on this 'Bayesian debug' tab in the Preview Pane. Copy/paste the results into a reply, it would be useful (just easier to read) if you use the CODE tags around that too..

[eyeofskye]

Hey Rusticdog, thank you for all that can I have a little while to think over the subscription thing while I wait to see what the problem is I have with 2010 or maybe phone a friend.

I'm pretty slow typing/thinking as well. Don't give it a second thought.

Any chance of changing Rattled Rabbit to Nervous Cockerel?

What spinners?

I'll have a look at the Bavarian bedbug as soon as I get more mail.

Ta

Brian (listening to the roof blowing away)

[rusticdog]

Of course, take all the time you need, and don't worry about the Rabbit, your one post away from being a Mystified Moa


If you don't notice the spinners then don't worry, they appear to the left of each message when an email is downloaded but not all the different Spam Tools have run their course over that message. If they were still showing it would mean the email evaluations weren't complete and would explain all the 0's in teh SideBar


If you want a test message to look at for the Bayesian Debug, try restoring a deleted spam email from the Recycle Bin, it's a good chance to check that restoring an email is working for you too

[eyeofskye]

Restoring mail works!
Checking some dead ringer spams none of them are exciting the Origin of spam or FirstAlert icons. Here is the Bayesian stuff from one of them:

Code: Select all

DECODED_BASE64_SUBJECT: 0
SUBJECT_WORDS: 14
FROM_WORDS: 4
CONTENT_TYPE: multipart/alternative;boundary="------------040908000406040204040902"
NUM_PARTS: 2
TYPE: multipart/alternative;boundary="------------040908000406040204040902"
ENCODING: 7bit
TYPE: text/html; charset=windows-1252
BYTES: 283
NUM_RAW_WORDS: 1
URL: http://80.91.176.144
IP: 80.91.176.144
MULT: 0.895338
COMB: 1.000000E-022
RAWSPAMYNESSE: 1.000000E+000
RAWSPAMYNESS: 1.00000
SPAMYNESSE: 3.674000E-001
SPAMYNESS: 0.36740
GOODCOUNT: 0
BADCOUNT: 13
GOODWORDCOUNT: 930
GGAIN: 0.930000
BADWORDCOUNT: 668
BGAIN: 0.668000
INTRESTINGWORDCOUNT: 20
WORDCOUNT: 11
TOTAL_WORDCOUNT_FACTOR: 0.5500
INTERESTING_WORDCOUNT_FACTOR: 0.5500
WORD: add_http://80.91.176.144	prob=0.990000       	occurrences=1
WORD: asubject_addicted   	prob=0.990000       	occurrences=1
WORD: asubject_consult    	prob=0.990000       	occurrences=1
WORD: asubject_doctor     	prob=0.990000       	occurrences=1
WORD: asubject_follow     	prob=0.990000       	occurrences=1
WORD: asubject_instructions	prob=0.990000       	occurrences=1
WORD: asubject_painkiller 	prob=0.990000       	occurrences=1
WORD: asubject_you'll     	prob=0.990000       	occurrences=1
WORD: from_"america       	prob=0.990000       	occurrences=1
WORD: from_<[email protected]>	prob=0.990000       	occurrences=1
WORD: ip_80.91.176.144    	prob=0.990000       	occurrences=3

Brian

[eyeofskye]

After 10,000 posts do you get to be a Weary Womble?

[rusticdog]

OK great. The RBL won't trigger as the email comes from your own service provider that time, when it's restored I mean.

Here's the snippets that matter :

Code: Select all

BADWORDCOUNT: 668
WORDCOUNT: 11
TOTAL_WORDCOUNT_FACTOR: 0.5500
INTERESTING_WORDCOUNT_FACTOR: 0.5500

MailWasher first needs 1000 bad words before it starts considering itself properly trained. You're almost there at 668, it won't take too much longer. So the result was scaled back because of this.

The other consideration for this particular email is the number of words found which is 11. We found around 20 words needed to be the best for long term accuracy, but there's no harm in dropping this. Because there was 11 words found, when 20 is preferred MailWasher has scaled back the result to 55% of the original.


Here's hopefully a brief idea of what happened....the stuff in Blue can be ignored if you don't care about such things.

The Bayesian's original thought was actually spot on. The email is spammy as they get....as the message scored at 1.0000. Anything scoring near -1.0000 is good, near 1.0000 is spam. If a message scored at 0.0000 it would be considered neutral.

But then MailWasher scaled back the result because it doesn't consider itself fully trained. So 1.0 became 0.668, still spammy...

Then MailWasher felt that because 20 words is better, but only 11 were there, it scaled it back some more. So 0.668 became 0.36740


This final result meant that MailWasher decided that the email was still spammy, but just not enough to conclusively say that it was a spam email for sure.


As for what you can do about it......just wait a little longer until your training take hold properly, as above, you're not far off. Had you been at your 1000 word threshhold this email would have come through marked as spam

Or, you could tell MailWasher that 20 words is too much, and under Settings >> Spam Tools >> Learning >> change the 'Interesting Word Count' to 15. That might give you a more immediate impact.


Or, you could do both

[rusticdog]

After 10,000 posts do you get to be a Weary Womble?

Hah!, love it.....you do now

[eyeofskye]

Here's what I got from a new bit of spam, a classic Chinese phishing scam that should have been in the databases.

Code: Select all

DECODED_BASE64_SUBJECT: 0
SUBJECT_WORDS: 6
FROM_WORDS: 4
ENCODING: 7bit
CONTENT_TYPE: text/plain;charset="Windows-1251"
NUM_PARTS: 1
ENCODING: 7bit
TYPE: text/plain;charset="windows-1251"
BYTES: 2599
BINARY_PESUDO_WORDS: 18
MULT: 1.000000
COMB: 1.000000
RAWSPAMYNESSE: 0.000000E+000
RAWSPAMYNESS: 0.00000
SPAMYNESSE: 0.000000E+000
SPAMYNESS: 0.00000
GOODCOUNT: 0
BADCOUNT: 0
GOODWORDCOUNT: 960
GGAIN: 0.960000
BADWORDCOUNT: 668
BGAIN: 0.668000
INTRESTINGWORDCOUNT: 20
WORDCOUNT: 20
TOTAL_WORDCOUNT_FACTOR: 1.0000
INTERESTING_WORDCOUNT_FACTOR: 0.0000

Nothing on the Origin of Spam or FirstAlert icons. Logs sent as suggested.

[ataraxy]

Nothing on the Origin of Spam or FirstAlert icons. Logs sent as suggested.

Does your email pass through your ISP's spam filter and does that spam filter add server records to the email headers? If that's the case, then this version of MailWasher won't get an Origin of Spam hit as it will be checking one of your ISP's servers. v6.x is able to look through the entries added by ISP's spam filtering and so is better in this regard.

Does v6.x get a FirstAlert! hit?