Subject: [os2genau_digest] No. 1888
From: Digest <deadmail>
To: "OS/2GenAu Digest"<deadmail>
Date: Sat, 12 Dec 2009 00:00:34 EST-10EDT,10,1,0,7200,4,1,0,7200,3600

**************************************************
Friday 11 December 2009
 Number  1888
**************************************************

Subjects for today
 
1  Re:  os2site dot com : Ian Manners" <deadmail>
2  Re:  Slow writing to USB 2.0 : Ian Manners" <deadmail>
3  Re:  os2site dot com : Ed Durrant <edurrant at durrant dot mine dot nu>
4  Re:  Slow writing to USB 2.0 : Wayne <datablitz.wayne at gmail dot com>
5  Re:  Slow writing to USB 2.0 : Wayne <datablitz.wayne at gmail dot com>

**= Email   1 ==========================**

Date:  Fri, 11 Dec 2009 00:55:22 +1100 (EDT)
From:  "Ian Manners" <deadmail>
Subject:  Re:  os2site dot com

Hi Ed,

>I wonder if there is any way to spot when a multiple file download is 
>occurring? i.e. someone is trying to mirror your site.

Problem thats been happening a lot, especially in the past 4-5
weeks is some smart arse has set themselves up to use different
UA (User Agents), on different IP address's from all over the world,
for a couple of hours each. Each one is below the blacklist
threshold (which I cant tighten up any more). So each block looks
like a normal user.

I can see the pattern occurring when I view the logs after sorting.

 I can setup a rule for that but it will impact other people to the extent
of making normal site usage impractical, not to mention very
annoying.

Users are blacklisted for X time if they 

a) use Y concurrent download threads
b) exceed Y amount of data
c) use various UA's
d) use a fake UA on a wrong IP (ie, Ukriane with a Google UA)
e) try to access or download trap files.

Those that keep doing it end up on the permanent blacklist in the
firewall.

>Perhaps supporting transfer only via a HTTP access from a "portal" on 
>your website, that would only let someone download one file at a time 
>could be an approach - a "door keeper" if you like.

I have it set for 4 threads per IP per file, anything lower leads to
problems. There is a preset limit that if exceeded leads to the
user being blacklisted for a period of time as well

>Probably turn off FTP straight away. Can you tell if the sites that are
>accessing are using HTTP or FTP ?

HTTP, you want FTP access, you have to ask me :-)

It's amazing how many people blast me over my access rules to
what I thought was my website, my answer is normally, you
pay for the data, you can download the data :o)

I figure as it is essentially a free service, the website can take
the rest of the month off while I think about it at my leisure, anyone
that wants urgent access can let me know and I'll email them
a username/password sometime next week, if its ultra urgent
I'll email them the file or email back a URL :-)

The next 3 days I'm going to be very busy on my real life things.

Cheers
Ian Manners
http://www.os2site dot com/

--------------------------------------------------
 
 http://www./melbpc/  -  The Melbourne OS/2 SIG
**= Email   2 ==========================**

Date:  Fri, 11 Dec 2009 00:58:07 +1100 (EDT)
From:  "Ian Manners" <deadmail>
Subject:  Re:  Slow writing to USB 2.0

Hi Peter,

>I can explain that part. The check to see whether the sender is a
>legitimate list member is done on the address in the SMTP command "MAIL
>FROM:" rather than on the "From:" header line. The reasoning here is
>that, although spammers can fake either address, there's a much higher
>probability that a spammer will fake the "From:".

Yes, now you mention it I can remember this coming up in the past and
I think you said the same thing :-)

Cheers
Ian Manners
http://www.os2site dot com/

--------------------------------------------------
 
 http://www./melbpc/  -  The Melbourne OS/2 SIG
**= Email   3 ==========================**

Date:  Fri, 11 Dec 2009 05:46:40 +1100
From:  Ed Durrant <edurrant at durrant dot mine dot nu>
Subject:  Re:  os2site dot com

Ian Manners wrote:
> Hi Ed,
>
>   
>> I wonder if there is any way to spot when a multiple file download is 
>> occurring? i.e. someone is trying to mirror your site.
>>     
>
> Problem thats been happening a lot, especially in the past 4-5
> weeks is some smart arse has set themselves up to use different
> UA (User Agents), on different IP address's from all over the world,
> for a couple of hours each. Each one is below the blacklist
> threshold (which I cant tighten up any more). So each block looks
> like a normal user.
>
> I can see the pattern occurring when I view the logs after sorting.
>
>  I can setup a rule for that but it will impact other people to the extent
> of making normal site usage impractical, not to mention very
> annoying.
>
> Users are blacklisted for X time if they 
>
> a) use Y concurrent download threads
> b) exceed Y amount of data
> c) use various UA's
> d) use a fake UA on a wrong IP (ie, Ukriane with a Google UA)
> e) try to access or download trap files.
>
> Those that keep doing it end up on the permanent blacklist in the
> firewall.
>
>   
>> Perhaps supporting transfer only via a HTTP access from a "portal" on 
>> your website, that would only let someone download one file at a time 
>> could be an approach - a "door keeper" if you like.
>>     
>
> I have it set for 4 threads per IP per file, anything lower leads to
> problems. There is a preset limit that if exceeded leads to the
> user being blacklisted for a period of time as well
>
>   
>> Probably turn off FTP straight away. Can you tell if the sites that are
>> accessing are using HTTP or FTP ?
>>     
>
> HTTP, you want FTP access, you have to ask me :-)
>
> It's amazing how many people blast me over my access rules to
> what I thought was my website, my answer is normally, you
> pay for the data, you can download the data :o)
>
> I figure as it is essentially a free service, the website can take
> the rest of the month off while I think about it at my leisure, anyone
> that wants urgent access can let me know and I'll email them
> a username/password sometime next week, if its ultra urgent
> I'll email them the file or email back a URL :-)
>
> The next 3 days I'm going to be very busy on my real life things.
>
> Cheers
> Ian Manners
> http://www.os2site dot com/
>
> --------------------------------------------------
>  
>  http://www./melbpc/  -  The Melbourne OS/2 SIG

>   
Its a real shame that someone would want to do this, without respect for 
the fact that you get stung for the network bandwidth used.

Another idea, presuming this is a "bot" doing this, how difficult would 
it be to implement a "catchya" feature (I think that's what it is 
called, where a human readable set of random letters and numbers are 
displayed and have to be entered before access to a file is given?

Needs some thinking through, so I agree with you closing off all access 
in the interim is probably a good idea - it may of itself stop the bot 
and they may give up in the meantime.

-- 
Cheers/2

Ed

eComStationAustralia podcast RSS feed http://eComStationAustralia.podbean dot com/feed or iTunes

--------------------------------------------------
 
 http://www./melbpc/  -  The Melbourne OS/2 SIG
===
**= Email   4 ==========================**

Date:  Fri, 11 Dec 2009 07:14:32 +0930
From:  Wayne <datablitz.wayne at gmail dot com>
Subject:  Re:  Slow writing to USB 2.0

** Reply to note from smee.wayne+caf_=datablitz=three dot com dot au at gmail dot com
Thu, 10 Dec 2009 07:56:57 +0800
>
> [attachments have been removed]
> --------------------------------------------------
>  
>  http://www./melbpc/  -  The Melbourne OS/2 SIG
===

Gmail does have problems at times.  This email  seems to be originated
from 1 of my gmail accounts but I didn't send it.  Nor have I sent anything
vaguely like it anywhere & nothing to os2genau.  Really seems like
gmail is at fault.  1 of my gmail accounts is forwarded to three dot com dot au.

Cheers
Wayne

--------------------------------------------------
 
 http://www./melbpc/  -  The Melbourne OS/2 SIG
===
**= Email   5 ==========================**

Date:  Fri, 11 Dec 2009 06:56:15 +0930
From:  Wayne <datablitz.wayne at gmail dot com>
Subject:  Re:  Slow writing to USB 2.0

** Reply to note from smee.wayne+caf_=datablitz=three dot com dot au at gmail dot com 
Thu, 10 Dec 2009 07:56:57 +0800
 >
 > [attachments have been removed]
 > --------------------------------------------------
 >  
 >  http://www./melbpc/  -  The Melbourne OS/2 SIG
 ===

Gmail does have problems at times.  This email  seems to be originated
from 1 of my gmail accounts but I didn't send it.  Nor have I sent anything
vaguely like it anywhere & nothing to os2genau.  Really seems like
gmail is at fault.  1 of my gmail accounts is forwarded to three dot com dot au.

Cheers
Wayne
--------------------------------------------------
 
 http://www./melbpc/  -  The Melbourne OS/2 SIG
===