Forgot your password?

Home > About Us > Modern Signal Blog >ColdFusion Rate Limiter

ColdFusion Rate Limiter

I was recently looking for a way to throttle the number of requests to a site from bad bots that sometimes flood a server with dozens of requests in a few seconds.  I found a very nice simple function by Charlie Arehart:
http://www.carehart.org/blog/client/index.cfm/2010/5/21/throttling_by_ip_address

Before proceeding, you might want to look over Charlie's article, which nicely lays out many of the considerations and caveats that come with limiting the number of requests by IP address.

Aside from all the caveats, Charlie's function had a couple things I wanted to adjust.  One was that, as written, it would block even "good" bots, like Googlebot, if they made requests more frequently than the "duration" parameter.  For instance, if you set it up to allow up to 6 requests every 3 seconds (as I did), it would end up blocking Googlebot from making a steady stream of requests at the frequency of once per second.  So even though Googlebot was only making 3 requests every 3 seconds it was getting blocked.

Adjusting Charlie's function to stop doing that was very easy.  But the other thing I was concerned about was whether keeping a list of IP addresses in an application variable would scale very well.  How long would the list have to be before it would affect performance?  I didn't really know, but felt it would be better to have a solution where the list would have old IP addresses pruned periodically.  I toyed with the idea of using a scheduled task to go through the list and removed old entries.  That seemed cumbersome, though.  In the end, since the site I was working on was running ColdFusion 9, I thought it would be a good opportunity to use ColdFusion 9's new caching functions.  Using ColdFusion's built-in caching functions means the I can have old IP addresses cleaned up automatically.

So here is my version of the rate limiter.  This is generally more forgiving than Charlie's version of the function.  In Charlie's version if a bot is blocked but continues to make requests (e.g. Googlebot), it will continue to be blocked until it pauses for the "duration".  In my version, the bot is blocked for the "duration" and then allowed to make more requests before being blocked again.  My main goal is to block the rogue bots that flood the system with 10 or 20 requests a second for a short time, and this does a pretty good job of that.

<cffunction name="limiter">
    <cfargument name="duration" type="numeric" default=3>
    <cfargument name="count" type="numeric" default=6>
    <cfset var cacheId = "rate_limiter_" & CGI.REMOTE_ADDR>
    <cfset var rate = cacheGet(cacheId)>

    <cfif isNull(rate)>
        <!--- Create cached object --->
        <cfset cachePut(cacheID, {attempts = 1, start = Now()}, createTimeSpan(0,0,1,0))>
    <cfelseif DateDiff("s", rate.start, Now()) LT arguments.duration>
        <cfif rate.attempts gte arguments.count>
            <cfoutput>
                <p>You are making too many requests too fast,
                please slow down and wait #arguments.duration# seconds</p>
            </cfoutput>
            <cfheader statuscode="503" statustext="Service Unavailable">
            <cfheader name="Retry-After" value="#arguments.duration#">
            <cflog file="limiter" text="#cgi.remote_addr# #rate.attempts# #cgi.request_method# #cgi.SCRIPT_NAME# #cgi.QUERY_STRING# #cgi.http_user_agent# #rate.start#">
            <cfif rate.attempts is arguments.count>
                <!--- Lock out for duration --->
                <cfset cachePut(cacheID, {attempts = rate.attempts + 1, start = Now()}, createTimeSpan(0,0,1,0))>
            </cfif>
            <cfabort>
        <cfelse>
            <!--- Increment attempts --->
            <cfset cachePut(cacheID, {attempts = rate.attempts + 1, start = rate.start}, createTimeSpan(0,0,1,0))>
        </cfif>
    <cfelse>
        <!--- Reset attempts --->
        <cfset cachePut(cacheID, {attempts = 1, start = Now()}, createTimeSpan(0,0,1,0))>
    </cfif>
</cffunction>


Comments

Craig's Globally Recognized Avatar Is it possible to use your code with CF8 or not really due to your use of CF9's caching tag. Wanted to try to implement your version, but am running CF8. Not sure how to modify it to make it work on my server, any ideas?

Posted on July 19, 2012 3:58:00 PM EDT by Craig

David Hammond's Globally Recognized Avatar Yeah, this won't run in CF8. You might want to get Charlie's version of the function. You could combine ideas from both functions by replacing the calls to cacheGet and cachePut in this function with application variables, as they are in Charlie's function.

Posted on July 19, 2012 5:01:16 PM EDT by David Hammond

Adam's Globally Recognized Avatar Any need to use CFLOCK to lock the cached variables since you could have multiple threads attempting to increment the shared counter?

Posted on August 22, 2012 7:22:52 AM EDT by Adam

David Hammond's Globally Recognized Avatar I hadn't thought about using CFLOCK. Race conditions are possible, but I don't think they are important since an occasional miscount shouldn't affect the effectiveness of the rate limiting much. One priority I had was in keeping the code a light-weight as possible.

Posted on August 22, 2012 8:08:11 AM EDT by David Hammond

Testimonials

  • Modern Signal has a professional staff that was very responsive to our needs during all phases - scoping, developing, implementing and maintaining - of our project.  We have been pleased with their ability to deliver quality work on time and on budget. If given the opportunity, I would work with them again.

    - The National Center for Safe Routes to School

  • Modern Signal significantly enhanced our site to be more efficient and user-friendly. They provide excellent customer service with timely and cost-effective solutions.

    - Center for Medicare Education

  • Modern Signal understands our business - from future needs to current limitations - so their solutions are always scalable, solid, and service-oriented.

    - National Association of Home Builders

  • I love working with Modern Signal! Their CMS is very easy to use and they are incredibly responsive to questions or challenges I bring them.

    - NALP

  • Modern Signal has been a great partner for us for over the past 10 years.  As our business grew and our needs changed, Modern Signal was able to work with us to adjust our website platform in the ever-changing online world.  Their service and response level has been second to none, and we've been never been happier with our relationship with them.

    - Charm City Run

  • Modern Signal worked with us to understand our needs and figure out what solution would work best for us. Our Lighthouse CMS is perfectly suited to our website goals. When we later needed to modify the CMS, they again took the time to understand exactly what was  needed and then built that functionality rather than delivering a cookie cutter solution.   

    - Ecosystem Investment Partners

  • I felt as if my company was their only client. They responded to my needs quickly and efficiently despite short turn around time and intense demands.

    - Teaching Strategies, Inc.