MscmpSystLimiter (mscmp_syst_limiter v0.1.0)

API for establishing rate limits for usage of finite system resources.

Online, multi-user systems can be unintentionally overwhelmed by aggressive service calls from external applications and systems or intentionally exploited by hostile actors seeking to defeat system protections though such actions as brute forcing user credentials or consuming all available computing resources of our application. One approach to mitigating the dangers of resource exhaustion or persistent illicit information seeking attempts is to reject excessive calls to system services.

This component limits the rate at which targeted services can be called by any one caller to a level which preserves the availability of resources to all users of the system, or makes brute force information gathering prohibitively time intensive to would be attackers of the system.

Third Party Functionality

This version of the MscmpSystLimiter component is primarily a wrapper around the third party Hammer library. MscmpSystLimiter offers a slightly different API to the wrapped library and changes some return values to be more consistent with the Muse Systems Business Management System standards and practices. We also reuse and incorporate some of the documentation from these projects into our own documentation as appropriate.

Concepts

MscmpSystLimiter implements a "Token Bucket" rate limiting algorithm. In a Token Bucket rate limit, for each user and request type a "bucket", called a "Counter" herein, with a finite number of tokens is created. As requests are made the Counter is checked to see if all the tokens are consumed and if not the request is allowed and a token consumed. If there are no tokens available at request time, then the request is denied until the Counter expires.

Over time, expired Counters are periodically deleted by the system. Both the expiry time and the cleanup schedule are configurable.

Summary

Rate Limiter

check_rate(counter_type, counter_id, scale_ms, limit)

Checks if a Counter is within it's permissible rate and increments the Counter if it is within it's permissible rate.

check_rate_with_increment(counter_type, counter_id, scale_ms, limit, increment)

Checks the rate same as MscmpSystLimiter.check_rate/4, but allows for a variable increment to be set for the call.

delete_counters(counter_type, counter_id)

Deletes a counter from the system.

get_check_rate_function(counter_type, scale_ms, limit)

Returns an anonymous function to simplify calls to check the rate for a specific counter type.

get_counter_name(counter_type, counter_id)

Creates a canonical name for each unique counter.

inspect_counter(counter_type, counter_id, scale_ms, limit)

Retrieves data about a currently used counter without counting towards the limit.

Rate Limiter

Link to this function

check_rate(counter_type, counter_id, scale_ms, limit)

@spec check_rate(
  MscmpSystLimiter.Types.counter_type(),
  MscmpSystLimiter.Types.counter_id(),
  integer(),
  integer()
) ::
  {:allow, count :: integer()}
  | {:deny, limit :: integer()}
  | {:error, MscmpSystError.t()}

Checks if a Counter is within it's permissible rate and increments the Counter if it is within it's permissible rate.

Parameters

  • counter_type - an atom representing the kind of counter for which the rate is being checked.

  • counter_id - the specific Counter ID of the type. For example if the counter_type is :user_login, the counter_id value may be a value like user@email.domain for the user's username.

  • scale_ms - the time in milliseconds of the rate window. For example, for the rate limit of 3 tries in one minute the scale_ms value is set to 60,000 milliseconds for one minute.

  • limit - the number of attempts allowed with the scale_ms duration. In the example of 3 tries in one minute, the limit value is 3.

Example 

iex> MscmpSystLimiter.check_rate(:check_rate_counter, "id1", 60_000, 3)
{:allow, 1}
iex> MscmpSystLimiter.check_rate(:check_rate_counter, "id1", 60_000, 3)
{:allow, 2}
iex> MscmpSystLimiter.check_rate(:check_rate_counter, "id1", 60_000, 3)
{:allow, 3}
iex> MscmpSystLimiter.check_rate(:check_rate_counter, "id1", 60_000, 3)
{:deny, 3}
Link to this function

check_rate_with_increment(counter_type, counter_id, scale_ms, limit, increment)

@spec check_rate_with_increment(
  MscmpSystLimiter.Types.counter_type(),
  MscmpSystLimiter.Types.counter_id(),
  integer(),
  integer(),
  integer()
) ::
  {:allow, count :: integer()}
  | {:deny, limit :: integer()}
  | {:error, MscmpSystError.t()}

Checks the rate same as MscmpSystLimiter.check_rate/4, but allows for a variable increment to be set for the call.

Parameters

  • counter_type - an atom representing the kind of counter for which the rate is being checked.

  • counter_id - the specific Counter ID of the type. For example if the counter_type is :user_login, the counter_id value may be a value like user@email.domain for the user's username.

  • scale_ms - the time in milliseconds of the rate window. For example, for the rate limit of 3 tries in one minute the scale_ms value is set to 60,000 milliseconds for one minute.

  • limit - the number of attempts allowed with the scale_ms duration. In the example of 3 tries in one minute, the limit value is 3.

Example 

iex> MscmpSystLimiter.check_rate_with_increment(
...>   :check_with_increment,
...>   "id1",
...>   60_000,
...>   10,
...>   7)
{:allow, 7}
iex> MscmpSystLimiter.check_rate_with_increment(
...>   :check_with_increment,
...>   "id1",
...>   60_000,
...>   10,
...>   2)
{:allow, 9}
iex> MscmpSystLimiter.check_rate(:check_with_increment, "id1", 60_000, 10)
{:allow, 10}
Link to this function

delete_counters(counter_type, counter_id)

@spec delete_counters(
  MscmpSystLimiter.Types.counter_type(),
  MscmpSystLimiter.Types.counter_id()
) ::
  {:ok, integer()} | {:error, MscmpSystError.t()}

Deletes a counter from the system.

Parameters

  • counter_type - an atom representing the kind of counter which is to be deleted.

  • counter_id - the specific Counter ID of the type. For example if the counter_type is :user_login, the counter_id value may be a value like user@email.domain for the user's username.

Example 

iex> MscmpSystLimiter.check_rate(:delete_test_counter, "id1", 60_000, 3)
{:allow, 1}
iex> MscmpSystLimiter.delete_counters(:delete_test_counter, "id1")
{:ok, 1}
Link to this function

get_check_rate_function(counter_type, scale_ms, limit)

@spec get_check_rate_function(
  MscmpSystLimiter.Types.counter_type(),
  integer(),
  integer()
) ::
  (counter_id :: MscmpSystLimiter.Types.counter_id() ->
     {:allow, count :: integer()}
     | {:deny, limit :: integer()}
     | {:error, MscmpSystError.t()})

Returns an anonymous function to simplify calls to check the rate for a specific counter type.

The returned function avoids requiring the parameters that are common between calls from being constantly supplied. The only parameter that the returned function requires is the counter_id value of the specific counter of the type to test; all other parameters typically required by MscmpSystLimiter.check_rate/4 are captured by the returned closure.

Parameters

  • counter_type - an atom representing the kind of counter that the returned function will be set to check.

  • scale_ms - the time in milliseconds of the rate window. For example, for the rate limit of 3 tries in one minute the scale_ms value is set to 60,000 milliseconds for one minute.

  • limit - the number of attempts allowed with the scale_ms duration. In the example of 3 tries in one minute, the limit value is 3.

Example

Note that The returned anonymous function is equivalent to making a call to the more verbose MscmpSystLimiter.check_rate/4:

iex> my_check_rate_function =
...>   MscmpSystLimiter.get_check_rate_function(
...>     :example_counter_get_func,
...>     60_000,
...>     3)
iex> my_check_rate_function.("id1")
{:allow, 1}
iex> MscmpSystLimiter.check_rate(
...>   :example_counter_get_func,
...>   "id1",
...>   60_000,
...>   3)
{:allow, 2}
iex> my_check_rate_function.("id1")
{:allow, 3}
Link to this function

get_counter_name(counter_type, counter_id)

@spec get_counter_name(
  MscmpSystLimiter.Types.counter_type(),
  MscmpSystLimiter.Types.counter_id()
) ::
  MscmpSystLimiter.Types.counter_name()

Creates a canonical name for each unique counter.

Parameters

  • counter_type - an atom representing the kind of counter being created.

  • counter_id - a value unique to the counter_type which identifies a specific counter.

Example 

iex> MscmpSystLimiter.get_counter_name(:example_counter_name, "123")
"example_counter_name_123"
Link to this function

inspect_counter(counter_type, counter_id, scale_ms, limit)

@spec inspect_counter(
  MscmpSystLimiter.Types.counter_type(),
  MscmpSystLimiter.Types.counter_id(),
  integer(),
  integer()
) ::
  {:ok,
   {count :: integer(), count_remaining :: integer(),
    ms_to_next_counter :: integer(), created_at :: integer() | nil,
    updated_at :: integer() | nil}}
  | {:error, MscmpSystError.t()}

Retrieves data about a currently used counter without counting towards the limit.

Parameters

  • counter_type - an atom representing the kind of counter which to inspect.

  • counter_id - the specific Counter ID of the type. For example if the counter_type is :user_login, the counter_id value may be a value like user@email.domain for the user's username.

  • scale_ms - the time in milliseconds of the rate window. For example, for the rate limit of 3 tries in one minute the scale_ms value is set to 60,000 milliseconds for one minute.

  • limit - the number of attempts allowed with the scale_ms duration. In the example of 3 tries in one minute, the limit value is 3.