Bump version to 0.5.0

Allow for configurable group separator

.rvmrc

@@ -1 +0,0 @@
-rvm gemset use redistat

.travis.yml

@@ -0,0 +1,8 @@
+language: ruby
+rvm:
+  - 1.8.7
+  - 1.9.2
+  - 1.9.3
+  - jruby-18mode
+  - jruby-19mode
+  - ree

LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2010 Jim Myhrberg.
+Copyright (c) 2011 Jim Myhrberg.
 
 Permission is hereby granted, free of charge, to any person obtaining
 a copy of this software and associated documentation files (the

README.md

@@ -1,85 +1,356 @@
-# Redistat
+# Redistat [![Build Status](https://secure.travis-ci.org/jimeh/redistat.png)](http://travis-ci.org/jimeh/redistat)
 
 A Redis-backed statistics storage and querying library written in Ruby.
 
-Redistat was originally created to replace a small hacked together statistics collection solution which was MySQL-based. When I started I had a short list of requirements:
+Redistat was originally created to replace a small hacked together statistics
+collection solution which was MySQL-based. When I started I had a short list
+of requirements:
 
 * Store and increment/decrement integer values (counters, etc)
 * Up to the second statistics available at all times
 * Screamingly fast
 
-Redis fits perfectly with all of these requirements. It has atomic operations like increment, and it's lightning fast, meaning if the data is structured well, the initial stats reporting call will store data in a format that's instantly retrievable just as fast.
+Redis fits perfectly with all of these requirements. It has atomic operations
+like increment, and it's lightning fast, meaning if the data is structured
+well, the initial stats reporting call will store data in a format that's
+instantly retrievable just as fast.
 
 ## Installation
 
     gem install redistat
 
-If you are using Ruby 1.8.x, it's recommended you also install the `SystemTimer` gem, as the Redis gem will otherwise complain.
+If you are using Ruby 1.8.x, it's recommended you also install the
+`SystemTimer` gem, as the Redis gem will otherwise complain.
 
-## Usage
+## Usage (Crash Course)
 
-The simplest way to use Redistat is through the model wrapper.
+view\_stats.rb:
 
-    class VisitorStats
+```ruby
-      include Redistat::Model
+require 'redistat'
-    end
 
-Before any of you Rails-purists start complaining about the model name being plural, I want to point out that it makes sense with Redistat, cause a model doesn't exactly return a specific row or object. But I'm getting ahead of myself.
+class ViewStats
+  include Redistat::Model
+end
 
-To store statistics we essentially tell Redistat that an event has occurred with a label of X, and statistics of Y. So let's say we want to store a page view event on the `/about` page on a site:
+# if using Redistat in multiple threads set this
-
+# somewhere in the beginning of the execution stack
-    VisitorStats.store('/about', {:views => 1})
+Redistat.thread_safe = true
-
+```
-In the above case "`/about`" is the label under which the stats are grouped, and the statistics associated with the event is simply a normal Ruby hash, except all values need to be integers, or Redis' increment calls won't work.
-
-To later retrieve statistics, we use the `fetch` method:
-
-    stats = VisitorStats.fetch('/about', 2.hour.ago, Time.now)
-    # stats => [{:views => 1}]
-    # stats.total => {:views => 1}
-
-The fetch method requires 3 arguments, a label, a start time, and an end time. Fetch returns a `Redistat::Collection` object, which is normal Ruby Array with a couple of added methods, like total shown above.
-
-For more detailed usage, please check spec files, and source code till I have time to write up a complete readme.
 
 
-## Some Technical Details
+### Simple Example
 
-To give a brief look into how Redistat works internally to store statistics, I'm going to use the examples above. The store method accepts a Ruby Hash with statistics to store. Redistat stores all statistics as hashes in Redis. It stores summaries of the stats for the specific time when it happened and all it's parent time groups (second, minute, hour, day, month, year). The default depth Redistat goes to is hour, unless the `depth` option is passed to `store` or `fetch`.
+Store:
 
-In short, the above call to `store` creates the following keys in Redis:
+```ruby
+ViewStats.store('hello', {:world => 4})
+ViewStats.store('hello', {:world => 2}, 2.hours.ago)
+```
 
-    VisitorStats//about:2010
+Fetch:
-    VisitorStats//about:201011
-    VisitorStats//about:20101124
-    VisitorStats//about:2010112401
 
-Each of these keys in Redis are a hash, containing the sums of each statistic point reported for the time frame the key represents. In this case there's two slashes, cause the label we used was “`/about`”, and the scope (class name when used through the model wrapper) and the label are separated with a slash.
+```ruby
+ViewStats.find('hello', 1.hour.ago, 1.hour.from_now).all
+  #=> [{'world' => 4}]
+ViewStats.find('hello', 1.hour.ago, 1.hour.from_now).total
+  #=> {'world' => 4}
+ViewStats.find('hello', 3.hour.ago, 1.hour.from_now).total
+  #=> {'world' => 6}
+```
 
-When retrieving statistics for a given date range, Redistat figures out how to do the least number of calls to Redis to fetch all relevant data. For example, if you want the sum of stats from the 4th till the last of November, the full month of November would first be fetched, then the first 3 days of November would be fetched and removed from the full month stats.
+
+### Advanced Example
+
+Store page view on product #44 from Chrome 11:
+
+```ruby
+ViewStats.store('views/product/44', {'count/chrome/11' => 1})
+```
+
+Fetch product #44 stats:
+
+```ruby
+ViewStats.find('views/product/44', 23.hours.ago, 1.hour.from_now).total
+  #=> { 'count' => 1, 'count/chrome' => 1, 'count/chrome/11' => 1 }
+```
+
+Store a page view on product #32 from Firefox 3:
+
+```ruby
+ViewStats.store('views/product/32', {'count/firefox/3' => 1})
+```
+
+Fetch product #32 stats:
+
+```ruby
+ViewStats.find('views/product/32', 23.hours.ago, 1.hour.from_now).total
+  #=> { 'count' => 1, 'count/firefox' => 1, 'count/firefox/3' => 1 }
+```
+
+Fetch stats for all products:
+
+```ruby
+ViewStats.find('views/product', 23.hours.ago, 1.hour.from_now).total
+  #=> { 'count'           => 2,
+  #     'count/chrome'    => 1,
+  #     'count/chrome/11' => 1,
+  #     'count/firefox'   => 1,
+  #     'count/firefox/3' => 1 }
+```
+
+Store a 404 error view:
+
+```ruby
+ViewStats.store('views/error/404', {'count/chrome/9' => 1})
+```
+
+Fetch stats for all views across the board:
+
+```ruby
+ViewStats.find('views', 23.hours.ago, 1.hour.from_now).total
+  #=> { 'count'           => 3,
+  #     'count/chrome'    => 2,
+  #     'count/chrome/9'  => 1,
+  #     'count/chrome/11' => 1,
+  #     'count/firefox'   => 1,
+  #     'count/firefox/3' => 1 }
+```
+
+Fetch list of products known to Redistat:
+
+```ruby
+finder = ViewStats.find('views/product', 23.hours.ago, 1.hour.from_now)
+finder.children.map { |child| child.label.me }
+  #=> [ "32", "44" ]
+finder.children.map { |child| child.label.to_s }
+  #=> [ "views/products/32", "views/products/44" ]
+finder.children.map { |child| child.total }
+  #=> [ { "count" => 1, "count/firefox" => 1, "count/firefox/3" => 1 },
+  #     { "count" => 1, "count/chrome"  => 1, "count/chrome/11" => 1 } ]
+```
+
+
+## Terminology
+
+### Scope
+
+A type of global-namespace for storing data. When using the `Redistat::Model`
+wrapper, the scope is automatically set to the class name. In the examples
+above, the scope is `ViewStats`. Can be overridden by calling the `#scope`
+class method on your model class.
+
+### Label
+
+Identifier string to separate different types and groups of statistics from
+each other. The first argument of the `#store`, `#find`, and `#fetch` methods
+is the label that you're storing to, or fetching from.
+
+Labels support multiple grouping levels by splitting the label string with `/`
+and storing the same stats for each level. For example, when storing data to a
+label called `views/product/44`, the data is stored for the label you specify,
+and also for `views/product` and `views`. You may also configure a different
+group separator using the `Redistat.group_separator=` method. For example:
+
+```ruby
+Redistat.group_separator = '|'
+```
+
+A word of caution: Don't use a crazy number of group levels. As two levels
+causes twice as many `hincrby` calls to Redis as not using the grouping
+feature. Hence using 10 grouping levels, causes 10 times as many write calls
+to Redis.
+
+### Input Statistics Data
+
+You provide Redistat with the data you want to store using a Ruby Hash. This
+data is then stored in a corresponding Redis hash with identical key/field
+names.
+
+Key names in the hash also support grouping features similar to those
+available for Labels. Again, the more levels you use, the more write calls to
+Redis, so avoid using 10-15 levels.
+
+### Depth (Storage Accuracy)
+
+Define how accurately data should be stored, and how accurately it's looked up
+when fetching it again. By default Redistat uses a depth value of `:hour`,
+which means it's impossible to separate two events which were stored at 10:18
+and 10:23. In Redis they are both stored within a date key of `2011031610`.
+
+You can set depth within your model using the `#depth` class method. Available
+depths are: `:year`, `:month`, `:day`, `:hour`, `:min`, `:sec`
+
+### Time Ranges
+
+When you fetch data, you need to specify a start and an end time. The
+selection behavior can seem a bit weird at first when, but makes sense when
+you understand how Redistat works internally.
+
+For example, if we are using a Depth value of `:hour`, and we trigger a fetch
+call starting at `1.hour.ago` (13:34), till `Time.now` (14:34), only stats
+from 13:00:00 till 13:59:59 are returned, as they were all stored within the
+key for the 13th hour. If both 13:00 and 14:00 was returned, you would get
+results from two whole hours. Hence if you want up to the second data, use an
+end time of `1.hour.from_now`.
+
+### The Finder Object
+
+Calling the `#find` method on a Redistat model class returns a
+`Redistat::Finder` object. The finder is a lazy-loaded gateway to your
+data. Meaning you can create a new finder, and modify instantiated finder's
+label, scope, dates, and more. It does not call Redis and fetch the data until
+you call `#total`, `#all`, `#map`, `#each`, or `#each_with_index` on the
+finder.
+
+This section does need further expanding as there's a lot to cover when it
+comes to the finder.
+
+
+## Key Expiry
+
+Support for expiring keys from Redis is available, allowing you too keep
+varying levels of details for X period of time. This allows you easily keep
+things nice and tidy by only storing varying levels detailed stats only for as
+long as you need.
+
+In the below example we define how long Redis keys for varying depths are
+stored. Second by second stats are available for 10 minutes, minute by minute
+stats for 6 hours, hourly stats for 3 months, daily stats for 2 years, and
+yearly stats are retained forever.
+
+```ruby
+class ViewStats
+  include Redistat::Model
+
+  depth :sec
+
+  expire \
+    :sec => 10.minutes.to_i,
+    :min => 6.hours.to_i,
+    :hour => 3.months.to_i,
+    :day => 2.years.to_i
+end
+```
+
+Keep in mind that when storing stats for a custom date in the past for
+example, the expiry time for the keys will be relative to now. The values you
+specify are simply passed to the `Redis#expire` method.
+
+
+## Internals
+
+### Storing / Writing
+
+Redistat stores all data into a Redis hash keys. The Redis key name the used
+consists of three parts. The scope, label, and datetime:
+
+    {scope}/{label}:{datetime}
+
+For example, this...
+
+```ruby
+ViewStats.store('views/product/44', {'count/chrome/11' => 1})
+```
+
+...would store the follow hash of data...
+
+```ruby
+{ 'count' => 1, 'count/chrome' => 1, 'count/chrome/11' => 1 }
+```
+
+...to all 12 of these Redis hash keys...
+
+    ViewStats/views:2011
+    ViewStats/views:201103
+    ViewStats/views:20110315
+    ViewStats/views:2011031510
+    ViewStats/views/product:2011
+    ViewStats/views/product:201103
+    ViewStats/views/product:20110315
+    ViewStats/views/product:2011031510
+    ViewStats/views/product/44:2011
+    ViewStats/views/product/44:201103
+    ViewStats/views/product/44:20110315
+    ViewStats/views/product/44:2011031510
+
+...by creating the Redis key, and/or hash field if needed, otherwise it simply
+increments the already existing data.
+
+It would also create the following Redis sets to keep track of which child
+labels are available:
+
+    ViewStats.label_index:
+    ViewStats.label_index:views
+    ViewStats.label_index:views/product
+
+It should now be more obvious to you why you should think about how you use
+the grouping capabilities so you don't go crazy and use 10-15 levels. Storing
+is done through Redis' `hincrby` call, which only supports a single key/field
+combo. Meaning the above example would call `hincrby` a total of 36 times to
+store the data, and `sadd` a total of 3 times to ensure the label index is
+accurate. 39 calls is however not a problem for Redis, most calls happen in
+less than 0.15ms (0.00015 seconds) on my local machine.
+
+
+### Fetching / Reading
+
+By default when fetching statistics, Redistat will figure out how to do the
+least number of reads from Redis. First it checks how long range you're
+fetching. If whole days, months or years for example fit within the start and
+end dates specified, it will fetch the one key for the day/month/year in
+question. It further drills down to the smaller units.
+
+It is also intelligent enough to not fetch each day from 3-31 of a month,
+instead it would fetch the data for the whole month and the first two days,
+which are then removed from the summary of the whole month. This means three
+calls to `hgetall` instead of 29 if each whole day was fetched.
+
+### Buffer
+
+The buffer is a new, still semi-beta, feature aimed to reduce the number of
+Redis `hincrby` that Redistat sends. This should only really be useful when
+you're hitting north of 30,000 Redis requests per second, if your Redis server
+has limited resources, or against my recommendation you've opted to use 10,
+20, or more label grouping levels.
+
+Buffering tries to fold together multiple `store` calls into as few as
+possible by merging the statistics hashes from all calls and groups them based
+on scope, label, date depth, and more. You configure the the buffer by setting
+`Redistat.buffer_size` to an integer higher than 1. This basically tells
+Redistat how many `store` calls to buffer in memory before writing all data to
+Redis.
 
 
 ## Todo
 
-* Proper/complete readme.
+* More details in Readme.
 * Documentation.
 * Anything else that becomes apparent after real-world use.
 
 
+## Credits
+
+[Global Personals](http://globalpersonals.co.uk/) deserves a thank
+you. Currently the primary user of Redistat, they've allowed me to spend some
+company time to further develop the project.
+
+
 ## Note on Patches/Pull Requests
- 
+
 * Fork the project.
 * Make your feature addition or bug fix.
 * Add tests for it. This is important so I don't break it in a
   future version unintentionally.
-* Commit, do not mess with rakefile, version, or history.
+* Commit, do not mess with rakefile, version, or history.  (if you want to
-  (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
+  have your own version, that is fine but bump version in a commit by itself I
+  can ignore when I pull)
 * Send me a pull request. Bonus points for topic branches.
 
 
 ## License and Copyright
 
-Copyright (c) 2010 Jim Myhrberg.
+Copyright (c) 2011 Jim Myhrberg.
 
 Permission is hereby granted, free of charge, to any person obtaining
 a copy of this software and associated documentation files (the

lib/redistat.rb

@@ -1,22 +1,32 @@
 
 require 'rubygems'
-require 'active_support'
-require 'active_support/hash_with_indifferent_access' if !{}.respond_to?(:with_indifferent_access) # Active Support 2.x and 3.x
-require 'redis'
 require 'date'
 require 'time'
-require 'time_ext'
-require 'json'
 require 'digest/sha1'
+require 'monitor'
+
+# Active Support 2.x or 3.x
+require 'active_support'
+if !{}.respond_to?(:with_indifferent_access)
+  require 'active_support/core_ext/hash/indifferent_access'
+  require 'active_support/core_ext/hash/reverse_merge'
+end
+
+require 'time_ext'
+require 'redis'
+require 'json'
+
+require 'redistat/mixins/options'
+require 'redistat/mixins/synchronize'
+require 'redistat/mixins/database'
+require 'redistat/mixins/date_helper'
 
-require 'redistat/collection'
 require 'redistat/connection'
-require 'redistat/database'
+require 'redistat/buffer'
+require 'redistat/collection'
 require 'redistat/date'
-require 'redistat/date_helper'
 require 'redistat/event'
 require 'redistat/finder'
-require 'redistat/finder/date_set'
 require 'redistat/key'
 require 'redistat/label'
 require 'redistat/model'
@@ -25,42 +35,72 @@ require 'redistat/scope'
 require 'redistat/summary'
 require 'redistat/version'
 
-require 'redistat/core_ext/date'
+require 'redistat/core_ext'
-require 'redistat/core_ext/time'
+
-require 'redistat/core_ext/fixnum'
-require 'redistat/core_ext/bignum'
 
 module Redistat
-  
+
   KEY_NEXT_ID = ".next_id"
   KEY_EVENT = ".event:"
-  KEY_LEBELS = "Redistat.lables:"
+  KEY_LABELS = "Redistat.labels:" # used for reverse label hash lookup
   KEY_EVENT_IDS = ".event_ids"
+  LABEL_INDEX = ".label_index:"
   GROUP_SEPARATOR = "/"
-  
+
   class InvalidOptions < ArgumentError; end
   class RedisServerIsTooOld < Exception; end
-  
+
   class << self
-    
+
+    def buffer
+      Buffer.instance
+    end
+
+    def buffer_size
+      buffer.size
+    end
+
+    def buffer_size=(size)
+      buffer.size = size
+    end
+
+    def thread_safe
+      Synchronize.thread_safe
+    end
+
+    def thread_safe=(value)
+      Synchronize.thread_safe = value
+    end
+
     def connection(ref = nil)
       Connection.get(ref)
     end
     alias :redis :connection
-    
+
     def connection=(connection)
       Connection.add(connection)
     end
     alias :redis= :connection=
-    
+
     def connect(options)
       Connection.create(options)
     end
-    
+
     def flush
       puts "WARNING: Redistat.flush is deprecated. Use Redistat.redis.flushdb instead."
       connection.flushdb
     end
-    
+
+    def group_separator
+      @group_separator ||= GROUP_SEPARATOR
+    end
+    attr_writer :group_separator
+
   end
 end
+
+
+# ensure buffer is flushed on program exit
+Kernel.at_exit do
+  Redistat.buffer.flush(true)
+end

lib/redistat/buffer.rb

@@ -0,0 +1,110 @@
+require 'redistat/core_ext/hash'
+
+module Redistat
+  class Buffer
+    include Synchronize
+
+    def self.instance
+      @instance ||= self.new
+    end
+
+    def size
+      synchronize do
+        @size ||= 0
+      end
+    end
+
+    def size=(value)
+      synchronize do
+        @size = value
+      end
+    end
+
+    def count
+      @count ||= 0
+    end
+
+    def store(key, stats, depth_limit, opts)
+      return false unless should_buffer?
+
+      to_flush = {}
+      buffkey = buffer_key(key, opts)
+
+      synchronize do
+        if !queue.has_key?(buffkey)
+          queue[buffkey] = { :key         => key,
+                             :stats       => {},
+                             :depth_limit => depth_limit,
+                             :opts        => opts }
+        end
+
+        queue[buffkey][:stats].merge_and_incr!(stats)
+        incr_count
+
+        # return items to be flushed if buffer size limit has been reached
+        to_flush = reset_queue
+      end
+
+      # flush any data that's been cleared from the queue
+      flush_data(to_flush)
+      true
+    end
+
+    def flush(force = false)
+      to_flush = {}
+      synchronize do
+        to_flush = reset_queue(force)
+      end
+      flush_data(to_flush)
+    end
+
+    private
+
+    # should always be called from within a synchronize block
+    def incr_count
+      @count ||= 0
+      @count += 1
+    end
+
+    def queue
+      @queue ||= {}
+    end
+
+    def should_buffer?
+      size > 1 # buffer size of 1 would be equal to not using buffer
+    end
+
+    # should always be called from within a synchronize block
+    def should_flush?
+      (!queue.blank? && count >= size)
+    end
+
+    # returns items to be flushed if buffer size limit has been reached
+    # should always be called from within a synchronize block
+    def reset_queue(force = false)
+      return {} if !force && !should_flush?
+      data = queue
+      @queue = {}
+      @count = 0
+      data
+    end
+
+    def flush_data(buffer_data)
+      buffer_data.each do |k, item|
+        Summary.update(item[:key], item[:stats], item[:depth_limit], item[:opts])
+      end
+    end
+
+    # depth_limit is not needed as it's evident in key.to_s
+    def buffer_key(key, opts)
+      # covert keys to strings, as sorting a Hash with Symbol keys fails on
+      # Ruby 1.8.x.
+      opts = opts.inject({}) do |result, (k, v)|
+        result[k.to_s] = v
+        result
+      end
+      "#{key.to_s}:#{opts.sort.flatten.join(':')}"
+    end
+
+  end
+end

lib/redistat/collection.rb

@@ -1,20 +1,20 @@
 module Redistat
   class Collection < ::Array
-    
+
     attr_accessor :from
     attr_accessor :till
     attr_accessor :depth
     attr_accessor :total
-    
+
     def initialize(options = {})
       @from = options[:from] ||= nil
       @till = options[:till] ||= nil
       @depth = options[:depth] ||= nil
     end
-    
+
     def total
       @total ||= {}
     end
-    
+
   end
-end
+end

lib/redistat/connection.rb

@@ -1,55 +1,80 @@
+require 'monitor'
+
 module Redistat
   module Connection
-    
+
     REQUIRED_SERVER_VERSION = "1.3.10"
-    
+    MIN_EXPIRE_SERVER_VERSION = "2.1.3"
+
+    # TODO: Create a ConnectionPool instance object using Sychronize mixin to replace Connection class
+
     class << self
-      
+
+      # TODO: clean/remove all ref-less connections
+
       def get(ref = nil)
         ref ||= :default
-        connections[references[ref]] || create
+        synchronize do
+          connections[references[ref]] || create
+        end
       end
-      
+
       def add(conn, ref = nil)
         ref ||= :default
-        check_redis_version(conn)
+        synchronize do
-        references[ref] = conn.client.id
+          check_redis_version(conn)
-        connections[conn.client.id] = conn
+          references[ref] = conn.client.id
+          connections[conn.client.id] = conn
+        end
       end
-      
+
       def create(options = {})
-        #TODO clean/remove all ref-less connections
+        synchronize do
-        ref = options.delete(:ref) || :default
+          options = options.clone
-        options.reverse_merge!(default_options)
+          ref = options.delete(:ref) || :default
-        conn = (connections[connection_id(options)] ||= connection(options))
+          options.reverse_merge!(default_options)
-        references[ref] = conn.client.id
+          conn = (connections[connection_id(options)] ||= connection(options))
-        conn
+          references[ref] = conn.client.id
+          conn
+        end
       end
-      
+
       def connections
         @connections ||= {}
       end
-      
+
       def references
         @references ||= {}
       end
-      
+
       private
-      
+
-      def check_redis_version(conn)
+      def monitor
-        raise RedisServerIsTooOld if conn.info["redis_version"] < REQUIRED_SERVER_VERSION
+        @monitor ||= Monitor.new
-        conn
       end
-      
+
+      def synchronize(&block)
+        monitor.synchronize(&block)
+      end
+
       def connection(options)
         check_redis_version(Redis.new(options))
       end
-      
+
       def connection_id(options = {})
-        options.reverse_merge!(default_options)
+        options = options.reverse_merge(default_options)
         "redis://#{options[:host]}:#{options[:port]}/#{options[:db]}"
       end
-      
+
+      def check_redis_version(conn)
+        raise RedisServerIsTooOld if conn.info["redis_version"] < REQUIRED_SERVER_VERSION
+        if conn.info["redis_version"] < MIN_EXPIRE_SERVER_VERSION
+          STDOUT.puts "WARNING: You MUST upgrade Redis to v2.1.3 or later " +
+            "if you are using key expiry."
+        end
+        conn
+      end
+
       def default_options
         {
           :host => '127.0.0.1',
@@ -58,7 +83,7 @@ module Redistat
           :timeout => 5
         }
       end
-      
+
     end
   end
-end
+end

lib/redistat/core_ext.rb

@@ -0,0 +1,5 @@
+require 'redistat/core_ext/bignum'
+require 'redistat/core_ext/date'
+require 'redistat/core_ext/fixnum'
+require 'redistat/core_ext/hash'
+require 'redistat/core_ext/time'

lib/redistat/core_ext/bignum.rb

@@ -1,8 +1,8 @@
 class Bignum
   include Redistat::DateHelper
-  
+
   def to_time
     Time.at(self)
   end
-  
+
 end

lib/redistat/core_ext/date.rb

@@ -1,8 +1,8 @@
 class Date
   include Redistat::DateHelper
-  
+
   def to_time
     Time.parse(self.to_s)
   end
-  
+
 end

lib/redistat/core_ext/fixnum.rb

@@ -1,8 +1,8 @@
 class Fixnum
   include Redistat::DateHelper
-  
+
   def to_time
     Time.at(self)
   end
-  
+
 end

lib/redistat/core_ext/hash.rb

@@ -0,0 +1,23 @@
+class Hash
+
+  def merge_and_incr(hash)
+    self.clone.merge_and_incr!(hash)
+  end
+
+  def merge_and_incr!(hash)
+    raise ArgumentError unless hash.is_a?(Hash)
+    hash.each do |key, value|
+      self[key] = value unless self.set_or_incr(key, value)
+    end
+    self
+  end
+
+  def set_or_incr(key, value)
+    return false unless value.is_a?(Numeric)
+    self[key] = 0 unless self.has_key?(key)
+    return false unless self[key].is_a?(Numeric)
+    self[key] += value
+    true
+  end
+
+end

lib/redistat/date.rb

@@ -1,6 +1,6 @@
 module Redistat
   class Date
-    
+
     attr_accessor :year
     attr_accessor :month
     attr_accessor :day
@@ -9,9 +9,9 @@ module Redistat
     attr_accessor :sec
     attr_accessor :usec
     attr_accessor :depth
-    
+
     DEPTHS = [:year, :month, :day, :hour, :min, :sec, :usec]
-    
+
     def initialize(input, depth = nil)
       @depth = depth
       if input.is_a?(::Time)
@@ -26,12 +26,12 @@ module Redistat
         from_integer(input)
       end
     end
-    
+
     def to_t
       ::Time.local(@year, @month, @day, @hour, @min, @sec, @usec)
     end
     alias :to_time :to_t
-    
+
     def to_d
       ::Date.civil(@year, @month, @day)
     end
@@ -41,7 +41,7 @@ module Redistat
       to_time.to_i
     end
     alias :to_integer :to_i
-    
+
     def to_s(depth = nil)
       depth ||= @depth ||= :sec
       output = ""
@@ -57,9 +57,9 @@ module Redistat
       output
     end
     alias :to_string :to_s
-    
+
     private
-    
+
     def from_time(input)
       DEPTHS.each do |k|
         send("#{k}=", input.send(k))
@@ -74,15 +74,15 @@ module Redistat
         send("#{k}=", 0)
       end
     end
-    
+
     def from_integer(input)
       from_time(::Time.at(input))
     end
-    
+
     def from_string(input)
       input += "19700101000000"[input.size..-1] if input =~ /^\d\d\d[\d]+$/i
       from_time(::Time.parse(input))
     end
-    
+
   end
 end

lib/redistat/event.rb

@@ -1,70 +1,58 @@
 module Redistat
   class Event
     include Database
-    
+    include Options
+
     attr_reader :id
     attr_reader :key
-    attr_reader :connection_ref
+
-    
     attr_accessor :stats
     attr_accessor :meta
-    attr_accessor :options
-    
-    def initialize(scope, label = nil, date = nil, stats = {}, options = {}, meta = {}, is_new = true)
-      @options = parse_options(options)
-      @connection_ref = @options[:connection_ref]
-      @key = Key.new(scope, label, date, @options)
-      @stats = stats ||= {}
-      @meta = meta ||= {}
-      @new = is_new
-    end
-    
-    def db
-      super(@connection_ref)
-    end
-    
-    def parse_options(options)
-      default_options.each do |opt, val|
-        options[opt] = val if options[opt].nil?
-      end
-      options
-    end
 
     def default_options
       { :depth => :hour,
         :store_event => false,
         :connection_ref => nil,
-        :enable_grouping => true }
+        :enable_grouping => true,
+        :label_indexing => true }
     end
-    
+
+    def initialize(scope, label = nil, date = nil, stats = {}, opts = {}, meta = {}, is_new = true)
+      parse_options(opts)
+      @key = Key.new(scope, label, date, @options)
+      @stats = stats ||= {}
+      @meta = meta ||= {}
+      @new = is_new
+    end
+
     def new?
       @new
     end
-    
+
     def date
       @key.date
     end
-    
+
     def date=(input)
       @key.date = input
     end
-    
+
     def scope
       @key.scope
     end
-    
+
     def scope=(input)
       @key.scope = input
     end
-    
+
     def label
       @key.label
     end
-    
+
     def label_hash
       @key.label_hash
     end
-    
+
     def label=(input)
       @key.label = input
     end
@@ -72,10 +60,10 @@ module Redistat
     def next_id
       db.incr("#{self.scope}#{KEY_NEXT_ID}")
     end
-    
+
     def save
       return false if !self.new?
-      Summary.update_all(@key, @stats, depth_limit, @connection_ref, @options[:enable_grouping])
+      Summary.update_all(@key, @stats, depth_limit, @options)
       if @options[:store_event]
         @id = self.next_id
         db.hmset("#{self.scope}#{KEY_EVENT}#{@id}",
@@ -90,21 +78,21 @@ module Redistat
       @new = false
       self
     end
-    
+
     def depth_limit
       @options[:depth] ||= @key.depth
     end
-    
+
     def self.create(*args)
       self.new(*args).save
     end
-    
+
     def self.find(scope, id)
       event = db.hgetall "#{scope}#{KEY_EVENT}#{id}"
       return nil if event.size == 0
       self.new( event["scope"], event["label"], event["date"], JSON.parse(event["stats"]),
-                JSON.parse(event["meta"]), JSON.parse(event["options"]), false )
+                JSON.parse(event["options"]), JSON.parse(event["meta"]), false )
     end
-    
+
   end
-end
+end

lib/redistat/finder.rb

@@ -1,38 +1,166 @@
+require 'redistat/finder/date_set'
+
 module Redistat
   class Finder
     include Database
-    
+
+    class << self
+      def find(*args)
+        new.find(*args)
+      end
+
+      def scope(scope)
+        new.scope(scope)
+      end
+
+      def label(label)
+        new.label(label)
+      end
+
+      def dates(from, till)
+        new.dates(from, till)
+      end
+      alias :date :dates
+
+      def from(date)
+        new.from(date)
+      end
+
+      def till(date)
+        new.till(date)
+      end
+      alias :untill :till
+
+      def depth(unit)
+        new.depth(unit)
+      end
+
+      def interval(unit)
+        new.interval(unit)
+      end
+    end
+
     attr_reader :options
-    
+
-    def initialize(options = {})
+    def initialize(opts = {})
-      @options = options
+      set_options(opts)
     end
-    
+
-    def db
+    def options
-      super(@options[:connection_ref])
+      @options ||= {}
     end
-    
+
-    def valid_options?
+    def all(reload = false)
-      return true if !@options[:scope].blank? && !@options[:label].blank? && !@options[:from].blank? && !@options[:till].blank?
+      @result = nil if reload
-      false
+      @result ||= find
     end
-    
+
-    def find(options = {})
+    def total
-      @options.merge!(options)
+      all.total
+    end
+
+    def each(&block)
+      all.each(&block)
+    end
+
+    def map(&block)
+      all.map(&block)
+    end
+
+    def each_with_index(&block)
+      all.each_with_index(&block)
+    end
+
+    def parent
+      @parent ||= self.class.new(options.merge(:label => options[:label].parent)) unless options[:label].nil?
+    end
+
+    def children
+      build_key.children.map { |key|
+        self.class.new(options.merge(:label => key.label.to_s))
+      }
+    end
+
+    def connection_ref(ref = nil)
+      return options[:connection_ref] if ref.nil?
+      reset! if options[:connection_ref] != ref
+      options[:connection_ref] = ref
+      self
+    end
+
+    def scope(input = nil)
+      return options[:scope] if input.nil?
+      reset! if !options[:scope].nil? && options[:scope].to_s != input.to_s
+      options[:scope] = Scope.new(input)
+      self
+    end
+
+    def label(input = nil)
+      return options[:label] if input.nil?
+      reset! if options.has_key?(:label) && options[:label].to_s != input.to_s
+      options[:label] = (!input.nil?) ? Label.new(input) : nil
+      self
+    end
+
+    def dates(start, finish)
+      from(start).till(finish)
+    end
+    alias :date :dates
+
+    def from(date = nil)
+      return options[:from] if date.nil?
+      reset! if options[:from] != date
+      options[:from] = date
+      self
+    end
+
+    def till(date = nil)
+      return options[:till] if date.nil?
+      reset! if options[:till] != date
+      options[:till] = date
+      self
+    end
+    alias :until :till
+
+    def depth(unit = nil)
+      return options[:depth] if unit.nil?
+      reset! if options[:depth] != unit
+      options[:depth] = unit
+      self
+    end
+
+    def interval(unit = nil)
+      return options[:interval] if unit.nil?
+      reset! if options[:interval] != unit
+      options[:interval] = unit
+      self
+    end
+
+    def find(opts = {})
+      set_options(opts)
       raise InvalidOptions.new if !valid_options?
-      if @options[:interval].nil? || !@options[:interval]
+      if options[:interval].nil? || !options[:interval]
         find_by_magic
       else
         find_by_interval
       end
     end
-    
+
-    def find_by_interval(options = {})
+    private
-      @options.merge!(options)
+
+    def set_options(opts = {})
+      opts = opts.clone
+      opts.each do |key, value|
+        self.send(key, opts.delete(key)) if self.respond_to?(key)
+      end
+      self.options.merge!(opts)
+    end
+
+    def find_by_interval
       raise InvalidOptions.new if !valid_options?
       key = build_key
-      col = Collection.new(@options)
+      col = Collection.new(options)
-      col.total = Result.new(@options)
+      col.total = Result.new(options)
       build_date_sets.each do |set|
         set[:add].each do |date|
           result = Result.new
@@ -46,13 +174,12 @@ module Redistat
       end
       col
     end
-    
+
-    def find_by_magic(options = {})
+    def find_by_magic
-      @options.merge!(options)
       raise InvalidOptions.new if !valid_options?
-      key = Key.new(@options[:scope], @options[:label])
+      key = build_key
-      col = Collection.new(@options)
+      col = Collection.new(options)
-      col.total = Result.new(@options)
+      col.total = Result.new(options)
       col << col.total
       build_date_sets.each do |set|
         sum = Result.new
@@ -64,15 +191,25 @@ module Redistat
       end
       col
     end
-    
+
+    def reset!
+      @result = nil
+      @parent = nil
+    end
+
+    def valid_options?
+      return true if !options[:scope].blank? && !options[:label].blank? && !options[:from].blank? && !options[:till].blank?
+      false
+    end
+
     def build_date_sets
-      Finder::DateSet.new(@options[:from], @options[:till], @options[:depth], @options[:interval])
+      Finder::DateSet.new(options[:from], options[:till], options[:depth], options[:interval])
     end
-    
+
     def build_key
-      Key.new(@options[:scope], @options[:label])
+      Key.new(options[:scope], options[:label])
     end
-    
+
     def summarize_add_keys(sets, key, sum)
       sets.each do |date|
         db.hgetall("#{key.prefix}#{date}").each do |k, v|
@@ -81,7 +218,7 @@ module Redistat
       end
       sum
     end
-    
+
     def summarize_rem_keys(sets, key, sum)
       sets.each do |date|
         db.hgetall("#{key.prefix}#{date}").each do |k, v|
@@ -90,80 +227,10 @@ module Redistat
       end
       sum
     end
-    
+
-    class << self
+    def db
-    
+      super(options[:connection_ref])
-      def find(*args)
-        new.find(*args)
-      end
-      
-      def scope(scope)
-        new.scope(scope)
-      end
-      
-      def label(label)
-        new.label(label)
-      end
-      
-      def dates(from, till)
-        new.dates(from, till)
-      end
-      alias :date :dates
-      
-      def from(date)
-        new.from(date)
-      end
-      
-      def till(date)
-        new.till(date)
-      end
-      alias :untill :till
-      
-      def depth(unit)
-        new.depth(unit)
-      end
-      
-      def interval(unit)
-        new.interval(unit)
-      end
-      
     end
-    
+
-    def scope(scope)
-      @options[:scope] = scope
-      self
-    end
-    
-    def label(label)
-      @options[:label] = label
-      self
-    end
-    
-    def dates(from, till)
-      from(from).till(till)
-    end
-    alias :date :dates
-    
-    def from(date)
-      @options[:from] = date
-      self
-    end
-    
-    def till(date)
-      @options[:till] = date
-      self
-    end
-    alias :until :till
-    
-    def depth(unit)
-      @options[:depth] = unit
-      self
-    end
-    
-    def interval(unit)
-      @options[:interval] = unit
-      self
-    end
-    
   end
-end
+end

lib/redistat/finder/date_set.rb

@@ -1,7 +1,7 @@
 module Redistat
   class Finder
     class DateSet < Array
-      
+
       def initialize(start_date = nil, end_date = nil, depth = nil, interval = false)
         if !start_date.nil? && !end_date.nil?
           find_date_sets(start_date, end_date, depth, interval)
@@ -9,6 +9,10 @@ module Redistat
       end
 
       def find_date_sets(start_date, end_date, depth = nil, interval = false)
+        if depth.nil? && interval.is_a?(Symbol)
+          depth = interval
+          interval = true
+        end
         start_date = start_date.to_time if start_date.is_a?(::Date)
         end_date = end_date.to_time if end_date.is_a?(::Date)
         if !interval
@@ -67,7 +71,7 @@ module Redistat
           end
           { :add => add, :rem => [] }
         elsif has_nunit
-          { :add => [end_date.beginning_of(nunit).to_rs.to_s(nunit)], 
+          { :add => [end_date.beginning_of(nunit).to_rs.to_s(nunit)],
             :rem => end_date.map_beginning_of_each(unit, :include_start => !lowest_depth).until(end_date.end_of(nunit)) { |t| t.to_rs.to_s(unit) } }
         else
           { :add => [], :rem => [] }
@@ -89,7 +93,7 @@ module Redistat
           { :add => [], :rem => [] }
         end
       end
-      
+
     end
   end
-end
+end

lib/redistat/key.rb

@@ -1,64 +1,84 @@
 module Redistat
   class Key
-    
+    include Database
-    attr_accessor :scope
+    include Options
-    attr_accessor :date
+
-    attr_accessor :options
+    def default_options
-    
+      { :depth => :hour }
-    def initialize(scope, label_name = nil, time_stamp = nil, options = {})
+    end
-      @options = default_options.merge(options || {})
+
-      @scope = scope
+    def initialize(scope, label_name = nil, time_stamp = nil, opts = {})
+      parse_options(opts)
+      self.scope = scope
       self.label = label_name if !label_name.nil?
       self.date = time_stamp ||= Time.now
     end
-    
+
-    def default_options
-      { :depth => :hour, :hashed_label => false }
-    end
-    
     def prefix
       key = "#{@scope}"
-      key << "/#{label}" if !label.nil?
+      key << "/#{label.name}" if !label.nil?
       key << ":"
       key
     end
-    
+
     def date=(input)
       @date = (input.instance_of?(Redistat::Date)) ? input : Date.new(input) # Redistat::Date, not ::Date
     end
-    
+    attr_reader :date
+
     def depth
-      @options[:depth]
+      options[:depth]
     end
-    
+
-    def label
+    # def scope
-      @label.name
+    #   @scope.to_s
+    # end
+
+    def scope=(input)
+      @scope = (input.instance_of?(Redistat::Scope)) ? input : Scope.new(input)
     end
-    
+    attr_reader :scope
-    def label_hash
+
-      @label.hash
-    end
-    
-    def label_groups
-      @label.groups
-    end
-    
     def label=(input)
       @label = (input.instance_of?(Redistat::Label)) ? input : Label.create(input, @options)
     end
-    
+    attr_reader :label
-    def groups
+
-      @groups ||= label_groups.map do |label_name|
+    def label_hash
-        self.class.new(@scope, label_name, self.date, @options)
+      @label.hash
+    end
+
+    def parent
+      @parent ||= self.class.new(self.scope, @label.parent, self.date, @options) unless @label.parent.nil?
+    end
+
+    def children
+      members = db.smembers("#{scope}#{LABEL_INDEX}#{@label}") || [] # older versions of Redis returns nil
+      members.map { |member|
+        child_label = [@label, member].reject { |i| i.nil? }
+        self.class.new(self.scope, child_label.join(Redistat.group_separator), self.date, @options)
+      }
+    end
+
+    def update_index
+      @label.groups.each do |label|
+        parent = (label.parent || "")
+        db.sadd("#{scope}#{LABEL_INDEX}#{parent}", label.me)
       end
     end
-    
+
+    def groups
+      @groups ||= @label.groups.map do |label|
+        self.class.new(@scope, label, self.date, @options)
+      end
+    end
+
     def to_s(depth = nil)
       depth ||= @options[:depth]
       key = self.prefix
       key << @date.to_s(depth)
       key
     end
-    
+
   end
-end
+end

lib/redistat/label.rb

@@ -1,53 +1,69 @@
 module Redistat
   class Label
     include Database
-    
+    include Options
-    attr_reader :raw
+
-    attr_reader :connection_ref
+    def default_options
-    
+      { :hashed_label => false }
-    def self.create(name, options = {})
-      self.new(name, options).save
     end
-    
+
-    def initialize(str, options = {})
+    def self.create(name, opts = {})
-      @options = options
+      self.new(name, opts).save
+    end
+
+    def self.join(*args)
+      args = args.map {|i| i.to_s}
+      self.new(args.reject {|i| i.blank? }.join(Redistat.group_separator))
+    end
+
+    def initialize(str, opts = {})
+      parse_options(opts)
       @raw = str.to_s
     end
 
-    def db
+    def to_s
-      super(@options[:connection_ref])
+      @raw
     end
-    
+
     def name
-      @options[:hashed_label] ? hash : @raw
+      @options[:hashed_label] ? hash : self.to_s
     end
-    
+
     def hash
-      @hash ||= Digest::SHA1.hexdigest(@raw)
+      @hash ||= Digest::SHA1.hexdigest(self.to_s)
     end
-    
+
     def save
-      @saved = (db.set("#{KEY_LEBELS}#{hash}", @raw) == "OK") if @options[:hashed_label]
+      @saved = db.hset(KEY_LABELS, hash, self.to_s) if @options[:hashed_label]
       self
     end
-    
+
     def saved?
+      return true unless @options[:hashed_label]
       @saved ||= false
     end
-    
+
+    def parent
+      @parent ||= groups[1] if groups.size > 1
+    end
+
+    def me
+      self.to_s.split(Redistat.group_separator).last
+    end
+
     def groups
-      return @groups if @groups
+      return @groups unless @groups.nil?
       @groups = []
       parent = ""
-      @raw.split(GROUP_SEPARATOR).each do |part|
+      self.to_s.split(Redistat.group_separator).each do |part|
         if !part.blank?
-          group = ((parent.blank?) ? "" : "#{parent}/") + part
+          group = ((parent.blank?) ? "" : "#{parent}#{Redistat.group_separator}") + part
-          @groups << group
+          @groups << Label.new(group)
           parent = group
         end
       end
       @groups.reverse!
     end
-    
+
   end
-end
+end

lib/redistat/mixins/database.rb

@@ -4,7 +4,8 @@ module Redistat
       base.extend(Database)
     end
     def db(ref = nil)
+      ref ||= @options[:connection_ref] if !@options.nil?
       Redistat.connection(ref)
     end
   end
-end
+end

lib/redistat/mixins/date_helper.rb

@@ -5,4 +5,4 @@ module Redistat
     end
     alias :to_rs :to_redistat
   end
-end
+end

lib/redistat/mixins/options.rb

@@ -0,0 +1,41 @@
+module Redistat
+  module Options
+
+    def self.included(base)
+      base.extend(ClassMethods)
+    end
+
+    module ClassMethods
+      def option_accessor(*opts)
+        opts.each do |option|
+          define_method(option) do |*args|
+            if !args.first.nil?
+              options[option.to_sym] = args.first
+            else
+              options[option.to_sym] || nil
+            end
+          end
+        end
+      end
+    end
+
+    def parse_options(opts)
+      opts ||= {}
+      @raw_options = opts
+      @options = default_options.merge(opts.reject { |k,v| v.nil? })
+    end
+
+    def default_options
+      {}
+    end
+
+    def options
+      @options ||= {}
+    end
+
+    def raw_options
+      @raw_options ||= {}
+    end
+
+  end
+end

lib/redistat/mixins/synchronize.rb

@@ -0,0 +1,52 @@
+require 'monitor'
+
+module Redistat
+  module Synchronize
+
+    class << self
+      def included(base)
+        base.send(:include, InstanceMethods)
+      end
+
+      def monitor
+        @monitor ||= Monitor.new
+      end
+
+      def thread_safe
+        monitor.synchronize do
+          return @thread_safe unless @thread_safe.nil?
+          @thread_safe = false
+        end
+      end
+
+      def thread_safe=(value)
+        monitor.synchronize do
+          @thread_safe = value
+        end
+      end
+    end # << self
+
+    module InstanceMethods
+      def thread_safe
+        Synchronize.thread_safe
+      end
+
+      def thread_safe=(value)
+        Synchronize.thread_safe = value
+      end
+
+      def monitor
+        Synchronize.monitor
+      end
+
+      def synchronize(&block)
+        if thread_safe
+          monitor.synchronize(&block)
+        else
+          block.call
+        end
+      end
+    end # InstanceMethods
+
+  end
+end

lib/redistat/model.rb

@@ -1,78 +1,77 @@
 module Redistat
   module Model
-    include Redistat::Database
+    include Database
-    
+    include Options
+
     def self.included(base)
       base.extend(self)
     end
-    
+
-    def store(label, stats = {}, date = nil, meta = {}, opts = {})
+
-      Event.new(name, label, date, stats, options.merge(opts), meta).save
+    #
+    # statistics store/fetch methods
+    #
+
+    def store(label, stats = {}, date = nil, opts = {}, meta = {})
+      Event.new(self.name, label, date, stats, options.merge(opts), meta).save
     end
     alias :event :store
-    
+
+    def fetch(label, from, till, opts = {})
+      find(label, from, till, opts).all
+    end
+    alias :lookup :fetch
+
+    def find(label, from, till, opts = {})
+      Finder.new( { :scope => self.name,
+                    :label => label,
+                    :from  => from,
+                    :till  => till }.merge(options.merge(opts)) )
+    end
+
+    def find_event(event_id)
+      Event.find(self.name, event_id)
+    end
+
+
+    #
+    # options methods
+    #
+
+    option_accessor :depth
+    option_accessor :scope
+    option_accessor :store_event
+    option_accessor :hashed_label
+    option_accessor :label_indexing
+
+    alias :class_name :scope
+
+    def expire(exp = nil)
+      if !exp.nil?
+        options[:expire] = exp.is_a?(Hash) ? exp : Hash.new(exp)
+      else
+        options[:expire]
+      end
+    end
+
     def connect_to(opts = {})
       Connection.create(opts.merge(:ref => name))
       options[:connection_ref] = name
     end
-    
+
+
+    #
+    # resource access methods
+    #
+
     def connection
       db(options[:connection_ref])
     end
     alias :redis :connection
-    
+
-    def fetch(label, from, till, opts = {})
-      Finder.find({
-        :scope => name,
-        :label => label,
-        :from  => from,
-        :till  => till
-      }.merge(options.merge(opts)))
-    end
-    alias :lookup :fetch
-    
-    def hashed_label(boolean = nil)
-      if !boolean.nil?
-        options[:hashed_label] = boolean
-      else
-        options[:hashed_label] || nil
-      end
-    end
-    
-    def class_name(class_name = nil)
-      if !class_name.nil?
-        options[:class_name] = class_name
-      else
-        options[:class_name] || nil
-      end
-    end
-    alias :scope :class_name
-    
-    def depth(depth = nil)
-      if !depth.nil?
-        options[:depth] = depth
-      else
-        options[:depth] || nil
-      end
-    end
-    
-    def store_event(boolean = nil)
-      if !boolean.nil?
-        options[:store_event] = boolean
-      else
-        options[:store_event] || nil
-      end
-    end
-    
-    def options
-      @options ||= {}
-    end
-    
-    private
-    
     def name
-      options[:class_name] || (@name ||= self.to_s)
+      options[:scope] || (@name ||= self.to_s)
     end
-    
+
   end
-end
+end

lib/redistat/result.rb

@@ -1,23 +1,18 @@
+require 'active_support/core_ext/hash/indifferent_access'
+
 module Redistat
   class Result < HashWithIndifferentAccess
-    
+
     attr_accessor :from
     attr_accessor :till
-    
+
     alias :date :from
     alias :date= :from=
-    
+
     def initialize(options = {})
       @from = options[:from] ||= nil
       @till = options[:till] ||= nil
     end
-    
+
-    
-    def set_or_incr(key, value)
-      self[key] = 0 if !self.has_key?(key)
-      self[key] += value
-      self
-    end
-    
   end
-end
+end

lib/redistat/scope.rb

@@ -1,18 +1,18 @@
 module Redistat
   class Scope
     include Database
-    
+
     def initialize(name)
       @name = name.to_s
     end
-    
+
     def to_s
       @name
     end
-    
+
     def next_id
       db.incr("#{@name}#{KEY_NEXT_ID}")
     end
-    
+
   end
-end
+end

lib/redistat/summary.rb

@@ -1,58 +1,89 @@
 module Redistat
   class Summary
     include Database
-    
+
-    def self.update_all(key, stats = {}, depth_limit = nil, connection_ref = nil, enable_grouping = nil)
+    class << self
-      stats ||= {}
+
-      return nil if stats.size == 0
+      def default_options
-      
+        {
-      depth_limit ||= key.depth
+          :enable_grouping => true,
-      enable_grouping = true if enable_grouping.nil?
+          :label_indexing  => true,
-      
+          :connection_ref  => nil,
-      if enable_grouping
+          :expire          => {}
-        stats = inject_group_summaries(stats)
-        key.groups.each { |k|
-          update_key(k, stats, depth_limit, connection_ref)
         }
-      else
-        update_key(key, stats, depth_limit, connection_ref)
       end
-    end
+
-    
+      def buffer
-    private
+        Redistat.buffer
-    
-    def self.update_key(key, stats, depth_limit, connection_ref)
-      Date::DEPTHS.each do |depth|
-        update(key, stats, depth, connection_ref)
-        break if depth == depth_limit
       end
-    end
+
-    
+      def update_all(key, stats = {}, depth_limit = nil, opts = {})
-    def self.update(key, stats, depth, connection_ref = nil)
+        stats ||= {}
-      stats.each do |field, value|
+        return if stats.empty?
-        db(connection_ref).hincrby key.to_s(depth), field, value
+
+        options = default_options.merge((opts || {}).reject { |k,v| v.nil? })
+
+        depth_limit ||= key.depth
+
+        update_through_buffer(key, stats, depth_limit, options)
       end
-    end
+
-    
+      def update_through_buffer(*args)
-    def self.inject_group_summaries!(stats)
+        update(*args) unless buffer.store(*args)
-      stats.each do |key, value|
+      end
-        parts = key.to_s.split(GROUP_SEPARATOR)
+
-        parts.pop
+      def update(key, stats, depth_limit, opts = {})
-        if parts.size > 0
+        if opts[:enable_grouping]
-          sum_parts = []
+          stats = inject_group_summaries(stats)
-          parts.each do |part|
+          key.groups.each do |k|
-            sum_parts << part
+            update_key(k, stats, depth_limit, opts)
-            sum_key = sum_parts.join(GROUP_SEPARATOR)
+            k.update_index if opts[:label_indexing]
-            (stats.has_key?(sum_key)) ? stats[sum_key] += value : stats[sum_key] = value
           end
+        else
+          update_key(key, stats, depth_limit, opts)
         end
       end
-      stats
+
+      private
+
+      def update_key(key, stats, depth_limit, opts = {})
+        Date::DEPTHS.each do |depth|
+          update_fields(key, stats, depth, opts)
+          break if depth == depth_limit
+        end
+      end
+
+      def update_fields(key, stats, depth, opts = {})
+        stats.each do |field, value|
+          db(opts[:connection_ref]).hincrby key.to_s(depth), field, value
+        end
+
+        if opts[:expire] && !opts[:expire][depth].nil?
+          db(opts[:connection_ref]).expire key.to_s(depth), opts[:expire][depth]
+        end
+      end
+
+      def inject_group_summaries!(stats)
+        summaries = {}
+        stats.each do |key, value|
+          parts = key.to_s.split(Redistat.group_separator)
+          parts.pop
+          if parts.size > 0
+            sum_parts = []
+            parts.each do |part|
+              sum_parts << part
+              sum_key = sum_parts.join(Redistat.group_separator)
+              (summaries.has_key?(sum_key)) ? summaries[sum_key] += value : summaries[sum_key] = value
+            end
+          end
+        end
+        stats.merge_and_incr!(summaries)
+      end
+
+      def inject_group_summaries(stats)
+        inject_group_summaries!(stats.clone)
+      end
+
     end
-    
-    def self.inject_group_summaries(stats)
-      inject_group_summaries!(stats.clone)
-    end
-    
   end
-end
+end

lib/redistat/version.rb

@@ -1,3 +1,3 @@
 module Redistat
-  VERSION = "0.1.0"
+  VERSION = "0.5.0"
 end

redistat.gemspec

@@ -18,13 +18,14 @@ Gem::Specification.new do |s|
   s.test_files    = `git ls-files -- {test,spec,features}/*`.split("\n")
   s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
   s.require_paths = ["lib"]
-  
+
   s.add_runtime_dependency 'activesupport', '>= 2.3.6'
   s.add_runtime_dependency 'json', '>= 1.4.0'
   s.add_runtime_dependency 'redis', '>= 2.1.0'
-  s.add_runtime_dependency 'time_ext', '>= 0.2.8'
+  s.add_runtime_dependency 'time_ext', '>= 0.2.9'
-  
+
+  s.add_development_dependency 'rake', '>= 0.8.7'
   s.add_development_dependency 'rspec', '>= 2.1.0'
-  s.add_development_dependency 'rcov', '>= 0.9.9'
   s.add_development_dependency 'yard', '>= 0.6.3'
+  s.add_development_dependency 'simplecov', '>= 0.6.1'
 end

spec/buffer_spec.rb

@@ -0,0 +1,159 @@
+require "spec_helper"
+
+describe Redistat::Buffer do
+
+  before(:each) do
+    @class  = Redistat::Buffer
+    @buffer = Redistat::Buffer.instance
+    @key = mock('Key', :to_s => "Scope/label:2011")
+    @stats = {:count => 1, :views => 3}
+    @depth_limit = :hour
+    @opts = {:enable_grouping => true}
+  end
+
+  # let's cleanup after ourselves for the other specs
+  after(:each) do
+    @class.instance_variable_set("@instance", nil)
+    @buffer.size = 0
+  end
+
+  it "should provide instance of itself" do
+    @buffer.should be_a(@class)
+  end
+
+  it "should only buffer if buffer size setting is greater than 1" do
+    @buffer.size.should == 0
+    @buffer.send(:should_buffer?).should be_false
+    @buffer.size = 1
+    @buffer.size.should == 1
+    @buffer.send(:should_buffer?).should be_false
+    @buffer.size = 2
+    @buffer.size.should == 2
+    @buffer.send(:should_buffer?).should be_true
+  end
+
+  it "should only flush buffer if buffer size is greater than or equal to buffer size setting" do
+    @buffer.size.should == 0
+    @buffer.send(:queue).size.should == 0
+    @buffer.send(:should_flush?).should be_false
+    @buffer.send(:queue)[:hello] = 'world'
+    @buffer.send(:incr_count)
+    @buffer.send(:should_flush?).should be_true
+    @buffer.size = 5
+    @buffer.send(:should_flush?).should be_false
+    3.times { |i|
+      @buffer.send(:queue)[i] = i.to_s
+      @buffer.send(:incr_count)
+    }
+    @buffer.send(:should_flush?).should be_false
+    @buffer.send(:queue)[4] = '4'
+    @buffer.send(:incr_count)
+    @buffer.send(:should_flush?).should be_true
+  end
+
+  it "should force flush queue irregardless of result of #should_flush? when #reset_queue is called with true" do
+    @buffer.send(:queue)[:hello] = 'world'
+    @buffer.send(:incr_count)
+    @buffer.send(:should_flush?).should be_true
+    @buffer.size = 2
+    @buffer.send(:should_flush?).should be_false
+    @buffer.send(:reset_queue).should == {}
+    @buffer.instance_variable_get("@count").should == 1
+    @buffer.send(:reset_queue, true).should == {:hello => 'world'}
+    @buffer.instance_variable_get("@count").should == 0
+  end
+
+  it "should #flush_data into Summary.update properly" do
+    # the root level key value doesn't actually matter, but it's something like this...
+    data = {'ScopeName/label/goes/here:2011::true:true' => {
+      :key => @key,
+      :stats => @stats,
+      :depth_limit => @depth_limit,
+      :opts => @opts
+    }}
+    item = data.first[1]
+    Redistat::Summary.should_receive(:update).with(@key, @stats, @depth_limit, @opts)
+    @buffer.send(:flush_data, data)
+  end
+
+  it "should build #buffer_key correctly" do
+    opts = {:enable_grouping => true, :label_indexing => false, :connection_ref => nil}
+    @buffer.send(:buffer_key, @key, opts).should ==
+      "#{@key.to_s}:connection_ref::enable_grouping:true:label_indexing:false"
+    opts = {:enable_grouping => false, :label_indexing => true, :connection_ref => :omg}
+    @buffer.send(:buffer_key, @key, opts).should ==
+      "#{@key.to_s}:connection_ref:omg:enable_grouping:false:label_indexing:true"
+  end
+
+  describe "Buffering" do
+    it "should store items on buffer queue" do
+      @buffer.store(@key, @stats, @depth_limit, @opts).should be_false
+      @buffer.size = 5
+      @buffer.store(@key, @stats, @depth_limit, @opts).should be_true
+      @buffer.send(:queue).should have(1).item
+      @buffer.send(:queue)[@buffer.send(:queue).keys.first][:stats][:count].should == 1
+      @buffer.send(:queue)[@buffer.send(:queue).keys.first][:stats][:views].should == 3
+      @buffer.store(@key, @stats, @depth_limit, @opts).should be_true
+      @buffer.send(:queue).should have(1).items
+      @buffer.send(:queue)[@buffer.send(:queue).keys.first][:stats][:count].should == 2
+      @buffer.send(:queue)[@buffer.send(:queue).keys.first][:stats][:views].should == 6
+    end
+
+    it "should flush buffer queue when size is reached" do
+      key = mock('Key', :to_s => "Scope/labelx:2011")
+      @buffer.size = 10
+      Redistat::Summary.should_receive(:update).exactly(2).times.and_return do |k, stats, depth_limit, opts|
+        depth_limit.should == @depth_limit
+        opts.should == @opts
+        if k == @key
+          stats[:count].should == 6
+          stats[:views].should == 18
+        elsif k == key
+          stats[:count].should == 4
+          stats[:views].should == 12
+        end
+      end
+      6.times { @buffer.store(@key, @stats, @depth_limit, @opts).should be_true }
+      4.times { @buffer.store(key, @stats, @depth_limit, @opts).should be_true }
+    end
+  end
+
+  describe "Thread-Safety" do
+    it "should read/write to buffer queue in a thread-safe manner" do
+
+      # Setting thread_safe to false only makes the spec fail with
+      # JRuby. 1.8.x and 1.9.x both pass fine for some reason
+      # regardless of what the thread_safe option is set to.
+      Redistat.thread_safe = true
+
+      key = mock('Key', :to_s => "Scope/labelx:2011")
+      @buffer.size = 100
+
+      Redistat::Summary.should_receive(:update).exactly(2).times.and_return do |k, stats, depth_limit, opts|
+        depth_limit.should == @depth_limit
+        opts.should == @opts
+        if k == @key
+          stats[:count].should == 60
+          stats[:views].should == 180
+        elsif k == key
+          stats[:count].should == 40
+          stats[:views].should == 120
+        end
+      end
+
+      threads = []
+      10.times do
+        threads << Thread.new {
+          6.times { @buffer.store(@key, @stats, @depth_limit, @opts).should be_true }
+          4.times { @buffer.store(key, @stats, @depth_limit, @opts).should be_true }
+        }
+      end
+
+      threads.each { |t| t.join }
+    end
+
+    it "should have specs that fail on 1.8.x/1.9.x when thread_safe is disabled"
+
+  end
+
+end

spec/collection_spec.rb

@@ -1,7 +1,7 @@
 require "spec_helper"
 
 describe Redistat::Collection do
-  
+
   it "should initialize properly" do
     options = {:from => "from", :till => "till", :depth => "depth"}
     result = Redistat::Collection.new(options)
@@ -9,12 +9,12 @@ describe Redistat::Collection do
     result.till.should == options[:till]
     result.depth.should == options[:depth]
   end
-  
+
   it "should have a total property" do
     col = Redistat::Collection.new()
     col.total.should == {}
     col.total = {:foo => "bar"}
     col.total.should == {:foo => "bar"}
   end
-  
+
-end
+end

spec/connection_spec.rb

@@ -2,60 +2,66 @@ require "spec_helper"
 include Redistat
 
 describe Redistat::Connection do
-  
+
+  before(:each) do
+    @redis = Redistat.redis
+  end
+
   it "should have a valid Redis client instance" do
     Redistat.redis.should_not be_nil
   end
 
   it "should have initialized custom testing connection" do
-    redis = Redistat.redis
+    @redis.client.host.should == '127.0.0.1'
-    redis.client.host.should == '127.0.0.1'
+    @redis.client.port.should == 8379
-    redis.client.port.should == 8379
+    @redis.client.db.should == 15
-    redis.client.db.should == 15
   end
-  
+
   it "should be able to set and get data" do
-    redis = Redistat.redis
+    @redis.set("hello", "world")
-    redis.set("hello", "world")
+    @redis.get("hello").should == "world"
-    redis.get("hello").should == "world"
+    @redis.del("hello").should be_true
-    redis.del("hello").should be_true
   end
-  
+
   it "should be able to store hashes to Redis" do
-    redis = Redistat.redis
+    @redis.hset("hash", "field", "1")
-    redis.hset("hash", "field", "1")
+    @redis.hget("hash", "field").should == "1"
-    redis.hget("hash", "field").should == "1"
+    @redis.hincrby("hash", "field", 1)
-    redis.hincrby("hash", "field", 1)
+    @redis.hget("hash", "field").should == "2"
-    redis.hget("hash", "field").should == "2"
+    @redis.hincrby("hash", "field", -1)
-    redis.hincrby("hash", "field", -1)
+    @redis.hget("hash", "field").should == "1"
-    redis.hget("hash", "field").should == "1"
+    @redis.del("hash")
-    redis.del("hash")
   end
-  
+
   it "should be accessible from Redistat module" do
     Redistat.redis.should == Connection.get
     Redistat.redis.should == Redistat.connection
   end
-  
+
   it "should handle multiple connections with refs" do
     Redistat.redis.client.db.should == 15
     Redistat.connect(:port => 8379, :db => 14, :ref => "Custom")
     Redistat.redis.client.db.should == 15
     Redistat.redis("Custom").client.db.should == 14
   end
-  
+
   it "should be able to overwrite default and custom refs" do
     Redistat.redis.client.db.should == 15
     Redistat.connect(:port => 8379, :db => 14)
     Redistat.redis.client.db.should == 14
-    
+
     Redistat.redis("Custom").client.db.should == 14
     Redistat.connect(:port => 8379, :db => 15, :ref => "Custom")
     Redistat.redis("Custom").client.db.should == 15
-    
+
     # Reset the default connection to the testing server or all hell
     # might brake loose from the rest of the specs
     Redistat.connect(:port => 8379, :db => 15)
   end
-  
+
-end
+  # TODO: Test thread-safety
+  it "should be thread-safe" do
+    pending("need to figure out a way to test thread-safety")
+  end
+
+end

spec/core_ext/hash_spec.rb

@@ -0,0 +1,30 @@
+require "spec_helper"
+
+describe Hash do
+
+  it "should #set_or_incr values" do
+    hash = {:count => 1}
+    hash.set_or_incr(:sum, 3).should be_true
+    hash.should == {:count => 1, :sum => 3}
+    hash.set_or_incr(:count, 4).should be_true
+    hash.should == {:count => 5, :sum => 3}
+    hash.set_or_incr(:count, 'test').should be_false
+    hash.set_or_incr(:view, 'test').should be_false
+    hash.should == {:count => 5, :sum => 3}
+    hash[:view] = 'test'
+    hash.set_or_incr(:view, 3).should be_false
+  end
+
+  it "should #merge_and_incr hashes" do
+    hash = { :count => 1, :city => 'hell', :sum => 3, :name => 'john' }
+
+    new_hash = { :count => 3, :city => 'slum', :views => 2 }
+    hash.clone.merge_and_incr(new_hash).should == { :count => 4, :city => 'slum', :views => 2,
+                                                    :sum   => 3, :name => 'john' }
+
+    new_hash = { :count => 'six', :city => 'slum', :views => 2, :time => 'late' }
+    hash.clone.merge_and_incr(new_hash).should == { :count => 'six', :city => 'slum', :views => 2,
+                                                    :sum   => 3, :name => 'john', :time => 'late' }
+  end
+
+end

spec/database_spec.rb

@@ -2,9 +2,9 @@ require "spec_helper"
 
 describe Redistat::Database do
   include Redistat::Database
-  
+
   it "should make #db method available when included" do
     db.should == Redistat.redis
   end
-  
+
-end
+end

spec/date_spec.rb

@@ -1,14 +1,14 @@
 require "spec_helper"
 
 describe Redistat::Date do
-  
+
   it "should initialize from Time object" do
     now = Time.now
     [Redistat::Date.new(now), now.to_rs].each do |rdate|
       Redistat::Date::DEPTHS.each { |k| rdate.send(k).should == now.send(k) }
     end
   end
-  
+
   it "should initialize from Date object" do
     today = Date.today
     [Redistat::Date.new(today), today.to_rs].each do |rdate|
@@ -16,7 +16,7 @@ describe Redistat::Date do
       [:hour, :min, :sec, :usec].each { |k| rdate.send(k).should == 0 }
     end
   end
-  
+
   it "should initialize from Fixnum object (UNIX Timestamp)" do
     now = Time.now.to_i
     time = Time.at(now)
@@ -24,13 +24,13 @@ describe Redistat::Date do
       [:year, :month, :day, :hour, :min, :sec].each { |k| rdate.send(k).should == time.send(k) }
     end
   end
-  
+
   it "should initialize from String object" do
     now = Time.now
     rdate = Redistat::Date.new(now.to_s)
     [:year, :month, :day, :hour, :min, :sec].each { |k| rdate.send(k).should == now.send(k) }
   end
-  
+
   it "should initialize from Redistat date String" do
     now = Time.now
     rdate = Redistat::Date.new(now.to_s)
@@ -38,25 +38,25 @@ describe Redistat::Date do
       rdate.to_s(k).should == Redistat::Date.new(rdate.to_s(k)).to_s(k)
     }
   end
-  
+
   it "should convert to Time object" do
     now = Time.now
     rdate = Redistat::Date.new(now)
     rdate.to_time.to_s.should == now.to_s
   end
-  
+
   it "should convert to Date object" do
     today = Date.today
     rdate = Redistat::Date.new(today)
     rdate.to_date.to_s.should == today.to_s
   end
-  
+
   it "should convert to Fixnum object (UNIX Timestamp)" do
     now = Time.now
     rdate = Redistat::Date.new(now)
     rdate.to_i.should == now.to_i
   end
-  
+
   it "should convert to string with correct depths" do
     today = Date.today
     now = Time.now
@@ -71,25 +71,25 @@ describe Redistat::Date do
       end
     end
   end
-  
+
   it "should add helper methods to Date, Time and Fixnum classes" do
     Date.today.to_time.should == Time.parse(Date.today.to_s)
     Time.now.to_i.to_time.should == Time.at(Time.now.to_i)
     Date.today.to_rs.to_date.should == Date.today
   end
-  
+
   it "should have a depth property" do
     now = Time.now
-    
+
     date = Redistat::Date.new(now)
     date.depth.should be_nil
     date.to_s.should == now.to_rs(:sec).to_s
     date.to_s.should == now.to_rs.to_s(:sec)
-    
+
     date = Redistat::Date.new(now, :hour)
     date.depth.should == :hour
     date.to_s.should == now.to_rs(:hour).to_s
     date.to_s.should == now.to_rs.to_s(:hour)
   end
-  
+
-end
+end

spec/event_spec.rb

@@ -2,23 +2,23 @@ require "spec_helper"
 
 describe Redistat::Event do
   include Redistat::Database
-  
+
   before(:each) do
     db.flushdb
     @scope = "PageViews"
     @label = "about_us"
     @label_hash = Digest::SHA1.hexdigest(@label)
-    @stats = {:views => 1}
+    @stats = {'views' => 1}
-    @meta = {:user_id => 239}
+    @meta = {'user_id' => 239}
     @options = {:depth => :hour}
     @date = Time.now
     @event = Redistat::Event.new(@scope, @label, @date, @stats, @options, @meta)
   end
-  
+
   it "should initialize properly" do
     @event.id.should be_nil
-    @event.scope.should == @scope
+    @event.scope.to_s.should == @scope
-    @event.label.should == @label
+    @event.label.to_s.should == @label
     @event.label_hash.should == @label_hash
     @event.date.to_time.to_s.should == @date.to_s
     @event.stats.should == @stats
@@ -33,15 +33,15 @@ describe Redistat::Event do
     @event.date = @date
     @event.date.to_time.to_s.should == @date.to_s
     # label
-    @event.label.should == @label
+    @event.label.to_s.should == @label
     @event.label_hash.should == @label_hash
     @label = "contact_us"
     @label_hash = Digest::SHA1.hexdigest(@label)
     @event.label = @label
-    @event.label.should == @label
+    @event.label.to_s.should == @label
     @event.label_hash.should == @label_hash
   end
-  
+
   it "should increment next_id" do
     event = Redistat::Event.new("VisitorCount", @label, @date, @stats, @options, @meta)
     @event.next_id.should == 1
@@ -49,7 +49,7 @@ describe Redistat::Event do
     @event.next_id.should == 2
     event.next_id.should == 2
   end
-  
+
   it "should store event properly" do
     @event = Redistat::Event.new(@scope, @label, @date, @stats, @options.merge({:store_event => true}), @meta)
     @event.new?.should be_true
@@ -59,15 +59,17 @@ describe Redistat::Event do
     keys.should include("#{@event.scope}#{Redistat::KEY_EVENT}#{@event.id}")
     keys.should include("#{@event.scope}#{Redistat::KEY_EVENT_IDS}")
   end
-  
+
   it "should find event by id" do
     @event = Redistat::Event.new(@scope, @label, @date, @stats, @options.merge({:store_event => true}), @meta).save
     fetched = Redistat::Event.find(@scope, @event.id)
-    @event.scope.should == fetched.scope
+    @event.scope.to_s.should == fetched.scope.to_s
-    @event.label.should == fetched.label
+    @event.label.to_s.should == fetched.label.to_s
     @event.date.to_s.should == fetched.date.to_s
+    @event.stats.should == fetched.stats
+    @event.meta.should == fetched.meta
   end
-  
+
   it "should store summarized statistics" do
     2.times do |i|
       @event = Redistat::Event.new(@scope, @label, @date, @stats, @options, @meta).save
@@ -79,5 +81,5 @@ describe Redistat::Event do
       end
     end
   end
-  
+
-end
+end

spec/finder/date_set_spec.rb

@@ -1,11 +1,11 @@
 require "spec_helper"
 
 describe Redistat::Finder::DateSet do
-  
+
   before(:all) do
     @finder = Redistat::Finder::DateSet.new
   end
-  
+
   it "should initialize properly" do
     t_start = Time.utc(2010, 8, 28, 22, 54, 57)
     t_end = Time.utc(2013, 12, 4, 22, 52, 3)
@@ -20,506 +20,508 @@ describe Redistat::Finder::DateSet do
       { :add => ["2011", "2012"],                         :rem => []             }
     ]
   end
-  
+
   it "should find date sets by interval" do
     t_start = Time.utc(2010, 8, 28, 18, 54, 57)
-    
+
     t_end = t_start + 4.hours
     result = Redistat::Finder::DateSet.new.find_date_sets(t_start, t_end, :hour, true)
     result[0][:add].should == ["2010082818", "2010082819", "2010082820", "2010082821", "2010082822"]
     result[0][:rem].should == []
-    
+    result.should == Redistat::Finder::DateSet.new(t_start, t_end, nil, :hour)
+
     t_end = t_start + 4.days
     result = Redistat::Finder::DateSet.new.find_date_sets(t_start, t_end, :day, true)
     result[0][:add].should == ["20100828", "20100829", "20100830", "20100831", "20100901"]
     result[0][:rem].should == []
+    result.should == Redistat::Finder::DateSet.new(t_start, t_end, nil, :day)
   end
-  
+
   it "should find start keys properly" do
-    
+
     #
     # Simple fetching
     # Dates: 22:54, 26th August, 2010 --> 22:52, 14th December, 2010
     #
-    
+
     t_start = Time.utc(2010, 8, 26, 22, 54, 57)
     t_end = Time.utc(2013, 12, 14, 22, 52, 3)
-  
+
     result = @finder.send(:find_start_keys_for, :sec, t_start, t_end)
     result[:add].should == ["20100826225458", "20100826225459"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :min, t_start, t_end)
     result[:add].should == ["201008262255", "201008262256", "201008262257", "201008262258", "201008262259"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :hour, t_start, t_end)
     result[:add].should == ["2010082623"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :day, t_start, t_end)
     result[:add].should == ["20100827", "20100828", "20100829", "20100830", "20100831"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :month, t_start, t_end)
     result[:add].should == ["201009", "201010", "201011", "201012"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :year, t_start, t_end)
     result[:add].should == ["2011", "2012"]
     result[:rem].should == []
-    
+
     #
     # Reverse / Inteligent fetching
     # Dates: 5:06, 4th April, 2010 --> 22:52, 14th February, 2011
     #
-    
+
     t_start = Time.utc(2010, 4, 4, 5, 6, 4)
     t_end = Time.utc(2011, 2, 14, 22, 52, 3)
-  
+
     result = @finder.send(:find_start_keys_for, :sec, t_start, t_end)
     result[:add].should == ["201004040506"]
     result[:rem].should == ["20100404050600", "20100404050601", "20100404050602", "20100404050603", "20100404050604"]
-    
+
     result = @finder.send(:find_start_keys_for, :min, t_start, t_end)
     result[:add].should == ["2010040405"]
     result[:rem].should == ["201004040500", "201004040501", "201004040502", "201004040503", "201004040504", "201004040505", "201004040506"]
-    
+
     result = @finder.send(:find_start_keys_for, :hour, t_start, t_end)
     result[:add].should == ["20100404"]
     result[:rem].should == ["2010040400", "2010040401", "2010040402", "2010040403", "2010040404", "2010040405"]
-    
+
     result = @finder.send(:find_start_keys_for, :day, t_start, t_end)
     result[:add].should == ["201004"]
     result[:rem].should == ["20100401", "20100402", "20100403", "20100404"]
-    
+
     result = @finder.send(:find_start_keys_for, :month, t_start, t_end)
     result[:add].should == ["2010"]
     result[:rem].should == ["201001", "201002", "201003", "201004"]
-    
+
     result = @finder.send(:find_start_keys_for, :year, t_start, t_end)
     result[:add].should == []
     result[:rem].should == []
-    
+
   end
-  
+
   it "should find end keys properly" do
-    
+
     #
     # Simple fetching
     # Dates: 22:04, 26th December, 2007 --> 5:06, 7th May, 2010
     #
-    
+
     t_start = Time.utc(2007, 12, 26, 22, 4, 4)
     t_end = Time.utc(2010, 5, 7, 5, 6, 3)
-  
+
     result = @finder.send(:find_end_keys_for, :sec, t_start, t_end)
     result[:add].should == ["20100507050600", "20100507050601", "20100507050602"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :min, t_start, t_end)
     result[:add].should == ["201005070500", "201005070501", "201005070502", "201005070503", "201005070504", "201005070505"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :hour, t_start, t_end)
     result[:add].should == ["2010050700", "2010050701", "2010050702", "2010050703", "2010050704"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :day, t_start, t_end)
     result[:add].should == ["20100501", "20100502", "20100503", "20100504", "20100505", "20100506"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :month, t_start, t_end)
     result[:add].should == ["201001", "201002", "201003", "201004"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :year, t_start, t_end)
     result[:add].should == []
     result[:rem].should == []
-    
+
     #
     # Reverse / Inteligent fetching
     # Dates: 22:04, 26th December, 2009 --> 22:56, 27th October, 2010
     #
-    
+
     t_start = Time.utc(2009, 12, 26, 22, 4, 4)
     t_end = Time.utc(2010, 10, 27, 22, 56, 57)
-  
+
     result = @finder.send(:find_end_keys_for, :sec, t_start, t_end)
     result[:add].should == ["201010272256"]
     result[:rem].should == ["20101027225657", "20101027225658", "20101027225659"]
-    
+
     result = @finder.send(:find_end_keys_for, :min, t_start, t_end)
     result[:add].should == ["2010102722"]
     result[:rem].should == ["201010272256", "201010272257", "201010272258", "201010272259"]
-    
+
     result = @finder.send(:find_end_keys_for, :hour, t_start, t_end)
     result[:add].should == ["20101027"]
     result[:rem].should == ["2010102722", "2010102723"]
-    
+
     result = @finder.send(:find_end_keys_for, :day, t_start, t_end)
     result[:add].should == ["201010"]
     result[:rem].should == ["20101027", "20101028", "20101029", "20101030", "20101031"]
-    
+
     result = @finder.send(:find_end_keys_for, :month, t_start, t_end)
     result[:add].should == ["2010"]
     result[:rem].should == ["201010", "201011", "201012"]
-    
+
     result = @finder.send(:find_end_keys_for, :year, t_start, t_end)
     result[:add].should == []
     result[:rem].should == []
-    
+
   end
-  
+
   it "should fetch start/end keys with limits" do
-    
+
     #
     # Simple fetching with Limits
     #
-    
+
     # seconds
     t_start = Time.utc(2010, 8, 26, 20, 54, 45)
     t_end = t_start + 4.seconds
-    
+
     result = @finder.send(:find_start_keys_for, :sec, t_start, t_end)
     result[:add].should == ["20100826205446", "20100826205447", "20100826205448"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :sec, t_start, t_end)
     result[:add].should == []
     result[:rem].should == []
-    
+
     t_start = Time.utc(2010, 8, 26, 20, 54, 4)
     t_end = t_start + 4.seconds
-    
+
     result = @finder.send(:find_start_keys_for, :sec, t_start, t_end)
     result[:add].should == ["20100826205405", "20100826205406", "20100826205407"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :sec, t_start, t_end)
     result[:add].should == []
     result[:rem].should == []
-    
+
     # minutes
     t_start = Time.utc(2010, 8, 26, 20, 54)
     t_end = t_start + 4.minutes
-    
+
     result = @finder.send(:find_start_keys_for, :min, t_start, t_end)
     result[:add].should == ["201008262055", "201008262056", "201008262057"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :min, t_start, t_end)
     result[:add].should == []
     result[:rem].should == []
-    
+
     t_start = Time.utc(2010, 8, 26, 20, 4)
     t_end = t_start + 4.minutes
-    
+
     result = @finder.send(:find_start_keys_for, :min, t_start, t_end)
     result[:add].should == ["201008262005", "201008262006", "201008262007"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :min, t_start, t_end)
     result[:add].should == []
     result[:rem].should == []
-    
+
     # hours
     t_start = Time.utc(2010, 8, 26, 20, 54)
     t_end = t_start + 2.hours
-  
+
     result = @finder.send(:find_start_keys_for, :min, t_start, t_end)
     result[:add].should == ["201008262055", "201008262056", "201008262057", "201008262058", "201008262059"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :hour, t_start, t_end)
     result[:add].should == ["2010082621"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :hour, t_start, t_end)
     result[:add].should == []
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :min, t_start, t_end)
     result[:add].should == ["2010082622"]
     result[:rem].should == ["201008262254", "201008262255", "201008262256", "201008262257", "201008262258", "201008262259"]
-    
+
     t_start = Time.utc(2010, 8, 26, 4, 54)
     t_end = t_start + 5.hours
-    
+
     result = @finder.send(:find_start_keys_for, :min, t_start, t_end)
     result[:add].should == ["201008260455", "201008260456", "201008260457", "201008260458", "201008260459"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :hour, t_start, t_end)
     result[:add].should == ["2010082605", "2010082606", "2010082607", "2010082608"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :hour, t_start, t_end)
     result[:add].should == []
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :min, t_start, t_end)
     result[:add].should == ["2010082609"]
     result[:rem].should == ["201008260954", "201008260955", "201008260956", "201008260957", "201008260958", "201008260959"]
-    
+
     # days
     t_start = Time.utc(2010, 8, 26, 20, 54)
     t_end = t_start + 2.day
-    
+
     result = @finder.send(:find_start_keys_for, :min, t_start, t_end)
     result[:add].should == ["201008262055", "201008262056", "201008262057", "201008262058", "201008262059"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :hour, t_start, t_end)
     result[:add].should == ["2010082621", "2010082622", "2010082623"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :day, t_start, t_end)
     result[:add].should == ["20100827"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :day, t_start, t_end)
     result[:add].should == []
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :hour, t_start, t_end)
     result[:add].should == ["20100828"]
     result[:rem].should == ["2010082820", "2010082821", "2010082822", "2010082823"]
-    
+
     result = @finder.send(:find_end_keys_for, :min, t_start, t_end)
     result[:add].should == ["2010082820"]
     result[:rem].should == ["201008282054", "201008282055", "201008282056", "201008282057", "201008282058", "201008282059"]
-    
+
     t_start = Time.utc(2010, 8, 6, 20, 54)
     t_end = t_start + 2.day
-    
+
     result = @finder.send(:find_start_keys_for, :min, t_start, t_end)
     result[:add].should == ["201008062055", "201008062056", "201008062057", "201008062058", "201008062059"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :hour, t_start, t_end)
     result[:add].should == ["2010080621", "2010080622", "2010080623"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :day, t_start, t_end)
     result[:add].should == ["20100807"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :day, t_start, t_end)
     result[:add].should == []
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :hour, t_start, t_end)
     result[:add].should == ["20100808"]
     result[:rem].should == ["2010080820", "2010080821", "2010080822", "2010080823"]
-    
+
     result = @finder.send(:find_end_keys_for, :min, t_start, t_end)
     result[:add].should == ["2010080820"]
     result[:rem].should == ["201008082054", "201008082055", "201008082056", "201008082057", "201008082058", "201008082059"]
-    
+
     # months
     t_start = Time.utc(2010, 8, 26, 20, 54)
     t_end = t_start + 3.months
-    
+
     result = @finder.send(:find_start_keys_for, :min, t_start, t_end)
     result[:add].should == ["201008262055", "201008262056", "201008262057", "201008262058", "201008262059"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :hour, t_start, t_end)
     result[:add].should == ["2010082621", "2010082622", "2010082623"]
     result[:rem].should == []
-        
+
     result = @finder.send(:find_start_keys_for, :day, t_start, t_end)
     result[:add].should == ["20100827", "20100828", "20100829", "20100830", "20100831"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :month, t_start, t_end)
     result[:add].should == ["201009", "201010"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :month, t_start, t_end)
     result[:add].should == []
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :day, t_start, t_end)
     result[:add].should == ["201011"]
     result[:rem].should == ["20101126", "20101127", "20101128", "20101129", "20101130"]
-    
+
     result = @finder.send(:find_end_keys_for, :hour, t_start, t_end)
     result[:add].should == ["20101126"]
     result[:rem].should == ["2010112620", "2010112621", "2010112622", "2010112623"]
-    
+
     result = @finder.send(:find_end_keys_for, :min, t_start, t_end)
     result[:add].should == ["2010112620"]
     result[:rem].should == ["201011262054", "201011262055", "201011262056", "201011262057", "201011262058", "201011262059"]
-    
+
     t_start = Time.utc(2010, 4, 26, 20, 54)
     t_end = t_start + 3.months
-    
+
     result = @finder.send(:find_start_keys_for, :min, t_start, t_end)
     result[:add].should == ["201004262055", "201004262056", "201004262057", "201004262058", "201004262059"]
     result[:rem].should == []
-        
+
     result = @finder.send(:find_start_keys_for, :hour, t_start, t_end)
     result[:add].should == ["2010042621", "2010042622", "2010042623"]
     result[:rem].should == []
-        
+
     result = @finder.send(:find_start_keys_for, :day, t_start, t_end)
     result[:add].should == ["20100427", "20100428", "20100429", "20100430"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :month, t_start, t_end)
     result[:add].should == ["201005", "201006"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :month, t_start, t_end)
     result[:add].should == []
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :day, t_start, t_end)
     result[:add].should == ["201007"]
     result[:rem].should == ["20100726", "20100727", "20100728", "20100729", "20100730", "20100731"]
-    
+
     result = @finder.send(:find_end_keys_for, :hour, t_start, t_end)
     result[:add].should == ["20100726"]
     result[:rem].should == ["2010072620", "2010072621", "2010072622", "2010072623"]
-    
+
     result = @finder.send(:find_end_keys_for, :min, t_start, t_end)
     result[:add].should == ["2010072620"]
     result[:rem].should == ["201007262054", "201007262055", "201007262056", "201007262057", "201007262058", "201007262059"]
-    
+
   end
-  
+
   it "should find inclusive keys on lowest depth" do
-    
+
     #
     # Simple start fetching
     # Dates: 22:54, 26th August, 2010 --> 22:52, 14th December, 2010
     #
-    
+
     t_start = Time.utc(2010, 8, 26, 22, 54, 57)
     t_end = Time.utc(2013, 12, 14, 22, 52, 3)
-  
+
     result = @finder.send(:find_start_keys_for, :sec, t_start, t_end, true)
     result[:add].should == ["20100826225457", "20100826225458", "20100826225459"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :min, t_start, t_end, true)
     result[:add].should == ["201008262254", "201008262255", "201008262256", "201008262257", "201008262258", "201008262259"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :hour, t_start, t_end, true)
     result[:add].should == ["2010082622", "2010082623"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :day, t_start, t_end, true)
     result[:add].should == ["20100826", "20100827", "20100828", "20100829", "20100830", "20100831"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :month, t_start, t_end, true)
     result[:add].should == ["201008", "201009", "201010", "201011", "201012"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_start_keys_for, :year, t_start, t_end, true)
     result[:add].should == ["2011", "2012", "2013"]
     result[:rem].should == []
-    
+
     #
     # Reverse / Inteligent start fetching
     # Dates: 5:06, 4th April, 2010 --> 22:52, 14th February, 2011
     #
-    
+
     t_start = Time.utc(2010, 4, 4, 5, 6, 4)
     t_end = Time.utc(2013, 2, 14, 22, 52, 3)
-  
+
     result = @finder.send(:find_start_keys_for, :sec, t_start, t_end, true)
     result[:add].should == ["201004040506"]
     result[:rem].should == ["20100404050600", "20100404050601", "20100404050602", "20100404050603"]
-    
+
     result = @finder.send(:find_start_keys_for, :min, t_start, t_end, true)
     result[:add].should == ["2010040405"]
     result[:rem].should == ["201004040500", "201004040501", "201004040502", "201004040503", "201004040504", "201004040505"]
-    
+
     result = @finder.send(:find_start_keys_for, :hour, t_start, t_end, true)
     result[:add].should == ["20100404"]
     result[:rem].should == ["2010040400", "2010040401", "2010040402", "2010040403", "2010040404"]
-    
+
     result = @finder.send(:find_start_keys_for, :day, t_start, t_end, true)
     result[:add].should == ["201004"]
     result[:rem].should == ["20100401", "20100402", "20100403"]
-    
+
     result = @finder.send(:find_start_keys_for, :month, t_start, t_end, true)
     result[:add].should == ["2010"]
     result[:rem].should == ["201001", "201002", "201003"]
-    
+
     result = @finder.send(:find_start_keys_for, :year, t_start, t_end, true)
     result[:add].should == ["2011", "2012", "2013"]
     result[:rem].should == []
-    
+
     #
     # Simple fetching
     # Dates: 22:04, 26th December, 2007 --> 5:06, 7th May, 2010
     #
-    
+
     t_start = Time.utc(2007, 12, 26, 22, 4, 4)
     t_end = Time.utc(2010, 5, 7, 5, 6, 3)
-  
+
     result = @finder.send(:find_end_keys_for, :sec, t_start, t_end, true)
     result[:add].should == ["20100507050600", "20100507050601", "20100507050602", "20100507050603"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :min, t_start, t_end, true)
     result[:add].should == ["201005070500", "201005070501", "201005070502", "201005070503", "201005070504", "201005070505", "201005070506"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :hour, t_start, t_end, true)
     result[:add].should == ["2010050700", "2010050701", "2010050702", "2010050703", "2010050704", "2010050705"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :day, t_start, t_end, true)
     result[:add].should == ["20100501", "20100502", "20100503", "20100504", "20100505", "20100506", "20100507"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :month, t_start, t_end, true)
     result[:add].should == ["201001", "201002", "201003", "201004", "201005"]
     result[:rem].should == []
-    
+
     result = @finder.send(:find_end_keys_for, :year, t_start, t_end, true)
     result[:add].should == ["2010"]
     result[:rem].should == []
-    
+
     #
     # Reverse / Inteligent fetching
     # Dates: 22:04, 26th December, 2009 --> 22:56, 27th October, 2010
     #
-    
+
     t_start = Time.utc(2009, 12, 26, 22, 4, 4)
     t_end = Time.utc(2010, 10, 27, 22, 56, 57)
-  
+
     result = @finder.send(:find_end_keys_for, :sec, t_start, t_end, true)
     result[:add].should == ["201010272256"]
     result[:rem].should == ["20101027225658", "20101027225659"]
-    
+
     result = @finder.send(:find_end_keys_for, :min, t_start, t_end, true)
     result[:add].should == ["2010102722"]
     result[:rem].should == ["201010272257", "201010272258", "201010272259"]
-    
+
     result = @finder.send(:find_end_keys_for, :hour, t_start, t_end, true)
     result[:add].should == ["20101027"]
     result[:rem].should == ["2010102723"]
-    
+
     result = @finder.send(:find_end_keys_for, :day, t_start, t_end, true)
     result[:add].should == ["201010"]
     result[:rem].should == ["20101028", "20101029", "20101030", "20101031"]
-    
+
     result = @finder.send(:find_end_keys_for, :month, t_start, t_end, true)
     result[:add].should == ["2010"]
     result[:rem].should == ["201011", "201012"]
-    
+
     result = @finder.send(:find_end_keys_for, :year, t_start, t_end, true)
     result[:add].should == ["2010"]
     result[:rem].should == []
-    
+
   end
-  
+
-end
+end

spec/finder_spec.rb

@@ -2,7 +2,7 @@ require "spec_helper"
 
 describe Redistat::Finder do
   include Redistat::Database
-  
+
   before(:each) do
     db.flushdb
     @scope = "PageViews"
@@ -10,56 +10,70 @@ describe Redistat::Finder do
     @date = Time.now
     @key = Redistat::Key.new(@scope, @label, @date, {:depth => :day})
     @stats = {"views" => 3, "visitors" => 2}
+    @two_hours_ago = 2.hours.ago
+    @one_hour_ago = 1.hour.ago
   end
-  
-  it "should initialize properly" do
-    two_hours_ago = 2.hours.ago
-    one_hour_ago = 1.hour.ago
-    options = {:scope => "PageViews", :label => "Label", :from => two_hours_ago, :till => one_hour_ago, :depth => :hour, :interval => :hour}
-    
-    finder = Redistat::Finder.new(options)
-    finder.options.should == options
 
-    finder = Redistat::Finder.dates(two_hours_ago, one_hour_ago).scope("PageViews").label("Label").depth(:hour).interval(:hour)
+  it "should initialize properly" do
-    finder.options.should == options
+    options = {:scope => "PageViews", :label => "Label", :from => @two_hours_ago, :till => @one_hour_ago, :depth => :hour, :interval => :hour}
-                                    
+
-    finder = Redistat::Finder.scope("PageViews").label("Label").from(two_hours_ago).till(one_hour_ago).depth(:hour).interval(:hour)
+    finder = Redistat::Finder.new
-    finder.options.should == options
+    finder.send(:set_options, options)
-    
+    finder.options[:scope].should be_a(Redistat::Scope)
-    finder = Redistat::Finder.label("Label").from(two_hours_ago).till(one_hour_ago).depth(:hour).interval(:hour).scope("PageViews")
+    finder.options[:scope].to_s.should == options[:scope]
-    finder.options.should == options
+    finder.options[:label].should be_a(Redistat::Label)
-    
+    finder.options[:label].to_s.should == options[:label]
-    finder = Redistat::Finder.from(two_hours_ago).till(one_hour_ago).depth(:hour).interval(:hour).scope("PageViews").label("Label")
+    finder.options.should == options.merge(:scope => finder.options[:scope], :label => finder.options[:label])
-    finder.options.should == options
+
-    
+    finder = Redistat::Finder.scope("hello")
-    finder = Redistat::Finder.till(one_hour_ago).depth(:hour).interval(:hour).scope("PageViews").label("Label").from(two_hours_ago)
+    finder.options[:scope].to_s.should == "hello"
-    finder.options.should == options
+    finder.scope.to_s.should == "hello"
-    
+
-    finder = Redistat::Finder.depth(:hour).interval(:hour).scope("PageViews").label("Label").from(two_hours_ago).till(one_hour_ago)
+    finder = Redistat::Finder.label("hello")
-    finder.options.should == options
+    finder.options[:label].to_s.should == "hello"
-    
+    finder.label.to_s.should == "hello"
-    finder = Redistat::Finder.interval(:hour).scope("PageViews").label("Label").from(two_hours_ago).till(one_hour_ago).depth(:hour)
+
-    finder.options.should == options
+    finder = Redistat::Finder.dates(@two_hours_ago, @one_hour_ago)
-    
+    finder.options[:from].should == @two_hours_ago
+    finder.options[:till].should == @one_hour_ago
+
+    finder = Redistat::Finder.from(@two_hours_ago)
+    finder.options[:from].should == @two_hours_ago
+    finder.from.should == @two_hours_ago
+
+    finder = Redistat::Finder.till(@one_hour_ago)
+    finder.options[:till].should == @one_hour_ago
+    finder.till.should == @one_hour_ago
+
+    finder = Redistat::Finder.depth(:hour)
+    finder.options[:depth].should == :hour
+    finder.depth.should == :hour
+
+    finder = Redistat::Finder.interval(true)
+    finder.options[:interval].should be_true
+    finder.interval.should be_true
+    finder = Redistat::Finder.interval(false)
+    finder.options[:interval].should be_false
+    finder.interval.should be_false
   end
-  
+
   it "should fetch stats properly" do
     first_stat, last_stat = create_example_stats
-    
+
     stats = Redistat::Finder.find({:from => first_stat, :till => last_stat, :scope => @scope, :label => @label, :depth => :hour})
     stats.from.should  == first_stat
     stats.till.should  == last_stat
     stats.depth.should == :hour
-    
+
     stats.total.should == { "views" => 12, "visitors" => 8 }
     stats.total.from.should == first_stat
     stats.total.till.should == last_stat
     stats.first.should == stats.total
   end
-  
+
   it "should fetch data per unit when interval option is specified" do
     first_stat, last_stat = create_example_stats
-    
+
     stats = Redistat::Finder.find(:from => first_stat, :till => last_stat, :scope => @scope, :label => @label, :depth => :hour, :interval => :hour)
     stats.from.should == first_stat
     stats.till.should == last_stat
@@ -75,29 +89,119 @@ describe Redistat::Finder do
     stats[4].should == {}
     stats[4].date.should == Time.parse("2010-05-14 16:00")
   end
-  
+
   it "should return empty hash when attempting to fetch non-existent results" do
     stats = Redistat::Finder.find({:from => 3.hours.ago, :till => 2.hours.from_now, :scope => @scope, :label => @label, :depth => :hour})
     stats.total.should == {}
   end
-  
+
   it "should throw error on invalid options" do
     lambda { Redistat::Finder.find(:from => 3.hours.ago) }.should raise_error(Redistat::InvalidOptions)
   end
-  
+
-  
+  describe "Grouping" do
+    before(:each) do
+      @options = {:scope => "PageViews", :label => "message/public", :from => @two_hours_ago, :till => @one_hour_ago, :depth => :hour, :interval => :hour}
+      @finder = Redistat::Finder.new(@options)
+    end
+
+    it "should return parent finder" do
+      @finder.instance_variable_get("@parent").should be_nil
+      @finder.parent.should be_a(Redistat::Finder)
+      @finder.instance_variable_get("@parent").should_not be_nil
+      @finder.parent.options[:label].to_s.should == 'message'
+      @finder.label('message')
+      @finder.instance_variable_get("@parent").should be_nil
+      @finder.parent.should_not be_nil
+      @finder.parent.options[:label].should be_nil
+      @finder.parent.parent.should be_nil
+    end
+
+    it "should find children" do
+      Redistat::Key.new("PageViews", "message/public/die").update_index
+      Redistat::Key.new("PageViews", "message/public/live").update_index
+      Redistat::Key.new("PageViews", "message/public/fester").update_index
+      members = db.smembers("#{@scope}#{Redistat::LABEL_INDEX}message/public") # checking 'message/public'
+      @finder.children.first.should be_a(Redistat::Finder)
+      subs = @finder.children.map { |f| f.options[:label].me }
+      subs.should have(3).items
+      subs.should include('die')
+      subs.should include('live')
+      subs.should include('fester')
+    end
+  end
+
+  describe "Lazy-Loading" do
+
+    before(:each) do
+      @first_stat, @last_stat = create_example_stats
+
+      @finder = Redistat::Finder.new
+      @finder.from(@first_stat).till(@last_stat).scope(@scope).label(@label).depth(:hour)
+
+      @match = [{}, {"visitors"=>"4", "views"=>"6"},
+                    {"visitors"=>"2", "views"=>"3"},
+                    {"visitors"=>"2", "views"=>"3"}, {}]
+    end
+
+    it "should lazy-load" do
+      @finder.instance_variable_get("@result").should be_nil
+      stats = @finder.all
+      @finder.instance_variable_get("@result").should_not be_nil
+
+      stats.should == @finder.find # find method directly fetches results
+      stats.total.should == @finder.total
+      stats.total.should == { "views" => 12, "visitors" => 8 }
+      stats.total.from.should == @first_stat
+      stats.total.till.should == @last_stat
+      stats.first.should == stats.total
+
+      @finder.all.object_id.should == stats.object_id
+      @finder.from(@first_stat + 2.hours)
+      @finder.instance_variable_get("@result").should be_nil
+      @finder.all.object_id.should_not == stats.object_id
+      stats = @finder.all
+      stats.total.should == { "views" => 6, "visitors" => 4 }
+    end
+
+    it "should handle #map" do
+      @finder.interval(:hour)
+      @finder.map { |r| r }.should == @match
+    end
+
+    it "should handle #each" do
+      @finder.interval(:hour)
+
+      res = []
+      @finder.each { |r| res << r }
+      res.should == @match
+    end
+
+    it "should handle #each_with_index" do
+      @finder.interval(:hour)
+
+      res = {}
+      match = {}
+      @finder.each_with_index { |r, i| res[i] = r }
+      @match.each_with_index { |r, i| match[i] = r }
+      res.should == match
+    end
+
+  end # "Lazy-Loading"
+
+
   # helper methods
-  
+
   def create_example_stats
     key = Redistat::Key.new(@scope, @label, (first = Time.parse("2010-05-14 13:43")))
-    Redistat::Summary.update(key, @stats, :hour)
+    Redistat::Summary.send(:update_fields, key, @stats, :hour)
     key = Redistat::Key.new(@scope, @label, Time.parse("2010-05-14 13:53"))
-    Redistat::Summary.update(key, @stats, :hour)
+    Redistat::Summary.send(:update_fields, key, @stats, :hour)
-    key = Redistat::Key.new(@scope, @label, Time.parse("2010-05-14 14:32"))
+    key = Redistat::Key.new(@scope, @label, Time.parse("2010-05-14 14:52"))
-    Redistat::Summary.update(key, @stats, :hour)
+    Redistat::Summary.send(:update_fields, key, @stats, :hour)
     key = Redistat::Key.new(@scope, @label, (last = Time.parse("2010-05-14 15:02")))
-    Redistat::Summary.update(key, @stats, :hour)
+    Redistat::Summary.send(:update_fields, key, @stats, :hour)
     [first - 1.hour, last + 1.hour]
   end
-  
+
 end

spec/key_spec.rb

@@ -1,24 +1,26 @@
 require "spec_helper"
 
 describe Redistat::Key do
-  
+  include Redistat::Database
+
   before(:each) do
+    db.flushdb
     @scope = "PageViews"
     @label = "about_us"
     @label_hash = Digest::SHA1.hexdigest(@label)
     @date = Time.now
     @key = Redistat::Key.new(@scope, @label, @date, {:depth => :hour})
   end
-  
+
   it "should initialize properly" do
-    @key.scope.should == @scope
+    @key.scope.to_s.should == @scope
-    @key.label.should == @label
+    @key.label.to_s.should == @label
     @key.label_hash.should == @label_hash
-    @key.label_groups.should == @key.instance_variable_get("@label").groups
+    @key.groups.map { |k| k.instance_variable_get("@label") }.should == @key.instance_variable_get("@label").groups
     @key.date.should be_instance_of(Redistat::Date)
     @key.date.to_time.to_s.should == @date.to_s
   end
-  
+
   it "should convert to string properly" do
     @key.to_s.should == "#{@scope}/#{@label}:#{@key.date.to_s(:hour)}"
     props = [:year, :month, :day, :hour, :min, :sec]
@@ -26,51 +28,102 @@ describe Redistat::Key do
       @key.to_s(props.last).should == "#{@scope}/#{@label}:#{@key.date.to_s(props.last)}"
       props.pop
     end
+    key = Redistat::Key.new(@scope, nil, @date, {:depth => :hour})
+    key.to_s.should == "#{@scope}:#{key.date.to_s(:hour)}"
   end
-  
+
   it "should abide to hashed_label option" do
     @key = Redistat::Key.new(@scope, @label, @date, {:depth => :hour, :hashed_label => true})
     @key.to_s.should == "#{@scope}/#{@label_hash}:#{@key.date.to_s(:hour)}"
     @key = Redistat::Key.new(@scope, @label, @date, {:depth => :hour, :hashed_label => false})
     @key.to_s.should == "#{@scope}/#{@label}:#{@key.date.to_s(:hour)}"
   end
-  
+
   it "should have default depth option" do
     @key = Redistat::Key.new(@scope, @label, @date)
     @key.depth.should == :hour
   end
-  
+
   it "should allow changing attributes" do
     # scope
-    @key.scope.should == @scope
+    @key.scope.to_s.should == @scope
     @scope = "VisitorCount"
     @key.scope = @scope
-    @key.scope.should == @scope
+    @key.scope.to_s.should == @scope
     # date
     @key.date.to_time.to_s.should == @date.to_s
     @date = Time.now
     @key.date = @date
     @key.date.to_time.to_s.should == @date.to_s
     # label
-    @key.label.should == @label
+    @key.label.to_s.should == @label
     @key.label_hash == @label_hash
     @label = "contact_us"
     @label_hash = Digest::SHA1.hexdigest(@label)
     @key.label = @label
-    @key.label.should == @label
+    @key.label.to_s.should == @label
     @key.label_hash == @label_hash
   end
-  
+
-  it "should create a group of keys from label group" do
+  describe "Grouping" do
-    label = 'message/public/offensive'
+    before(:each) do
-    result = [ "message/public/offensive",
+      @label = "message/public/offensive"
-               "message/public",
+      @key = Redistat::Key.new(@scope, @label, @date, {:depth => :hour})
-               "message" ]
+    end
-    
+
-    key = Redistat::Key.new(@scope, label, @date, {:depth => :hour})
+    it "should create a group of keys from label group" do
-    
+      label = 'message/public/offensive'
-    key.label_groups.should == result
+      result = [ "message/public/offensive",
-    key.groups.map { |k| k.label }.should == result
+                 "message/public",
+                 "message" ]
+
+      key = Redistat::Key.new(@scope, label, @date, {:depth => :hour})
+
+      key.groups.map { |k| k.label.to_s }.should == result
+    end
+
+    it "should know it's parent" do
+      @key.parent.should be_a(Redistat::Key)
+      @key.parent.label.to_s.should == 'message/public'
+      Redistat::Key.new(@scope, 'hello', @date).parent.should be_nil
+    end
+
+    it "should update label index and return children" do
+      db.smembers("#{@scope}#{Redistat::LABEL_INDEX}#{@key.label.parent}").should == []
+      @key.children.should have(0).items
+
+      @key.update_index                                                  # indexing 'message/publish/offensive'
+      Redistat::Key.new("PageViews", "message/public/die").update_index  # indexing 'message/publish/die'
+      Redistat::Key.new("PageViews", "message/public/live").update_index # indexing 'message/publish/live'
+
+      members = db.smembers("#{@scope}#{Redistat::LABEL_INDEX}#{@key.label.parent}") # checking 'message/public'
+      members.should have(3).item
+      members.should include('offensive')
+      members.should include('live')
+      members.should include('die')
+
+      key = @key.parent
+      key.children.first.should be_a(Redistat::Key)
+      key.children.should have(3).item
+      key.children.map { |k| k.label.me }.should == members
+
+      members = db.smembers("#{@scope}#{Redistat::LABEL_INDEX}#{key.label.parent}") # checking 'message'
+      members.should have(1).item
+      members.should include('public')
+
+      key = key.parent
+      key.children.should have(1).item
+      key.children.map { |k| k.label.me }.should == members
+
+      members = db.smembers("#{@scope}#{Redistat::LABEL_INDEX}") # checking ''
+      members.should have(1).item
+      members.should include('message')
+
+      key.parent.should be_nil
+      key = Redistat::Key.new("PageViews")
+      key.children.should have(1).item
+      key.children.map { |k| k.label.me }.should include('message')
+    end
   end
-  
+
-end
+end

spec/label_spec.rb

@@ -2,47 +2,85 @@ require "spec_helper"
 
 describe Redistat::Label do
   include Redistat::Database
-  
+
   before(:each) do
     db.flushdb
     @name = "about_us"
     @label = Redistat::Label.new(@name)
   end
-  
+
   it "should initialize properly and SHA1 hash the label name" do
     @label.name.should == @name
     @label.hash.should == Digest::SHA1.hexdigest(@name)
   end
-  
+
   it "should store a label hash lookup key" do
     label = Redistat::Label.new(@name, {:hashed_label => true}).save
     label.saved?.should be_true
-    db.get("#{Redistat::KEY_LEBELS}#{label.hash}").should == @name
+    db.hget(Redistat::KEY_LABELS, label.hash).should == @name
-    
+
     name = "contact_us"
     label = Redistat::Label.create(name, {:hashed_label => true})
     label.saved?.should be_true
-    db.get("#{Redistat::KEY_LEBELS}#{label.hash}").should == name
+    db.hget(Redistat::KEY_LABELS, label.hash).should == name
   end
-  
-  it "should separate label names into groups" do
-    name = "message/public/offensive"
-    label = Redistat::Label.new(name)
-    label.name.should == name
-    label.groups.should == [ "message/public/offensive",
-                             "message/public",
-                             "message" ]
 
-    name = "/message/public/"
+  it "should join labels" do
-    label = Redistat::Label.new(name)
+    include Redistat
-    label.name.should == name
+    label = Label.join('email', 'message', 'public')
-    label.groups.should == [ "message/public",
+    label.should be_a(Label)
-                             "message" ]
+    label.to_s.should == 'email/message/public'
-
+    label = Label.join(Label.new('email'), Label.new('message'), Label.new('public'))
-    name = "message"
+    label.should be_a(Label)
-    label = Redistat::Label.new(name)
+    label.to_s.should == 'email/message/public'
-    label.name.should == name
+    label = Label.join('email', '', 'message', nil, 'public')
-    label.groups.should == [ "message" ]
+    label.should be_a(Label)
+    label.to_s.should == 'email/message/public'
   end
-  
+
-end
+  it "should allow you to use a different group separator" do
+    include Redistat
+    Redistat.group_separator = '|'
+    label = Label.join('email', 'message', 'public')
+    label.should be_a(Label)
+    label.to_s.should == 'email|message|public'
+    label = Label.join(Label.new('email'), Label.new('message'), Label.new('public'))
+    label.should be_a(Label)
+    label.to_s.should == 'email|message|public'
+    label = Label.join('email', '', 'message', nil, 'public')
+    label.should be_a(Label)
+    label.to_s.should == 'email|message|public'
+    Redistat.group_separator = Redistat::GROUP_SEPARATOR
+  end
+
+  describe "Grouping" do
+    before(:each) do
+      @name = "message/public/offensive"
+      @label = Redistat::Label.new(@name)
+    end
+
+    it "should know it's parent label group" do
+      @label.parent.to_s.should == 'message/public'
+      Redistat::Label.new('hello').parent.should be_nil
+    end
+
+    it "should separate label names into groups" do
+      @label.name.should == @name
+      @label.groups.map { |l| l.to_s }.should == [ "message/public/offensive",
+                                                   "message/public",
+                                                   "message" ]
+
+      @name = "/message/public/"
+      @label = Redistat::Label.new(@name)
+      @label.name.should == @name
+      @label.groups.map { |l| l.to_s }.should == [ "message/public",
+                                                   "message" ]
+
+      @name = "message"
+      @label = Redistat::Label.new(@name)
+      @label.name.should == @name
+      @label.groups.map { |l| l.to_s }.should == [ "message" ]
+    end
+  end
+
+end

spec/model_helper.rb

@@ -2,29 +2,30 @@ require "redistat"
 
 class ModelHelper1
   include Redistat::Model
-  
+
-  
+
 end
 
 class ModelHelper2
   include Redistat::Model
-  
+
   depth :day
   store_event true
   hashed_label true
-  
+
 end
 
 class ModelHelper3
   include Redistat::Model
-  
+
   connect_to :port => 8379, :db => 14
-  
+
 end
 
 class ModelHelper4
   include Redistat::Model
-  
+
-  class_name "FancyHelper"
+  scope "FancyHelper"
-  
+  expire :hour => 24*3600
-end
+
+end

spec/model_spec.rb

@@ -3,25 +3,43 @@ require "model_helper"
 
 describe Redistat::Model do
   include Redistat::Database
-  
+
   before(:each) do
+    @time = Time.utc(2010, 8, 28, 12, 0, 0)
     ModelHelper1.redis.flushdb
     ModelHelper2.redis.flushdb
     ModelHelper3.redis.flushdb
     ModelHelper4.redis.flushdb
   end
-  
+
   it "should should name itself correctly" do
     ModelHelper1.send(:name).should == "ModelHelper1"
     ModelHelper2.send(:name).should == "ModelHelper2"
   end
-  
+
+  it "should return a Finder" do
+    two_hours_ago = 2.hours.ago
+    one_hour_ago  = 1.hour.ago
+    finder = ModelHelper1.find('label', two_hours_ago, one_hour_ago)
+    finder.should be_a(Redistat::Finder)
+    finder.options[:scope].to_s.should == 'ModelHelper1'
+    finder.options[:label].to_s.should == 'label'
+    finder.options[:from].should  == two_hours_ago
+    finder.options[:till].should  == one_hour_ago
+  end
+
+  it "should #find_event" do
+    Redistat::Event.should_receive(:find).with('ModelHelper1', 1)
+    ModelHelper1.find_event(1)
+  end
+
   it "should listen to model-defined options" do
     ModelHelper2.depth.should == :day
     ModelHelper2.store_event.should == true
     ModelHelper2.hashed_label.should == true
-    ModelHelper2.class_name.should be_nil
+    ModelHelper2.scope.should be_nil
-    
+    ModelHelper2.expire.should be_nil
+
     ModelHelper1.depth.should == nil
     ModelHelper1.store_event.should == nil
     ModelHelper1.hashed_label.should == nil
@@ -37,69 +55,147 @@ describe Redistat::Model do
     ModelHelper1.depth.should == nil
     ModelHelper1.store_event.should == nil
     ModelHelper1.hashed_label.should == nil
-    
+
-    ModelHelper4.class_name.should == "FancyHelper"
+    ModelHelper4.scope.should == "FancyHelper"
     ModelHelper4.send(:name).should == "FancyHelper"
+    ModelHelper4.expire.should == {:hour => 24*3600}
   end
-  
+
   it "should store and fetch stats" do
-    ModelHelper1.store("sheep.black", {:count => 6, :weight => 461}, 4.hours.ago)
+    ModelHelper1.store("sheep.black", {:count => 6, :weight => 461}, @time.hours_ago(4))
-    ModelHelper1.store("sheep.black", {:count => 2, :weight => 156})
+    ModelHelper1.store("sheep.black", {:count => 2, :weight => 156}, @time)
-    
+
-    stats = ModelHelper1.fetch("sheep.black", 2.hours.ago, 1.hour.from_now)
+    stats = ModelHelper1.fetch("sheep.black", @time.hours_ago(2), @time.hours_since(1))
     stats.total["count"].should == 2
     stats.total["weight"].should == 156
     stats.first.should == stats.total
-    
+
-    stats = ModelHelper1.fetch("sheep.black", 5.hours.ago, 1.hour.from_now)
+    stats = ModelHelper1.fetch("sheep.black", @time.hours_ago(5), @time.hours_since(1))
     stats.total[:count].should == 8
     stats.total[:weight].should == 617
     stats.first.should == stats.total
-    
+
-    ModelHelper1.store("sheep.white", {:count => 5, :weight => 393}, 4.hours.ago)
+    ModelHelper1.store("sheep.white", {:count => 5, :weight => 393}, @time.hours_ago(4))
-    ModelHelper1.store("sheep.white", {:count => 4, :weight => 316})
+    ModelHelper1.store("sheep.white", {:count => 4, :weight => 316}, @time)
-    
+
-    stats = ModelHelper1.fetch("sheep.white", 2.hours.ago, 1.hour.from_now)
+    stats = ModelHelper1.fetch("sheep.white", @time.hours_ago(2), @time.hours_since(1))
     stats.total[:count].should == 4
     stats.total[:weight].should == 316
     stats.first.should == stats.total
-    
+
-    stats = ModelHelper1.fetch("sheep.white", 5.hours.ago, 1.hour.from_now)
+    stats = ModelHelper1.fetch("sheep.white", @time.hours_ago(5), @time.hours_since(1))
     stats.total[:count].should == 9
     stats.total[:weight].should == 709
     stats.first.should == stats.total
   end
-  
+
+  it "should store and fetch grouping enabled stats" do
+    ModelHelper1.store("sheep/black", {:count => 6, :weight => 461}, @time.hours_ago(4))
+    ModelHelper1.store("sheep/black", {:count => 2, :weight => 156}, @time)
+    ModelHelper1.store("sheep/white", {:count => 5, :weight => 393}, @time.hours_ago(4))
+    ModelHelper1.store("sheep/white", {:count => 4, :weight => 316}, @time)
+
+    stats = ModelHelper1.fetch("sheep/black", @time.hours_ago(2), @time.hours_since(1))
+    stats.total["count"].should == 2
+    stats.total["weight"].should == 156
+    stats.first.should == stats.total
+
+    stats = ModelHelper1.fetch("sheep/black", @time.hours_ago(5), @time.hours_since(1))
+    stats.total[:count].should == 8
+    stats.total[:weight].should == 617
+    stats.first.should == stats.total
+
+    stats = ModelHelper1.fetch("sheep/white", @time.hours_ago(2), @time.hours_since(1))
+    stats.total[:count].should == 4
+    stats.total[:weight].should == 316
+    stats.first.should == stats.total
+
+    stats = ModelHelper1.fetch("sheep/white", @time.hours_ago(5), @time.hours_since(1))
+    stats.total[:count].should == 9
+    stats.total[:weight].should == 709
+    stats.first.should == stats.total
+
+    stats = ModelHelper1.fetch("sheep", @time.hours_ago(2), @time.hours_since(1))
+    stats.total[:count].should == 6
+    stats.total[:weight].should == 472
+    stats.first.should == stats.total
+
+    stats = ModelHelper1.fetch("sheep", @time.hours_ago(5), @time.hours_since(1))
+    stats.total[:count].should == 17
+    stats.total[:weight].should == 1326
+    stats.first.should == stats.total
+  end
+
   it "should connect to different Redis servers on a per-model basis" do
     ModelHelper3.redis.client.db.should == 14
-    
+
-    ModelHelper3.store("sheep.black", {:count => 6, :weight => 461}, 4.hours.ago)
+    ModelHelper3.store("sheep.black", {:count => 6, :weight => 461}, @time.hours_ago(4), :label_indexing => false)
-    ModelHelper3.store("sheep.black", {:count => 2, :weight => 156})
+    ModelHelper3.store("sheep.black", {:count => 2, :weight => 156}, @time, :label_indexing => false)
-    
+
     db.keys("*").should be_empty
     ModelHelper1.redis.keys("*").should be_empty
     db("ModelHelper3").keys("*").should have(5).items
     ModelHelper3.redis.keys("*").should have(5).items
-    
+
-    stats = ModelHelper3.fetch("sheep.black", 2.hours.ago, 1.hour.from_now)
+    stats = ModelHelper3.fetch("sheep.black", @time.hours_ago(2), @time.hours_since(1), :label_indexing => false)
     stats.total["count"].should == 2
     stats.total["weight"].should == 156
-    stats = ModelHelper3.fetch("sheep.black", 5.hours.ago, 1.hour.from_now)
+    stats = ModelHelper3.fetch("sheep.black", @time.hours_ago(5), @time.hours_since(1), :label_indexing => false)
     stats.total[:count].should == 8
     stats.total[:weight].should == 617
-    
+
     ModelHelper3.connect_to(:port => 8379, :db => 13)
     ModelHelper3.redis.client.db.should == 13
-    
+
-    stats = ModelHelper3.fetch("sheep.black", 5.hours.ago, 1.hour.from_now)
+    stats = ModelHelper3.fetch("sheep.black", @time.hours_ago(5), @time.hours_since(1), :label_indexing => false)
     stats.total.should == {}
-    
+
     ModelHelper3.connect_to(:port => 8379, :db => 14)
     ModelHelper3.redis.client.db.should == 14
-    
+
-    stats = ModelHelper3.fetch("sheep.black", 5.hours.ago, 1.hour.from_now)
+    stats = ModelHelper3.fetch("sheep.black", @time.hours_ago(5), @time.hours_since(1), :label_indexing => false)
     stats.total[:count].should == 8
     stats.total[:weight].should == 617
   end
-  
+
+  describe "Write Buffer" do
+    before(:each) do
+      Redistat.buffer_size = 20
+    end
+
+    after(:each) do
+      Redistat.buffer_size = 0
+    end
+
+    it "should buffer calls in memory before committing to Redis" do
+      14.times do
+        ModelHelper1.store("sheep.black", {:count => 1, :weight => 461}, @time.hours_ago(4))
+      end
+      ModelHelper1.fetch("sheep.black", @time.hours_ago(5), @time.hours_since(1)).total.should == {}
+
+      5.times do
+        ModelHelper1.store("sheep.black", {:count => 1, :weight => 156}, @time)
+      end
+      ModelHelper1.fetch("sheep.black", @time.hours_ago(5), @time.hours_since(1)).total.should == {}
+
+      ModelHelper1.store("sheep.black", {:count => 1, :weight => 156}, @time)
+      stats = ModelHelper1.fetch("sheep.black", @time.hours_ago(5), @time.hours_since(1))
+      stats.total["count"].should == 20
+      stats.total["weight"].should == 7390
+    end
+
+    it "should force flush buffer when #flush(true) is called" do
+      ModelHelper1.fetch("sheep.black", @time.hours_ago(5), @time.hours_since(1)).total.should == {}
+      14.times do
+        ModelHelper1.store("sheep.black", {:count => 1, :weight => 461}, @time.hours_ago(4))
+      end
+      ModelHelper1.fetch("sheep.black", @time.hours_ago(5), @time.hours_since(1)).total.should == {}
+      Redistat.buffer.flush(true)
+
+      stats = ModelHelper1.fetch("sheep.black", @time.hours_ago(5), @time.hours_since(1))
+      stats.total["count"].should == 14
+      stats.total["weight"].should == 6454
+    end
+  end
+
 end

spec/options_spec.rb

@@ -0,0 +1,36 @@
+require "spec_helper"
+
+describe Redistat::Options do
+
+  before(:each) do
+    @helper = OptionsHelper.new
+    @helper.parse_options(:wtf => 'dude', :foo => 'booze')
+  end
+
+  it "should #parse_options" do
+    @helper.options[:hello].should == 'world'
+    @helper.options[:foo].should == 'booze'
+    @helper.options[:wtf].should == 'dude'
+    @helper.raw_options.should_not have_key(:hello)
+  end
+
+  it "should create option_accessors" do
+    @helper.hello.should == 'world'
+    @helper.hello('woooo')
+    @helper.hello.should == 'woooo'
+  end
+
+end
+
+class OptionsHelper
+  include Redistat::Options
+
+  option_accessor :hello
+
+  def default_options
+    { :hello => 'world',
+      :foo => 'bar' }
+  end
+
+
+end

spec/result_spec.rb

@@ -1,14 +1,14 @@
 require "spec_helper"
 
 describe Redistat::Result do
-  
+
   it "should should initialize properly" do
     options = {:from => "from", :till => "till"}
     result = Redistat::Result.new(options)
     result.from.should == "from"
     result.till.should == "till"
   end
-  
+
   it "should have set_or_incr method" do
     result = Redistat::Result.new
     result[:world].should be_nil
@@ -17,5 +17,5 @@ describe Redistat::Result do
     result.set_or_incr(:world, 8)
     result[:world].should == 11
   end
-  
+
-end
+end

spec/scope_spec.rb

@@ -2,20 +2,20 @@ require "spec_helper"
 
 describe Redistat::Scope do
   include Redistat::Database
-  
+
   before(:all) do
     db.flushdb
   end
-  
+
   before(:each) do
     @name = "PageViews"
     @scope = Redistat::Scope.new(@name)
   end
-  
+
   it "should initialize properly" do
     @scope.to_s.should == @name
   end
-  
+
   it "should increment next_id" do
     scope = Redistat::Scope.new("Visitors")
     @scope.next_id.should == 1
@@ -23,5 +23,5 @@ describe Redistat::Scope do
     @scope.next_id.should == 2
     scope.next_id.should == 2
   end
-  
+
-end
+end

spec/spec_helper.rb

@@ -2,6 +2,12 @@
 $LOAD_PATH.unshift(File.dirname(__FILE__))
 $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
 
+require 'simplecov'
+SimpleCov.start do
+  add_filter '/spec'
+  add_filter '/vendor'
+end
+
 # require stuff
 require 'redistat'
 require 'rspec'

spec/summary_spec.rb

@@ -2,7 +2,7 @@ require "spec_helper"
 
 describe Redistat::Summary do
   include Redistat::Database
-  
+
   before(:each) do
     db.flushdb
     @scope = "PageViews"
@@ -10,28 +10,43 @@ describe Redistat::Summary do
     @date = Time.now
     @key = Redistat::Key.new(@scope, @label, @date, {:depth => :day})
     @stats = {"views" => 3, "visitors" => 2}
+    @expire = {:hour => 24*3600}
   end
-  
+
   it "should update a single summary properly" do
-    Redistat::Summary.update(@key, @stats, :hour)
+    Redistat::Summary.send(:update_fields, @key, @stats, :hour)
     summary = db.hgetall(@key.to_s(:hour))
     summary.should have(2).items
     summary["views"].should == "3"
     summary["visitors"].should == "2"
-    
+
-    Redistat::Summary.update(@key, @stats, :hour)
+    Redistat::Summary.send(:update_fields, @key, @stats, :hour)
     summary = db.hgetall(@key.to_s(:hour))
     summary.should have(2).items
     summary["views"].should == "6"
     summary["visitors"].should == "4"
-    
+
-    Redistat::Summary.update(@key, {"views" => -4, "visitors" => -3}, :hour)
+    Redistat::Summary.send(:update_fields, @key, {"views" => -4, "visitors" => -3}, :hour)
     summary = db.hgetall(@key.to_s(:hour))
     summary.should have(2).items
     summary["views"].should == "2"
     summary["visitors"].should == "1"
   end
-  
+
+  it "should set key expiry properly" do
+    Redistat::Summary.update_all(@key, @stats, :hour,{:expire => @expire})
+    ((24*3600)-1..(24*3600)+1).should include(db.ttl(@key.to_s(:hour)))
+    [:day, :month, :year].each do |depth|
+      db.ttl(@key.to_s(depth)).should == -1
+    end
+
+    db.flushdb
+    Redistat::Summary.update_all(@key, @stats, :hour, {:expire => {}})
+    [:hour, :day, :month, :year].each do |depth|
+      db.ttl(@key.to_s(depth)).should == -1
+    end
+  end
+
   it "should update all summaries properly" do
     Redistat::Summary.update_all(@key, @stats, :sec)
     [:year, :month, :day, :hour, :min, :sec, :usec].each do |depth|
@@ -45,7 +60,16 @@ describe Redistat::Summary do
       end
     end
   end
-  
+
+  it "should update summaries even if no label is set" do
+    key = Redistat::Key.new(@scope, nil, @date, {:depth => :day})
+    Redistat::Summary.send(:update_fields, key, @stats, :hour)
+    summary = db.hgetall(key.to_s(:hour))
+    summary.should have(2).items
+    summary["views"].should == "3"
+    summary["visitors"].should == "2"
+  end
+
   it "should inject stats key grouping summaries" do
     hash = { "count/hello" => 3, "count/world"   => 7,
              "death/bomb"  => 4, "death/unicorn" => 3,
@@ -55,7 +79,7 @@ describe Redistat::Summary do
                     "death" => 7,  "death/bomb"  => 4, "death/unicorn" => 3,
                     "od"    => 15, :"od/sugar"   => 7, :"od/meth"      => 8 }
   end
-  
+
   it "should properly store key group summaries" do
     stats = {"views" => 3, "visitors/eu" => 2, "visitors/us" => 4}
     Redistat::Summary.update_all(@key, stats, :hour)
@@ -66,58 +90,87 @@ describe Redistat::Summary do
     summary["visitors/eu"].should == "2"
     summary["visitors/us"].should == "4"
   end
-  
+
   it "should not store key group summaries when option is disabled" do
     stats = {"views" => 3, "visitors/eu" => 2, "visitors/us" => 4}
-    Redistat::Summary.update_all(@key, stats, :hour, nil, false)
+    Redistat::Summary.update_all(@key, stats, :hour, {:enable_grouping => false})
     summary = db.hgetall(@key.to_s(:hour))
     summary.should have(3).items
     summary["views"].should == "3"
     summary["visitors/eu"].should == "2"
     summary["visitors/us"].should == "4"
   end
-  
+
   it "should store label-based grouping enabled stats" do
     stats = {"views" => 3, "visitors/eu" => 2, "visitors/us" => 4}
     label = "views/about_us"
     key = Redistat::Key.new(@scope, label, @date)
     Redistat::Summary.update_all(key, stats, :hour)
 
-    key.groups[0].label.should == "views/about_us"
+    key.groups[0].label.to_s.should == "views/about_us"
-    key.groups[1].label.should == "views"
+    key.groups[1].label.to_s.should == "views"
     child1 = key.groups[0]
     parent = key.groups[1]
-    
+
     label = "views/contact"
     key = Redistat::Key.new(@scope, label, @date)
     Redistat::Summary.update_all(key, stats, :hour)
-    
+
-    key.groups[0].label.should == "views/contact"
+    key.groups[0].label.to_s.should == "views/contact"
-    key.groups[1].label.should == "views"
+    key.groups[1].label.to_s.should == "views"
     child2 = key.groups[0]
-    
+
     summary = db.hgetall(child1.to_s(:hour))
     summary["views"].should == "3"
     summary["visitors/eu"].should == "2"
     summary["visitors/us"].should == "4"
-    
+
     summary = db.hgetall(child2.to_s(:hour))
     summary["views"].should == "3"
     summary["visitors/eu"].should == "2"
     summary["visitors/us"].should == "4"
-    
+
     summary = db.hgetall(parent.to_s(:hour))
     summary["views"].should == "6"
     summary["visitors/eu"].should == "4"
     summary["visitors/us"].should == "8"
   end
-  
+
+  it "should store label-based grouping enabled stats using a different group separator" do
+    Redistat.group_separator = '|'
+    stats = {"views" => 3, "visitors|eu" => 2, "visitors|us" => 4}
+    label = "views|about_us"
+    key = Redistat::Key.new(@scope, label, @date)
+    Redistat::Summary.update_all(key, stats, :hour)
+
+    key.groups[0].label.to_s.should == "views|about_us"
+    key.groups[1].label.to_s.should == "views"
+    child1 = key.groups[0]
+    parent = key.groups[1]
+
+    label = "views|contact"
+    key = Redistat::Key.new(@scope, label, @date)
+    Redistat::Summary.update_all(key, stats, :hour)
+
+    key.groups[0].label.to_s.should == "views|contact"
+    key.groups[1].label.to_s.should == "views"
+    child2 = key.groups[0]
+
+    summary = db.hgetall(child1.to_s(:hour))
+    summary["views"].should == "3"
+    summary["visitors|eu"].should == "2"
+    summary["visitors|us"].should == "4"
+
+    summary = db.hgetall(child2.to_s(:hour))
+    summary["views"].should == "3"
+    summary["visitors|eu"].should == "2"
+    summary["visitors|us"].should == "4"
+
+    summary = db.hgetall(parent.to_s(:hour))
+    summary["views"].should == "6"
+    summary["visitors|eu"].should == "4"
+    summary["visitors|us"].should == "8"
+    Redistat.group_separator = Redistat::GROUP_SEPARATOR
+  end
+
 end
-
-
-
-
-
-
-
-

spec/synchronize_spec.rb

@@ -0,0 +1,125 @@
+require "spec_helper"
+
+module Redistat
+  describe Synchronize do
+
+    let(:klass) { Synchronize }
+
+    describe '.included' do
+      it 'includes InstanceMethods in passed object' do
+        base = mock('Base')
+        base.should_receive(:include).with(klass::InstanceMethods)
+        klass.included(base)
+      end
+    end # included
+
+    describe '.monitor' do
+      it 'returns a Monitor instance' do
+        klass.monitor.should be_a(Monitor)
+      end
+
+      it 'caches Monitor instance' do
+        klass.monitor.object_id.should == klass.monitor.object_id
+      end
+    end # monitor
+
+    describe '.thread_safe' do
+      after { klass.instance_variable_set('@thread_safe', nil) }
+
+      it 'returns value of @thread_safe' do
+        klass.instance_variable_set('@thread_safe', true)
+        klass.thread_safe.should be_true
+      end
+
+      it 'defaults to false' do
+        klass.thread_safe.should be_false
+      end
+
+      it 'uses #synchronize' do
+        klass.monitor.should_receive(:synchronize).once
+        klass.thread_safe.should be_nil
+      end
+    end # thread_safe
+
+    describe '.thread_safe=' do
+      after { klass.instance_variable_set('@thread_safe', nil) }
+
+      it 'sets @thread_safe' do
+        klass.instance_variable_get('@thread_safe').should be_nil
+        klass.thread_safe = true
+        klass.instance_variable_get('@thread_safe').should be_true
+      end
+
+      it 'uses #synchronize' do
+        klass.monitor.should_receive(:synchronize).once
+        klass.thread_safe = true
+        klass.instance_variable_get('@thread_safe').should be_nil
+      end
+    end # thread_safe=
+
+    describe "InstanceMethods" do
+      subject { SynchronizeSpecHelper.new }
+
+      describe '.monitor' do
+        it 'defers to Redistat::Synchronize' do
+          klass.should_receive(:monitor).once
+          subject.monitor
+        end
+      end # monitor
+
+      describe '.thread_safe' do
+        it ' defers to Redistat::Synchronize' do
+          klass.should_receive(:thread_safe).once
+          subject.thread_safe
+        end
+      end # thread_safe
+
+      describe '.thread_safe=' do
+        it 'defers to Redistat::Synchronize' do
+          klass.should_receive(:thread_safe=).once.with(true)
+          subject.thread_safe = true
+        end
+      end # thread_safe=
+
+      describe 'when #thread_safe is true' do
+        before { subject.stub(:thread_safe).and_return(true) }
+
+        describe '.synchronize' do
+          it 'defers to #monitor' do
+            subject.monitor.should_receive(:synchronize).once
+            subject.synchronize { 'foo' }
+          end
+
+          it 'passes block along to #monitor.synchronize' do
+            yielded = false
+            subject.synchronize { yielded = true }
+            yielded.should be_true
+          end
+        end # synchronize
+      end # when #thread_safe is true
+
+      describe 'when #thread_safe is false' do
+        before { subject.stub(:thread_safe).and_return(false) }
+
+        describe '.synchronize' do
+          it 'does not defer to #monitor' do
+            subject.monitor.should_not_receive(:synchronize)
+            subject.synchronize { 'foo' }
+          end
+
+          it 'yields block' do
+            yielded = false
+            subject.synchronize { yielded = true }
+            yielded.should be_true
+          end
+        end # synchronize
+      end # when #thread_safe is false
+
+    end
+
+  end # Synchronize
+end # Redistat
+
+class SynchronizeSpecHelper
+  include Redistat::Synchronize
+end

spec/thread_safety_spec.rb

@@ -2,13 +2,13 @@ require "spec_helper"
 
 describe "Thread-Safety" do
   include Redistat::Database
-  
+
   before(:each) do
     db.flushdb
   end
-  
+
   #TODO should have more comprehensive thread-safe tests
-  
+
   it "should incr in multiple threads" do
     threads = []
     50.times do
@@ -19,7 +19,7 @@ describe "Thread-Safety" do
     threads.each { |t| t.join }
     db.get("spec:incr").should == "50"
   end
-  
+
   it "should store event in multiple threads" do
     class ThreadSafetySpec
       include Redistat::Model
@@ -35,5 +35,5 @@ describe "Thread-Safety" do
     result.total[:count].should == 50
     result.total[:rand].should <= 250
   end
-  
+
-end
+end