Path: | README.rdoc |
Last Update: | Mon Oct 15 02:55:14 +0000 2012 |
This is the Geni memcached client. It started out as a fork of fiveruns/memcache-client, which was a fork of seattle.rb‘s memcache-client, but over time, our client has diverged, and I‘ve rewritten the majority of the code. Of course, a lot of credit is due to those whose code served as a starting point for this code. Thanks to fauna for the idea to include the libmemcached source and the code to make it compile when installing the gem. Thanks to develon for the improvements to SegementedServer.
cache = Memcache.new(:server => "localhost:11211") cache.set('stuff', [:symbol, 'String', 1, {:bar => 5}]) cache.get('stuff') => [:symbol, "String", 1, {:bar => 5}] cache['things'] = {:foo => '1', :bar => [1,2,3]} cache['things'] => {:foo => "1", :bar => [1,2,3]}
Like fiveruns/memcache-client and fauna/memcached, memcache (shown in italics when I am referring to this library) is a memcached client, but it differs significantly from these clients in several important ways.
I tried to keep the basic interface as similar as I could to memcache-client. In some cases, memcache can be a near drop-in replacement for memcache-client. However, I did rename the main class from MemCache to Memcache to prevent confusion and to force those switching to memcache to update their code. Here are the notable interface changes:
cache.set('foo', :a, :expiry => 10.minutes) cache.set('bar', :b, :expiry => Time.parse('5:51pm Nov 24, 2018')) cache.set('baz', 'c', :expiry => 30.minutes, :raw => true)
cache.get('foo') # => :a cache.get(['foo', 'bar']) # => {"foo"=>:a, "bar"=>:b} cache.get(['foo']) # => {"foo"=>:a}
cache.get('foo', :expiry => 1.day)
cache.set('foo', :aquatic, :flags => 0b11101111) value = cache.get('foo') => :aquatic value.memcache_flags.to_s(2) => "11101111" cache.set('foo', 'aquatic', :raw => true, :flags => 0xff08) cache.get('foo', :raw => true).memcache_flags.to_s(2) => "1111111100001000"
cache.add('foo', 1) cache.add('foo', 0) cache.get('foo') => 1 cache.replace('foo', 2) cache.get('foo') => 2 cache.write('foo', 'bar') ## shortcut for cache.set('foo', 'bar', :raw => true) cache.append('foo', 'none') ## append and prepend only works on raw values cache.prepend('foo', 'foo') ## cache.read('foo') ## shortcut for cache.get('foo', :raw => true) => "foobarnone"
value = cache.get('foo', :cas => true) cache.cas('foo', value.upcase, :cas => value.memcache_cas) cache.get('foo') => "FOOBARNONE" value = cache.get('foo', :cas => true) cache.set('foo', 'modified') cache.cas('foo', value.downcase, :cas => value.memcache_cas) cache.get('foo') => "modified"
The underlying architechture of memcache is more modular than memcache-client. A given Memcache instance has a group of servers, just like before, but much more of the functionality is encapsulated inside the Memcache::Server object. Really, a Server object is a thin wrapper around an remote memcached server that takes care of the socket and protocol details along with basic error handling. The Memcache class handles the partitioning algorithm, marshaling of ruby objects and various higher-level methods.
By encapsulating the protocol inside the Server object, it becomes very easy to plug-in alternate backend server implementations. Right now, there are three basic, alternate servers:
Memcached limits the size of values to 1MB. This is done to reduce memory usage, but it means that large data structures, which are also often costly to compute, cannot be stored easily. We solve this problem by providing an additional server called Memcache::SegmentedServer. It inherits from Memcache::Server, but includes code to segment and reassemble large values. Mike Stangel at Geni originally wrote this code as an extension to memcache-client and I adapted it for the new architecture.
You can use segmented values either by passing SegmentedServer objects to Memcache, or you can use the segment_large_values option.
server = Memcache::SegmentedServer.new(:host => 'localhost', :port => 11211) cache = Memcache.new(:server => server) cache = Memcache.new(:server => 'localhost:11211', :segment_large_values => true)
We handle errors differently in memcache than memcache-client does. Whenever there is a connection error or other fatal error, memcache-client marks the offending server as dead for 30 seconds, and all calls that require that server fail for the next 30 seconds. This was unacceptable for us in a production environment. We tried changing the retry timeout to 1 second, but still found our exception logs filling up with failed web requests whenever a network connection was broken.
So, the default behavior in memcache is for reads to be stable even if the underlying server is unavailable. This means, that instead of raising an exception, a read will just return nil if the server is down. Of course, you need to monitor your memcached servers to make sure they aren‘t down for long, but this allows your site to be resilient to minor network blips. Any error that occurs while unmarshalling a stored object will also return nil.
Writes, on the other hand, cannot just be ignored when the server is down. For this reason, every write operation is retried once by closing and reopening the connection before finally marking a server as dead and raising an exception. We will not attempt to read from a dead server for 5 seconds, but a write will always attempt to revive a dead server by attempting to connect.
Unlike the other ruby memcache clients, keys in memcache can contain spaces. This is possible because the backend transparently enscapes all space characters, and is especially important if you are using method_cache or record_cache. Memcache::Server implements this escaping using gsub and it adds a slight performance penalty when escaping is necessary. NativeServer implements this escaping directly in C, and the performance overhead is negligible.
You can also partition your keys into different namespaces for convenience. This is done by prefixing all keys in the backend server with "namespace:". However, the hash keys returned by multi gets do not contain the prefix. In this way, the namespace can be totally transparent to your code. You can also determine whether the prefix is used for hashing with the following option:
The Memcache::NativeServer backend provides native bindings to libmecached. This is significantly faster than using Memcache::Server as demonstrated by runnning bench/benchmark.rb. NativeServer encapsulates a set of remote servers and allows you to use the various hashing methods in libmemcached.
You can use native bindings either by passing NativeServer objects to Memcache, or you can use the native option. Native bindings are compatible with segmented values through the SegmentedNativeServer object or by combining the native option with segment_large_values.
server = Memcache::NativeServer.new(:servers => ['localhost:11211', 'localhost:11212']) cache = Memcache.new(:server => server) cache = Memcache.new(:servers => ['localhost:11211', 'localhost:11212'], :native => true)
NativeServer also accepts a few other options:
:default :md5 :crc :fnv1_64 :fnv1a_64 :fnv1_32 :fnv1a_32 :jenkins :hsieh :murmur.
NOTE: Even though there is a libmemcached method named :default (which is actually Jenkins‘s one-at-a-time hash), the default hashing method if you don‘t specify one is :crc.
:modula :consistent :ketama :ketama_spy
NOTE: :modula is the default. internally, :consistent is an alias for :ketama, and :ketama_spy provides compatibility with the SPY Memcached client for Java.
$ sudo gem install memcache --source http://gemcutter.org
Copyright (c) 2010 Justin Balthrop, Geni.com; Published under The MIT License, see the LICENSE file.
ext/extconf.rb Copyright (c) 2010 Cloudburst, LLC, licensed under the AFL3 license, and used with permission; see the ext/LICENSE_AFL3 file.