Class MCollective::RPC::Client
In: lib/mcollective/rpc/client.rb
Parent: Object

The main component of the Simple RPC client system, this wraps around MCollective::Client and just brings in a lot of convention and standard approached.

Methods

Attributes

agent  [R] 
client  [R] 
config  [RW] 
ddl  [R] 
discovery_timeout  [RW] 
filter  [RW] 
limit_targets  [R] 
output_format  [R] 
progress  [RW] 
stats  [R] 
timeout  [RW] 
verbose  [RW] 

Public Class methods

Creates a stub for a remote agent, you can pass in an options array in the flags which will then be used else it will just create a default options array with filtering enabled based on the standard command line use.

  rpc = RPC::Client.new("rpctest", :configfile => "client.cfg", :options => options)

You typically would not call this directly you‘d use MCollective::RPC#rpcclient instead which is a wrapper around this that can be used as a Mixin

[Source]

    # File lib/mcollective/rpc/client.rb, line 19
19:             def initialize(agent, flags = {})
20:                 if flags.include?(:options)
21:                     options = flags[:options]
22: 
23:                 elsif @@initial_options
24:                     options = Marshal.load(@@initial_options)
25: 
26:                 else
27:                     oparser = MCollective::Optionparser.new({:verbose => false, :progress_bar => true, :mcollective_limit_targets => false}, "filter")
28: 
29:                     options = oparser.parse do |parser, options|
30:                         if block_given?
31:                             yield(parser, options)
32:                         end
33: 
34:                         Helpers.add_simplerpc_options(parser, options)
35:                     end
36: 
37:                     @@initial_options = Marshal.dump(options)
38:                 end
39: 
40:                 @stats = Stats.new
41:                 @agent = agent
42:                 @discovery_timeout = options[:disctimeout]
43:                 @timeout = options[:timeout]
44:                 @verbose = options[:verbose]
45:                 @filter = options[:filter]
46:                 @config = options[:config]
47:                 @discovered_agents = nil
48:                 @progress = options[:progress_bar]
49:                 @limit_targets = options[:mcollective_limit_targets]
50:                 @output_format = options[:output_format] || :console
51:                 @force_direct_request = false
52: 
53:                 agent_filter agent
54: 
55:                 @client = MCollective::Client.new(@config)
56:                 @client.options = options
57: 
58:                 @collective = @client.collective
59: 
60:                 # if we can find a DDL for the service override
61:                 # the timeout of the client so we always magically
62:                 # wait appropriate amounts of time.
63:                 #
64:                 # We add the discovery timeout to the ddl supplied
65:                 # timeout as the discovery timeout tends to be tuned
66:                 # for local network conditions and fact source speed
67:                 # which would other wise not be accounted for and
68:                 # some results might get missed.
69:                 #
70:                 # We do this only if the timeout is the default 5
71:                 # seconds, so that users cli overrides will still
72:                 # get applied
73:                 begin
74:                     @ddl = DDL.new(agent)
75:                     @timeout = @ddl.meta[:timeout] + @discovery_timeout if @timeout == 5
76:                 rescue Exception => e
77:                     Log.debug("Could not find DDL: #{e}")
78:                     @ddl = nil
79:                 end
80: 
81:                 STDERR.sync = true
82:                 STDOUT.sync = true
83:             end

Public Instance methods

Sets the agent filter

[Source]

     # File lib/mcollective/rpc/client.rb, line 302
302:             def agent_filter(agent)
303:                 @filter["agent"] << agent
304:                 @filter["agent"].compact!
305:                 reset
306:             end

Sets the class filter

[Source]

     # File lib/mcollective/rpc/client.rb, line 278
278:             def class_filter(klass)
279:                 @filter["cf_class"] << klass
280:                 @filter["cf_class"].compact!
281:                 reset
282:             end

Sets the collective we are communicating with

[Source]

     # File lib/mcollective/rpc/client.rb, line 421
421:             def collective=(c)
422:                 @collective = c
423:                 @client.options[:collective] = c
424:             end

Set a compound filter

[Source]

     # File lib/mcollective/rpc/client.rb, line 316
316:             def compound_filter(filter)
317:                 @filter["compound"] = Matcher::Parser.new(filter).execution_stack
318:                 reset
319:             end

Constructs custom requests with custom filters and discovery data the idea is that this would be used in web applications where you might be using a cached copy of data provided by a registration agent to figure out on your own what nodes will be responding and what your filter would be.

This will help you essentially short circuit the traditional cycle of:

mc discover / call / wait for discovered nodes

by doing discovery however you like, contructing a filter and a list of nodes you expect responses from.

Other than that it will work exactly like a normal call, blocks will behave the same way, stats will be handled the same way etcetc

If you just wanted to contact one machine for example with a client that already has other filter options setup you can do:

puppet.custom_request("runonce", {}, ["your.box.com"], {:identity => "your.box.com"})

This will do runonce action on just ‘your.box.com’, no discovery will be done and after receiving just one response it will stop waiting for responses

If direct_addressing is enabled in the config file you can provide an empty hash as a filter, this will force that request to be a directly addressed request which technically does not need filters. If you try to use this mode with direct addressing disabled an exception will be raise

[Source]

     # File lib/mcollective/rpc/client.rb, line 232
232:             def custom_request(action, args, expected_agents, filter = {}, &block)
233:                 @ddl.validate_request(action, args) if @ddl
234: 
235:                 if filter == {} && !Config.instance.direct_addressing
236:                     raise "Attempted to do a filterless custom_request without direct_addressing enabled, preventing unexpected call to all nodes"
237:                 end
238: 
239: 
240:                 @stats.reset
241: 
242:                 custom_filter = Util.empty_filter
243:                 custom_options = options.clone
244: 
245:                 # merge the supplied filter with the standard empty one
246:                 # we could just use the merge method but I want to be sure
247:                 # we dont merge in stuff that isnt actually valid
248:                 ["identity", "fact", "agent", "cf_class", "compound"].each do |ftype|
249:                     if filter.include?(ftype)
250:                         custom_filter[ftype] = [filter[ftype], custom_filter[ftype]].flatten
251:                     end
252:                 end
253: 
254:                 # ensure that all filters at least restrict the call to the agent we're a proxy for
255:                 custom_filter["agent"] << @agent unless custom_filter["agent"].include?(@agent)
256:                 custom_options[:filter] = custom_filter
257: 
258:                 # Fake out the stats discovery would have put there
259:                 @stats.discovered_agents([expected_agents].flatten)
260: 
261:                 # Handle fire and forget requests
262:                 if args.include?(:process_results) && args[:process_results] == false
263:                     return fire_and_forget_request(action, args, custom_filter)
264:                 end
265: 
266:                 # Now do a call pretty much exactly like in method_missing except with our own
267:                 # options and discovery magic
268:                 if block_given?
269:                     call_agent(action, args, custom_options, [expected_agents].flatten) do |r|
270:                         block.call(r)
271:                     end
272:                 else
273:                     call_agent(action, args, custom_options, [expected_agents].flatten)
274:                 end
275:             end

Disconnects cleanly from the middleware

[Source]

    # File lib/mcollective/rpc/client.rb, line 86
86:             def disconnect
87:                 @client.disconnect
88:             end

Does discovery based on the filters set, if a discovery was previously done return that else do a new discovery.

Alternatively if identity filters are given and none of them are regular expressions then just use the provided data as discovered data, avoiding discovery

Discovery can be forece if direct_addressing is enabled by passing in an array of hosts with :hosts or JSON data like those produced by mcollective rpc JSON output

Will show a message indicating its doing discovery if running verbose or if the :verbose flag is passed in.

Use reset to force a new discovery

[Source]

     # File lib/mcollective/rpc/client.rb, line 348
348:             def discover(flags={})
349:                 flags.include?(:verbose) ? verbose = flags[:verbose] : verbose = @verbose
350: 
351:                 verbose = false unless @output_format == :console
352: 
353:                 unless @discovered_agents
354:                     # if either hosts or json is supplied try to figure out discovery data from there
355:                     # if direct_addressing is not enabled this is a critical error as the user might
356:                     # not have supplied filters so raise an exception
357:                     if flags[:hosts] || flags[:json]
358:                         raise "Can only supply discovery data if direct_addressing is enabled" unless Config.instance.direct_addressing
359: 
360:                         hosts = []
361: 
362:                         if flags[:hosts]
363:                             hosts = Helpers.extract_hosts_from_array(flags[:hosts])
364:                         elsif flags[:json]
365:                             hosts = Helpers.extract_hosts_from_json(flags[:json])
366:                         end
367: 
368:                         raise "Could not find any hosts in discovery data provided" if hosts.empty?
369: 
370:                         @discovered_agents = hosts
371:                         @force_direct_request = true
372: 
373:                     # if an identity filter is supplied and it is all strings no regex we can use that
374:                     # as discovery data, technically the identity filter is then redundant if we are
375:                     # in direct addressing mode and we could empty it out but this use case should
376:                     # only really be for a few -I's on the cli
377:                     #
378:                     # For safety we leave the filter in place for now, that way we can support this
379:                     # enhancement also in broadcast mode
380:                     elsif options[:filter]["identity"].size > 0
381:                         regex_filters = options[:filter]["identity"].select{|i| i.match("^\/")}.size
382: 
383:                         if regex_filters == 0
384:                             @discovered_agents = options[:filter]["identity"].clone
385:                             @force_direct_request = true if Config.instance.direct_addressing
386:                         end
387:                     end
388:                 end
389: 
390:                 # All else fails we do it the hard way using a traditional broadcast
391:                 unless @discovered_agents
392:                     @stats.time_discovery :start
393: 
394:                     STDERR.print("Determining the amount of hosts matching filter for #{discovery_timeout} seconds .... ") if verbose
395:                     @discovered_agents = @client.discover(@filter, @discovery_timeout)
396:                     @force_direct_request = false
397:                     STDERR.puts(@discovered_agents.size) if verbose
398: 
399:                     @stats.time_discovery :end
400:                 end
401: 
402:                 @stats.discovered_agents(@discovered_agents)
403:                 RPC.discovered(@discovered_agents)
404: 
405:                 @discovered_agents
406:             end

Sets the fact filter

[Source]

     # File lib/mcollective/rpc/client.rb, line 285
285:             def fact_filter(fact, value=nil, operator="=")
286:                 return if fact.nil?
287:                 return if fact == false
288: 
289:                 if value.nil?
290:                     parsed = Util.parse_fact_string(fact)
291:                     @filter["fact"] << parsed unless parsed == false
292:                 else
293:                     parsed = Util.parse_fact_string("#{fact}#{operator}#{value}")
294:                     @filter["fact"] << parsed unless parsed == false
295:                 end
296: 
297:                 @filter["fact"].compact!
298:                 reset
299:             end

Returns help for an agent if a DDL was found

[Source]

    # File lib/mcollective/rpc/client.rb, line 91
91:             def help(template)
92:                 if @ddl
93:                     @ddl.help(template)
94:                 else
95:                     return "Can't find DDL for agent '#{@agent}'"
96:                 end
97:             end

Sets the identity filter

[Source]

     # File lib/mcollective/rpc/client.rb, line 309
309:             def identity_filter(identity)
310:                 @filter["identity"] << identity
311:                 @filter["identity"].compact!
312:                 reset
313:             end

Sets and sanity checks the limit_targets variable used to restrict how many nodes we‘ll target

[Source]

     # File lib/mcollective/rpc/client.rb, line 428
428:             def limit_targets=(limit)
429:                 if limit.is_a?(String)
430:                     raise "Invalid limit specified: #{limit} valid limits are /^\d+%*$/" unless limit =~ /^\d+%*$/
431:                     @limit_targets = limit
432:                 elsif limit.respond_to?(:to_i)
433:                     limit = limit.to_i
434:                     limit = 1 if limit == 0
435:                     @limit_targets = limit
436:                 else
437:                     raise "Don't know how to handle limit of type #{limit.class}"
438:                 end
439:             end

Magic handler to invoke remote methods

Once the stub is created using the constructor or the RPC#rpcclient helper you can call remote actions easily:

  ret = rpc.echo(:msg => "hello world")

This will call the ‘echo’ action of the ‘rpctest’ agent and return the result as an array, the array will be a simplified result set from the usual full MCollective::Client#req with additional error codes and error text:

{

  :sender => "remote.box.com",
  :statuscode => 0,
  :statusmsg => "OK",
  :data => "hello world"

}

If :statuscode is 0 then everything went find, if it‘s 1 then you supplied the correct arguments etc but the request could not be completed, you‘ll find a human parsable reason in :statusmsg then.

Codes 2 to 5 maps directly to UnknownRPCAction, MissingRPCData, InvalidRPCData and UnknownRPCError see below for a description of those, in each case :statusmsg would be the reason for failure.

To get access to the full result of the MCollective::Client#req calls you can pass in a block:

  rpc.echo(:msg => "hello world") do |resp|
     pp resp
  end

In this case resp will the result from MCollective::Client#req. Instead of returning simple text and codes as above you‘ll also need to handle the following exceptions:

UnknownRPCAction - There is no matching action on the agent MissingRPCData - You did not supply all the needed parameters for the action InvalidRPCData - The data you did supply did not pass validation UnknownRPCError - Some other error prevented the agent from running

During calls a progress indicator will be shown of how many results we‘ve received against how many nodes were discovered, you can disable this by setting progress to false:

  rpc.progress = false

This supports a 2nd mode where it will send the SimpleRPC request and never handle the responses. It‘s a bit like UDP, it sends the request with the filter attached and you only get back the requestid, you have no indication about results.

You can invoke this using:

  puts rpc.echo(:process_results => false)

This will output just the request id.

[Source]

     # File lib/mcollective/rpc/client.rb, line 179
179:             def method_missing(method_name, *args, &block)
180:                 # set args to an empty hash if nothings given
181:                 args = args[0]
182:                 args = {} if args.nil?
183: 
184:                 action = method_name.to_s
185: 
186:                 @stats.reset
187: 
188:                 @ddl.validate_request(action, args) if @ddl
189: 
190:                 # Handle single target requests by doing discovery and picking
191:                 # a random node.  Then do a custom request specifying a filter
192:                 # that will only match the one node.
193:                 if @limit_targets
194:                     target_nodes = pick_nodes_from_discovered(@limit_targets)
195:                     Log.debug("Picked #{target_nodes.join(',')} as limited target(s)")
196: 
197:                     custom_request(action, args, target_nodes, {"identity" => /^(#{target_nodes.join('|')})$/}, &block)
198:                 else
199:                     # Normal agent requests as per client.action(args)
200:                     call_agent(action, args, options, &block)
201:                 end
202:             end

Creates a suitable request hash for the SimpleRPC agent.

You‘d use this if you ever wanted to take care of sending requests on your own - perhaps via Client#sendreq if you didn‘t care for responses.

In that case you can just do:

  msg = your_rpc.new_request("some_action", :foo => :bar)
  filter = your_rpc.filter

  your_rpc.client.sendreq(msg, msg[:agent], filter)

This will send a SimpleRPC request to the action some_action with arguments :foo = :bar, it will return immediately and you will have no indication at all if the request was receieved or not

Clearly the use of this technique should be limited and done only if your code requires such a thing

[Source]

     # File lib/mcollective/rpc/client.rb, line 118
118:             def new_request(action, data)
119:                 callerid = PluginManager["security_plugin"].callerid
120: 
121:                 {:agent  => @agent,
122:                  :action => action,
123:                  :caller => callerid,
124:                  :data   => data}
125:             end

Provides a normal options hash like you would get from Optionparser

[Source]

     # File lib/mcollective/rpc/client.rb, line 410
410:             def options
411:                 {:disctimeout => @discovery_timeout,
412:                  :timeout => @timeout,
413:                  :verbose => @verbose,
414:                  :filter => @filter,
415:                  :collective => @collective,
416:                  :output_format => @output_format,
417:                  :config => @config}
418:             end

Resets various internal parts of the class, most importantly it clears out the cached discovery

[Source]

     # File lib/mcollective/rpc/client.rb, line 323
323:             def reset
324:                 @discovered_agents = nil
325:             end

Reet the filter to an empty one

[Source]

     # File lib/mcollective/rpc/client.rb, line 328
328:             def reset_filter
329:                 @filter = Util.empty_filter
330:                 agent_filter @agent
331:             end

[Validate]