processi

about processes and engines

Archive for the ‘scheduling’ Category

rufus-scheduler 2.0.12 released

rufus-scheduler is a thread-based scheduler written in Ruby. It lets you write code like:

require 'rufus-scheduler'

s = Rufus::Scheduler.start_new

s.every '10m' do
  puts 'open the window for the cat'
end

s.at '2012-01-01 12:00' do
  puts 'reminder: wife's birthday'
end

s.cron '0 22 * * 1-5' do
  puts 'activate the security system'
end

s.join # in case of stand-alone script...

The main addition brought by this release is the :mutex attribute when scheduling blocks of code. I was seeing people misusing :blocking => true to exclude block execution overlapping. It works but the scheduler is blocked as well, and crons might get skipped:

s.every '10m', :blocking => true do
  puts 'doing this...'
  sleep 60 * 60 # 1 hour
  puts 'done.'
end

# if the scheduler is in the blocking task above, crons will get skipped...
s.at '2012-01-01 12:00' do
  puts 'do that.'
end

My advice was to use mutexes instead:

$m = Mutex.new

s.every '10m' do
  $m.synchronize do
    puts 'doing this...'
    sleep 60 * 60 # 1 hour
    puts 'done.'
  end
end

# if the scheduler is in the blocking task above, crons will get skipped...
s.at '2012-01-01 12:00' do
  $m.synchronize do
    puts 'do that.'
  end
end

For those of you who use such mutexes and are OK with them wrapping the whole block, rufus-scheduler 2.0.12 introduces the :mutex attribute:

s.every '10m', :mutex => 'my_mutex_name' do
  puts 'doing this...'
  sleep 60 * 60 # 1 hour
  puts 'done.'
end

# if the scheduler is in the blocking task above, crons will get skipped...
s.at '2012-01-01 12:00', :mutex => 'my_mutex_name'  do
  puts 'do that.'
end

Where rufus-scheduler receives a mutex name and manages it for you.

When one wants more control over the granularity, it’s OK to do:

$m = Mutex.new

s.every '10m', :mutex => $m do
  puts 'doing this...'
  sleep 60 * 60 # 1 hour
  puts 'done.'
end

# if the scheduler is in the blocking task above, crons will get skipped...
s.at '2012-01-01 12:00' do
  puts 'do that'
  $m.synchronize do
    puts 'and that.'
  end
end

 

Remember that rufus-scheduler is not a cron replacement. Many thanks to all the people who complained or helped in the development of this piece of software over the years.

 

source: https://github.com/jmettraux/rufus-scheduler
issues: https://github.com/jmettraux/rufus-scheduler/issues
mailing list: http://groups.google.com/group/rufus-ruby
irc: freenode #ruote

 

Written by John Mettraux

October 28, 2011 at 11:27 am

rufus-scheduler 2.0, with em flavour

This blog is turning into a release gallery.

Anyway. Just released rufus-scheduler 2.0. It’s a complete rewrite. The only new “feature” is an EventMachine mode.

The plain scheduler uses a thread waking up every 0.330s (by default) to check for jobs to trigger, while the em-based scheduler uses an EventMachine timer.

Other flavour for the core loop are possible, why not something 1.9 fiber based in the short term ?

This release 2.0 is mostly backward compatible. Only some corner cases and advanced usages (block arity) have changed (they have been simplified). Having a look at the readme should help.

require 'rubygems'
require 'rufus/scheduler' # sudo gem install rufus-scheduler

s = Rufus::Scheduler.start_new

will still work, it will return an instance of Rufus::Scheduler::PlainScheduler, but if an EM reactor is running, it will stick to it and return a Rufus::Scheduler::EmScheduler instance.

source : http://github.com/jmettraux/rufus-scheduler/
rdoc : http://rufus.rubyforge.org/rufus-scheduler/
ml : http://groups.google.com/group/rufus-ruby

Many thanks to Klaas Jan Wierenga for contributing nice stress tests (for the 1.0 and the 2.0 branches), thanks as well to Kenneth Kalmer for its daemon-kit + rufus-scheduler effort.

 

Written by John Mettraux

May 7, 2009 at 2:27 am

Posted in ruby, rufus, scheduling

rufus-scheduler 1.0.12 released

Just released version 1.0.12 of the rufus-scheduler gem.

It contains a few improvements, like a more generic find_jobs() methods, but the main new feature is the :timeout attribute, which, when present, limits the time allocated for a triggered job.

require 'rubygems'
require 'rufus/scheduler' # sudo gem install rufus-scheduler

s = Rufus::Scheduler.start_new

s.every "10h30m", :timeout => "3h" do
  do_that_long_job()
end

Every 10 hours and 30 minutes the ‘long job’ will get triggered, if after 3 hours it isn’t done, it will get interrupted via a Rufus::TimeOutError.

Previously, users were wrapping the job inside a ‘Timeout’ construct which used a second thread. The :timeout attribute simply schedules a rufus job for the expiration, leveraging its own infrastructure.

Thanks to Xianhang Zhang, K Liu and Tim Uckun for their feedback.

rufus : http://rufus.rubyforge.org
the scheduler : http://rufus.rubyforge.org/rufus-scheduler
source : http://github.com/jmettraux/rufus-scheduler
mailing list : http://groups.google.com/group/rufus-ruby

Written by John Mettraux

December 18, 2008 at 3:11 pm

Posted in ruby, rufus, scheduling

the scheduler and the active_record

update :

I have had issues with ActiveRecord > 2.2, a bunch of jobs scheduled where “missing the train” (MySQL gone).

I placed

  ActiveRecord::Base.verify_active_connections!

at the beginning of my scheduled blocks and the issue seem to have vanished. Rails does that connection verification for each request, not for things outside of requests, have to do it by oneself.


The rufus-scheduler cares about scheduling, it’s written in Ruby. Some people use it in a RubyOnRails environment. I’ve seen people having trouble with ActiveRecord connections and the scheduler. The last instance of that was with Jim.

My initial reaction was “the rufus-scheduler has nothing to do with ActiveRecord’s connection management, it’s a dumb ruby library, why do you post that here ?”, but I didn’t word that reaction and I simply pointed at the ‘connection management’ issue. That paid well, since Jim came up with a solution, closing the connection at the end of each of scheduled job (ActiveRecord was implicitely opening a new connection for the job).

One of the great things with Ruby and Open Source is that you can look at the source. So I glanced at how ActiveRecord does the connection management and learnt a thing or two. (And with Ruby there is far less code to scan before reaching the objective)

The rufus-scheduler has one thread for the scheduling and then one thread for each job it triggers. One thread per job so that the scheduling thread doesn’t get blocked by the job executions. In its “thread safe” mode, ActiveRecord stores its database connections in a hash whose key is the current Thread object_id. So if you have lots of job that do business with ActiveRecord, you’ll have lots of threads and a[n open] connection for each of those threads.

Thanks to Jim for triggering this little active_record study and for sharing his solution.

Know thyself (and your tools).I ha

Written by John Mettraux

September 14, 2008 at 6:15 am

rufus-scheduler goes to the confs

The rufus-scheduler is a spin off of ruote, a workflow and BPM engine dearly needs a scheduler, for timeouts or simply for triggering an activity at the right time. Rufus-scheduler is the chronograph of ruote.

Rufus-scheduler alone, as a ruby gem, has a wider audience than ruote has. It even goes to conferences.

I should have written about it sooner, but Tomek Stachewicz talked about it in its Euroko2008 talk “Sharing The Load“. Sorry Tomek, I wrote a scheduler but I’m late on this one.

On the 5th of september, Mike Subelsky talked about “Ruby in the Cloud” at the Lone Star Ruby Conference 2008.

Thanks a ton guys !

Written by John Mettraux

September 8, 2008 at 12:37 am

Posted in ruby, rufus, scheduling

rufus-scheduler 1.0.9 released

just released the “rufus-scheduler” 1.0.9 [gem].

It fixes a few bugs, mainly making jobs visible via get_job(job_id) even when they are right in the middle of execution.

It also introduces 4 new shorter aliases : “at”, “in”, “every” and “cron” (to “schedule_at”, “schedule_in”, “schedule_every” and “schedule” respectively) :

require 'rubygems'
require 'rufus/scheduler' # gem 'rufus-scheduler'

scheduler = Rufus::Scheduler.start_new

scheduler.in("3d") do
  regenerate_monthly_report()
end
  #
  # will call the regenerate_monthly_report method
  # in 3 days from now

scheduler.every "10m10s" do
  # every 10 minutes and 10 seconds
  check_score(favourite_team) 
end

scheduler.cron "0 22 * * 1-5" do
  log.info "activating security system..."
  activate_security_system()
end

scheduler.at "Sun Oct 07 14:24:01 +0900 2009" do
  init_self_destruction_sequence()
end

Changelog :

– bug #21262 : brought back (proxy) duration_to_f to Rufus::Scheduler
– todo #21251 : added ‘at’, ‘cron’, ‘in’ and ‘every’ shortcut methods
– todo #21203 : keeping track of At and EveryJob until last trigger is over
– todo #20823 : now raising exception in case of bad ‘at’ parameter
– todo #21194 : added trigger_thread to Job class
– todo #21193 : spinned CronLine out to rufus/cronline.rb
– bug #20893 : sometimes unschedule(every_job) not working when job was active (performing). Fixed.

Links :

http://github.com/jmettraux/rufus-scheduler
http://rubyforge.org/projects/rufus/
http://rufus.rubyforge.org/rufus-scheduler/files/README_txt.html
http://groups.google.com/group/rufus-ruby

Thanks to Sean Liu, Adam and Rael for their help.

Written by John Mettraux

July 18, 2008 at 4:52 am

bike geolocation with Ruby and RMagick

I had a pleasant surprise today, Thibaut mailed me about his use of the Rufus::Scheduler in his Paris bike geolocation application, for calling his image generation code every twenty minutes.

Read more about how Thibaut leverages Hpricot and RMagick in his post entitled Data Visualization with Ruby and RMagick – Where Are Those Bikes ?

(Seems like I’m using geolocation quite liberally, are those bikes internet devices ? No the docking stations are, they feed their info about bikes availability to their center, which then provides the information via a website)

Written by John Mettraux

April 1, 2008 at 1:28 am

Posted in ruby, rufus, scheduling