the scheduler and the active_record
update :
I have had issues with ActiveRecord > 2.2, a bunch of jobs scheduled where “missing the train” (MySQL gone).
I placed
ActiveRecord::Base.verify_active_connections!
at the beginning of my scheduled blocks and the issue seem to have vanished. Rails does that connection verification for each request, not for things outside of requests, have to do it by oneself.
The rufus-scheduler cares about scheduling, it’s written in Ruby. Some people use it in a RubyOnRails environment. I’ve seen people having trouble with ActiveRecord connections and the scheduler. The last instance of that was with Jim.
My initial reaction was “the rufus-scheduler has nothing to do with ActiveRecord’s connection management, it’s a dumb ruby library, why do you post that here ?”, but I didn’t word that reaction and I simply pointed at the ‘connection management’ issue. That paid well, since Jim came up with a solution, closing the connection at the end of each of scheduled job (ActiveRecord was implicitely opening a new connection for the job).
One of the great things with Ruby and Open Source is that you can look at the source. So I glanced at how ActiveRecord does the connection management and learnt a thing or two. (And with Ruby there is far less code to scan before reaching the objective)
The rufus-scheduler has one thread for the scheduling and then one thread for each job it triggers. One thread per job so that the scheduling thread doesn’t get blocked by the job executions. In its “thread safe” mode, ActiveRecord stores its database connections in a hash whose key is the current Thread object_id. So if you have lots of job that do business with ActiveRecord, you’ll have lots of threads and a[n open] connection for each of those threads.
Thanks to Jim for triggering this little active_record study and for sharing his solution.
Know thyself (and your tools).I ha
Nice summary of my issue.
Thanks again for your help with the solution!
Jim
September 15, 2008 at 9:01 pm
When using Rails 2.2 and allow_concurrency=true, is it still suggested to remove the AR connection manually at the end of each job?
Steven
November 5, 2008 at 3:56 pm
Hi Steven,
if you look at :
http://github.com/rails/rails/tree/v2.1.0/activerecord/lib/active_record/connection_adapters/abstract/connection_specification.rb#L26
then yes.
It seems that if you don’t remove the connection, AR will pick it up the next time. And if it has gone stale meanwhile, an error will get reported. If AR finds no connection, it will create a new, fresh, one. (That’s my interpretation of the AR code).
Best regards,
John
John Mettraux
November 5, 2008 at 11:56 pm
Yeah, thats 2.1, but what about 2.2? I think you end up at http://github.com/rails/rails/tree/v2.2.0/activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb#L319 instead…
Steven
November 9, 2008 at 1:15 am
Hi Steven,
sorry for overlooking the version info you gave.
Seems the connection_pool does the job of verifying the connection for us.
http://github.com/rails/rails/tree/v2.2.0/activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb#L173-195
http://github.com/rails/rails/tree/v2.2.0/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb#L133-138
things got easier. Now testing that is still required.
Best regards,
PS, the mailing list for rufus[-scheduler] is at http://groups.google.com/group/rufus-ruby
John Mettraux
November 9, 2008 at 3:02 am
John, thanks a lot.
Steven
November 9, 2008 at 9:04 am
Thanks for the tip on the thread that opens another connection, I was having issues with an every minute schedule, btw, the rufus rocks.
andres
March 4, 2009 at 10:54 pm